Basic HDFS Commands for Begginers

1. To display files and directories in HDFS
  $ hadoop fs -ls

 2. To create a directory in HDFS
$ hadoop fs -mkdir HDFSPractice

3. To dispaly the contents of the directory
$ hadoop fs -ls <Directory_Name>  
           

Since, our directory is new we don't have any files in the directory. So let's add some files to our directory.



4. Loading files into HDFS from our local file system
 $hadoop fs -copyFromLocal /usr/hadoopPractice/employee.txt HDFSPractice/
   

Note:  1. hadoop is case sensitive so copyFromLocal is different from copyfromLocal.
      
Now lets display the contents of the HDFSPractice directory as in point 3. 

5. Remove/Delete file from HDFS
$ hadoop fs -rm HDFSPractice/employee.txt

6. Loading the file in our HDFS to our local file system
$  hadoop fs -copyToLocal HDFSPractice/employee.txt /usr/





Goals of HDFS
1. Very large distributed file system
          ---- 10k nodes, 10PB data, 100 million files.
2. User Space, runs on heterogeneous OS
3.Optimized for batch processing
          -----  Locations of data exposed so that the computations can move to where data resides
          -----  Provides very high aggregate bandwidth
4.Assumes commodity hardware
         -----  Files are replicated to handle hardware failures.
         -----  Detects the failures and recovers from them.