1. To display files and directories in HDFS
$ hadoop fs -ls
2. To create a directory in HDFS
$ hadoop fs -mkdir HDFSPractice
3. To dispaly the contents of the directory
$ hadoop fs -ls <Directory_Name>
Since, our directory is new we don't have any files in the directory. So let's add some files to our directory.
4. Loading files into HDFS from our local file system
$hadoop fs -copyFromLocal /usr/hadoopPractice/employee.txt HDFSPractice/
Note: 1. hadoop is case sensitive so copyFromLocal is different from copyfromLocal.
Now lets display the contents of the HDFSPractice directory as in point 3.
5. Remove/Delete file from HDFS
$ hadoop fs -rm HDFSPractice/employee.txt
6. Loading the file in our HDFS to our local file system
$ hadoop fs -copyToLocal HDFSPractice/employee.txt /usr/
Goals of HDFS
1. Very large distributed file system
---- 10k nodes, 10PB data, 100 million files.
2. User Space, runs on heterogeneous OS
3.Optimized for batch processing
----- Locations of data exposed so that the computations can move to where data resides
----- Provides very high aggregate bandwidth
4.Assumes commodity hardware
----- Files are replicated to handle hardware failures.
----- Detects the failures and recovers from them.
