Tuesday, 21 January 2014

Hadoop Basic Commands

Hadoop Commands
No
Command
Command Usage
Description
1
cat
hadoop dfs -cat <path>
prints the file contents
2
chgrp
hadoop dfs -chgrp [-R] GROUP URI [URI …]
Change group association of files. With -R, make the change recursively through the directory structure. The user must be the owner of files, or else a super-user.
3
chmod
hadoop dfs -chmod [-R] <MODE[,MODE]... | OCTALMODE> URI
4
chown
hadoop dfs -chown [-R] [OWNER][:[GROUP]] URI [URI ]
5
copyFromLocal
hadoop dfs -copyFromLocal <localsrc> URI
6
copyToLocal
hadoop dfs -copyToLocal [-ignorecrc] [-crc] URI
7
count
hadoop dfs -count [-q] <paths>
8
cp
hadoop dfs -cp URI [URI …] <dest>
9
du
hadoop dfs -du URI [URI …]
Get size of each file in dir
10
dus
hadoop dfs -dus URI [URI …]
Gets total file size
11
expunge
hadoop dfs -expunge
Empty the Trash
12
get
hadoop dfs -get [-ignorecrc] [-crc] <src> <localdst>
13
getmerge
hadoop dfs -getmerge <src> <localdst> [addnl]
Takes a source directory and a destination file as input and concatenates files in src into the destination local file. Optionally addnl can be set to enable adding a newline character at the end of each file
14
ls
hadoop dfs -ls path
15
lsr
hadoop dfs -lsr <args>
Recursive version of ls. Similar to Unix ls -R
16
mkdir
17
moveFromLocal
18
moveToLocal
19
mv
20
put
21
rm
22
rmr
hadoop dfs -rmr [-skipTrash] URI [URI …]
23
setrep
hadoop dfs -setrep [-R] <path>
Changes the replication factor of a file. -R option is for recursively increasing the replication factor of files within a directory.
24
stat
hadoop dfs -stat URI [URI …]
25
tail
hadoop dfs -tail [-f] URI
Displays last kilobyte of the file to stdout. -f option can be used as in Unix
26
test
hadoop dfs -test -[ezd] URI
e check to see if the file exists. Return 0 if true.
z check to see if the file is zero length. Return 0 if true.
-d check to see if the path is directory. Return 0 if true
27
text
hadoop dfs -text <src>
Takes a source file and outputs the file in text format
28
touchz
hadoop dfs -touchz URI [URI …]
Create a file of zero length.
29
jar
hadoop jar
The hadoop jar command runs a JAR file.
30
fsck
hadoop fsck
HDFS supports the fsck command to check for various inconsistencies. It it is designed for reporting problems with various files, for example, missing blocks for a file or under-replicated blocks.
31
job
hadoop job -list


No comments:

Post a Comment