Skip to main content

Hadoop for beginners

I just completed by hadoop fundamentals course from Udemy.com . The videos were very well organized so that you will get the glance of what is this world of big data and how hadoop framework can play a major role is processing this big data. The course was insisting in downloading hortonworks hadoop development sandbox and working with it. Hortonworks are providing the hadoop environment setup to download and we can load it in a virtual machine. I have downloaded the virtual box sandbox file.

The course gave a string insight on hadoop architecture and buzz words around it. It gave a in depth idea of hive and pig tools and how they play the key role in storing and processing data in the framework.

Labels:

Comments

Post a Comment

Popular posts from this blog

UNIX : How to ignore lines with certain names

Sometimes we need to ignore multiple lines with certain words and get the list out of the file. usually it will be a log file to read . The below grep command can be used to ignore multiple words present in a text file. Lets say the file contain $ cat list.txt apple orange apple banana papaya Now we need to ignore line with orange , banana and papaya . So we can use the below grep command. $ cat list.txt | grep -Ev "orange|banana|papaya" apple apple It will ignore lines with the words in -v part of grep.
Post a Comment
Read more

UNIX : unpack and pack using tar

Tarball or tar is a utility in linux to unpack or pack the files. Lets say we have a file name hello.tar.gz which is packed and compressed. To unpack the file use the below command > tar xvzf hello.tar.gz This is unpack the compressed tarball file to the directory hello in current location. To pack a list of files to a compressed tarball file , we can follow the below steps. Let say we have the following list of files in Hello directory. > cd Hello > ls helloworld.py README.txt Move back to parent directory and give the below command. > cd .. > tar zcvf hello.tar.gz Hello This will create the compressed tar file in the current location
Post a Comment
Read more