[{"_1":2,"_46":-5,"_47":-5},"loaderData",{"_3":4,"_14":15,"_44":-5,"_45":-5},"root",{"_5":-5,"_6":-5,"_7":-5,"_8":9,"_10":11},"user","shoppingCartId","toast","locale","en","constants",{"_12":13},"paypalClientId","AenrHU49peF-ZUH-Yq7AXLLNZYGUGlml474U16NCjB4ZznmQCEjiliN3whyuVZmkb7VxEzmTALgtKEKE","routes/_store+/_readers+/email_author.$slug/route",{"_16":17,"_43":-5},"book",{"_18":19,"_20":21,"_22":23,"_24":25,"_26":27,"_28":29},"id","Z2lkOi8vbGVhbnB1Yi9TdG9yZTo6Qm9va1Byb2R1Y3QvMjg0NzI","title","Learn Data with Bash Shell","state","published","contactAuthorMethod","email","aboutText","

Bash may not the best way to handle all kinds of data! But, there often comes a time when you are provided with a pure Bash environment, such as what you get in the common Linux based super computers and you just want an early result or view of the data before you drive into the real programming, using Python, R and SQL, SPSS, and so on.

Expertise in these data-intensive languages also comes at the price of spending a lot of time on them. In contrast, bash scripting is simple, easy to learn and perfect for mining textual data! Particularly if you deal with genomics, microarrays, social networks, life sciences, and so on. It can help you to quickly sort, search, match, replace, clean and optimise various aspect of your data, and you wouldn’t need to go through any tough learning curves.

There are several examples of practical data mining that will have a flow of importing specific data resources into flat text-type files. Bash can run different programs (grep, sort, sed, and so on) on those files, clean, optimise and extract preliminary views (cut, csvlook, view, cat, head, etc.) of the data. There is one part of data mining, which involves unstructured data and then transforming it into a structured one (awk, shell). A scripting language like Bash can be very useful for doing the transformation. We strongly believe, learning and using Bash shell scripting should be the first step if you want to say, Hello Big Data!

This book starts with some practical bash-based flat file data mining projects involving:

University ranking data [Previews: Part I, Part II, Part III] Sample video lectures.
Facebook data [Previews: Part I, Part II]
Crime Data
Shakespeare-era plays and poems data

If you haven’t used Bash before, feel free to skip the projects and get to the tutorials part. Read the tutorials and then come back to the projects again. The tutorial section will introduce with bash scripting, regular expressions, AWK, sed, grep and so on.

Finally, it gives you a concise beginner friendly guide to the big data landscape including an overview of the critical Big Data tools such as HDFS, MapReduce, YARN, Flume, Hive and more. The book finishes with a near-complete list of references to all the relevant command line and Big data tools.

Get the interactive version!

","authors",[30],{"_18":31,"_32":33,"_5":34},"Z2lkOi8vbGVhbnB1Yi9TdG9yZTo6UHJvZHVjdEF1dGhvci8yOTY1Mw","name","Scientific Programmer",{"_35":36},"avatar",{"_37":38,"_39":40,"_41":42},"url","https://d308did4coypzi.cloudfront.net/avatars/975976/medium/62b4519a67f08.png?1677564955","styleName","medium","exists",true,"currentUser","routes/_store+/_layout","routes/_store+/_readers+/_layout","actionData","errors"]

Email the Author