BASH Scripting for idiots

ally_uk

Does such a thing exist? Skill wise currently I have written a basic script that sets up two variables and runs basic commands, i.e yum update and does alot of basic echo stuff i.e we are now going to update the system. I want to take it to the next level and learn about functions, loops etc.

I have read the Sorbell Book which covers some of scripting, But it's pretty high level stuff and goes over my head, are there any basic tutorials that have a structured approach i.e create this first script and the next script we take it further and do this.

How did you guys become scripting gods? i'm feeling pretty disheartened at the moment because everything I have read starts off ok then just gets to a level where it's like wtf? lol

powmia

Which Sobell book? I would recommend, "A practical guide to Linux: commands editors and shell programming"... but that might be the Sobell book you're referring to.

fiftyo

ally_uk wrote: »

I have read the Sorbell Book which covers some of scripting, But it's pretty high level stuff and goes over my head, are there any basic tutorials that have a structured approach i.e create this first script and the next script we take it further and do this.

Then this is your book! Unix Shell Programming (3rd Edition): Stephen G. Kochan, Patrick Wood: 9780672324901: Amazon.com: Books
Otherwise I'd suggest reading the tldp manual, although I'd view it as more of a reference manual.
http://www.tldp.org/LDP/Bash-Beginners-Guide/Bash-Beginners-Guide.pdf
And just in general, play around with bash will take you pretty far.

ally_uk wrote: »

i'm feeling pretty disheartened at the moment because everything I have read starts off ok then just gets to a level where it's like wtf? lol

I felt like that too when I began scripting, but the more you tinker with it, the more you'll understand, it's sort of laying a puzzle where you create the pieces.
A good way to learn scripting is to assign yourself a task, such as "I need to shoot this update to this set of machines". Then you break the task into smaller pieces, in this case you'd need 1) a list of hostnames/ips 2) Something which will handle logins to the machines and 3) commands to shoot.
Another good practice is to always ask why and understand what the code does, instead of pasting in chunks of code from google.(Which might be a bit obvious but it's worthwhile!)

powmia

If it's the actual programming logic that you're trying to learn, like you said... functions, loops, etc.. you may want to learn an actual programming/scripting language instead of going straight to shell scripting without having that foundation. Just food for thought. If that's something you would consider, I recommend picking an easy, clean language that will let you focus on the logic and not the syntax:

Python Programming: An Introduction to Computer Science 2nd Edition: John Zelle: 9781590282410: Amazon.com: Books

ally_uk

Yeah I have the Sorbell book the chapter on BASH scripting isn't noob freindly sure he shows a example of a hello world script but the rest of the stuff after is just jargon and goes over a beginners head,

I wish there was a book or guide that teaches you a basic script then gradually adds concepts to it, Instead most of the guides seem to be quite high level and all you end up with is a headache lol

powmia

Unfortunately, most don't use shell languages to teach fundamental programming/scripting. Most guides assume they are just teaching you a language, not programming/scripting.

ally_uk

So should I skip BASH altogether and learn a programming language? if so what is the easiest one to start with? Python, Perl, Ruby? what would build a good solid foundation and understanding of programming concepts

zxshockaxz

Bash is a lot of fun. I'm definitely not a god though. If you're working with linux machines (obviously, because this question is about bash), then I'd recommend learning python. I'm not a god at that either, but I'm better at it. Python is way powerful, and pretty easy to learn as well. A great place to start is codeacademy.com, it'll teach you all of the basics, like functions and loops and all that.
There was a presentation on securitytube.net call "bash scripting for penetration testers." I believe it was about an hour or an hour and a half. It actually taught me quite a bit. he goes over loops and all of that. you should be able to use knowledge gained from that and use it for what you need. just takes a little debugging.

lsud00d

I love bash scripting!

This is an online guide I commonly referenced when starting with Bash:

Bash Guide for Beginners

This website has a bunch of awesome tidbits:

Bash Shell Programming HowTos – nixCraft

powmia

Don't skip it altogether. If you're working with unix/linux systems, it is a valuable skillset that can save you a ton of time. I'm just saying that if you're completely new to the concepts, it may not be the best to start with bash.

ally_uk

So in terms of using Python for Linux what kind of stuff can you do? i.e write scripts for administration?

I am pretty new to programming concepts, I kind of understand variables i.e they hold data, But if you mention functions, If, Do, While, Parameters that's when the s**t hits the fan and I am left with a headache lol

From a CLI level I am pretty comfortable can navigate around use vi, Use tools such as grep, Can configure environmental variables / set up Alias, I also taught myself how to use Tar and rsync nothing fancy just basic stuff.

Bash Script level I can write basic stuff i.e echo stuff, run commands and setup basic variables i.e Colour="black"

So that's the level I am currently am at, I have made progress considering I knew nothing about Linux.

powmia

You could use python to print "hello world" to your screen, to hack your neighbor, or to power a search engine Administration would fall in there somewhere as well.

ally_uk

lol I meant more along the lines of combing Python with rsync and commands i.e to create a backup script or something.

ptilsen

I would really start with a guide, walkthrough, class, book, or whatever that teaches programming concepts. If there's a good one that's Bourne/BASH-centric, great. Otherwise, Java or Python are great starting points. You need to learn loops, selections, functions/methods, classes, data types, error handling, regex and string manipulation, and basic logic. The book powmia posted looks really, really good, and learning Python specifically will help you in professional Linux scripting.

I was a decent cross-platform scripter for years. I could do BASH when I needed to, VBScript, Windows batch scripting, PowerShell, whatever. But to do something more advanced was often a painful Google-and-figure-it-out experience. Actually studying programming for programming's sake has immensely improved the quality of my code, my capabilities, and the speed at which I write scripts. The problem with doing it the hard way (as I did and you have) is that you will only ever manage to infer a subset of the concepts. You will not learn them fully. Furthermore, functional languages like PowerShell and BASH do so much of the dirty work for you that you won't have much concept of what's really happening. They also don't give as good of feedback when there are bugs, so even when you do fix something, you might not understand why the fix worked.

I can't stress enough how much just studying Java from a computer science perspective has improved my ability to write good, clean, reliable, effective, and powerful shell scripts, particularly in PowerShell, which frankly is just BASH for Windows (keeping in mind that there's a such thing as actual BASH for Windows). Look at Python or Java from a CS perspective, and I think BASH is going to get a lot easier for you.

powmia wrote: »

Don't skip it altogether. If you're working with unix/linux systems, it is a valuable skillset that can save you a ton of time. I'm just saying that if you're completely new to the concepts, it may not be the best to start with bash.

Agree with this completely. BASH is worth knowing, but I haven't seen a lot of material that teaches programming concepts using BASH effectively. Python is a really good starting point. So is Java, but Python is probably going to be more accessible and usable for someone currently doing systems administration. Java can be a bit frustrating if you are confident you don't want to be a software or web developer, because there's very little gratification since it's not very useful in systems administration. Still great for teaching concepts, especially object-oriented, but Python is definitely a better fit for a sysadmin.

zxshockaxz

ally_uk wrote: »

So in terms of using Python for Linux what kind of stuff can you do? i.e write scripts for administration?

Python, especially on linux, can automate just about anything and everything. BASH is great, but Python is more powerful. Plus its an easy language to read and use.

lsud00d

Good points ptilsen.

One to add regarding Java--even if you won't be writing it, it's good to understand the basics. As a sys admin I've worked with a lot of java-based apps utilizing JVM's, and having some java knowledge greatly increased my ability to diagnose and address java errors.

powmia

ally_uk wrote: »

lol I meant more along the lines of combing Python with rsync and commands i.e to create a backup script or something.

I look at it like this. Bash, is a shell. What is a shell? It's a rudimentary language used primarily to execute the utilities that are in the file system of an OS. grep... not a part of bash, that's a utility. diff, cat, ssh, ls, etc... those are utilities of a linux distro, not bash.

In python, you can execute the same utlities that are on the file system of a host you are running your python script on. So... those same utilities that your are accessing with bash, can be accessed by python. The difference, is that you are receiving and manipulating that output with a language that is 1000 times as powerful. It does make one wonder why the heck you would ever use a shell for anything other than interactive use, and it isn't uncommon to see people write a script that does nothing but call the built-in utilities of linux.. but wrapped in python or perl if/while/for statements and what not.

BUT... I'm not contradicting myself. You still need to know bash. The quick, repetitive tasks, that just need a small amount of script (sometimes just a really long one-liner) are where it really pays to know a shell language.

just my opinion.

UnixGuy

I recommend this book to understand the topics, read them in this order:
Linux Shell Scripting with Bash: Ken O. Burtch: 9780672326424: Amazon.com: Books

Then this book to put what you learned to practice:
Amazon.com: Bash Cookbook: Solutions and Examples for Bash Users (Cookbooks (O'Reilly)) (978059652678: Carl Albing, JP Vossen, Cameron Newham: Books

Read them both, and practice what's in the books and you will get your skills to the next level

UnixGuy

With regards to Perl/Python and even Java, I think since all you need to do is automate system administration tasks, I think they're not really necessary at this point. Bash is your basic scripting language. IMHO, master BASH, and use it! Then when (and if) the need come to learn a higher level programming language, you can pick and choose

ally_uk

Such as awesome forum generates such interesting discussion. Although I now have a inner conflict part of me is saying stick with BASH the other is saying get some Python!! (Adam Sandler voice lol)

In terms of Python since I have never touched it in my life what's the best starting point. How do you invoke a python script do I have to install or compile something? Any good books or resources.

Unix guy thanks for the links I'm going to keep reading up on BASH it would be a silly move to drop it I enjoy using it but this python thing seems to be also interesting decisions decisions lol

ptilsen

To UnixGuy's point, BASH is enough, and you can do well just learning BASH. However, I don't think focusing on BASH for systems administration is going to be an effective way to learn programming concepts. I wrote shell scripts and Vbscript for years, and I got pretty good at it, but I never got the concepts as well as when I started studying Java. Studying programming is going to get the concepts so ingrained that using BASH will be easy. And I really don't think learning Python is going to take you away from BASH for very long. The syntactic differences are mostly trivial. And while I will defer to those deeper into *nix-focused careers, from what I've heard Python has become more common for even administrative scripting at many organizations, which means it might make a lot of sense for you to learn it anyway.

Slithy

@ptilsen (This is a little offtopic for the thread) For some reason, i want to say that i remember regular expressions being simpler in bash compared to perl/c++, while the core concepts are similar how do you think bash and java compare in constructing and using regex?

ptilsen

Regex is probably easier in BASH than anything except maybe query languages. In general, the lower level you go, the more complex escape characters and such become, and the messier regex gets, at least in my fairly limited knowledge. I do not like regex in Java. I don't recall having issues with it in BASH, and I use it in PowerShell scripts and SCCM collection queries. At the end of the day, though, regex is not terribly complicated in any language I know of; it's just a matter of learning the syntax in that particular language.

CodeBlox

ally_uk wrote: »

Such as awesome forum generates such interesting discussion. Although I now have a inner conflict part of me is saying stick with BASH the other is saying get some Python!! (Adam Sandler voice lol)

In terms of Python since I have never touched it in my life what's the best starting point. How do you invoke a python script do I have to install or compile something? Any good books or resources.



Unix guy thanks for the links I'm going to keep reading up on BASH it would be a silly move to drop it I enjoy using it but this python thing seems to be also interesting decisions decisions lol

Python has some nice documentation on their website: The Python Tutorial — Python v3.3.2 documentation

onesaint

Learn Shell scripting for one liners and small tasks/program execution. Learn Python / Perl for anything beyond that.

A good BASH reference is the Advanced Bash Scripting Guide

MIT offers their intro to programming course based on Python for free. It might be worth looking into.

The course description:
"This subject is aimed at students with little or no programming experience. It aims to provide students with an understanding of the role computation can play in solving problems. It also aims to help students, regardless of their major, to feel justifiably confident of their ability to write small programs that allow them to accomplish useful goals. The class will use the Python™ programming language."

Code Academy is also quite good for learning Python and how lists, loops, and things like tuples work.

Slithy

ptilsen wrote: »

Regex is probably easier in BASH than anything except maybe query languages. In general, the lower level you go, the more complex escape characters and such become, and the messier regex gets, at least in my fairly limited knowledge. I do not like regex in Java. I don't recall having issues with it in BASH, and I use it in PowerShell scripts and SCCM collection queries. At the end of the day, though, regex is not terribly complicated in any language I know of; it's just a matter of learning the syntax in that particular language.

It may just be that it was just a matter of preferring one language's syntax over the other in the circumstance for me. Thank you for the reply.

UnixGuy

ptilsen: I agree with you, learning Bash alone is not enough to get programming but I thought OP wants to learn scripting for system administration, so maybe learning Bash first was a priority.

Expect

In my opinion, learning from working shell scripts examples is the best way to understand how things work in bash.
great examples are located at nixCraft website.

ally_uk

Unix Guy is right I originally was looking at BASH scripting as a means to automate of chain a set of commands together for administration purposes, I.e create a rysnc backup script.

Again you are also right with currently I do not understand Programming concepts like loops, functions, parameters I guess if you are a veteran programmer this stuff just comes naturally but with myself it's kind of hard especially when tackling BASH scripting I think the best approach as some people have mentioned is to learn Python, learn programming concepts then apply them to BASH once I have developed the understanding it's like a piece of the jigsaw is currently missing. I haven't really been exposed to programming concepts or languages.

I assume the majority of BASH guide's assume you have familiarity with Programming concepts know how to work variables, Know how to setup expressions, loops and god knows what else so i'm jumping in the deep end.

I won't be giving up on BASH currently run Red Hat based system and use BASH a hell of alot so will continue to tinker away with it but I am seriously considering looking at this Python stuff now and building the foundation.

UnixGuy

ally_uk

I'm not sure about bash scripting books assuming any programming background, they seem to start from scratch. Learning Python to be able to write Python programs is good, but learning Python so that you can apply the concepts to Bash might be confusing, I don't know.

Good bash books will explain loops, variables, functions,..etc with examples, and then you can apply those principles to your bash scripts directly. The bash cookbook is a great way to practice your skills. If a concept seems unclear, you can Google it, there are thousands & thousands of examples online.

If you want Bash scripting skills, then grab a Bash scripting book (or more) and start practicing, it will get easier, but initially it will be little bit slow. Google is your best friend.

ptilsen

UnixGuy, do you have any good BASH resources that explain programming concepts? I really have nothing against starting with BASH if it's explained in the right context. I just haven't seen any myself, while Python and Java conversely have lots of good material. I think ally_uk would do just fine with a resource that is explaining the concepts and using Bash. I think his problem, and the same problem almost everyone will have, is that he's learned BASH for BASH, and had to try to infer concepts. You can do that, but it's slower and will almost always leave an incomplete picture.