Python...LOVE IT!

the_Grinch

For the first time at work they are looking to automate basically everything we do. Our current project involves a lot of pieces, but once completed the insight we'll gain will be game changing (as noted by our consultant, if successful we will have done something no one else does). For my part, I needed to write a program (or script) that connects to a secure ftp, downloads a zip file, unzips said file, and confirms that the file containing the sha1 of the report was created within 3 minutes of the report being generated. Ultimately, if it fails, I will move the failed file to a folder and we'll generate an alert off of it. Below is my code:

#!/usr/bin/python
#This program will connect to the denoted ftp, download the required zip files
#unzip the files and then confirm that the checksum was performed within 3 minutes
#of the file being created
#Created by
#
#This program will connect to the ftp and download the required files
#Required ftp module, zip module, os module
import pysftp
import os

#This connects to the ftp with the username and password provided
srv = pysftp.Connection(host="host",username="",password="")

#Have to change to the correct directory
srv.chdir("public")

#File we need to download
srv.get("test.zip")

#Closes the ftp connection
srv.close()

print "File downloaded successfully"

#This will unzip the file after downloading
os.system("unzip test.zip")

print "Files extracted"


#This program will look for a difference in the creation time of the files

#This gets the time in seconds of the original files creation time
org = os.stat('file1').st_mtime

#This gets the time in seconds of the checksum file
csfile = os.stat('file2').st_mtime

#This gets the difference and alerts if it is over 180 seconds
tdiff = csfile - org

print tdiff

#Test to see if time is 3 minutes or less
if tdiff <= 180:
print "Time difference in file is equal to or less then 3 minutes"
else:
print "Time difference in file is greater then 3 minutes"

A little more background. The files need to be zipped because if they were downloaded directly from the FTP the times would change. By being zipped they maintain their creation times. Mtime is used because it only changes when the data within the file has been modified. Also, mtime shows the creation date in the form of seconds so that I can actually do the math.

Thexzeno

on the same boat here as well, i started learning python during my 1 year unemployment hiatus and never thought id use it during my job. Well i recently got my first it job at a help desk and lo and behold i managed to create a script that cuts of 30 secs from call time by automating the process pinging the 2nd lvl team through mscommunicator or lync2010.

I honestly suggest everyone take the time to learn python as its easy to read,very easy to learn and has a robust community with API'S for basically every application. if you do enough research i dont believe theres nothing you couldnt automate.


Also, looking at your code you should add a variable for user input that specifies the zipfile to be downloaded


for example:


Print "Please input name of the zipfile blahblahblah"
Zip_file = input() #it'll be raw_input if using python 2.7 i use 3 though

#File we need to download
srv.get(Zip_file)


not sure if youve already thought of that or what your requirements for the script are exactly though

the_Grinch

Yup, you are definitely correct. As of right now we haven't decided on the naming convention for the file (I suspect it will be something like company.report.date.zip. Once I get that then I can code it in as a variable appending the date to the normal file name for the purposes of downloading it. Excellent suggestion either way! You are right so many modules out there for integrating with various pieces of software.

DevilWAH

I was playing with Python recently, found it quite friendly, need to play a bit more with it to get the effect I want

darkerz

+1 for Python!

Anything that follows a process flow or a standard should be automated, then maintained as the operation changes over time. Not just for configurations, but for general office "stuff".

Eventually you find yourself attempting to automate everything, hotkeying script executions and becoming a sort of wizard. You begin touching other languages, perhaps wanting to build your own program or interface with an existing API.

One day you wake up and realize you've become a DevOps guy who's into networking, systems, security, everything - the time you've saved on repetitive tasks and repeating variables + general efficiency allows you to dive into mutiple specialties.

On this forum, the above sounds normal, doesn't it? Just like how we obsessively go after certs. Most of the work force is NOT like this, I would say safely 97%+ of folks in our field will not do this. For those who do, it's a bright future.

It's exciting to think about.

the_Grinch

Hit the nail on the head darkerz! When we had our meeting about the project they asked who would write it? I said if I can do it in Python then I shouldn't have any trouble. Gave us a deadline of April 9th and finished it this morning. It is definitely much easier to learn when you have an end goal though. I've tried learning Python on a few occasions and always stopped because I couldn't apply it to what I was doing. Now things have changed for sure because I can work through the logic to see exactly how something should unfold. The biggest help thus far has been a buddy of mine who is in DevOps. He uses Python on a daily basis and before he moved I asked him how should I get started? He simply said "start writing code". Until recently I thought he was nuts, but then I had to write code and found out he was right on the money.

UnixGuy

Great work The_Grinch!! I'd love to get into DevOps as well...been configuring puppet for a while now, but it's taking ages! I used to write everything in Perl, but I found that Perl is one of those things that if you don't use (VERY OFTEN) you end up forgetting stuff. I have a programming background that makes things easier though, but DevOps seems to be the future of sysadmins!

YFZblu

Another +1 for Python - It's truly a joy to work with.

I noticed you have 'password' fields in your script there - Are you currently storing them in cleartext in the script? Obviously I don't know your environment and it's possible you NEED to do it this way, and the risk has been accepted; however have you considered using the 'GetPass' module? It would allow you to prompt the User for the password, without storing it in the script. The added benefit of using 'GetPass' over raw_input("Password: ") is that GetPass doesn't reflect the User's input back to the screen.

Obviously, if you need to run this script all the time throughout the day, manually running it and entering user input would cause a ton of overhead. Once again going back to the fact that I don't know the exact situation here. Just my two cents.

Bokeh

I just got a book in today on learning Python the Hard Way. I have nothing on the books for this weekend, so looks like I can start digging into this.

JustFred

I just bought the python book a few weeks ago and I'm still trying to make time and go through it. It's been on my list for a long time now. I'm definitely going to start learning soon. First need to get some stuff out of the way. I think every engineer should learn Python

ally_uk

Very interesting thread

I would like to get into learning the art of Python myself I have done the basics in the past i.e variable declaration print output and getting the program to ask for user input ( nothing to hardcore)

The Program / script's you have written have caught my attention and I immediately thought this is exactly what I would like to do, I am not interested in developing games or spending the next 10 decades writing hello world

So the question is how do you take the next stage and learn about how to use Python in conjunction with either Windows or Linux Commands and create automated scripts for administrate for example creating a backup script or a logging script.

Do any of the books or guides teach you this kind of thing?

Many Thanks

MSP-IT

I started with Python in late Oct of last year. If you have the ability, I highly recommend looking at Object Oriented Design with Simon Allardice. I didn't get a good handle on understanding how classes worked until I got into Java, but that short series, along with a good baseline understanding of the language can get you far.

FloOz

For the network engineers out there:

Show 176 - Intro to Python & Automation for Network Engineers - Packet Pushers Podcast

ally_uk

Any good sites / guides to learn python windows / Linux related stuff

the_Grinch

Honestly I did a lot of Googling to find what was available. I knew I needed to connect to an sftp so I good python sftp module. Necessity is the mother of invention as they say. At my last job they wanted a script to do backups of a file. I had to go about breaking down exactly what they were looking for:

1. Copy file
2. Move the file to another server to be backed up

From there (since it was Windows) I looked up the commands I would need to accomplish that. Then they added a wrinkle and said delete any file older then a week. So I had to check the creation times and then delete the oldest ones.

I do have the SecurityTube course, which I've watched pieces of to get some knowledge and I have a few books that I have read partly through.

ally_uk wrote: »

Very interesting thread

I would like to get into learning the art of Python myself I have done the basics in the past i.e variable declaration print output and getting the program to ask for user input ( nothing to hardcore)

The Program / script's you have written have caught my attention and I immediately thought this is exactly what I would like to do, I am not interested in developing games or spending the next 10 decades writing hello world

So the question is how do you take the next stage and learn about how to use Python in conjunction with either Windows or Linux Commands and create automated scripts for administrate for example creating a backup script or a logging script.

Do any of the books or guides teach you this kind of thing?

Many Thanks

GoodBishop

I'm reminded of this - xkcd: Python

linuxlover

Interesting, our company uses mostly Perl for automation.

YFZblu

^ Ours too - One of my goals at work this year is to see all of those '.pl' extensions slowly outnumbered by '.py'

UnixGuy

Perl is famous in the UK. I used to do everything with Perl but I haven't used it in a while. In Australia, everyone seems to ask for Python!

linuxlover wrote: »

Interesting, our company uses mostly Perl for automation.

the_Grinch

Perl always seemed very cumbersome to me, where as Python has the "order" I understand. Today I wrote a program to query the Maxmind GeoIPLite2 database (they give this out for free) to trace where an IP is located. Took about 15 minutes! I have to expand it heavily to include reading a database, checking the ip, and then writing the information to said database. But overall a very good start! That being said I was looking at Coding for Penetration Testers and boy is his code full of misprints and bugs. I had issues with his bash code and his Python code....yikes! Hoping Violent Python is better written.

sizeon

Can't you do this with powershell?

YFZblu

^ This task can be accomplished with any number of languages; however sanity always leads me back to Python...

YFZblu

the_Grinch wrote: »

Perl always seemed very cumbersome to me, where as Python has the "order" I understand. Today I wrote a program to query the Maxmind GeoIPLite2 database (they give this out for free) to trace where an IP is located. Took about 15 minutes! I have to expand it heavily to include reading a database, checking the ip, and then writing the information to said database. But overall a very good start! That being said I was looking at Coding for Penetration Testers and boy is his code full of misprints and bugs. I had issues with his bash code and his Python code....yikes! Hoping Violent Python is better written.

Duuuuude. I'm currently reading Violent Python. Care to trade notes as we go along? It's a great read, btw

NovaHax

YFZblu wrote: »

Duuuuude. I'm currently reading Violent Python. Care to trade notes as we go along? It's a great read, btw

Meh...I honestly wasn't impressed with Violent Python. I recall seeing serious problems with a lot of those scripts. For example...the author has a SYN flood script with Scapy...but doesn't even multi-thread the script. There is no way you can produce any kind of denial of service by sending SYN packets in sequence.

I wrote a similar script that allows multiple concurrent half-open connections by randomly generating an integer value between 0 and 65535...then using the random value as the source port. Alternatively, you could loop through them in sequence and then pull each subsequent source port value from a queue as each process is spun up.

YFZblu

I haven't reached the SYN flood yet, so I don't have context on that - I'll have to keep an eye out for issues. I see Violent Python as a book to get you thinking beyond the typical tools of the trade; not an end-all-be-all to penetration testing with Python.

NovaHax

Yeah...it'll definitely get you thinking in the right direction. And actually, can probably make for some good exercises if you improve upon the scripts, since most of them are using the correct libraries and have the right idea...just poor implementation in some of them.

YFZblu

Agreed - I can absolutely see this book being less valuable to those who already have a deeper understanding of Python or those who won't take the concepts and run with them.

For the next few weeks i'm sort of using Violent Python to stave off my desire to purchase the OSCP training material until I have more time...I had my finger over the offsec purchase button last week, but I'd like to shoot for 30 days instead of 60, so the timing is critical for me.

UnixGuy

I don't like programming languages religious debates, but Python vs. Perl debate has been beaten to death. I still prefer Perl when there is a need and the shell otherwise, but I'm planning on learning Python because that's what the market demands

the_Grinch

Honestly, each language does serve it's purpose and PERL does some amazing things. If I actually sat down and tried to learn it I don't doubt I would enjoy it.

YFZblu - Definitely would like to trade notes once I start reading it. I heard it's a great companion for the Securitytube Python Scripting Expert which is why I picked it up.

emerald_octane

I've done a little bit of Java, Shell and Perl, but not enough to where it'd be on my resume. Lo and behold I can tell it's definitely holding me back in the marketplace, so i'm knee deep in "Learn Python the Hard Way" and "Violent Python". Excited to eventually get into the fold.

the_Grinch

Early in my career I had Java and Visual Basic on my resume because I had used them in the past and if I saw code I could definitely see what it was doing. But after the third interview where they went down the path of asking me programming related questions I knew I had to remove it. I do plan to put Python on there at some point, but not until I can do a significant amount of coding without looking things up.