A True Programmer: Best Tool for the Job or Your Preferred Language?

the_Grinch

Slowly (but surely) I am moving down the programming path and while working on the flow of some scripting I am going to do it got me thinking. Is the train of thought use the best tool for the job (i.e. quickest way to accomplish your goal) or use your preferred language to complete the job? As an example, in my case I am trying to dive deep into Python (along with Java and Javascript at some point). Working with Elasticsearch I have a couple API options (Python, Java, Javascript, and Bash). The long and short of it is this:

1. Connect to Elasticsearch
2. Find all the indices older then 90 days
3. Store each name in a list or array
4. Close each of those indices
5. Move the related files to another folder
6. From that folder transfer those files to HDFS

I'm pretty sure this could easily be scripted with BASH (the array might be really large on the first run, but after that it would be one file a day). But given that I'm trying to dive deep into Python it would probably be a great exercise to get a better grip on the language. So thoughts?

apr911

When looking at a new project requiring me to choose a language, I look at 5 aspects:
1. Personal preference
2. Extensibility
3. Future Care/Maintenance
4. Employer Requirements/Preferences
5. Vendor Requirements

1. As you develop your programming skills often times the best tool for the job and the quickest tool for the job will collapse into one such that its really a matter of preference (and skill) that determines the language you use.

Your personal preferences for one language over another will develop for a number of reasons (skill level, security, ease of use, etc)

Personally I really like python but as a bit of a structure-**** (apparently the board blocks this out... Its a German-political party circa 1920-1945) myself, the whitespace aware aspects of python particularly appeal to me and though I can read, write and understand java, Ive come to hate java and have therefore not written in java in sometime; to the extent that my skills have most certainly atrophied and I'd probably be hard pressed to write a java program today.

2. Beyond personal preference and skill however is another classification that has less to do with speed or which is the best tool to use but rather which is the most extensible.

I look at bash and see a great language for quick scripts and one-liners that are intended to gather information for one-time use. Sure you could write a shell script that can be reused if you only need to see the data but to make further manipulations of the data, bash is not very extensible (you could pass your data from one script to another but that is an "expensive" operation). Bash is also great for system operations (like file moves) since the command set is native to the language (unlike python where you have to import subprocess and write a bash command inside the subprocess functions)

A true programming language (like python or java) is more extensible thanks to inheritance which will allow other functions and classes to make use of your data in different places. It is also more versatile in that python and java are both OS independent (bash is distinctly linux as powershell is distinctly windows) and can therefore be written to run on either system (again making it more extensible).

Additionally, most full programs are written using a higher order language which may allow your code to be integrated into a larger program as an additional module.

3. Care and maintenance.

This part blends somewhat with extensibility as it always important to remember business requirements change and with that change comes the need to do more with the data they've collected, at which point, you may find yourself rewriting a bash script into another language for that sole purpose.

In addition however, there is the care and maintenance considerations from a security perspective and a language perspective. There are entire classes and modules in python that have been deprecated or removed in python 3 just as there are in the various versions of Java, C++, C#, C, etc. Java is also notorious for security issues that may impact you either directly from a security perspective or indirectly through the disabling of modules needed in your program. You'll need to plan updates to your code as the underlying language changes.

Finally, ideally you want to be sure the next person after you is able to maintain, or at least understand, what the script is doing. Comments are good as is clear concise code. Chances are that if a major update is ever required, the next person is likely to rewrite your script in a language of their choosing but clean concise code that is easy to understand will save the next guy (who may one day be you) a lot of headaches and will be much appreciated. I may not remember all of the "good" programmers I've worked with over the years but I most certainly remember some of the "worst" programmers I've worked with and if their name ever showed up on a resume, job interview (either looking for a job or on the interview panel) alarm bells would go off in my head.

You want to choose a language you can program in efficiently, that you are likely to keep an interest in (so you are still aware and skilled in the language when you have to do updates on the code), that has a clear forward progression path into the future (seriously, dont pick a near dead language [i.e. Basic/LISP/Fortran] just because you can, only if you have to) and never forget the future programmer (again dont pick a language just because you can, nobody will appreciate your "genius" for writing a script in ERLANG that could have been done in python, most likely they'll curse you for doing so).

4. The next piece to the puzzle is the question of employer preference. If you're the only python programmer there, chances are your employer wont be happy with a python script that no one but you can maintain. Additionally, if all of their programs are written in Java, writing the program in python will be of no benefit to the employer (and again likely make them not very happy) since the data has to be exported from python and imported into java, losing all extensibility aspects of the code.

5. The final piece to the puzzle is the one which you and your employer have the least control over and that is what the vendor chooses to implement. Its all well and good if your application is written in python and is fully integrated into a larger piece of software the employer uses but if the vendor's API doesnt support python, well you're SOL. IMO its one of the big reasons why rest-API has become so popular, it is extremely robust and it provides a programmer friendly framework for datastructures that allow the programmer to get data from a source and put it into another language however they would like and I dont know of a modern programming language that isn't able to fetch and parse a structured webpage.



In this case, you aren't necessarily bound by the vendor's API and it sounds like your employer doesnt have a large software function this data needs to tie into and it also sounds like you've already discussed other language preferences with your employer which truly leaves the choice up to you. You could write it in python as a proof of concept and to build your skills in python but it sounds like it will take significantly longer than if you were to write the program in bash which means its up to you and how much time you want to spend on it when it comes to choosing the language.

the_Grinch

This has got to be one of the most comprehensive answer ever! Thank you so much for it! Thankfully, this is strictly a close an index (it's a simple one line command) and then move to a folder. That being said I am probably going to go with Python for it since if I ever do move on I think the chances someone knows Python are pretty good. Plus my boss has a background in computer science and I am pretty sure he can figure it out. Again thank you for the awesome answer!

seigex

DevilWAH

Cant really argue with arp911.

The other thing to consider is what the rest of the solution is coded in. if the previous person has been using python, then you bolt on some PHP, and the next guy a bit of java it all gets a bit messy. Where possible a single solution/project should keep to a single language.

I do think that programmers that stick to a single or few languages write much clearer/cleaner code, simple because they have the practice and experience. Ones that switch back and forth as apr911 said might keep a good understanding but forget the finer points and this can lead to in efficient and poorly structured code. So its all a balance if you need to write a short script to pull some data from AD accounts it might be worth "learning" powershell til you have enough to do what you need, but if the task is more complete where you are getting in to logical operations and multiply objects, it might be best to stick to a language you al ready know well.

tecke

My 2¢

Python's requests, json and subprocess librarys will help you with quickly building this script.

Requests: HTTP for Humans — Requests 2.7.0 documentation
https://docs.python.org/2/library/json.html
https://docs.python.org/2/library/subprocess.html

GL!