D-R-Y, Automation has you covered.
Automation is a word that most of us just hear out in the world, in seminars, developer conferences and places of the sort. Another place you see the word often is in your engineering textbooks — yes, I assumed you are an engineer for a moment there.
Automation should be the strong suite of any developer or programmer out there. Of course, anyone who can automate, should automate, but I’ll focus the rest of the content here keeping in mind the state of folks like me.
I don’t usually blog. But this was something I just had to put on paper, err… on Medium.
The first question we ask ourselves, or others, as beginners to programming, is what languages do we need to learn. I have been posed this question quite a number of times and my standard reply used to be, “Do not learn language just because you have to. Learn it for a particular requirement. That’ll help you excel in it better.”. Of course, I always suggest them to be pretty good in at least C or C++. But now, if I was posed that question again, I would append to that answer and emphasise the need to know at least one scripting language like Python or Shell. Why? Well, keep reading.
Before I tell you why I had this sudden thought of adding a scripting language to my answer, I want to tell you about what inspired this notion. It was the result of my experience in Robert Bosch as a Machine Learning Intern. While I’ll try not to bore you with the details of the work I did, I want you to focus on the subtleties of the work I did there. Like every intern, we (I had a friend with me, yeah) were clueless. And like every clueless intern we did all the work manually for a while. Be it, as complex as marking and editing images in a software or as simple as copy pasting it, we did it manually thinking it’ll be the only time we need to do it. This was the first mistake we learnt the hard way. And as a rule of thumb, always keep in mind that, “The tasks you do at work, none of them will be maiden”.
We primarily worked around Deep Neural Networks. Hence, we had to deal with a lot of data. This led to the requirement of moving them from a directory to another, renaming them, changing extensions and more of the like. This is easily do able, you say, and you’ll do it manually. But what if you had to do this every single day of the internship? Things don’t seem pleasant that way. This is where scripting languages become your Superman! Write a simple script once, use it any day of the week with a single command to execute it. I’m sure you wouldn’t appreciate the need of a script to copy stuff from one location to another. But take my word for it, it’s worth it. The intuition for such an automation makes more sense if you understand another not-so-easy task we had to do. Well, the task is dead easy, but time consuming, and well — boring. It was to open up every directory with images in it, and rename some of them based on conditions. This is an ideal play ground for automation. If you ask me what other examples I could offer, it could be finding duplicate files/data across directories, deleting files based on conditions, resizing them and plenty more.
It is for these kind of scenarios the knowledge of a scripting language comes to be mandatory. The ideal one according to me would be to know the Python Programming Language. Learning the Shell scripting language is an added advantage. We all work on Linux, don’t we?
When I say automation, it could be a very simple script. As simple as renaming files in particular directory:
import os
import sys# get the directory where you want to perform this renaming
directory = sys.argv[1] # parse through file list in the directory
for filename in os.listdir(directory):
# if found
if filename.find(“_superman”) > 0:
# convert _superman to _batman
newfilename = filename.replace(“_batman”, “”)
print directory + ‘/’ + newfilename
# rename the file
os.rename(directory + ‘/’ + filename, directory + ‘/’ + newfilename)
Or something quite extensive as a GUI automation that my friend Knhash implemented to automate a image editing tool procedure.
This video shows the automation of a GUI process of saving the ROI (Region of Interest) from the image and also saving the black and white mask of the shapes. Doing this task for hundreds of images is tiresome. But we could set the script to do the heavy lifting for us while we had our coffee breaks.
So, following the principle of DRY (Don’t Repeat Yourself), make sure you are proficient in at-least one scripting language so that you don’t repeat yourself.
