Use Bash to Automate Checking for that Dream Job

by Jordan Spencer Cunningham on

!#/bin/bashWhile I’m currently not on an active job search, I keep my eyes open for potential opportunities all the time; it’s a competitive market, and my services will go to the highest bidder (within reason). I try to keep up on some of the “dream job” companies to see if there are any openings for which I might be qualified, but remembering to regularly check multiple sites’ careers pages seems to be beyond my capability now that I have, you know, actual responsibilities. The beauty of working as a sysadmin is that you learn how to automate as many tasks as possible, and there is no reason that this philosophy shouldn’t apply to all aspects of life. The following can be adapted to really any careers web page so you, too, can be alerted when a position for your dream job opens up.

Edit: Someone pointed out that Glassdoor is a pretty good jobs aggregator, so I’d recommend anyone go there. This is more fun, of course, plus if anyone managing each site’s web servers happens to look at the logs, they’ll see my user agent and instantly revere my glory.

BambooHR

BambooHR is a company with which I interviewed about a year ago for a DevOps position despite never really touching DevOps before. The product seems superbly built, but the main reason I was interested in them is that the more I read about how their employees are treated, and after touring part of the building to meet some of the employees and see the kind of environment that’s been established there, the more I wanted to be one of those employees. I’d ride the train down to Layton for that kind of job.

Apparently someone with more DevOps experience landed the job, but I’ve still kept my eye on the company ever since. It was the hapless tedium of checking their careers page every so often that first inspired me to build the following Bash script to check their page for me:

I didn’t know exactly what kind of job titles I should be looking for, so I decided to take a dragnet approach: download a copy of their careers page each time the script is run and compare it with the previous check’s copy; if they match, do nothing. If there is a difference between the two (thus indicating a job position change), email that difference to me so I can determine if it’s worth looking at.

BambooHR’s careers page’s section listing current openings is populated via JavaScript. I quickly realized this when I found that wget did not “see” the list of openings when downloading the page– it only saw a jumble of JavaScript. However, it was quick work to locate the script loading that list and find the PHP page from which that script populates its data; this is why I’m hitting /jobs/embed.php instead of  /about/careers.php.

BambooHR on KSL.com Jobs

I had originally found and applied for BambooHR on KSL.com’s jobs website— a local Utah site similar to and more popular than Craigslist– so I figured I had better have a script check KSL as well. KSL turned out to be tricky, though, because theirs is strictly a search function; as far as I could discover, there was no page generated especially for BambooHR that listed all current positions that I could copy and paste the above code to check. After much research and probing KSL.com/jobs, I couldn’t find any information on or any indication of an API, either, though I did find this, which was at least interesting and may come in handy one day. I did eventually discover (I don’t remember how) that I needed to send my requests to /jobs/save rather than the default search page, /jobs.

My attempts at first were fruitless. Using cURL to send a POST request utilizing the search textbox seemed to invalidate the contents of the textbox and simply searched for everything; attempting to have cURL send POST requests utilizing different form fields showed the same problem. Then I tested disabling cookies in my browser, and voila– the KSL site in the browser was behaving exactly the same way as the site as loaded by cURL: all job results were returned without any filter as if I hadn’t entered anything into the search textbox at all. I realized that KSL probably wanted to discourage just what I was trying to do, but two can play at this game.

For one of the few times in my life I utilized Firefox’s Firebug to discover the session information for the cookies I had saved for KSL. I simply parsed that information into cURL header form and ran a test, and all was well at last.

This worked well, at least until that session expired, which was probably no longer than a month. The results are now back to nixing the “bamboo” filter I place upon them; at this point, I tend not to care since my first BambooHR jobs parser is working well, and I’ve gotten an email every time they’ve made an adjustment to their page.

Automattic

For years I’ve daydreamed of working for Automattic, makers of WordPress. Back in the old days I fancied myself as becoming a PHP developer; while I’ve taught myself a hefty bit of PHP and have coded some relatively large applications in PHP, I determined that strictly being a developer was not my can of chili, as it were. During that developer’s dream time of mine, I did apply for Automattic once or twice, just to see what would happen. It took a couple of months, but both times I was rejected, though not without some helpful pointers on how I might make myself more attractive to Automattic in the future, which was nice.

Now that I’ve taken the sysadmin path, I know I’ll likely never be a Code Wrangler (what they call developers over at Automattic), but my experience is mounting up enough that I’d like to try my hand at scoring a Systems Wrangler position (more or less a sysadmin position). I knew exactly what Automattic titled this job. Bash scripting yet again came to my aid, this time fairly straightforward:

The only caveat around which I had to work was the fact that there was a comment in one of the JavaScript code sections on the /work-with-us/ page that contained the words “Systems Wrangler”. I simply had to pipe my original egrepped output to an inverse grep to essentially say “filter out any lines with this pattern in them”, which effectively removed that comment from any grep results. Now I will receive an email whenever someone at Automattic posts a job for Systems Wrangler.

Before setting these scripts to run on a schedule, I ran chmod +x  on each of them, of course.

I’ve set these scripts to run four times a day every six hours on my Synology so that there won’t be much time in between a job posting and my knowing about it. One can just as easily do the same thing on any Linux-based machine using cron.

A wise man once told me that a lazy sysadmin is a good sysadmin.

I still need to create parsers for DigitalOcean, Namecheap, Stack Exchange, Malwarebytes, and the Wikimedia Foundation. All in good time.