What’s Your Geek Number? My Points System To Rate Software Engineers (without a full technical interview)

The ability to quickly and accurately rate the ability of software engineers after reading a résumé and conducting a relatively brief screening (as compared to full interviews) is very useful for those of us who recruit technologists.  This skill is one of the elements that separates great recruiters from the rest, and it is admittedly not an exact science.

As an experienced recruiter, I often work with candidates that I have represented in the past, so I will have the benefit of historical data to assess how strong a candidate’s skills are and how well he/she will perform.  This data typically will include feedback from past job interviews, referrals from other engineers, discretionary ‘word on the street’ reference checks, and knowledge of a candidate’s job history.

But how do I make a judgment on a new candidate that has no experience or history with me?  I receive a résumé, we have a conversation, and I have to make the call as to whether this person is a top candidate.  Without going into a deep technical interview, how do I surmise how strong this engineer’s technical skills are?  This is one of the reasons clients hire me, as my screening should save them time to weed out those that are not qualified.

After fourteen years in the business, I have noticed some distinct patterns regarding behaviors, traits, and small details that are shared by many of the top software engineers.  Conversely, there are some noticeable trends that are indicative of engineers that are probably not in the top tier.  For the sake of clarity, my definition of ‘top’ engineers is based on successful job interviews with clients, on the job performance while working for clients, and substantial anecdotal evidence from an engineer’s peers (the peer piece often being the most substantial factor).

Below I have compiled a ‘points system’ for software engineers, with an assigned value for each characteristic.  As you can see, some items are worth positive points, while other traits take points away.  Again, this is by no means an exact science, and I imagine we will get some false positives (i.e. it is possible that some average engineers could score high), but I sincerely doubt many great engineers would score low.  Although I have used some of these characteristics to help loosely evaluate engineering talent in the past, I have never assigned any specific points values until now.

What does my points score mean?  A score in the high 20′s should be where we start seeing engineers that probably take their career pretty seriously and are willing and able to invest some time in bettering themselves.  A score in the 30′s would certainly be indicative of a candidate I would expect to perform well more often than not, and above 40 I would expect all candidates to be in the top tier.  I know a few engineers who score in the 50 range and higher, and I don’t think you will find any individuals that are not widely admired and respected at this point level.

To be clear, I’m certainly not saying that engineers with blogs are always technically superior, or that having a five page résumé somehow makes you a bad engineer.  The assigned points value are not how much that characteristic should be ‘valued’ by an employer, but rather the strength of the indicator.  (Note the highest point value is assigned to Linux kernel work, which I have found in my experience to be the strongest indicator of good engineers – not all good engineers hack the kernel, but almost all engineers who do kernel work have been good.)  These are simply patterns that I have observed while talking to thousands of engineers, and I’m sure many of you will disagree.  Critiques are welcome – even encouraged (just be polite).  If you have found indicators of great (or poor) engineers, I’d love to hear them – even if they are odd.  Let’s debate!

(As my business is focused on engineers working within a specific set of languages primarily geared towards the open source world, this list is focused on fit for my purposes.  Those who work in Microsoft shops or develop embedded software, for example, might not score as well due to the open source leanings here.)

POSITIVE POINTS

  • Linux kernel hacking, just for fun  +7
  • Committer/contributor to open source project(s) +6
  • Technical patent +6
  • Speaker at conferences/group events +5
  • Blog about tech (even if only occasionally) +4
  • Attend user groups/meetups at least once a month +4
  • Regular reader of tech books/blogs +4
  • Speaker at in-house company trainings +4
  • Master’s in Computer Science  +3
  • Tech hobby with mobile platforms, robotics, Arduino +3
  • Run Linux or other Unix flavor at home +3
  • Published author/editor of book or web material +3 articles +8 books
  • Start-up/entrepreneurial experience +3
  • Active projects on GitHub, BitBucket similar +2 each
  • In addition to your main ‘paid’ language/platform, some level of fluency in any of the following: Java, C++, Scala, Clojure, Lisp, Ruby, Erlang, Python, Haskell, iPhone/Android platform +5 per language

QUICK ASIDE:  Another element I tend to include in the ‘plus’ category is work experience for (+5) or a job offer (+2) from a certain subset of employers.  Why?  Because I know firsthand that these employers have very high standards in hiring and rarely make bad hires, so odds are if you have worked for or have been offered employment by one of these companies you are very likely good at what you do.  I can’t make this list public, but I share this just to let you know another contributing factor.

NEGATIVE POINTS

  • Worked over 12 years for current employer (few exceptions)  -10
  • Worked over 6 years for current employer  (few exceptions) -5
  • Only run Windows at home  -5
  • 40+ tech buzzwords or acronyms on your résumé -5
  • AOL email address -4
  • Résumé is over three pages -4
  • Having three or more permanent jobs in a given two year period -4
  • GPA/SAT score on résumé and 10+ years experience -2

If you found this post useful, you may find my ebook Job Tips For GEEKS: The Job Search very helpful.  You can also follow Job Tips For Geeks on FacebookTwitter, and Google +.

10 comments

  1. He who has banned recruiters

    I was actually going to leave an extensive comment on your latest post but having read through this I’ve decided against it. You claim to be a good recruiter but this comical test only tells me you haven’t got a clue. I don’t mean to be rude in any fashion but this is so wrong on so many levels that it doesn’t make sense to even attempt to comment.

    I suppose it’s just a different personality. Unless you are a good software developer (or have been) it will be impossible for you to understand what it takes to be one. Specifics of our profession.

    I think I will keep my permanent ban on recruiters for some more years.

    Btw, you may be interested to know that many good developers (actually the best ones) in very different countries quit their jobs and go independent, primarily because of the hiring practices and annoyance with recruiters.

    • fecak

      Thanks for the comment. Please, by all means, let me know what you disagree with on this system. If you care to comment on my other blog post, I’d be happy to hear your point of view on that as well. As opposed to just taking the time to say you disagree, please add something to the discussion.

      The fact that you have a permanent ban on recruiters must work for you, and I don’t know if I should expect to get a fair assessment of my ideas from someone who has a policy not to work with anyone in my profession.

      I know many good developers that have gone independent, and many of them have actually worked with and for me (I place on both a permanent and contract basis with clients).

  2. Pingback: What’s Your Geek Number? My Points System To Rate Software Engineers (without a full technical interview) | Carnet de notes
  3. Mark

    Hello,
    i just came across this post and as a beginner I would have to ask a few questions. I’m not a US citizen and English is not my primary language so excuse me for any mistakes in advance.

    I have never wrote a CV before, even thou I have a 3 years of working experience as Java developer (2 companies). Firstly, I was recruited directly from the university. Secondly, I was proposed a job after presenting a speech at the local Java based conference. After reading your post I decided to finally write my first CV and as I’m confirmed perfectionist i have a lot of doubts.

    1. Certifications.
    I know that certifications aren’t worth much, but does it mean that they shouldn’t be put in CV? You haven’t mention these in your point system, so I’m wondering: mention certificates or not. If yes, where should it be positioned? At the bottom of last page (as not so important), or at the top in the overview section, just a short info like: “certifications: SCJP, ABC, BLABLA.” (as short as possible)

    2. Organizations.
    I’m a Mensa member and heard from other people that it’s better not to mention such organizations in CV. It wont be a problem (to don’t present this info) except the fact that I’m not a passive member. I’m an administrator of the computer infrastructure of my country’s organization part, so (even as this is not a programming exp) i thought it’s worth mentioning in “experience” part of CV.

    3. Abandoned studies.
    I was a PhD student for 2 years and quit it as I get the second job (it requires relocation and was worth it). It would be better to omit this in my CV?

    4. It’s better to provide used technologies aside with projects I was working on? Or I should wrap known technologies in some separate section and let a projects (working experience) section to contain just a short briefing about my role at the project itself?

    5. Is it a good practice to provide a links to online projects I am working on? Link to blog etc.?

    6. If I wrote an article or made a speech at a conference should I provide a detailed list containing name of the conference, date, subject? Or just a short info like: “Articles and conference speeches in subject of concurrent programming”.

    • fecak

      Mark – Thanks for reading. Regarding your CV, certifications that are relevant should be included. Participation in organizations, with few exceptions, are also good to include. I would think that your doctoral coursework should be mentioned assuming it was in a technical discipline. Always include some information on technologies used on each project. Links to online projects are appropriate. If you have given speeches at events or even if you have attended technical events, it would be helpful to include the info – for speeches, give the name of your presentation, conference name, city, and year. Good luck

  4. Jude Osborn

    Good points here. I live in Australia where employers expect long detailed resumes, so the point about he resume length doesn’t really apply, but I think the rest is really good. I’d add a couple more plusses your list, though I’m not sure how many points I’d add:

    - Test driven development. Not just writing unit tests, but writing virtually all their code by starting with a test. This is a really good indicator that someone has experience in a quality development environment and/or has a love for quality coding.

    - High end JavaScript skills. I don’t just mean doing a few fade in animations with JQuery, but application level JavaScript development and/or experience with node.js or serious frameworks like Backbone.js or AngularJS. JavaScript has become an important player in recent years, and it really shouldn’t be ignored by any engineer these days.

    • fecak

      Jude – Thanks. TDD could have easily made the list. I tried to avoid specific languages and frameworks, but JavaScript is certainly a very desired skill set these days. I’ve been telling my network that for a while now. Thanks for the comments.

  5. bigdaveyl

    Sorry to be late to the party.

    I agree in principle, but how does one sidestep the whole “stagnation” issue?

    For example, I would score in the 20′s but that would be negated by the fact I have been with my current employer for a decade. Obviously, I have attempted to keep up on trends in my free time, otherwise, I wouldn’t have gotten 20+ points.

    The problem with my situation is that I took a job that seemed interesting at the time, I was just out of college (was graduating during the bust), and survived the dot-com bust and was profitable. I am doing “programming,” just not anything mainstream (it’s a niche product for a certain space) and I’m getting some experience that can be applied to other jobs (like bash scripting). I have worked my way up the ladder so I am making a decent salary, and would have more than doubled my salary if the 2007-2008 financial crisis hadn’t taken place.

    Anyways, it seems like most of the positions available locally (I have branched out at times) require paid experience, and obviously recruiters are bound by their clients needs, which I understand. It seems like the most consistent type of phone or email screen/interview I will get pinged for is “technical support analyst” positions for mainly B2B software. In other words, it’s tech support but you aren’t dealing with the unwashed masses. The obvious problem with these jobs are:
    (1) The pay is crappy
    (2) no real career progression path to get into something that I would enjoy doing and may make better use of my skills (you do a bit of soul searching because you keep doubting yourself and wondering if you’re doing the right thing)
    (3) At least I am “programming” at my current job
    In other words, what do these jobs offer me, and/or how do they put me in a better place going forward?

    Do you have any suggestions?

    • fecak

      If we’re talking beyond just the context of this article, one avoids stagnation by moving once their position does not provide the opportunity to learn or grow. By ‘requiring paid experience’, I assume you are referring to paid experience with a certain technology. So for example, if your day job is doing tech support but you do Android programming at night, you are saying that companies won’t look at you for an Android job? That isn’t really my personal experience, but you often have to be able to demonstrate your value (which makes having apps available very valuable to mobile devs, just as one example). This is where things like GitHub come in.

      You are getting pinged for things you don’t want. Instead of waiting around to be found, you should target some companies you’d like to work for and then contact people there and justify why you could be a good hire. If you’re just posting resumes on Monster and Dice, you probably aren’t going to be getting discovered by the kinds of places you may want to work. Don’t be reactive, be proactive.

      • bigdaveyl

        Thank you for your reply.

        I do have a github project, stack exchange, networked, trying to target specific places, volunteered, etc, so I have been a bit proactive on my end. Could I be better at doing that stuff? Yeah! Who couldn’t be?

        You’re right on about the paid experience. It seems like the run of the mill HR person, recruiter/head hunter, hiring manager, etc. seems not to give two craps about these things, at least in these parts. It is quite possible that I am targeting the wrong spots and have to kiss a few frogs. For example, there is a recruiter that does a lot of recruiting for telecommunications, networking, administration and the like. I heard he had an opening through the grape vine, only required a minimum amount of experience. I sent him a message stating that I thought I had experience in a lot of the things he was looking for and pointed him to the specific places/portfolio on the web that would show that along with my resume. Of course, I do not hear anything.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s