Software engineer goals: fairness and career development

Introduction

As you might have surmised from the previous post, I care a lot about getting the right goals in place for software developers.
There are several reasons for that, and I'll unfold most of them as this post finds its way to conclusion. The first and the main principle is already revealed in the title though. It is fairness.

Goals are not just for people whom I manage. They are also for me: they make sure that everyone is assessed fairly based on their results, and not by personal bias or most recent memories. They also draw a fair, uniform, target that determines rewards and career progression based on performance rather than visibility, luck and various other subjective factors.

Software engineers: what do they do?

Ok, time to get off the soap box. Let's talk trivialities, and start by asking a semi-rhetorical question. What do software engineers get paid for? Well, they get paid for coding (or, if we hop back on the soap box for a second, they draw their paycheck for constructing software).

So, if we try and be as simplistic as possible, the more software is written in a mythical man-month with the right quality, the more rewarded should the employee be. Yes, I know, writing software is not flipping burgers, and we are way too simplistic here. That will be fixed before this article is over, but for now please hold the indignation: we all have to start somewhere. And, truth be told, there is strong evidence that there is a 10x difference in productivity factor among developers. If one guy creates ten times more value than his/her neighbour, shouldn't there be some kind of reward? So, let's develop the topic of

Productivity

Assuming we answer the previous question with an emphatic 'yes', we have to understand how productivity is to be measured. Now, this is a much taller hurdle. In fact, there is no acceptable answer - many have struggled to define various productivity metrics, and many have failed.
Just counting lines of code produced (LOC metrics) is too easy. LOC differs based on language, complexity of the task, and encourages bad copy/paste practices, verbosity etc. As an extreme example, imagine a hard defect that is resolved by a single line change after a couple of days of surgeon-like troubleshooting, and compare it to a copy/pasted test harness code. There are more cases such as these two where counting LOC fails, and fails hard.
There are slightly better productivity metrics: story points, functional points, and requirements implemented. Unfortunately, better does not mean good. Story points are subjective estimates, and it's not uncommon for one developer to price a task at 5 SPs, and for another 13 SPs. Functional points and requirements are often too bulky, and don't work well when multiple developers collaborate on the same requirement. As always, we need to be wary of side effects, where engineers motivated by speed with which they implement new requirements, and thus, eschew maintenance work or unit tests.

To avoid all this nastiness, I've arrived to fluffy comparative goals, which go along these lines:

Complete tasks at a pace consistent with engineers at level X

Yes, I know, very fluffy, not smart (upper- and lower-case), and prone to misinterpretation. However, it does provide a talking point if someone does not pull their own weight, or vice versa, if someone is indeed very productive. Future goals in this article will be better though - promise. For now, let's say that we want to talk about productivity, but not necessarily measure it with fine instruments. With that in mind, we can turn to greener pastures, and talk about

Quality

This is an easier one. Defects can be measured, and the more defects created the lower is the quality (yes, captain obvious is yet again at work). Obviously, one way to avoid defects is to avoid writing software, but hopefully the productivity goal, even in its vague form, puts a boundary around that.

So how are we going to measure good versus bad? One approach is to do a similar cop out as with the productivity goal, and say something like this:

Do not create more defects than expected from engineers at level X

This is better than nothing, but I feel we can do better. Sometimes, one bad defect that caused an exec-level aggravation is big enough to register, while five minor defects not visible to naked human eye don't. For this reason, we can take it one level forward:

Do not create any showstopper defects, not more than A major and B minor defects as expected from engineers at level X

This is a bit better, but in some situations we can take it further. Catching defects at code review vs. QA vs. customer differs by orders of magnitude in organizational costs. In other words, we want to encourage strong review and QA practices to create better software at lower costs. The main point though is that it is not just the manager's job to facilitate that. The developer should be the main stakeholder in catching defects as early as possible. They should be motivated to have a thorough code review, and be happy when their peer finds an issue in their work. They should take active interest in the QA plans, and appreciate testers who find issues.
For many people, this is self-evident, but definitely not for everyone. Assuming the point of goals is to encourage particular behaviour, we can go for this:

Do not create any showstopper defects, not more than A major defects found in code review, B found in QA, and C found by customers as expected from engineers at level X. Also, do not create more than D minor defects.

Maintainability

Before we move off this topic altogether, let me just touch upon an important and slightly controversial point. Defect is not just a case where the product does not behave as per specification. It is also a case where the code is so brittle that a defect is liable to appear once someone else touches it. Poor encapsulation, 100+ LOC functions, lack of OOD, absence of test coverage are an issue in themselves, even if the code (usually by accident) functions the right way. Hence, I tend to treat those as defects during code reviews. In case you disagree, think of a builder who uses hollow bricks here and there in your new house. Sure, the house does not have any defects today in the strict sense of the word. Is this house well built though, and wouldn't you like to reflect your opinion in the builder's assessment?

Tracking

Before we race too far though, let's talk about the evil twin of elaborate goals: performance tracking. If we work off a goal which enumerates a number of defects found during code review, minor/major and so on, we need to somehow measure these numbers over the performance period.
The more goals and numbers are there, the more there is to measure, and there's unlikely to be a friendly PA or accountant that would take care of the business. No, it would have to be the goals' creator, i.e. you, the manager.

Hence, there are two valid options:

1. Intentionally dumb down the goals, and rely more on your subjective opinion and working relationship with the employee when it comes to reviewing.
2. Work with approximate numbers as much as possible, but still have SMART definitions.

There's no right or wrong here. Usually, I go for the second option, and do lightweight tracking every week or so by registering any "abnormal" results, i.e. whenever a complex work is done to great quality or vice versa. In the vast majority of the cases, this suffices to provide a fair review. Rarely, when I and the reviewee have different perception of the work done, I might have to dig deeper and get more accurate numbers on demand for a task or two. Just make sure that your code review system, SCM and quality system are there to back you up.

Proficiency

So far, I've concentrated on the bread and butter of software engineering: i.e. creating software. However, this is not always the entire story. Software professionals also contribute by solving problems, reviewing code of others, responding to technical queries, designing solutions and so on. The higher an engineer is on the technical career ladder the less they code and the more they deal with aforementioned activities. This, in turn, means that we need more goals as coders become developers and architects.

However, measuring proficiency again bumps into lack of metrics. (For example, counting the number of technical emails answered is emphatically not the way to go.

One goal I tend to use goes along these lines:

Be the technical point of contact for components X, Y, and Z

Sometimes this can be extended into collective responsibility where an engineer is accountable in some degree for all defects in X, Y and Z. This usually happens further along the career curve, and has to come with informal and formal authority.

Adjusting

I've spent quite a bit of your and my time talking about what goals should be. However, now I'd like to pop that balloon and again say something controversial. The important part of goals is not what they are, but that they are a living and breathing mechanism.

If the manager reviews the goals every couple of months, checks whether they fairly assess their team's work, and adjusts them as necessary, then he/she would do a reasonable job, even if the starting point was off the mark. And vice versa, preparing the perfect goals, putting them in the fridge for 12 months, and unfreezing them come review time in many cases will yield a slightly spoiled result.

Of course, there are always developers who are happy where they are and do something very similar year after year, but they are usually not the focus of elaborate goal construction. It's those that develop their skillset and have aspirations that are the focus of this article, and for them goals must be constantly revised: same as their career.

Project-specific goals

One dilemma that came back to me year after year revolved around project-specific goals. For example:

Complete refactoring for the management front-end by Q3, release it to customers by Q4, with fewer than X major regressions

There's something very tempting about these goals: they cascade directly business goals to individuals. At the end of the day, I care less about amount of code produced, or the amount of entries in the ticketing system. What I care about is whether customers got the feature on time, and whether they are happy with it, and such goals allow expressing just that. 
However, they might not be less tempting for the individual, as sometimes not all factors are in their control (e.g. the business does not prioritize this feature enough, or there's no right QA skillset present, and defects escape). This is why I oscillated between adding and omitting those; at this point, I'm using them sparingly with a number of caveats to ensure that they are achievable and fair.

Summary

As usual, this post is less about providing answers, and more about sharing thoughts. I went through a fair amount of trial and error when managing engineers, and that process led me in a particular direction. There's no single size that fits all, and my last advice for today will be to use this post as food for thought rather than a template.

Software engineer goals: when SMART is not smart

This time around I'd like to talk about creating goals for software engineers.
First things first, let's get the well-known body of work behind us. You probably know about SMART goals. If you do not, the link to Wikipedia is in the previous sentence. One way or another, by the time we went past this paragraph, we're all on the same page with regards to specific, measurable, achievable, relevant, and time-bound (or finite, if smarf were a good acronym) goals.

However, once we leave this familiar territory behind us, we get into the big lawless desert that separates the SMART ideal from real, flesh and blood, people. As managers, we intuitively know what we'd like to get from our team, and we know what SMART means. What we don't always know is how to relate one to another.
In this mini-article I'll try dispelling the notion that goals can be generated in an algorithmic way, by ticking specific boxes.

Let's kick things off with a few counter-examples:

SMART goals that aren't smart

Counter Example #1

Resolve at least 10 bugs in the next 3 months

This goal is specific, measurable, achievable, relevant, and time-bound. It still has one big problem that should banish it into neverland.

Here's the thing: bugs are not born equal. A typo is a bug, and a subtle kernel packet routing problem is a bug. Expertise and time required for dealing with these two issues are incomparable. By setting such a goal, we encourage the reviewee to seek out the easier bugs: after all, they have to hit a number. And, by extension, we nudge their career into technical comfort zone, and inhibit progress.

Counter Example #2

Raise 2 patents in the next year

This goal also ticks all the 5 boxes, though one can argue that the "A" (achievable) is a bit shaky. However, just to make our discussion simpler, let's assume that the reviewee is working on projects that have potential for patents. There's still a big chasm however, one which is happily supplied by the legal system. It takes years to fully approve a patent, and until those have elapsed, it's hard to tell whether the patent brought benefit to the business. Internally, there are few people who can serve as the quality gate before the patent goes out to the bureau. So, we might get 2 patents, but those may waste our internal resources, employee's time or both - it's hard to control and assess their value.

Counter Example #3

Find 15 defects

I've seen this one a number of times, and it is one of the worse sins to be stamped on a QA engineer's goals' spreadsheet. Firstly, it creates antagonism between developers and testers: if a developer does their job well, then there are fewer defects to find. Hence, maybe the tester now is less interested in having strong quality before getting their hands on the new feature. Also, as per the first counter-example, bugs are not born equal. Often we'll value a single non-trivial, high impact defect detection over 10 typo findings.

Thinking about side effects

In each of these three counter-examples, we had a particular outcome in mind, dressed it in SMART clothing, but the result was full of side-effects that subverted our original aim far and wide. The intents are hard to argue with: we want developers to fix defects at a reasonable pace, we want them to be innovative, and we want our testers to release quality software. The problem is not with them, but with how they were expressed. The obvious question now comes: how should they be expressed in a smart (lowercase) way?

Revised examples

Counter Example #1

Resolve at least 10 bugs in the next 3 months

What was the original goal here? To be blunt, we want the developer to be productive, and not sit on a single work item for weeks while his teammates are sweating through the backlog. However, as described above, we solved that problem by creating several others.
The sad truth is that the number of defects resolved do not codify productivity. What does codify productivity is the total value or complexity of tasks completed. If we operate in an Agile-like environment, we could express this via percentage of tasks completed versus committed (which can be more than 100%):

Complete at least 90% of committed tasks each sprint

Of course, the word sprint can be replaced with month, or any other recurring time unit. Note that the goal does not prevent people from under-committing. The good news is that we are neither lawyers nor sportsmen. We don't have to legislate everything, and we can prevent under-commitment when planning our work, and ensuring that everyone gets their fair share.

Counter Example #2

Raise 2 patents in the next year

Here, we were obviously after innovation. There's a senior engineer on the payroll, and we'd like him/her to generate sound ideas, and not just implement what they're told. However, we well know that innovation does not come via patents, and the value of those patents can be hardly measured come review time. Perhaps it's time to detach ourselves from numbers and go for something like this:

Proactively suggest design improvements and/or feature enhancements

Now this goal is not SMART, as it is not measurable. However, given the right description, it sets clear expectations, and it provides enough to talk about. We probably should say that suggestions accepted by the team, including product management, count more, and that we're going after quality rather than number. Ideas, among other concepts, are hard to measure and count, and it's more honest and productive to accept that rather than retrofit measurable goals for the sake of measurable goals.

Counter Example #3

Find 15 defects

The intent here is simple enough: we don't want to get defects out. However, setting a goal of finding X defects is akin to setting a goal of catching Y criminals. For sure, the bugs/criminals will be found, but they are likely to be the wrong ones, while the right ones will escape.
And, this is exactly what matters: how many criminals bugs escape, not how many are detected. Also, bugs are not born equal, so a fair goal would go along these lines:

Have no more than 3 critical, 5 high, 7 medium and 9 trivial defects escape under area of your responsibility

Summary

This mini-article had examples, counter-examples, numbers and absence of numbers, but little in a way of recipes and templates. That is fully intentional: you know your team better than I do, so I can't tell what goals you should set. This is exactly why I used the word  crafting in the title, and not generating: people are unique, and so are their goals. SMART is not a rule or an axiom, but a guideline, and guidelines always have exceptions.

Should managers code?

Intro

Let's start with a question: is there a point in the career ladder where we stop contributing individually, and only coordinate (or facilitate) others?



My view is that this point is like the proverbial asymptotic curve; it can get as close as we like to zero, but will never quite reach it. Or, coming back to plain terms - everyone should and can contribute; it's only a question of how much and what for. The line between a manager and an individual contributor is not the same as a geographical border, which draws a distinct line between "us" and "them". Rather, it is a long blur, and throughout that blur we should care about what we bring to the team. As careers traverse these shades and their accompanying titles, personal contributions follow along that curve getting close to a zero, but - the main point - there are still good reasons to code/test/review. They become indirect: a manager rolls up his/her sleeve not to add manpower, but for other reasons, which I'll go into very shortly.

Matching personal experience and ability to influence

Let's go back (or forward) in time to that moment where you (will have) got the desired promotion letter. Presumably, you reached that moment by doing your job well, and gaining the recognition of your peers and managers, and again, presumably, you have strong experience with the technology and ability to influence. (You might have met managers who answer none of the conditions in this sentence, but they are neither the topic nor the audience for this article). With all of that, it makes little sense to go and abandon all of that useful experience on the next day.
So, there's a dilemma: on one hand, you want to and can code, on another, you're no longer paid to do that. Or, is it actually true? Let's start with:

Leadership by example

Most respected, though not necessarily the longest-lived, generals lead their troops and take part in battles. IT managers benefit from a quieter lifestyle; apart from an occasional coffee overdose, or a minor heart attack due to a misdirected e-mail, they are usually safe from workplace hazards. However, there is one parallel: both soldiers and software engineers respect leaders who from time to time can stand shoulder to shoulder with them and demonstrate that they can pull their own weight.

To start with, it breaks some of that barrier between "us", the workers in the trenches, and "them", the generals who command from above. There's more of that important team feeling, especially if the manager helps personally during crunch time. On the other hand, personal experience makes it easier to understand software engineers' grievances and requests for refactoring, switching toolsets and so on.
In my experience, there were a number of instances where personally seeing and maintaining particularly bad legacy code gave me the right mindset and conviction to push upper management for the right investment in refactoring. There was also a case or two where using development toolset helped lobbying for easier processes, and resisting introduction of cumbersome workflows that always seem fine to a manager, but not to someone who has to endure them every day.

The good news is that it's enough to do a few tasks every month to get all the benefit from the above. People remember the general that led them to a single battle; they don't necessarily expect him to do it on each and every one. Solving 3-4 small bugs every month shouldn't take more than a couple of work days; i.e. not more than ten percent of office time. This small investment can buy respect of your reports, as well as the right mindset and awareness to help them with their day-to-day job. However, it can also help you personally:

Experience and career continuity

Let's become egoistical for a moment, and think of our livelihood, i.e. ensuring that paycheck continues arriving month after month. Those that have been in the industry for long enough, know that nothing in this world is forever. Companies' fortunes turn around, bubbles pop, and financial crises appear every decade or so. Of course, the more indispensable you are, the less likely you are to be affected by the cataclysms and storms that rage outside, but at the end of the day the mantra is the same: nothing is forever, and nobody is untouchable. We must always think of our skillset and consider how we'll fare in the greater beyond should the worst happen.

Now, here's the thing: managers have it tough out there. The ratio of individual contributors to managers vacancies is 1-to-10 (or thereabouts), and software developers/testers simply have more openings to apply for. Moreover, a large proportion of employers look nowadays for managers with strong technical chops. In other words, a strategy of shying away from hands-on work is a dangerous one. In today's climate, it can leave one unemployed for a while should a disaster strike.
Of course, "hands-on" and coding is not one of the same. There are great leaders out there who never code, but yet understand and speak technology every day. However, if you're in middle management, you can count on being asked coding questions during interviews (thanks Google), and those 3-4 defects a month can make a big difference in keeping your toolset sharpened.
Just in case you haven't been fully convinced, let me throw another argument:

Fair reviews and career development

Providing team members with fair reviews and gently nudging their careers in the right direction is one of main responsibilities that we, managers, have. What are developers doing most of their time? Right, they code, design and review software. So, when the time comes, the bulk of their assessment should be based on how well they've done just that.

Unfortunately, "should" is often a word removed from reality; usually we review best only what we observe, and managers that distance themselves from coding can't observe. Instead, they review using peer feedback coupled with their personal interactions with the team, which tend to be e-mails, meeting participation, and general visibility. These are all fine, but remind slightly of an art critic that forms her opinion by not seeing the art in question, but reading second-hand opinions.
Hence, those personal coding tasks, the occasional code review, if chosen judiciously, can play an important role in observing and providing fair performance viewpoint. And, before I depart onward, let's not forget about the art of giving praise. It's one thing to praise using generic phrases: "hey, you've delivered feature X one week ahead of time, attaboy", and another to do so while being very specific: "you've delivered feature X week ahead of time by reusing module Y, which didn't occur to any of us, and then you had another nice shortcut with introducing an adaptor layer on top of existing interface framework". This difference can mean a lot to a developer who genuinely and rightfully takes pride in their work.

Avoiding time-critical tasks

It's not all plain sailing though. Though it's not too common, managers can err on the other side, and do too much hands-on work. The one rule to remember is: your role is to help others perform. Coding is often easier, and more gratifying than answering a long e-mail, preparing a budget, or crafting a personal review. The unfortunate reality is that these "dull" activities always must take priority; they are what we signed up for when the promotion came through. An addendum to that unfortunate reality is unpredictability; we can never tell when a critical e-mail will drop in the mailbox, or when we have a crisis to resolve.
Continuing the same train of thought: coding and context switches are not friends. Everyone in the team gets interrupted, but a developer usually has the mandate to park questions for an hour or two while they are putting finishing touches on a task. A manager cannot play this card; his/her official job description is to facilitate others, not to code. Hence, when a manager looks for tasks to take over for all the aforementioned reasons, these should not be time-critical. Ideally, if they are put on ice for a day or two, nobody will raise an alarm, and team's performance won't get affected.

Margin of error and personal responsibility

So far, I've been lumping coding and code reviews together, as both are technical, personal contribution, tasks. However, reviews are special in a way, so I'll tease them apart for the course of a couple of paragraphs.
While I was going on above about coordination, e-mails, budgets etc. etc, the truth is that management is first and foremost about responsibility; for R&D managers, responsibility over software their teams produce. Here, there is a whole spectrum of options of ensuring quality: from trusting the team all the way to personally reviewing each and every output.

As always in life, extremes have pronounced weaknesses. Reviewing each semicolon typed by the team does not scale. Trusting the team is fine up to a point where a particularly bad decision misfires. It's not hard to find core parts of the product where mistakes simply cost too much: think something along the lines of Facebook's personal wall, JavaScript inside google.com, packet management inside a web proxy etc. I'm sure if you give it a bit of thought, you can find similar business-sensitive zones in your product(s).
Personally watching over these areas has both the effect of adding another pair of eyeballs, and technical awareness which would allow you to react quicker if something does eventually go wrong.

Summary

Hopefully, this article has put an argument for managers not to divorce themselves from source code. Or, if you're not a manager, it may explain why your boss keeps looking over your should from time to time, and makes an appearance in your SCM tool, and why it's not necessarily a bad thing.