Five Reasons Why I left Startup Life

Ever since I left the startup life, I’ve been quiet about why.  There are some things I shouldn’t talk about (since I still heart everyone at that startup), but there are some things I can talk about; and I’d like to focus on these, because they aren’t specific to any one startup, and they’re issues you’ll need to face sooner than later if you join as an early stage startup employee.
This critique is focused on early stage startups where you are or will be joining as the first or as employee 1-5.  It’s especially focused on startups that are either still going through their seed round or just finished the seed round.
Startups have no administrative support. Startups are new businesses, and the lack of administration that goes with that.  There are several things you take for-granted in a business, things like insurance, clockwork payroll, T&E systems, and an administrative support system that enables you to focus on doing your job.
None of that exists in a startup. If you’re lucky (and if the startup has the funding), you’ll get some sort of group insurance; but if not, you may be buying it on the individual market (or your company will pay your COBRA, as happened in my case).  If you’re employee #1, they’ll likely just be setting up payroll, and that is fraught with the same problems that any new system has: a lack of use means a lack of habits, which means mistakes. You’ll also probably be the first person to use T&E for your business, which means receipts and spreadsheets.  There are programs that seek to make this easier (ZenPayroll, Gusto, Expensify) but they aren’t free, and they still have the same issues each new system has: integration cost, time, and money. Every time a founder is setting up those systems or working with those systems, they aren’t working on anything else.
Every business person becomes a single point of failure because if they’re out, there aren’t yet processes in place to make sure someone else can handle it.
Being Employee #1 means being the squeaky wheel. If you’re Employee #1 in the company, you are the one that will encounter any cultural, process, or feedback issues, and you’ll encounter them first. Founders have a fight? You’ll know about it (it’s too small not to). Fundraising is hard? You’ll know. What’s the company’s process for paying for your home internet or cellphone that you’ve been using for work? Oh, they said one thing but didn’t follow through? You get to be the one to bring it up. Memorial Day is coming up: Do we have those days off or not? What days do we have off? Do we actually have vacation? How many days? Is your boss missing something fundamental? Are they doing something you wish they’d stop or change?  I hope you can tell them because there isn’t anyone else to, and there’s no one else to talk to about it.   Are they presuming you’ll pay for your trips and then get reimbursed? (Have they forgotten you work for a startup, and by definition make under market value?)
It’s really easy to be micromanaged because everything you do is visible.  In a startup, you have an outsized impact on the outcome. As a software developer, the business impact of what we do is invisible. Set up unit tests? Invisible. Create script to deploy a site? Invisible. However, that time you spent is very much visible. Since you’re obstensibly there to ship the product, those things that make future you’s life easier (like setting up a deploy process or documentation) is de-prioritized.
When I was starting the firmware work, my initial approach was to develop it using TDD. It’s methodical, helps me keep bugs from encroaching, and would enable the architecture to handle our desire to allow user code (by decoupling the architecture).  So I brought up that I was going to spend some time setting up the TDD environment and porting the existing demo code (that I didn’t really understand, to be honest) over to that.
I received a lot of flack from that, including from unexpected places.  Keep in mind, there were only 4 of us total, so to have one person complain about it starts off the second person complaining about it (Yes, GroupThink even hits small groups), and they put the kibosh on that.
As a direct result of that conversation, I was instructed to ‘not worry about setting up the project for TDD’. I spent far more time (easily 3x) fixing bugs incurred by not using TDD than I would have if I had set the project up to be TDD-able from the start.
I need to be clear; this came directly from a combination of the two issues I mentioned above: It’s really easy for a Founder to micromanage one person; and it’s really hard to push back (and yes, I should have pushed back. Not pushing back ended up adversely affecting the project in other ways).
Startups lack reliable funding, especially startups in unproven industries. I won’t say much on this except to say that there was never a moment when we had a year of runway left when I was at the startup; and the number was often much lower than that. This was due to a combination of factors, but the two most prevalent were: 1) we were doing something new and 2) hardware startups are… different. Personally, if Hardware startups have “never been easier” (according to Ben’s closing quote), then I’d hate to have see them before now.
Your life will be filled with constant stress and there’s no one support structure inside your startup to combat that. About two weeks after I left the startup, I visited relatives, and they remarked immediately how less ragged I looked.  I hadn’t noticed, but thinking back to that time; my days were like so:

  1. Get up at 6am. check email, make sure site isn’t down. Get the kids out the door.
  2. Take a shower. Check email.
  3. Get in front of computer around 7am.
  4. Work on development (Firmware, app, website, whatever).
  5. Encounter bug. No one to bounce ideas off of.
  6. Thrash. Sometimes for minutes, sometimes for hours.
  7. Have standup at 10. Explain there’s a bug and I don’t know how long it will take to fix.
  8. Endure sighs from teammates (none of whom either have the time to help, or could really help on this)
  9. go back to figuring out the bug.
  10. Ask Hardware Lead to sit on a hangout to figure out the bug (with no other developers, this was as good as it got)
  11. Figure out bug by 3 or so. Eat lunch.
  12. Work until around 6pm. Eat dinner.
  13. Work again from 8-10, or until bug was fixed / measurable progress was made.
  14. Go to bed.
  15. Rinse, lather, repeat.

Being the only software developer at a company where most of the visible pieces are software is difficult; but even more difficult is not having another developer to work with to solve problems. The Hardware lead was amazing and helped out a lot (I don’t think we would have shipped had he not been my rubber duck); but it’s hard to replace a classically trained software developer, and there was definitely time and money lost due to that.
I burned out three times when working at a startup, and the first time was after I worked through my vacation. (see those dark commits during the two week period at the end of July/Beginning of August, that was my vacation).
Due to issues like developer health and burnout, I firmly believe there should never be a software developer working alone, and this experience only solidified that. Software Development necessarily means building something new, and the cost lost to thrashing and burnout is far more than the cost of hiring a second software developer. This is especially true if you’re asking the developer to build something in a stack they’ve never touched before (for me, Embedded C).
But, I couldn’t very well complain to my teammates, could I? After all, they had their own full time issues; fundraising, building hardware, growth; what could I say to them? Ultimately, I was sacrificing my life and health in a situation where I was just an employee. There was a mismatch of devotion and position. If you work 12+ hour days for a year and a half, but at the end of the day you’re just an employee with the option to buy stock, you’re not going to feel valued.
If I had to do it all over again, I would have adjusted my performance and expectations to be just an employee and put my family and my health first.  I would still do it; but I would not have sacrificed what I did to do it.  We talk a lot about self-care in this industry (though far less than we probably should), but startups and investors still value the worker be that sacrifices their life and health for the “good of the startup”.
I am fully responsible for all the actions I took. I chose to put myself through this because I fully believed in what I was building and I had a mismatch of expectations, thinking that I was more than just an employee of a company.
So you’ve read all this and you are thinking you want to join a startup, great!  Here are some tips I have:

  1. Clear expectations first. This means getting everything in writing and being explicit about your role in the company.
  2. Set boundaries. These could be personal boundaries, it could be professional boundaries. In my case, I should have (nicely) told the other party to pound sand when they told me not to architect the firmware a certain way. I was ultimately responsible for the time and cost overrun due to that; so I should have taken a firmer stand.
  3. Remember, this is a marathon, not a sprint. You are in control of whether you burn out, so make sure to see the warning signs and act before it’s too late.
  4. Negotiate for a package that makes you feel valued, and stick to what you negotiated. If they promise $X raise after Y milestone, bring it up. These were the expectations you were hired with, and it’s no one else’s responsibility but yours to ensure you’re treated according to your expectations.
  5. Unless you’re a founder, you are just an employee. Your devotion and your passion will be used against you, sometimes by you. If you really believe in a project, that’s a warning sign that you should take a step back and rationally think through it. Ask a friend who isn’t emotionally invested.  There’s a reason why car dealers ask how you feel about a car after you test drive it. If you fall in love with the car, you’re more likely to make a worse deal than if you were dispassionate about it. At the end of the day, no matter how passionate you are about your mission, you’re still an employee. You are at least third in line for priorities; sometimes fourth. (Investors, Company, Customers, and then you). If something causes an upset in the balance, you are fourth on the list of people to worry about for the founders.

Joining a startup can be the most rewarding thing you ever do, just make sure you jump into it with eyes wide open.
Update #2:  Four days after I wrote this, I wrote about Five Reasons Why You Should Join a Startup. It’s the other side of the coin, as it were.
Update #1: Removed the name of the startup. Yes, it’s easy to find; but these lessons aren’t specific to any one startup.

How do I require specific text in a git commit message?

You have your own git server (either on-premise gitlab or something else), and you want to require your developers to have certain text in their git commit messages, like the work item number it falls under, or the ticket number.
To solve this, you need to create a git hook, which is just a file that goes into your gitlab server. The file goes into your /data/opt/gitlab/git-data/repositories/my_project.git/custom_hooks directory. You may need to use sudo -i to cd to this directory.
The hook you’ll create is called a pre-receive hook, and it is specifically meant to check all commits before adding them to the remote repository.
pre-receive hook that requires certain text to be in the commit message is below:

#!/bin/python
import sys
import re
import subprocess
#Format: "oldref newref branch"
line = sys.stdin.read()
(base, commit, ref) = line.strip().split()
new_branch_push = re.match(r'[^1-9]+', base)
branch_deleted = re.match(r'[^1-9]+', commit)
contains_commit_msg = False
if not new_branch_push:
    revs = base + "..." + commit
    proc = subprocess.Popen(['git', 'rev-list','--oneline','--first-parent', revs], stdout=subprocess.PIPE)
    lines = proc.stdout.readlines()
    if lines:
        for line in lines:
            rev = str(line)
            match = re.search(r'TICKET-[0-9]{2,5}|#NEX-[0-9]{2,5}|HOTFIX|FORCE', rev)
            if match is not None:
                contains_commit_msg = True
if contains_commit_msg or new_branch_push or branch_deleted:
    exit(0)
else:
    print "Commit does not contain the story associated with the commit in the format: TICKET-123 or #TICKET-123"
    exit(1)

The code is in python.
The pre-receivehook will receive a list of commit hashes in the form of:
ab3c291835832c309 b2fed56890cab348 new-branch
where the first hash is the old commit, the second is the new commit, and the branch name is the third.  This also means you can use this hook to ensure branch names meet a certain criteria.
Deleted branches will have zeros for the second commit, and new branches will have all zeros for the first commit. and those would cause errors (since new branches really don’t have a commit message; and deleted branches don’t have one either); and this code handles that.
If you want specific text to be in the ticket, this line handles that:

match = re.search(r'TICKET-[0-9]{2,5}|#TICKET-[0-9]{2,5}|HOTFIX|FORCE', rev)

Just change the text in r''to your own python regular expression. You can test your python regular expression at pythex.org.
This regular expression says that the ticket must have the word “Ticket”, followed by a dash, followed by a 2-5 digit number, or it must have a pound sign in front of “Ticket” followed by a dash, followed by a 2-5 digit number, or it must have the word HOTFIXin all caps, or it must have the word FORCE in all caps.
Once you have your pre-receive hook in the right directory, you must do three things:

  1. Change the owner to the gituser (the user account that will run this command):

chown git:git pre-receive

  1. Make the command executable.

chmod +x pre-receive

  1. Change the owner of the custom_hooksdirectory to git.

chown git:git pre-receive /data/opt/gitlab/gitdata/repositories/my_project.git/custom_hooks
I keep an up-to-date version of this pre-receive hook on my github page.

Nulls Are Not The Problem

I recently saw this promoted tweet:
screen-shot-2017-01-03-at-10-31-28-am
Of course, this isn’t the first time Null has been brought up as a source of woe for programmers. There’s a wiki on why Null is Considered Harmful, and the most recognizable argument against null is from its inventor:

I call it my billion-dollar mistake. It was the invention of the null reference in 1965. At that time, I was designing the first comprehensive type system for references in an object oriented language (ALGOL W). My goal was to ensure that all use of references should be absolutely safe, with checking performed automatically by the compiler. But I couldn’t resist the temptation to put in a null reference, simply because it was so easy to implement. This has led to innumerable errors, vulnerabilities, and system crashes, which have probably caused a billion dollars of pain and damage in the last forty years. — Tony Hoare, 2009

Since ‘null’ is clearly the source of our problems (it’s right there in the name “NullReferenceException”), then we should get rid of Nulls. Problem Solved!
Or, as H.L. Mencken once said:

Explanations exist; they have existed for all time; there is always a well-known solution to every human problem — neat, plausible, and wrong.

Absolutist positions in programming immediately make me question the source. The absolute ‘badness’ of NULL is a sacred cow that needs to be slaughtered for the good of our industry.
Null, in its simplest form, is the absence of a value.  In the same way Evil is the absence of Good, or a donut hole is the absence of a donut, null is the absence of a value.
That absence, however, has meaning.  And this is where we as programmers falter.  We give that meaning several distinct meanings that depend on context. We make decisions based on that meaning without clearly communicating our intent, and we forget to check whether there is an absence of a value.  How many times have you heard a programmer say “There’s no possible way that method could fail!” only to have it fail in production? I’ve personally lost count.
If we banished null to the “never use” bin of object oriented programming because we use it incorrectly, then there are a whole lot of things we need to get rid of with that reasoning.
The second issue with banning null is that the absence of a value has meaning contextually.  In a purely closed system it’s not an issue, but any system that interfaces with the outside world (meaning any program other than yours) is going to have times where it doesn’t work, or times where (for whatever reason), it doesn’t return the data you need.
If we tell programmers to write code to handle failure, why wouldn’t we tell programmers to check for data failure as well? That’s what the basis of Object Oriented Protocol is, after all (check the name, “Object Oriented Programming”): it’s data and behavior mixed together in one cohesive unit.
So, if you write code that will never interface with anything else, feel free to ignore anything I’ve said. After all, you can program against that uncertainty.  But for the rest of us, Null is an important part of our toolkit.
In three different systems I’ve built, I had good reason to use Null:

  • Sometimes I don’t get data back from the source.  Not getting this data back means I shouldn’t reason about it negatively; but its absence is a decision point for other paths in my program
  • Parsing text is a common failure;  should I always and forever represent that as something other than what it is? Is the failure to create an object from a parse an actual object itself?  It’s the absence of an object. After all, I failed to create it!
  • They haven’t set a date for an action.  Should I treat that as a value?  Or the absence of a value?

In each case; I used null (wrapped in a method to adequately describe what null means). According to the Null detractors, I should have returned a constant, or thrown an exception, or something.
Sadly, none of the ways to get around null fix the problem, they only paper over the issue. Let’s take his first example, returning a constant instead of a null, using the null object pattern:

public Employee getByName(String name) {
  int id = database.find(name);
  if (id == 0) {
    return Employee.NOBODY;
  }
  return Employee(id);
}

If we don’t find an employee, it’s purely semantical to return a constant Employee.NOBODY, and even worse is that it’s potentially harmful downstream. If our payment processing software (as a purely contrived example) gets an employee object back, but doesn’t check that that ‘nobody’ isn’t an actual employee, then they’ll go ahead and use the default values for the properties in their calculations.  A salary of $0 really skews reports.
“But, they’ll check against Employee.Nobody”, you say.  That’s the same as checking against Null.  The only difference is one will throw a runtime exception and the other one won’t — they both still have logical errors if you insist on filling it with default values (and not null).
The second argument against null is to use exceptions as flow control:

public Employee getByName(String name) {
  int id = database.find(name);
  if (id == 0) {
    throw new EmployeeNotFoundException(name);
  }
  return Employee(id);
}

Except, of course, Exceptions should be used for handling Exceptional things (it’s in the name), and not for common problems.Not finding an employee (for instance, in the context of searching) should not be considered exceptional.  If anything, it’s normal. People mistype names all the time. Humans make mistakes.
It’s even another ‘best practice’ to not use Exceptions as Flow Control. While I happen to agree with that practice, I put best practice in scare quotes believe all best practices are conditional on context. I haven’t yet encountered an occasion where exceptions as flow control is good; but that doesn’t mean there isn’t one, and I won’t tell someone “don’t do that” unless I’m looking at the code with them when I’m saying it, and I’m saying it for that particular circumstance.
Matt Sherman, a development manager at Stack Overflow (and someone I greatly respect) has said this about Nulls:

I submit that we can avoid a large majority of null-related errors if we follow these two principles:

  • Passing a null is a programming error
  • Branching on null is a programming error

I think most of our problems trace back to imbuing null with meaning, i.e., we use it as a signal.

I submit that null is the absence of meaning. It is not a de facto Boolean describing existence. Rather, it should be treated non-deterministically.

This implies that most functions should not “handle” null. Because null is the absence of meaning, we can not use it to make a decision.

These are great ideas, but they only work for a closed system, which is a system where you control all inputs and outputs, and aren’t dependent upon any external systems or processes that might fail for unknowable reasons.
Passing a null in a system is contextual (and maybe it shouldn’t be): If an external system passed me null; I may have to make a decision about that; I can’t simply say, “It doesn’t exist”, because it may exist (I argue that if you want a certainty for truthiness, then you need true or false; but if there is a degree of uncertainty, null is useful for conveying that).
For instance, if I get passed null for an Employee (to go back to our previous reason, because another system couldn’t find an employee), then I have to make a decision based on that, so I have to branch on null.
My point is that we can’t wish away null or just chide programmers for using it. It’s here, and it has meaning.  What I believe we should do is to reduce the amount of meanings it has; and that doesn’t mean not using it or programming it away completely; but it does mean reducing uses of null to mean “I don’t know, and don’t make any assumptions on why I don’t know“.
You should read all of Matt’s post. He brings up some great common sense approaches for reducing null and where it could be a code smell:

A common idiom in C# is to use optional parameters, with null as the default value:

void Foo(int id, Bar bar = null, Baz baz = null)

When I see a method signature like this, I have $5 that says the method’s implementation branches on those parameters. Which means that parameters are no longer just values, but are control flow. Bad semantics.

Instead, consider creating several methods with good names, and which accept and require only the parameters they use – no more optionals. “Branching” is done via method resolution, about which we can more easily reason.

Also, you should follow Matt on Twitter; he’s pretty awesome.
But, back to slaying sacred cows. Another sacred cow for Yegor are that static methods are bad:

A dry summary of [the above listed arguments] is that utility classes are not proper objects; therefore, they don’t fit into object-oriented world. They were inherited from procedural programming, mostly because we were used to a functional decomposition paradigm back then.

And much like the arguments against null, none of this is prefaced with context. So if you say  statics are bad in OOP, you should then say why they’re good:

[Stack Overflow] goes to great lengths to reduce garbage collection costs, skipping practices like TDD, avoiding layers of abstraction, and using static methods. While extreme, the result is highly performing code. When you’re doing hundreds of millions of objects in a short window, you can actually measure pauses in the app domain while GC runs. These have a pretty decent impact on request performance.

Null, much like static; has its place.  We should not minimize its importance, rather we should write our software to handle it. For developers this means:

  • Don’t shy away from using null/returning null if it fits your business case.
  • Do ensure you’re documenting (preferably through great naming) what null means in the context.
  • Don’t abstract null away to a ‘default’ reference type if your system has consumers; they may treat a default value differently from not knowing the value; and that’s a source of bugs.
  • In your own closed systems, reduce the amount of knowledge someone needs to interact with your code. You can architect out chaos in a truly closed system and it’s worth the extra time to do that.  This may mean throwing exceptions or having a default type. This works because you control the inputs and outputs. It works because you’re in a closed system.
  • In a system where you communicate with the outside world (read: anyone other than your own program), you will encounter failure. You will encounter the absence of data. Prepare for it.
  • If you have downstream clients, then don’t pass them null for multiple reasons (if you choose to use null); and if you can possible pass something that conveys more information, do so.  If null means “I don’t know” in your system, then it should mean that everywhere.

Programming is a means of representing and automating life interactions with computers, and life is not nearly as well ordered as programmers want it to be.  To make better software, we should prepare ourselves and our code for external forces, not chastise them for existing.