I could be wrong, but I think this is a question that quite a few startups (and even more established players) grapple with at various points in time – whether its the initial launch, subsequent updates etc. The question tends to be more critical for startups because it could very easily be a life-and-death scenario, and startups typically cannot afford a sizeable Quality Assurace organization to tell them if the quality of their offering is up to snuff. A lot of factors go into such a decision – Features, Quality, Competition and Timing amongst others. And please bear in mind that this post is written in the context of our experience, which was building a consumer internet website. Depending on your product/service, some of these points might not apply.
At the time of planning the release, in addition to an ideal target date, you hopefully have the right set of features identified as well as prioritized into Must-Haves and Nice-to-Haves. It might be tempting to go for more fine-grained categorization, but I think that’s overkill especially when you are a startup. It will also be tempting to put a lot more features into the Must-have bucket, but when you are on an aggressive schedule and have limited resources, it is very important to be practical and have goals that can actually be accomplished. And while more features seems more exciting, it also means more code, more testing, more stuff for your users to grok, and most importantly, if you can’t make a good first impression with some of the features, it will hurt more than help the cause. The target date is likely governed by business constraints, so you’d like that to be a fixed target – if that’s the case, then you have to be refactor your feature list appropriately. If the planned launch is a big event, you are likely to spend a good deal of money and effort in promoting it, so it’s all the more important to plan realistically, allow for contingencies and ensure that you do meet the target date.
So you know what features are being targeted and you have a rough schedule for when you expect the work to be complete – make sure the milestones are clearly identified and you have allowed time for fixing bugs, stabilizing the product etc. As the development team is probably busy implementing them, a plan to test the new offering needs to be in place, and more importantly, start getting executed in phases. Initially, its probably just the development team testing it…then the rest of the core team starts to give things a go, and eventually, the broader team starts to get involved. Everytime we were getting ready to launch an upgrade for our site, we would have every staff member test it out and make sure that the functionality on the site was intact before it left the door – new features work as advertised, no breakage in the key functions on the site, no obvious performance degradation etc. When you are small team, you better be able to do that. And if you can build some basic automation around these key elements as you go along, you are going to save yourself a lot of effort and money down the road, especially if you plan on doing a few iterations of this cycle. If your offering has integration points with other partner sites, then you most certainly want to ensure that those are well-tested and validated as well. It is important to track quality as you go – all you need is a simple, centralized trackable list and a sound process to keep your team moving thru that list. We used Google Docs to file our bugs/work items, assign ownership and track progress. Granted it isn’t as granular as we’d like it to be and doesn’t provide a lot of tracking support, but it gets the job done. Get your team to be discplined about entering and updating issues into the list and review the list on a daily basis to keep things moving.
Once your satisfaction level with the quality increases, you should consider sending notification out to a limited audience (called friends and family) and get some external validation thru them before you go truly public – we have managed to find and fix quite a few issues with that approach (because they accurately reflect the end customer, come with a fresh, unbiased and un-cookied perspective and also allow you to test a wide variety of browsers, browser versions etc) but their time is valuable for sure, so go thru this exercise only after you get your product quality above a certain bar. And when they report issues, somoene needs to own that relationship, respond to the finder and ensure that those reported issues are tracked and addressed.
As you go along, it is also important to keep the target date in mind and take stock of where you are. If everything is going well and things are on track, then great. But they typically won’t be, software is hand-made after all. You will have decision points at various stages to decide whether the date gets moved or whether features get cut, and what the implications of each of those decisions need to be factored in. For instance, we had a big update planned in time for Valentine’s Day because of anticipated traffic but still faced some issues with our payment module and had to postpone – it meant losing a lot of potential PR, but the right thing to do was to postpone the release. So stay on top of your list of open issues/bugs, make the right decisions and make them quickly as the need arises and prepare for the launch. Once the site is ready and certified, then the hard work truly starts especially if you are a servicing business like ours. More on that in the next post