Two months ago our customers were using our prototype “product”: an app built in Excel with their data manipulated into working mockups of our suite of data intelligence reports. This was perfect for quick, cheap iterations of the report screens but lacked scalability since we were manually loading the data & sending the files out.
Two weeks ago we had finished converting those into a webapp. We tested this version of the product by emailing screenshots of the reports but still didn’t give customers their own access because we wanted to isolate the feedback on the new webapp format without worrying about the new environment’s stability.
Last week we handed out keys. The production environment had proven solid so we removed ourselves out as an intermediate buffer and let our customers have their own access. The feedback, both along the way and now that customers can access it themselves has been fantastic. We have great customers.
Fans of agile product development and rapid iteration will note the themes:
- Do the cheapest possible test at each phase, e.g. prototyping in Excel. If the test isn’t way cheaper than building it for real then what’s the point. Lots of companies don’t get this.
- Match the test to the question e.g. testing the webapp’s format without testing its stability. Behave like a scientist and design your tests so they are isolated from factors that aren’t relevant to the question you are trying to answer.
- Iterate quickly, e.g. hourly and daily and weekly. Your learning rate is a function of the quality * frequency of your tests. There are sharp diminishing returns on quality after a certain threshold so you can instead maximize your education rate by upping the frequency of tests.