By Cheng Zou, CTO & Co-Founder
At one time, developing applications on a subscription business model used to be incredibly difficult. Seven years ago, I was at WebEx managing a team of 30 developers building our billing system. Despite this big team, we were never able to keep up with quickly evolving market requirements.
It was a hot time for web conferencing and competition was fierce. We were forced to respond quickly to threats but speed was a challenge. For example, it took us 11 months of development work to support a free trial model in our prosumer market and we were too late to have an impact.
We were also slow to deploy routine things like pricing configuration change. It took four teams coordinating across two releases and multiple system syncs to implement a simple price change. We also were hit with scalability limits in our rating system and we could not handle the exponential volume growth of teleconference usage data. As a result, it took close to 24 hours to process a single day’s usage.
For billing & payments at WebEx, we built a dedicated operations team focused on the tedious task of processing production usage data, producing invoices and processing payment files through the CyberSource gateway (via our own integration that we built). At one point, this team grew to 7-8 people and was still barely able to keep up with all these manual operations tasks to satisfy our internal needs.
Later on we added customer account management (yet another isolated application) to the puzzle and had to sort out the myriad of issues and integration challenges. FInally, we struggled in producing business intelligence reports for our product management team and sales to upsell our most promising customers and to prevent others from churning.
Fast-forward to today, where subscription businesses have exploded in almost every industry and these once heavy-lifting operational requirements are shared by all. This opportunity is what inspired me to co-found Zuora. Now, as a company enabling over 600 customers to be successful in the Subscription Economy, we have learned to win ourselves by developing a monthly innovation cycle that delivers increasing value to our cusotmers. This has not been easy given the diverse requirements across a diverse set of customer industries, sizes and business models.
What are the keys to our development success? Our development systems and processes have to be comprehensive, adaptive and agile to deal with these challenges. But most importantly, we have to be really responsive to customers. Many of our most popular features came directly from customer feature requests.
Just a few of my favorites (hard to choose from 50 releases):
- We added support for tier 1 payment gateways Paypal and Cybersource because our customers wanted Zuora to handle the downstream business operation of payment collections in an integrated way.
- Our Saleforce integration, which initially solved the need for mid-market customers who wanted to sync their back office data from Zuora back to Salesforce so they could have a complete view of their business. We then built the Z-Force product that is built natively on force.com with a 360 sync capability.
- REST APIs. We responded to customer requests for an API that is use case based (to complement our SOAP APIs) to help them easily integrate their web service front-end to their Zuora back office. We produced 22 REST based APIs that addresses the need end to end.
Responding to customer requests quickly is the benefit or outcome of our ever improving, and now highly predictable and reliable development process. Through this monthly cycle of continuous innovation, we have built & improved our engineering delivery model on multiple fronts.
Here are some highlights worth sharing:
- First, we have become agile. We have to align our project management practice and release operation seamlessly to enable us to release every month (and in on-demand fashion if there is an urgent fix to deploy). This involves managing multiple release projects in parallel and cohesive coordination across product management, engineering, QA, documentation, tech operations, and customer care.
- Second, we have become truly automated. We have to set up a continuous integration environment for our build and testing operations so that we can run through 50,000+ test cases every few hours and ensure there is no regression introduced in the check-in code. Have I mentioned that deployment and migration script verification is also a part of this automation?
- Third, we have become operation aware. All project team members must understand the production and non-functional impacts a new feature or a bug fix will have and they must address those concerns and risks in their design and implementation, while our QA team must define adequate test plans too. In addition, to support these non-functional requirements, we developed performance lab and various API sandbox and staging environments to ensure the quality of our code in production like environment.
- Last but not least, we have become quality minded. We adopted the MECE framework in our testing design and made significant investment in developing test automation infrastructure. Now we have more than 50,000+ automated test cases covering UI, API, batch operations and scenario tests. Every code check-in requires passing of baseline test and full regression test runs twice every day. When dealing with customer reported bugs and escalations, we have a well-defined process to identify root cause, review solutions and add test cases to ensure we don’t make same mistake twice.
I’m incredibly proud of what we have accomplished in 5 short years at Zuora, impacting not one customer but close to 600 customers around the world. Today’s news is a wonderful milestone and is a product of the ingenuity, passion and creativity of our world-class team. I want to thank many employees of Zuora around the world, each of whom have put in countless hours to deliver on our commitment to customers.