Missed time to market window! Really?

We know project management is all about the juggling the three balls of time, cost and quality. A project is successfull if it meets the functional and non-functional requirements within predetermined time, cost and quality constraints.

The traditional project management approach (and hence 99% of the tools) focus on completing the defined work within given time constraints and cost limits. However, the recent focus has been shifting more to the quality of the final output!!

Let's look at some examples:

• Google. Didn't google missed the time to market long before it released its search engine?
• Apple iPOD. Had it made a difference if iPOD was delayed another 6 months?
• Toyota Prius in 2000. Missed the TTM by three years! (Audi released its first hybrid in 1997 and Honda released its hybrid in 1999)

More and more companies are realising the fact that quality rocks!! If your product is high quality, it doesn't really matter if you are a year or two late to the market. Every product has its life, but if it is of high quality it tends to live longer - which changes the whole Net Present Value (NPV) calculation, in case you are using it to calculate the validity of your projects and releases.

Project requirements can be divided into functional and non-functional buckets. Functional requirements are the core (and supplementry) features of your product. Non-functional requirements are the systemic qualities, which encapsulates all "illities" - Availability, Scalability, Reliability, Flexibility, Extensibility, Interoperability, Compatibility, Testability, Understandability, Load and Performance, Stability, Resiliency, Manageability, Mantainability, Security, Supportability, Adaptability, Configurability and Usability! Note: Not all illities are applicable to all product offerings.

Your product may have over thousand functionalities, but just pick a handful of core ones (maybe 3 to 5) and all of the non-funtional requirements for your first release. A high quality product markets itself: word-of-mouth is the most effective marketing tool. Once a customer is hooked-in, slowly roll-out new features. That way you'll have the relationship going and you can get a continuous inflow of money - easy from SEC's perspective and no hassle of accounting manipulations either! That's what is driving software as a service (SaaS) market today.

SOA is the SaaS enabler and it is changing the way software is released. SOA brings business agility. However, our project management tools are still old-fashioned. Project managers are still focused on TTM and CTM concepts. They are still chasing deadlines and pennies. Quality awareness is forcing ALM companies to come up with more sophesticated tools that stitches the SOA fabric.

For innovation and quality, you are never late to the market!

It all boils down to Metrics!!

Setting up a goal is one thing, but how do we know that we have achieved our goal? Software engineering is becoming more of an art than science. Success is a relative term! A project manager with exceptional artistic and articulative skills can sell a project, which is on road to failure, as a successful investment to the executives. In absence of real numbers, the darkness prevails. And under this darkness, all decisions lead to the path of failure.

Snapshot:

"We have a GA date approaching. PPM calls a Projet-Team meeting and takes a vote of confidence, which decides the fate of the software!!"

"A P1 bug is not a show-stopper if it already exists in production. The release will not degrade the production quality!!"

"QA gives a conditional GO with list of risks. By the time decision propogates to executives, the attachment is dropped and the Conditional-Go turns into a Sure-shot-Go!!"

"PM: The problem is not in our piece of the code. The issue is because of the other component that we are dependent on!!"

Sounds Familiar? Interesting, isn't it?

Let's face it, we need sophisticated tools that can generate real time metrics for anyone to make informative decisions. People often mix product quality with process quality. Even though a high quality process generates a high qualiy product (TQM principle), I believe the metrics for the two should be different. For example, higher percentage of test automation improves QA process quality and doesn't directly improve underlying product quality! Note: the automation of processes in the early ALM cycle would have a more direct impact on the product quality.

Here are the list of questions, that metrics should be able to answer:

• Q1. What is the overall Quality Index (QI) of the product. QI for a particular feature or requirements? What is the QI of different components?
  
• Consistency of the processes and measures is the key here.
• It is easy to fabricate a QI model that concentrates on intrinsic product quality!
• Q2. What's our Release Readiness? What's the risk if we release our product today?
  
• Q3. What's the QI trend for differenet releases and different builds?
  
• Trend is more important than actual QI snapshot.
• Errors in the QI (if any) cancel out when you read trends
  
• Q4. What's the Dependency Matrix (DM)? How does other SOA components impact my product? How does my product impact other offerings in the organization?
  
• Current snapshot from QI perspective
  
• Roadmap overlap for future releases. Cross-project Backlog Management.
  
• Q5. What's the realtime Coverage graphs?
• Test Coverage (test validating requirements)
  
• Code Coverage (tests validating code)
• Q6. Testing Strategy Automation.
  
• When files A, B and C change, which features get impacted. What test cases and configurations should a test lead plan for next build?
• Q7. Process Quality. How productive is my team?
• Measures of test automation.
  
• Comparisons with baseline (and manual testing)
  

With above metrics in hand, I can easily make statements like:

• We are ready for the release!! Our product QI crossed 85% in the last build.
• Because of TTM (time to market) pressures, we have decided to release our product with 65% QI. To mitigate the risk, we have also decided to increase our customer support resources.
  
• Our product is not ready for GA because we have a dependency on products B, C and D, and product B has a QI of only 30%. Since B is tightly coupled with our core, we are not in a position to release our product.
• I can effecively utilized my QA resources to concentrate on only the impacted features in a build. We don't have to regress every build every time. We can validate a build with handfull of fixes in less than two hours, and that too with over 95% confidence!!
• We can now sell SLAs and QLAs around certan metrics because we have a consistent (and automated) way of capturing them.
• I can trace a customer escalation all the way back to requirements, because we have end-to-end integration of ALM tools with excellent search facilities.
  

Follow on: Development vs. QA - Why disagree?

I like the bold questions raised by jason in his blog "Development vs. QA - Why disagree?". For the last half decade (especially after dot-bust), quality has gained an overwhelming visibility in the software industry and the awareness is growing day-by-day. Numberous studies have proven the exponential relationship between the life of bug and associated cost. Sustaining costs are increasing far beyond original development costs. Therefore, companies are trying to crush new bugs, as soon as they find their way into the code. And hence, the pressure is on developers to test their own code!

I don't see the testing goals between development and QA as conflicting. I see the conflict more because of differences in role, availability of test beds, and more importantly the will! Developers generally don't want to do testing - they always write the perfect code!

The QA is way too on the other side of the wall. 99% of QA teams are involved in black-box testing of features as customer sees them. So, Quality organization is always more close to the end-customer as compared to the development.

To me - both practices are inefficient. We all know that by testing in the end, QA cannot build quality in, it's the development team that needs to write a quality code to start with. I am a firm believer of TQM principles and Deming 14 points. To improve quality, all processes must be standardized, engineering principles must be put in place, and there should be tools that ease the adoption of all these processes. Processes without tools create too much work and chaos!.

The solution is to have a QE team, a team that is more close to development (report to the same director, or even the same manager!) , responsible for all the functional testing. QE team can catch bugs early in development cycle and QA team can focus only on non-functional requirements as part of the system testing.

Developers don't want to be QA!! They restrict themselves to Unit testing and some basic functional unit testing. Another complecated issue is the deployment that is generally not automated. Lack of automated deployment breaks down the Continuous Integration cycle and developers' motivation to automate post deployment test scenarios. Using tools like MockStrutsTestCase and CactusStrutsTestCase, developers have started to look into some level of pre-deployment functional unit testing (including in-container testing) - but again, that's streatching the limits of Unit testing, as jason said.

QA is black box. But there is a limit to what QA can test - with limited resources, especially time. Once feature freeze is done and all the code is checked-in, nobody wants to give QA couple of months just to make sure they can complete the test cycle. The complicated nature of software, with all the reuseable code and interdependencies, fixing one bug late in the game has a huge potential to give birth to two or more bugs (it's like the Samuel monster from Helloboy!)

Watch Watts S. Humphrey's video for more info on why testing in the end is a bad bad thing to do.

Trackback URL: Follow on: Development vs. QA - Why disagree?