When it comes to quality, the bottom line is a satisfied customer. From a business perspective, certain organizations are tasked with setting customer expectations on the demand-side, typically marketing, sales, and support, while other organizations, such as software development and professional services, are in charge of fulfilling them. The essence of customer-driven quality comes down to ensuring that all of these teams work in unison to meet or exceed customer expectations along the entire fulfilment chain.

The quality organization should always take an end-to-end(E2E) perspective to ensure that the right things are happening along the entire chain, from product management through software development and on to support. This means proactively looking up and down the chain for potential expectation mismatches. It means engaging and influencing other organizations to proactively make adjustments to eliminate expectation conflicts. It most importantly, it means building robust E2E tests that cover all the scenarios that are critical for customer success.

Achieving customer-driven quality means focusing on the right existing customer resources and proactively seeking out new sources of information. Here are some guidelines on resources that produce a balanced customer perspective. The best results are achieved when you use a mixture of them all.

• Customer Defect Data: Look at this data if the software is already in production. Besides showing you specific bugs, this data often reveals important usage patterns previously unseen by the QA team.

• Functional Specification: Use the spec in moderation. A well written spec is always a great asset, but it can drown a team in details, causing them to overlook important usage scenarios. A poorly written spec is worse, most often because it is incomplete. The written word is seductive and QA teams often become over-dependent on it as their golden standard.
• Internal Defect Data: Internally discovered defects are useful as long as they are justified and prioritized against customer realities.

• Caution: Don’t become exclusively dependent on these sources. They provide the least amount of information about how customers actually use their software.

• Customer Proxies: A customer proxy is someone who can truly stand in the shoes of customers. Usually these people are business analysts, account managers, and inbound marketing and sales personnel. They are a vital information source especially if the software is new and not yet in production.
• Technical Support: Use tech support to find out about "defects" that are not being reported because workarounds exist.
• Customer Profiles: Look at future events at customer sites that could break the software. Usually these are due to planned expansion, technology changes, and shifts in load.

Quality organizations, engineering teams, and outsourcers are often guilty of ignoring the interests of customers by focusing on their own inward-looking quality metrics. While good internal processes are essential to the successful operation of a business, bear in mind that quality is ultimately about creating an excellent customer experience.

Case Study

Let’s look at a case where an existing QA team I acquired started off hopelessly bogged down and incapable of operating with a customer-driven perspective.

• No customer defect data existed yet because the software was still too new.
• A technical specification existed, but it was grossly inaccurate and always lagged behind the implementation.
• Culturally, the QA team was fixated on deriving all of their work from the spec. The team resisted getting started until the spec was "nailed down once and for all".
• The implementation was new and the numerous internal defects were still elementary. Over-concentration on internal bugs was a huge distraction from creating customer-focused test scenarios.
• The QA team had not cultivated any relationships with experts on the customer demand-side of the business. QA was essentially isolated within software development and at the mercy of a broken spec.

• First and foremost, we developed working relationships with customer proxies, in particular product managers and professional services consultants who really knew about the customer's needs.
• Discovered who the lead customers were and we worked with them directly to develop their end-to-end(E2E) deployment profiles. This enabled the QA team to develop effective usage scenarios.
• Paired proxies with QA professionals to develop test plans and test case matrixes based on real knowledge of what the product would need to do. Enabled the QA team to prioritize their test development.
• Established design review meetings to get developers in the room talking to the QA team about the implementation and its soft spots. Developers did walkthroughs of functionality using mockups. It turned out that both the developers and QA loved this approach. The QA team was able to build tests despite the absence of a spec. In many cases, developers tweaked their software to enable testing to happen sooner.
• Introduced a test automation platform that the team used to build repeatable tests.
• Restructured team to move the work to more assertive and technically proficient developers.

• The schedule was significantly compressed, chiefly because the tests concentrated on areas that really mattered to the intended customer base. Leaner and meaner tests were built first, and less relevant test cases were deferred to later releases.
• Demand-side thought leaders became part of the QA team and acted as customer advocates.
• Highly productive tests were created even though the spec was flawed. Scenario-based testing focused the team on finding defects that truly impacted customers.
• Tests were automated giving the business a reusable asset for releases that occurred in immediate succession after the software was rolled out.

In the end, these solutions broke the deadlock, accelerated the schedule, and produced a high quality product for our customers. Success couldn't have happened without shifting to a customer-driven approach.