How to recognize problems in outsourcing? A customer perspective

Last blogpost we discussed some of the challenges the suppliers may experience. This time some challenges will be discussed from the customer’s perspective.

In our book Quality Level Management- Managing Quality in Outsourcing we mention several major challenges customers experience in obtaining desired product quality:

  • No idea about the quality (that will be) achieved by the supplier;
  • Time-to-market is too long;
  • Maintenance or operation is not possible due to quality issues.

Unknown product quality

Many outsourcing projects start with the requirements being part of the contract. But it is not at all clear what is being expected about product quality. Sometimes acceptance tests are scheduled that are to be performed by the customer after delivery. In other cases it is stated in the contract that the supplier must perform system and integration tests. It is nice to be able to state something about quality afterwards, but what about the quality during development? Which measures have been taken to maintain a certain product quality? Is this enough to fulfil the business case requirements?

In general it is best to cooperate tight and to have short release cycles. This way the customer can check the product quality more often. See also the figure below.

Frequent releases vs big bang

The figure shows that the quality will be at a higher level when several releases are being delivered and tested during the project instead of just 1 test after a complete release (in this case indicated by release 4). However this has its drawbacks. The customer must invest in time and money by constantly being actively involved. He must be present for the project if needed. Furthermore several tests must be performed by the customer instead of just one big test.

An important (positive) side-effect of the frequent releases is that customer and supplier have to cooperate more resulting in a much better relationship and mutual understanding. For some customers as well as suppliers this is however a horrifying scenario as they want as little contact as possible with each other. At a test factory we decided to introduce witness testing. Some suppliers said: okay, this will give us defects in an earlier stage and helps us to deliver faster, i.e. a win-win situation. Other suppliers reacted: why do you want to look behind the scenes? Frequently those suppliers turned out to have a double agenda or not being organized.

An even more frequent problem is the lack of cooperation between different systems in the chain. On their own the systems work fine, but together they are not working. This can be due to performance problems, clashing interoperability issues etc.

Product to be released later and later

Having all requirements ready at the start should make life easier. The supplier knows what to make and the customer just has to check if it is okay. Alas, life is not that easy. Things change over time and the customer has new needs. This point has already been discussed from the supplier’s perspective. But also the customer must keep up pace. With changing requirements and changing priorities the product will keep changing and delivered later and later. A clear example is the Betuwe trajectory; a railway trajectory going from the port of Rotterdam to the border with Germany. Originally meant for cargo trains, the politicians decided that the trains should also be capable of passenger transport. This multiplicated the number of requirements resulting in several years of delay and a huge cost overrun. This was done after the contract and project plans had already been agreed!

Unmaintainable product

The product has been released and luckily it meets the user’s quality expectations and can be used. A problem that may turn out is that the configuration files are very difficult to maintain. Each time a configuration item has to be changed, the supplier must fly in for their corresponding fare. This is the case with a large camera application to monitor the highways in the Netherlands.

In general it is very important for a customer to describe the technical environment before contracting. The subscribers than have to show their capability to link into this environment and to handle the architectural restrictions. Therefore it is very important to write the requirements together with the architect taking care of architectural issues, but also non-functional requirements.

An example

We will illustrate the previous following the example of the previous blog, the student and personnel registration system of the police, see

The contract was written and discussed between the managers, purchasers and sales men. No technical staff was involved, at least not to discuss architectural issues, non-functionals together with all technical complexities. Furthermore the contract discussions focussed on the regulatory issues. Measures to make the cultural gap smaller and to cooperate during the project were not taken. It was them versus us.

To make things worse the quality check was at the end of the project. No quality checks were made during the project. Both sides were already making up the archives for a battle in court.

So the main problem in this contract were in the lack of cooperation. This also resulted in little involvement and immature processes (like defect management). Together with regular scope changes a perfect recipe for disaster. Mirrored against the QLM-model it shows problems on all 3 levels. There was a lack of cooperation (organizational level), immature processes (process level) and a lot of scope changes (product level).

Blog Symptoms QLM - customer - QLM-model

Using the QLM model appropriate measures could have been selected. The measures have been mapped onto the QLM-model in the figure above.

  • Involvement of architects and designers early in the project, preferably during the contract phase.
  • Agreement about clear processes between customer and supplier, like defect management and requirements management.
  • More frequent releases using iterative development, like agile methodologies. This way the quality of the product is assessed more frequently. Furthermore it is easier to make changes to incorporate changing requirements.
  • The customer could be more involved in checking in-between products using document reviews and witness testing. Also intermediate demos can be scheduled possibly even inviting (real) users to validate the product early in the process.

In this blog we spoke about customers having to deal with several challenges in obtaining desired product quality.  Next blogs we’ll look at the subject where both partners can work together in achieving quality goals.

About Eduard Hartog

Eduard Hartog has studied Computer Sciences at Haagse Hogeschool and now is a consultant at Improve Quality Services. Eduard has more than 20 years of experience in development and management in the IT industry and has specialized in quality and testing. Lately Eduard was an auditor for several tunnel projects and he has developed the ISTQB Expert level course ITTP.
This entry was posted in Quality Level Management and tagged , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s