“Off-shoring Do’s and Dont’s” : Supplier Selection & Experiences in Outsourcing

During the ImproveQS ViK (Vakmanschap in Kwaliteit) event “Do’s and Dont’s in Outsourcing”, our guest speaker Ronald Hogenboom gave an insight in his experiences with Supplier Selection processes and attention points to consider when Outsourcing. He described the Supplier selection process, the start of Outsourcing, observations made and improvement areas in outsourcing. In this blog, I’ll share a brief overview of his presentation with you.

The need:

How was the need for outsourcing identified?

As always, Outsourcing within Ronald’s company was invoked due to some needs that were identified. These were:

• Time to Market
• Flexibility to ramp up/ramp down
• Access to skilled resources
• Cost savings (long term)

Especially getting skilled resources was an issue since it often took 2-3 months to recruit an internal resource. Hiring contractors was getting more and more expensive.

Considering these factors, outsourcing was agreed upon. A Supplier Selection project team was formed and a RFP to some shortlisted suppliers was sent out. Responsibilities were identified based on the experience of individuals and the teams where the maximum need for number of resources with the lowest risk of exposing Company confidential data was recognized. The area of choice was clear — testing activities would be outsourced.

The selection process:

Why were the candidate supplier(s) selected? Why were they considered as the best match? How were the negotiations entered?

We already use scrum – why do we need Collaborative Business Ownership?

Good Scrum teams are able to realize more and more user stories per sprint. But sometimes the Product Owner can’t cope with that velocity. Collaborative Business Ownership helps you to deliver good quality user stories at a proper pace for everyone.

In a Scrum context the development team frequently delivers software that is high value. A good scrum team develops each time a higher velocity to deliver more in each iteration. In this ideal picture the business is able to create user stories for the team at that same increasing velocity. Apparently, team and business understand each other completely and responds perfectly to each other’s needs to realize valuable delivery.

In practice however, the business people often struggle in generating enough user stories of sufficient quality. Sometimes the development team accepts work not yet properly refined, thus they have to refine it themselves within the sprint. In other occasions a team refuses the work assignments as being not ready to take into development. One way or the other, it slows the velocity of delivering quality software. And the preparation work done by the business people deflates in value (if you are looking for experiences, please google ‘definition of ready’ it results in many experiences and opinions).

There is another assumption in the ideal Scrum picture described above. The development team should be familiar with the business issues the software is meant to support. But if there’s a tiny crack between their image of the business issue and the image the business people have then there is a real risk of realizing the wrong software. This is already mentioned quite long ago currently resulting in the practices ATDD and Specification by Example (see also Liz Keogh’s comments on the history of these).

In the ideal Scrum picture some people make another assumption.: the acceptance by the business people is an isolated action at the end of an iteration. But it is not. This acceptance is the result of a continuous involvement of the business people.
The Collaborative Business Ownership concept strengthens the collaboration of development team and stakeholders on taking responsible action towards the generation and realization of valuable delivery to the business by all people involved. Collaborative Business Ownership (CBO in short) addresses the quality of user stories ready on time, the knowledge transfer from stakeholders to development team and informing stakeholders continuously on the quality of the delivery. It does so, by sharing the ownership of the Business with the development team and the business people.

Within the described matter, one word must be added on the role of the Product Owner. In Scrum this role operates as the authoritative representative of the business also ensuring the knowledge needed by the team. In practice we have seen that this role is not able to convey the (tacit) context of all real customer people involved with the product. Therefore one needs to reinforce the sharing of this knowledge.

So, in addition to applying Scrum we need
• The development team to communicate satisfactorily on business knowledge with all people involved in the software;
• The business people inform themselves timely and satisfactorily on the software currently in development; and
• Just in time work assignments ready for development in the sprint.

Collaborative Business Ownership facilitates these by involving the development team in the generation of work assignments by the business people and involving the real customer people in the development work of the team.

Scrum gives you many clues on the collaboration within the development team but it only implies practices between team and business people.

Concluding: To enhance to use of Scrum, practicing CBO needs you to create options for the involvement for both team and business people. As you may have noticed there exist several practices – for several years now – that imply the CBO concept. If you already practice one of those, please share this experience in relation to CBO! In case you wonder how to enable CBO practices, please leave a note and we will share our experiences with you!

ImproveQS ViK (Vakmanschap in Kwaliteit) event: “Off-shoring: do’s and dont’s”

On September 25^th, the ImproveQS ViK (Vakmanschap in Kwaliteit) event will be held in Waalre.

On this evening, organized by the QLM workgroup of Improve QS, Ronald Hogenboom of TomTom International will share his experiences with outsourcing of testing activities. After his presentation, together with the audience, we will discuss some real-life examples and map them to the QLM model.

Presentation: Supplier Selection & Experiences in Outsourcing

Organizations today tend to jump into outsourcing testing activities with expectations of significant cost savings, reduced cycle times, and higher quality. In this presentation, Ronald will assist in preparing you to address these expectations by providing off-shoring “do’s and don’ts”. He will provide lessons learned and help you to avoid, or at least be better prepared for, the testing outsourcing challenges. Ronald will explain things to take in account when your company is considering outsourcing or already has outsourcing activities and give practical examples on how to manage the quality level in this customer-supplier relationship. The goal of any sourcing engagement is for both parties to have a mutually beneficial and long lasting arrangement. This presentation will improve your odds of making this happen.

How to recognize problems in outsourcing? A customer perspective

Last blogpost we discussed some of the challenges the suppliers may experience. This time some challenges will be discussed from the customer’s perspective.

In our book Quality Level Management- Managing Quality in Outsourcing we mention several major challenges customers experience in obtaining desired product quality:

• No idea about the quality (that will be) achieved by the supplier;
• Time-to-market is too long;
• Maintenance or operation is not possible due to quality issues.

Unknown product quality

Many outsourcing projects start with the requirements being part of the contract. But it is not at all clear what is being expected about product quality. Sometimes acceptance tests are scheduled that are to be performed by the customer after delivery. In other cases it is stated in the contract that the supplier must perform system and integration tests. It is nice to be able to state something about quality afterwards, but what about the quality during development? Which measures have been taken to maintain a certain product quality? Is this enough to fulfil the business case requirements?

In general it is best to cooperate tight and to have short release cycles. This way the customer can check the product quality more often. See also the figure below.

The figure shows that the quality will be at a higher level when several releases are being delivered and tested during the project instead of just 1 test after a complete release (in this case indicated by release 4). However this has its drawbacks. The customer must invest in time and money by constantly being actively involved. He must be present for the project if needed. Furthermore several tests must be performed by the customer instead of just one big test.

An important (positive) side-effect of the frequent releases is that customer and supplier have to cooperate more resulting in a much better relationship and mutual understanding. For some customers as well as suppliers this is however a horrifying scenario as they want as little contact as possible with each other. At a test factory we decided to introduce witness testing. Some suppliers said: okay, this will give us defects in an earlier stage and helps us to deliver faster, i.e. a win-win situation. Other suppliers reacted: why do you want to look behind the scenes? Frequently those suppliers turned out to have a double agenda or not being organized.

An even more frequent problem is the lack of cooperation between different systems in the chain. On their own the systems work fine, but together they are not working. This can be due to performance problems, clashing interoperability issues etc.

Product to be released later and later

Having all requirements ready at the start should make life easier. The supplier knows what to make and the customer just has to check if it is okay. Alas, life is not that easy. Things change over time and the customer has new needs. This point has already been discussed from the supplier’s perspective. But also the customer must keep up pace. With changing requirements and changing priorities the product will keep changing and delivered later and later. A clear example is the Betuwe trajectory; a railway trajectory going from the port of Rotterdam to the border with Germany. Originally meant for cargo trains, the politicians decided that the trains should also be capable of passenger transport. This multiplicated the number of requirements resulting in several years of delay and a huge cost overrun. This was done after the contract and project plans had already been agreed!

Unmaintainable product

The product has been released and luckily it meets the user’s quality expectations and can be used. A problem that may turn out is that the configuration files are very difficult to maintain. Each time a configuration item has to be changed, the supplier must fly in for their corresponding fare. This is the case with a large camera application to monitor the highways in the Netherlands.

In general it is very important for a customer to describe the technical environment before contracting. The subscribers than have to show their capability to link into this environment and to handle the architectural restrictions. Therefore it is very important to write the requirements together with the architect taking care of architectural issues, but also non-functional requirements.

An example

We will illustrate the previous following the example of the previous blog, the student and personnel registration system of the police, see http://nos.nl/artikel/509248-strop-dreigt-voor-politieacademie.html.

The contract was written and discussed between the managers, purchasers and sales men. No technical staff was involved, at least not to discuss architectural issues, non-functionals together with all technical complexities. Furthermore the contract discussions focussed on the regulatory issues. Measures to make the cultural gap smaller and to cooperate during the project were not taken. It was them versus us.

To make things worse the quality check was at the end of the project. No quality checks were made during the project. Both sides were already making up the archives for a battle in court.

So the main problem in this contract were in the lack of cooperation. This also resulted in little involvement and immature processes (like defect management). Together with regular scope changes a perfect recipe for disaster. Mirrored against the QLM-model it shows problems on all 3 levels. There was a lack of cooperation (organizational level), immature processes (process level) and a lot of scope changes (product level).

Using the QLM model appropriate measures could have been selected. The measures have been mapped onto the QLM-model in the figure above.

• Involvement of architects and designers early in the project, preferably during the contract phase.
• Agreement about clear processes between customer and supplier, like defect management and requirements management.
• More frequent releases using iterative development, like agile methodologies. This way the quality of the product is assessed more frequently. Furthermore it is easier to make changes to incorporate changing requirements.
• The customer could be more involved in checking in-between products using document reviews and witness testing. Also intermediate demos can be scheduled possibly even inviting (real) users to validate the product early in the process.

In this blog we spoke about customers having to deal with several challenges in obtaining desired product quality.  Next blogs we’ll look at the subject where both partners can work together in achieving quality goals.

How to recognize problems in outsourcing? A supplier perspective

The most common goals in outsourcing are improving customer (user) satisfaction, cost reduction and access to more flexible or capable human resources.  When everything goes well these goals are achieved: both customer and supplier thrive by the relation. Unfortunately many outsourcing partners struggle in achieving these goals. What are common symptoms and causes of failing to achieve these goals?

This question can be answered from three perspectives: customer, supplier and together (both customer and supplier). This blog post focusses on some of the challenges that suppliers experience. In the following posts the other perspectives will be discussed.

In our book Quality Level Management- Managing Quality in Outsourcing we mention several major challenges suppliers experience in obtaining desired product quality:

• Time and budget are fixed, but the scope keeps changing, which makes it impossible to achieve or sustain the quality desired;
• It is unclear what the customer actually wants. Requirements are unclear and ambiguous.

Scope changes

Anyone who has been part of a customer supplier relation will recognize the discussion whether an issue is an error or a change.  Having such discussions is healthy and part of the game. Too much discussion is however counterproductive. Also for the supplier! Asking more money for each scope change is often a supplier strategy to earn more money…

… but the supplier is not always to blame for not delivering in time or what’s needed. In public tenders suppliers often make unrealistic offers to get the deal. Customers primarily select suppliers by price, but mistakenly expect to get the best quality. Customers who only take and never give should not be surprised suppliers ask money for each change. Discussions and indecisiveness at customer side can frustrate the schedules, quality and moral at supplier side enormously. The 2e Maasvlakte was much cheaper than budgeted; partly because there were no long (legal) fights over scope and the suppliers could do the work effectively.

 Unclear and ambiguous requirements

Customers often do not know what they want. Part of that is human nature. Not expecting to have any new insights during a project is an illusion. For this the agile methodology or change management are common practices. Having said that, customers very often seem too immature to make this work and implement these processes well. Customers have difficulty managing their long list of stakeholders, requirements and wishes. Sometimes because the customer has little experience in managing IT projects. Sometimes because the customer organisation consists of several companies or is very hierarchical. Sometimes because the only focus is on budget and schedule, until reality kicks in during acceptance testing and quality becomes more important. And sometimes the customer has too much money to spend. In the latter suppliers are most willing to help. In the other cases Quality Level Management can help both customers and suppliers.

The first step to improvement is awareness on how both organisations work and how realistic expectations are. The second step is selecting appropriate measures according to the QLM model, see our previous post on QLM. As the causes mentioned in this blog are on several levels (product, process and organisation), measures should be taken at the appropriate level as well.

An example

The previous can be illustrated by the student and personnel registration system of the police, see http://nos.nl/artikel/509248-strop-dreigt-voor-politieacademie.html. Three years after the start of the project customer and supplier have ended up in a legal discussion. The first symptom of problems ahead was the poor quality in the system test (and probably even earlier having unclear requirements). The supplier claims this is caused by little involvement of the customer (organisational level), immature defect management (process level) and regular scope changes (product level).

Using the QLM model appropriate measures could have been selected.

• Stakeholder management, e.g. defining a stakeholder web, would have made the lack of involvement of the customer more clear.
• Agile development, in which small pieces of functionality would have been delivered in several iterations, would have decreased the effect of scope changes and problems in quality would have been detected much earlier.
• Proper defect management could have been conducted as soon as it became clear that there was an issue in analysing defects.
• Clear requirements would have made it easier to make the right product. Although this is primarily the task of the customer, the supplier could (and should) have asked critical questions.
• Witness testing in an early stage by relevant stakeholders would helped to detect poor quality much earlier.

In this blog we spoke about suppliers having to deal with several challenges in obtaining desired product quality.  Without good governance of the customer this is not an easy task, but several measures can be taken. Next blogs we’ll look at the subject from a client perspective and how both partners can work together in achieving quality goals.

Are you delivering Business Quality?

Do you deliver potentionally shippable software each sprint? What does potentionally shippable mean for you? Do you have a shared understanding about that? Do you know what your customers want? How do you know? Do you as a team ever talk to them? When are your stories done? What does done mean to you? Do you recognize struggling with tests in a sprint? Why do you struggle? Do you know the amount of product risk that goes with particular stories? What about your internal quality? Do you service technical debt regularly or even prevent it from building up? Do you perform code reviews? Why? Do you reach 80% code coverage with your unit tests? Why not 70? Or 90? Why leave 20 percent out? Do you ever say no to stories as a Product Owner? Remember 60% of features will probably never be used by your customers. Are you quality infected? What does that mean?

These questions (and their answers) help scrum teams in gaining a better collective understanding about delivering quality software that really is potentionally shippable each sprint. We will elaborate on these questions in forthcoming blog posts on Business Quality. Stay tuned…

Is your supplier playing games? Does your customer know what he wants?

Do you recognize the feeling that your supplier is playing games with you? That he’s not delivering what’s agreed upon? And perhaps you suspect that not delivering quality is a strategic choice? In the meanwhile you spend at least twice as much in performing acceptance tests? And lose even more time on discussing issues that are obviously defects?

Or are you more familiar with the case that your customer actually does not know what he wants? That every ‘stakeholder’ has a different opinion? Or worse, your stakeholder, has a different opinion every time you meet? And thinks you have all the time in the world to act upon these wishes? Simply forgetting that your profit margins are minimal, as this was the only way to get the order.

If you recognize either situation and could use some help, Quality Level Management, QLM for short, is the answer for you.

In this first blog I will answer four basic questions on Quality Level Management (QLM):

• Why QLM?
• What is QLM?
• How does QLM work?
• Why is QLM so useful?

In the following posts I’ll dive into concrete issues both customers and supplier have to deal with. If customers and suppliers recognize problems early, it becomes easy to implement appropriate measures early.

Why Quality Level Management?

To manage and control quality is difficult to achieve within one party. Problems seem to escalate in case more than one party is involved. Physical distance, cultural differences and difference in vision and strategy lead to misinterpretation of requirements, too little involvement of the business and poor quality processes. All are known causes for poor product quality. Quality level management helps to overcome these problems.

What is Quality Level Management?

Quality Level Management (QLM) is managing quality in the case of outsourcing of IT-related products and services.

Managing Quality:

There is always a balance (or struggle) between time, budget, scope and quality as depicted in the iron triangle.

The time, budget and scope are quantifiable and often much easier to define and measure than quality. Therefore time and money are primarily used to select suppliers and to control projects. How to manage and control quality when it is so hard to pinpoint what the desired quality is?

 

Outsourcing:

In Quality Level Management the term outsourcing is used to denote the fact that several parties are involved: at least a customer and a supplier. This can be a supplier abroad, but also another department in the same company. When using the term outsourcing in case of QLM this also refers to:

• Off-shoring;
• Near shoring;
• Joint venture;
• Fully owned;
• Turnkey;
• Indirect;
• Right sourcing;
• Multi sourcing;
• Best shoring;
• Back shoring.

In all cases the difficulty is that different parties are involved. Different parties may have different goals. The basic notion of us-versus-them causes parties to behave in their own interest instead of working together as partners striving for a win-win.

IT-related products and services:

Quality Level Management focuses on IT-related products and services. This can range from a bank application, to a tunnel, to a medical device to a printer. Although QLM focusses on IT many measures that can be taken as part of QLM are also applicable to other domains.

How does Quality Level Management work?

Quality Level Management first looks at the context of the outsourcing relationship.

• Type of products outsourced;
• Type of services outsourced (design, development, testing, maintenance or a combination);
• Type of organisations;
• Type of contracts;
• The (cultural) differences between customer and supplier;
• The desired level of quality.

Then it identifies problems at three levels: product, process and organisation. Based on the context and the problems at hand appropriate measures are selected. For this the QLM-model is used. The QLM-model helps to determine a balanced outsourcing strategy that focuses on meeting goals and expectations in terms of time, budget and last but not least in terms of quality.

The model for Quality Level Management has two main dimensions:

• Levels at which measures can be taken: the organisation, processes and products;
• Types of measures: preventive, detective and corrective.

A balanced strategy uses a selection of quality measures out of the nine areas of attention in the QLM-model.

Why is Quality Level Management so useful?

Most existing models focus only on organisational and process aspects. The QLM-model focuses on measures to be taken at all levels; including the product level. As problems at product level often are caused by problems at organisational level, this complete focus helps both customers and suppliers to pinpoint the actual problem and select appropriate measures.

The QLM model can be applied at any point in time. At the start of an outsourcing relation most focus will be on preventive measures and defining detective measures. As the relation prolongs the focus shifts to corrective measures. Being able and willing to make necessary changes is crucial for obtaining a fruitful relationship and to obtain the desired level of quality.

Next posts I will discuss some problems customers and suppliers have in managing quality in outsourcing of IT-related products.

Jeanne Hofmans