There are some key factors which will determine how useful a project evaluation will be:
- Performance should be evaluated at all stages of the consulting engagement
- The performance of the contracting organisation as well as the consulting partner should be evaluated
- There should be a standard list of questions used to evaluate performance
- The learning must be subsequently applied to all other areas of the buying process, especially when considering whether to use a consulting firm for additional projects
Before you introduce an evaluation process it is important to clarify the different roles required, the purpose of the evaluation itself and the method of implementation.
Roles in the Process
- Are you clear who the stakeholders are? Clearly the internal client but how about the finance function, the procurement team – possibly the consultant supplier too?
- The role of stakeholders. To what extent will the stakeholders (all or only a selection) be involved in the evaluation process? How involved should they be in order to get good insight? How do you balance this against their time commitments and availability?
- The role of procurement. Did they have a role in the project themselves and is this to be part of the evaluation process?
The purpose of the evaluation?
- to evaluate only
- to improve
- to find out what went wrong
- to increase competition
- to drive down price next time
- to improve specifying/commissioning
- to improve selection; to make the buying organisation a “smarter client”, etc
How will the results be communicated and to whom?
- The culture of the organisation
- How does the evaluation process fit with other lessons learned processes, e.g. continuous improvement
- The nature of the project/assignment
- The process needs to ensure proportionality with impact, risk, strategic importance – or was the project a BAU or operational project
- The nature of the supplier
- Were they a one off supplier, a new supplier, a regular/frequent supplier? How should a co-supplier or a sub-contractor be evaluated?
- The type of information required
- Will it be just quantitative or will it be qualitative too?
- The method of gathering information
- Should it be survey, interviews and/or workshops? Who should be included? Should it be just internal colleagues or external providers too?
- The timescale of the process
- Is it appropriate to schedule the evaluation for some months after project completion in some cases, so the results can be analysed and understood?
- The publication of the results
- How to ensure they are received and used by appropriate stakeholders to enhance future sourcing and management decisions; and also as a source of data for wider consultancy firm/client review meetings.
The Core Questions
Good practice would be for each consulting project to be evaluated against the following 7 dimensions:
- Was the project delivered to time?
- Did the outcome(s) of the project represent value for money?
- Were the outcome(s) of the project delivered to quality?
- Were the anticipated benefits of the project delivered?
- Was the necessary level of knowledge transferred?
- Would you consider using this consulting firm again?
- Would you recommend this consulting firm to a colleague?
You can download these core questions here.
A Deeper Dive
In some cases a much deeper level of evaluation would be beneficial to identify those learning points that can maximise the value of any related future expenditure.
Here are a range of supplementary questions that can be adapted and applied to the evaluation process depending upon the level of learning required. These questions cover five different stages of a project and within each stage there are two sets of questions:
- One set for the internal client to answer
- One set for the consultant supplier to answer
For both the core and supplementary questions the following rating scale is recommended.
Strongly Disagree/ Disagree/ Slightly Disagree/ Neutral/ Slightly Agree/ Agree/ Strongly Agree / Not Applicable