By Max Langosco, Project Management Expert
So how do you measure the value of training? Ideally of course we would like to see the impact on the business - the return on investment in training.
In practice that is challenging: imagine investing €20,000 in a training session for a class full of PMs who run tens or hundreds of millions worth in projects. Even if only one in every three of them saves a one-digit percentage point off their budget in avoided or well managed risks, the ROI is huge… but hard to measure. How can we know that the success was actually due to the class? Could it have happened anyway? Maybe it is due to new tools which are being used, or simply thanks to a better choice of PM?
How does one measure the impact of a successfully managed risk anyway? Since we don’t have the alternative event to compare against?
A commonly talked about (and less commonly used) approach is the Kirkpatrick model. The Kirkpatrick model defines four levels of evaluation – Reaction, Learning, Application and Impact, with level 1 evaluation commonly referred to within the training industry as Happy Sheet.
However, there is a different approach that we may choose to take. Training is a support activity to change. If there is no change expected from the participants, then there is no need to send them to a training session. So an important question to ask, when evaluating the success of investment in training, is: “Has the mindset shifted?”
Training is a support activity to change.
Following up on mindset shifts is crucial in the development of social and political movements, so I was curious of what they think about it at the Monitor Institute (part of Deloitte Consulting). They suggest a set of “metrics for every aspect of movement-building.” Then, they create categories/areas (community organizing, engagement, leadership, alliance building, campaigns, communications and framing, media…). “In each area they suggest telling a story of progress using two types of metrics: transactional and transformational. They define them as follows:
Transactions involve the quantifiable markers both internal (e.g., how much funding, how many members, etc.) and external to the organization (e.g., voter turnout, policies passed, etc). While the data is not always easy to collect (especially with transient or mobile groups), such measures tend to be easier to track because they are more tangible. But transactions only tell part of the story and tend to skip over the richness of experience and momentum that can be precursors to big change.
Transformations, on the other hand, are the vital but sometimes “invisible” work. They show how people, organizations, and movements have been altered through the collective efforts. Taking the transformation further, they can show how societal and political views have shifted or been impacted by movement building.”
So if we apply this logic to the mindset shift which we are fostering within a corporation, we could redefine the transactional and the transformational criteria (T&T), and see if we can use them to find out if training is actually enabling the expected mindset shift. Let’s give it a try:
Assume we want to shift the corporate mindset from the existing ad-hoc project management approach to a structured PMI style approach, and that we have set up a program of classes through which our PMs are all going.
Step 1: Create categories/areas:
These could be: management support, engagement… (Or we could even use the Kirkpatrick model: Reaction, Learning, Application and Impact).
Step 2: Tell a story in each area using T&T:
• Transactional:Management Support: trend of approved funding for classes, certifications, PMI membership, and PM community building activities.
• Engagement: number of participants in class, number of PMP certified project managers…
• Transformational:What has changed?
• Are PMs behaving differently?
• Which benchmarks are being used?
• Are best practices shared?
However we look at it, it is not going to be easy.
There is a trap in which we risk falling. “Happy Sheets” or feedback forms tend to be as easy to retrieve, track, and draw nice little graphs from, as they are of limited use. Limited to the “customer satisfaction” aspects, but telling us nothing about the transformative impact of the training efforts, and certainly nothing about the ROI.
Any thoughts? Ideas?
* Max Langosco is a respected expert in project management and has trained over 1,500 project managers in Europe, Middle East and Africa and has supported the implementation of Project Management Offices (PMO), and project management best practices, in corporations from the telecom, energy, IT, insurance, and banking industries.
Max has spoken at several public and corporate Project Management venues, offers a range of project management and risk management consulting and training services and his practices integrate project risk with business risk, extending from the traditional managing of single risks, to a more holistic approach to project management by building corporate and project “systems” which are able to benefit from uncertainty.
This article was originally published at http://crowrider.com/blog/