Project Report Requirement
Objectives
This study is aimed at testing the feasibility of a Return on Investment (ROI) model within the organization so that the success of a training programme or a set of training programmes can be measured more accurately in monetary terms, thus enabling the management to understand the direct impact on the business bottom-line as a causal effect of the training imparted.
Training
and development
Individual
Assignment: The objective of the assignment is gaining hands-on-learning
experience on training evaluation from reaction assessment to return on
investment. You have to choose a training program conducted by any of the
organizations and evaluate the programme.
Assignment Template
Section 1: Introduction
1. Name of the
organization ( if it is confidential, a fictitious name may be given)
2. Nature of business
3. Ownership
4. Geographical
orientation
5. Headcount in the
location where the study was conducted
6. L&D department
structure and personnel
7. The L&D policy (briefly)
(This section should not more than 400 words)
Section 2: The Program
1. Need and relevance
of the program for the organization and the participants
2. Objectives of the
program
3. Program design
(including content, methodology, duration, resource persons etc.)
4. Participants (number
of participants , why this program for these participants)
5. Program delivery
(when and where it was conducted, who conducted &how was it conducted (in a
block or in intermittent phases)
6. Why ROI assessment ( the reasons for picking up this
program for ROI assessment)
(This section should not be more than 700 words)
Section 3: Reaction and Action Plans
1. The feedback form
used
2. When the feedback
data collected
3. How was it collected
( online or offline)
4. Who collected the
data
5. The scores on each
item and the overall average (present in a tabular form)
6. Analysis of
subjective questions, if any, in the feedback form
7. Your views on
overall feedback
8. Did the participants
create Action plans at the end of the program
9. If ‘ yes’ mention the essence of the action plans and show
one action plan which was created by of the participants
(This section should not be more than 700 words)
Section 4: Learning Evaluation
1. The learning
assessment test/s used (detailed description of the test/s, written and
practical)
2. When the learning
data collected
3. How was it collected
( online or offline)
4. Who collected the
data
5. What were the score
of each participant
6. How the decision of pass or fail taken
(This section should not be more than 1000 words)
Section 5: Job Application
1. How did you collect data of applying the skills (the
method/s used)
2. When this data was
collected
3. Who gave you the
data
4. How was it collected
5. Who collected the
data
6. Detailed
presentation of application of knowledge and skills learning on the job (
participant-wise)
7. What were the barriers and enablers for participants had
in applying the knowledge and skills
(This section should not be more than 1000 words)
Section 6: Business Results
1. What are the
performance items on which you targeted to derive business results
8. How did you collect
business results (the method/s used)
2. Who gave the data
3. Who collected the
data
4. When this data was
collected
5. Detailed presentation of business results (
participant-wise)
(This section should not be more than 1200 words)
Section 7: ROI Analysis
7.1 Isolating the
effects of training
1. What method you
followed to isolate the effects of training
2. Detailed description
of the method
7.2. Converting business results into Monterey values
1. What Business results you converted into Monterey benefit
3. What methods you
used to convert the business results monetary values
4. How monetary values
were assigned
5. Who assigned
Monterey values
6. Detailed
presentation of business results and their Monterey values (participant-wise)
7.3. Tabulating Program Costs
1. What are the costs
you considered
2. Present the cost table with figures
7.4. ROI
1. What was the ROI
generated (present the calculations)
2. Your views on the ROI generated
(This section should not be more than 1200 words)
Section 7: Recommendations
1. What are the
barriers and enablers you had in the entire process of evaluation
2. If you have to do
ROI assessment next time what areas to be revised/more concentrated
3. Your recommendations
(This
section should not be more than 500 words)
Introduction
Return on investment (ROI) has become one of the
most challenging and intriguing issues facing the human resources development
(HRD) and performance improvement fields. With ROI, decision makers evaluate
investments by comparing the magnitude and timing of expected gains to the
magnitude and timing of investment costs. A good ROI means that investment
returns compare favorably to investment costs. This study is aimed at testing
the feasibility of a Return on Investment (ROI) model in the context of a
training programme in Bharat oil, a government organization, to see whether the
success of a training programme can be measured more accurately in monetary
terms. It is meant to enable the management to understand the direct impact on
the business bottom line as a causal effect of the training imparted and to
decide whether to continue or discontinue the programmes. Bharat Oil conducted
in-house training programmes in 2004 which included a blend of behavioral and
functional training. The programmes conducted at this organization were
evaluated using the most comprehensive evaluation models and discussed in terms
of return on investment. The two programmes chosen for the study were: -The
Threshold Programme and the Excellence in Project Management, each having 30
participants.
Bharat Oil conducts on an average approximately
2,000 training programmes per year which are conducted in-house, leveraging
expert faculty from renowned institutes/organizations and well-trained business
leaders from within the company. The training plan contains a variety of
programmes which meet intrinsic needs of different functions. Some of the
programmes are made mandatory at the corporate level such as ‘Vigilance
Awareness Programme,’ ‘Communication & Presentation Skills,’ ‘Gender
Sensitivity Programme,’ and ‘Security Sensitization’. However, majority of the
Divisions carry out additional programmes based on ‘Here and Now Needs’ of the
customer divisions such as ‘Safety and Hazardous Operations,’ ‘Safety
Integrated Learning’, ‘Hydro treatment Processes for Middle Distillates,’ and
‘Treasury Management.’
Training Strategy
· Align
and integrate learning with the company’s business.
· Identify
skill and competency gaps at all levels and create groups for delivering
in-house trainings.
· Identify
courses outside IOC/India and depute key employees handling new technology,
practices or new business to attend these courses.
· Set
up a world-class institute for creating critical mass of highly trained middle
and senior level executives for occupying top level positions to the
organization and the industry/nation.
· Set
up a learning centre for training of marketing executives on
customer-orientation, marketing of products and other marketing specific
programmes.
· Organize
cutting-edge training on quality, cost, attitude, and customer service and
communication network.
· Conduct
brain-storming sessions with the key executives, senior and middle management
groups, un-ions and working levels, for deciding strategies for the changed
scenario and involvement of employees.
· Organize
regular interaction with top management for deliberations on specific issues in
an organized way.
· Impart
training to senior top managers in advance technologies and latest managerial
tools and techniques for maintaining cutting edge of the organization and
developing leaders with all-round qualities.
· Have
multi-pronged communication about the imminent competition across the
organization on a regular basis and Enterprise Resource Planning So-lutions and
related training.
· Focus
on quality, ISO Certification/accreditation, adoption of Business Excellence
Model by strategic business units and benchmarking with national/international
training providers.
· Train
critical position holders extensively in India and abroad to handle business ambiguities
and complexities and gear up for the new business outside India.
· Develop
in-house Petroleum based international MBA programme with in-built foreign
module to create a global mindset and entrepreneurial qualities among
executives.
· Depute
sizable number of executives for MBA courses in other Institutes of India for
creating a mix of executives with general MBA and petroleum-based MBA.
· Conduct
Management Development Training Programmes in niche business schools in India
and abroad for understanding global business trends.
· Make
a paradigm shift from ‘Supply and Distribution’ to ‘Marketing Strategies and
CRM’
· Train
Executives for business integration, diversification, and entering into joint
ventures and handling newly acquired companies.
Training Vision
· Be
a learning organization, and inculcate skills and competencies required for
becoming a transnational, integrated energy company and for value creation for
the stakeholders.
Training Mission
· Develop
employees by enhancing their knowledge, skills, and competencies in functional
areas – general, strategic management, and advanced functional management, in
line with the business of the company.
· Provide
post-experience management education for renewal of executives to global
standards.
· Achieve
and sustain customer respect through world-class products and services.
· Sustain
learning environment through creativity, competence, and recognition of
contribution.
Research Methodology
Objectives
This study is aimed at testing the feasibility of a
Return on Investment (ROI) model within the organization so that the success of
a training programme or a set of training programmes can be measured more
accurately in monetary terms, thus enabling the management to understand the
direct impact on the business bottom-line as a causal effect of the training
imparted.
Variables
Independent Variable:
The effects of
behavioral and functional training imparted to the employees have been coined
as independent variable. Bharat Oil used training as a major tool to change
the attitude sets and skill levels of its employees. It conducted 2,434
in-house training programmes which included a blend of behavioral and
functional training. The training was intended to develop employees by enhancing
their knowledge, skills, and competencies in functional areas/general/strategic
management and advanced functional management, in line with the business of the
company.
Dependent Variable:
Change in the attitude sets and of the employees and
direct impact on the business bottom-line have been considered as dependent
variable in the present study. It includes change in knowledge, skills, and
competencies enhancement in functional areas as well as change in attitude and
the behavioral aspects of the employees.
Programme/Sample
The two programmes chosen for the study were:
1. Threshold Programme (30 Participants)
2. Excellence in Project Management Programme (30 Participants)
Threshold Programme
This particular programme was chosen for the purpose
of ROI calculation since this is one of Indian Oil’s most elite programmes. The
programme has been designed to enhance the general management skills of senior
man-agers of the company and to help them acquire a comprehensive strategic
perspective so that they are prepared to lead the organization with confidence
to meet the emerging challenges. This is the programme where the best of the
best are trained to enable them to cross over from middle management into the
rungs of top management.
Excellence in Project Management Programme
Due to the increased investment envisaged by the
corporation to meet the current challenges in technological, environmental, and
customer expectations in terms of quality in delivery, the Excellence in
Project Management Programme becomes one of the most vital ingredients in the recipe
for success.
Collecting Post-Programme Data
The questionnaires for “Threshold Programme
Evaluation” and “Excellence in
Project Management Programme” have been implemented to ascertain the degree of
success in meeting the objectives of the training programme at Bharat Oil.
Converting Data into Monetary Benefits
Several methods such as calculating the cost of
quality, using historical costs, and internal and external experts’ inputs and
converting employee time could have been used but due to lack of time and data,
the method chosen for this study was to use the estimates from participants. In some situations,
programme participants are capable of estimating the value of a soft data
improvement. This method seems appropriate when participants are capable of
providing estimates of the cost (or value) of the unit of measure improved by
applying the skills learned in the programme. The advantage of this approach is
that the individuals closest to the improvements are also often the ones most
capable of providing the most reliable estimates of its values.
Why Measure ROI of a Training Programme
Several issues are driving the increased interest
in, and application of, the ROI process, the most common being:
· the
pressure from clients and senior managers which show that the return on their
training investment is probably the most influential drive
· the
competitive economic pressures that are causing intense scrutiny of all
expenditures
· the
general trend towards accountability with all staff support groups that is
causing some HRD departments to measure their contribution
· to
justify the existence of the training department by showing how it contributes
to the organization’s objectives and goals
· to
decide whether to continue or discontinue the training programmes.
Benefits of ROI
Measures
Contribution – ROI makes it possible for the HRD staff to know the specific
contribution from a select
number of programmes. It can determine if the benefits of the programme,
expressed in monetary terms, have outweighed the costs and thus whether it has
made a contribution and is actually a good investment or not.
Sets
Priorities –
By calculating ROIs in different areas, one
can determine which programmes contribute the most to the organization,
allowing priorities to be established for high-impact training.
Focuses
on Results – Measurement
of ROI is a result-based process which brings a focus on results with all
programmes. The process requires instructional design-ers, facilitators,
participants, and support groups to concentrate on measurable objectives – what
the programme is attempting to accomplish. Thus, the process has the added
benefit of improving the effectiveness of all the training programmes.
Alters
Management Perceptions of Training – The ROI process,
when applied consistently and comprehensively, can convince the management
group that training is an investment and not an expense. Managers will see
training as making a viable contribution to their objectives, thus increasing
the respect for the function. This is an important step in building partnership
with management.
Five Levels of ROI Evaluation
The evaluation levels categorize data, reporting a
chain of impact as reaction leads to learning, to application, to impact, and
to return on investment.
|
|
ROI
Evaluation Level
|
Measurement
Focus
|
|
1.
|
Reaction
& Planned Action
|
Measures
participant satisfaction with the programme and captures planned actions
|
|
2.
|
Learning
|
Measures
changes in knowledge, skills, and attitudes
|
|
3.
|
Application
and Implementation
|
Measures
changes in on-the-job behavior and progress with application
|
|
4.
|
Business
Impact
|
Captures
changes in business impact measures
|
|
5.
|
Return
on Investment
|
Compares
programme monetary benefits to the programme costs.
|
Estimating Programme Costs
Most of the data relating to programme costs was
avail-able from the Project Completion Reports of the programmes. The only
additional cost which has been included is the cost of participants’ time which had not been previously taken into
account. This element has been considered while trying to estimate programme
costs because this represents the time that the participants have spent
attending the programme which they would have otherwise spent at the workplace.
The CTC (cost to company) of the participants represent an expense that should
be included. For situations where the programme has been conducted, these costs
can be estimated using average or midpoint values for salaries in typical job
classifications (Tables 1 and 2).
Table1: Costs Considered in the Excellence in Project Management Programme at Indian Oil
|
No.
|
Cost
Element
|
Consolidated
Amount (Rs.)
|
|
1
|
Faculty
honorarium
|
19,200
|
|
2
|
Stay
@ Rs. 1,500 per day/participant
|
1,68,000
|
|
3
|
Catering
@ Rs. 125 per day/participant
|
14,875
|
|
4
|
Photocopying,
stationary and brochure
|
27,500
|
|
5
|
Air
and local travel
|
34,755
|
|
6
|
Photography
and books
|
3,850
|
|
7
|
Cost
of participants’ time @ Rs. 8,000
|
2,40,000
|
|
|
per
participant (assumed average CTC)
|
|
|
|
Total
|
5,08,180
|
Table2: Costs Considered in the Threshold Programme at Indian Oil
|
No.
|
Cost
Element
|
Amount
(Rs.)
|
Consolidated
Amount (Rs.)
|
|
1
|
Faculty
Honorarium:
|
|
|
|
|
Vedanta
|
7,500
|
|
|
|
Thomas
Intl.
|
29,160
|
|
|
|
APTECH
|
12,900
|
|
|
|
Comp.
Update
|
20,000
|
|
|
|
IIM-A
|
11,98,800
|
12,68,360
|
|
2
|
Stay
@ Rs. 1,500/- per day/participant for 22 SMs/CMs and @
|
8,01,600
|
|
|
Rs.
2,400/- for 8 DGMs for 28 days
|
|
|
3
|
Catering
@ Rs. 125/- per day/participant for 30 persons for 28 days
|
1,05,000
|
|
4
|
Photocopying
|
43,000
|
|
5
|
Stationary
incl. Ties and Bags
|
18,220
|
|
6
|
Air
Travel and Local Travel (If borne by IIPM)
|
5,000
|
|
7
|
Others:
(Foreign Module)
|
|
|
|
Air
Fare @ Rs. 35,726 for 31 people
|
11,07,506
|
|
|
|
Visa
+ Ins. @ Rs. 2,700 for 31 people
|
8,37,000
|
|
|
|
Foreign
Allowance (27 people + 1 coordinator)
|
24,14,250
|
43,58,756
|
|
8
|
Cost
of participants’ time @ Rs. 90,000 per participant (assumed average CTC)
|
|
27,00,000
|
|
|
|
|
92,999,36
|
|
Source: HR (Dept.), Indian Oil
|
|
|
|
|
|
Return Calculation
The return has been calculated in three different
ways:
1.Benefits/Costs Ratio
One of the earliest methods for evaluating training
investments is the benefits/costs ratio (BCR). This method compares the
benefits of the programme to the costs in a ratio.
BCR = Programme Benefits / Programme Costs
In simple terms, BCR compares the annual economic benefits
of the programme to the costs of the programme. A BCR of one means that the
benefits equal the costs. A BCR of two, usually written as 2:1, indicates that
for each rupee spent on the programme, two rupees were returned as benefits.
2.ROI (Return on Investment)
Perhaps the most appropriate formula for evaluating
training investments is net programme benefits divided by cost. The ratio is
usually expressed as percentage when the fractional values are multiplied by
100. ROI can thus be expressed as:
ROI (%) = [(Programme Benefits — Programme Costs) /
Programme Costs] x 100
The ROI value is related to the BCR by a factor of
one. This means, for example, that a BCR of 2.56 is the same as an ROI value of
156 per cent. An ROI on a training investment of 60 per cent means that an
additional 60 per cent of the costs are reported as ‘earnings’. An ROI on
training investment of 150 per cent indicates that the costs have been
recovered and an additional 1.5 multiplied by the costs is captured as
‘earnings’.
3.Payback Period
The payback period is a common method for evaluating
capital expenditures. With this approach, the annual cash proceeds (savings)
produced by the investment are equated to the original cash outlay required by
the investment to arrive at some multiple of cash proceeds equal to the
original investment. Measurement is usually in term of years or months.
Payback Period = Total Investment / Annual Savings
Return Calculation on The Threshold Programme
1) Benefits/Costs Ratio BCR = Programme Benefits / Programme Costs = 32, 50,000 / 15, 49,989 =
2.097
2) ROI (%) = [(Programme Benefits — Programme Costs) /
Programme Costs ] x 100
= [(32, 50,000 – 15,49,989) / 15,49,989] x 100 = 109.68%
3) Payback
Period = Total Investment / Annual Savings
= 15,49,989 / 17,00,011 = 0.91 years or 11 months (approx)
Return Calculation on Excellence in Project Management Programme
1) Benefits/Costs Ratio
BCR = Programme Benefits / Programme Costs
9, 87,750 / 1,86,333 = 5.3
2) ROI
(%) = [(Programme Benefits — Programme
Costs) / Programme Costs ] x 100
[ (9,87,750 – 1,86,333) / 1,86,333] x 100 = 430% (approx)
3) Payback Period = Total Investment / Annual
Savings
= 1,86,333 /
8,01,417
= 0.23 years or 3 months (approx)
Data Analysis and Results
The calculations are based on a series of impact
questions. Five adjustments are made to ensure that the data is credible and
accurate:
1) The participants who do not complete the
questionnaire or provide usable data on the impact questions are assumed to
have made no improvement.
2) Extreme and unrealistic data have been omitted.
3) Only annualized values are used as requested in
the responses.
4) The values are adjusted to reflect the confidence
level of participants.
5) The values are adjusted for the amount of the
improvement related directly to the programme.
These five adjustments create a very credible value
that is usually considered to be an understatement of the benefits accrued due
to the training imparted. Tables 3 and 4 give the estimates of training impact
from participants in the different programmes.
Table3: Estimates of Training Impact from Participants in the Threshold Programme
|
Participant
|
Improvement
(Rs.)
|
Basis
|
Confidence
Level
(%)
|
Isolated
Effect %age
|
Conservative
Integration (Rs.)
|
|
1
|
30,00,000
|
Better
decision making
|
30
|
50
|
4,50,000
|
|
2
|
25,00,000
|
Efficient
management
|
60
|
40
|
6,00,000
|
|
3
|
20,00,000
|
Quick
and accurate decisions
|
40
|
80
|
6,40,000
|
|
4
|
40,00,000
|
Efficient
handling of situations and
|
30
|
70
|
8,40,000
|
|
|
|
increased
teamwork within department
|
|
|
|
|
5
|
30,00,000
|
Good
decisions
|
60
|
40
|
7,20,000
|
|
|
|
Total
Benefits
|
|
|
32,50,000
|
Table4: Estimates of Training Impact from Participants in Excellence in Project Management Programme
|
Participant
|
Improvement
(Rs.)
|
Basis
|
Confidence
Level (%)
|
Isolated
Effect %age
|
Conservative
Integration (Rs.)
|
|
1
|
1,60,000
|
Additional
Sales
|
90
|
50
|
72,000
|
|
2
|
3,00,000
|
Decreased
time overruns
|
60
|
50
|
90,000
|
|
3
|
3,00,000
|
Reduced
cost of rework
|
70
|
60
|
1,26,000
|
|
4
|
2,00,000
|
Minimizing
startup hiccups
|
50
|
50
|
50,000
|
|
5
|
2,75,000
|
Reduced
maintenance problems
|
60
|
40
|
66,000
|
|
6
|
3,00,000
|
Increased
site effectiveness
|
70
|
50
|
1,05,000
|
|
7
|
4,00,000
|
Reduced
maintenance problems
|
80
|
40
|
1,28,000
|
|
8
|
2,00,000
|
Reduced
project time
|
70
|
50
|
70,000
|
|
9
|
3,50,000
|
Reduced
on-site problems
|
85
|
50
|
1,48,750
|
|
10
|
1,40,000
|
Efficient
time and labour management
|
50
|
60
|
42,000
|
|
11
|
1,50,000
|
Under
budget
|
100
|
60
|
90,000
|
|
|
|
Total
Improvement
|
|
|
9,87,750
|
Limitations of the Study
The method chosen to isolate the impact of training
is meant to obtain information directly from the programme participants. The
effectiveness of this approach rests on the assumption that participants are
capable of determining or estimating how much of a performance improvement is
related to the training programme. This is one of the major methodological
limitations of such studies. Erroneous, incomplete, and extreme information
could have distorted the analysis. While calculating ROI through this method,
the participants should know how much of the change was caused by applying their
learning from the programme.
Apart from this, the following limitations have been
figured out in the present study:
1) The exact figure for cost of participants’ time
was not available. Hence there is a possibility that the estimations made are
not accurate.
2) Suitable responses from all the participants were
not available. This means that the effectiveness of both training programmes
taken into consideration has been an extrapolation of the responses, which were
made available. The assumption here is that similar benefits would have accrued
from the participants who did not make their response available.
3) There was no possibility of even experimenting
with the control groups since the project commenced post training.
4) The participants of both the programmes were
posted all over the country, making distance a barrier for conducting in-depth
interviews.
The presence of participants across the nation also
made it difficult to obtain post-training data from more than one source, i.e.,
the participants themselves. This was a unique problem that was uncovered
particularly with the Threshold Programme participants. Since they primarily
occupy high leadership positions, they do not have any activity within their
purview which directly impacts the bottom line of the organization. It is
generally their ability to handle the employees, who directly boost the bottom
line well, which is their real contribution. This indirect effect makes
quantifying the benefits derived due to training rather difficult.
Recommendations and Implications
If appropriate and feasible, participants should
receive prior communication about the requirement for a follow up
questionnaire. This minimizes some of the resistance to the process, provides
an opportunity to explain in some more detail the circumstances surrounding the
evaluation, and positions the follow-up evaluation as an integral part of the
programme – not as an add-on activity that someone initiated three months after
the programme.
Management involvement at the local level might
prove critical to the response rate success. Managers can distribute the
questionnaires themselves, make reference to the questionnaire at staff
meetings, follow up to see if the questionnaire has been completed and
generally show the support for completing the questionnaire. This direct
supervisor support might cause some participants to respond with usable data.
Even if it is an abbreviated form, participants
should see the results of their study. More importantly, participants must
understand that they will receive a copy of the study when they are asked to
provide the data. This promise might increase the response rate, as some
individuals want to see the results of the entire group along with their
particular input.
It is difficult to evaluate an entire HRD function
such as management development, career development, executive education or
technical training within the ROI umbrella. ROI is more effective when applied
to one programme that can be linked to a direct payoff.
For this reason, ROI evaluation must be a
micro-level activity that will usually focus on a single programme or a few
tightly integrated programmes. This decision to evaluate several programmes or
just one programme should include consideration of the objectives and timing of
the programme. Attempting to evaluate a group of programmes conducted over a
long period becomes quite difficult. The cause and effect relationship becomes
more confusing and complex.
Incentives might be used to obtain a greater
response results from the participants. For example, one might pin a ten rupee
note to the questionnaire and add a heading on the questionnaire which states
“Please fill out this form over a cup of coffee”. Another incentive might be to
send a pen along with the questionnaire along with a note which states “Kindly
use this pen to fill out the form”. These methods have been known to boost
response rates.
Conclusion
Any organization wanting to improve systems,
procedures, or even attitudes must plan accordingly. This planning must be
integrated and aligned with the business. The effort to use training as a tool
to help balance an organization in a dynamic environment must be continuous.
There must be constant matching of individual and
organizational needs in a system which is biased towards neither. Once this
matching is completed and training programmes carried out, it is then not only
important to evaluate the training but to realize how much it has boosted the
bottom line. Here is where the ROI comes to the fore possibly on helping management
realize that training is truly an investment and not an expense.
Training can no more be a mundane task which exists
because it always has. It is now strategically imperative that training be
conducted with the clear understanding that if people are truly the
organization’s greatest asset, then training is beyond doubt, the greatest
investment and must hence be utilized wisely.
By evaluating training programmes with the ROI in
mind, training functions can be perceived in a more credible light. Programmes
aligned with organization strategy are offered, while others that add little
value are redesigned and sometimes eliminated. With proper planning around a
proven framework, realistic evaluation targets, and shared responsibilities for
major steps, the ROI process can be implemented in a cost-effective, systemic
manner and can assist the resource-constrained training function to present
their work in terms of financial benefits that leaders understand and have come
to expect. Specifically, part one of this ROI series emphasized the following
cost-savings approaches:
1. Plan
for evaluation early in the process
2. Build
evaluation into the training process
3. Share
the responsibilities for evaluation
4. Require
participants to conduct major steps
5. Use
short-cut methods for major steps.
Part two will continue to describe practical
application of five additional cost-saving approaches to ROI implementation:
6. Use
sampling to select the most appropriate programmes for ROI analysis
7. Use
estimates in the collection and analysis of data
8. Develop
internal capability
9. Streamline
reporting
10. Utilize technology.
Annexure
1: Questionnaires for Programme Evaluation
‘Threshold’
Programme Evaluation
1.
Listed below are the objectives of the ‘Threshold’ programme. After reflecting
on the programme, please indicate the degree of success in meeting the
objectives:
|
As a result of this
Programme, participants will be able to:
|
Failed
|
Limited Success
|
Generally Successful
|
Completely Successful
|
|
|
|
|
|
|
|
a. Develop a general
management orientation through highlighting the inter-linkages across
decisions in functional area
|
|
|
|
|
|
b. Strengthen the
understanding of key issues and challenges in strategy formulation and
implementation
|
|
|
|
|
|
c. Enhance the
awareness about the threats, constraints and opportunities that have arisen
from operating in increasingly deregulated product, services and resource
markets
|
|
|
|
|
|
d. Sensitize the
participants to the need to create value through appropriate corporate
actions for long-term survival and growth of the organization
|
|
|
|
|
2. Please rate on a scale of 1-5, the relevance of
each of the Programme elements to your job by indicating (1) not relevant and
(5) very relevant
|
|
1
|
2
|
3
|
4
|
5
|
|
a.
Interactive Activities
|
|
|
|
|
|
|
b.
Group Discussions
|
|
|
|
|
|
|
c.
Networking Opportunities
|
|
|
|
|
|
|
d.
Reading Materials / Video
|
|
|
|
|
|
|
e.
Programme Content
|
|
|
|
|
|
3. Please indicate the degree to which your use of
the following actions was enhanced as a result of your participation in the
training Programme:
|
|
No
Change
|
Little
Change
|
Some
Change
|
Significant
Change
|
Momentous
Change
|
No
Opportunity to use Skills
|
|
a.
Taking decisions with a sound analysis of the same
|
|
|
|
|
|
|
|
b.
Making the unit/department more market-oriented
|
|
|
|
|
|
|
|
c.
Focusing on customer analysis and customer value
|
|
|
|
|
|
|
|
d.
Effectively using the information technology within the organization
|
|
|
|
|
|
|
|
e.
Understanding and managing co-workers and superiors
|
|
|
|
|
|
|
|
f.
Effectively measuring and managing risk
|
|
|
|
|
|
|
|
g.
Engaging in vertical integration, diversification and/or mergers and
acquisitions
|
|
|
|
|
|
|
|
h.
Ability to envision for your organization
|
|
|
|
|
|
|
|
i.
Enhancing your leadership traits
|
|
|
|
|
|
|
|
j.
Better handling the competition faced by your company
|
|
|
|
|
|
|
4. Please identify any specific accomplishments that
you can link to this training Programme)? What specific suggestions do you have
for improving the Programme? (on time schedules, project completion, response
times, etc)
5. What specific value
in Indian Rupees can be attributed to the above accomplishments/ improvements
(use first year values only)? While this is a difficult question, try to think
of specific ways in which the above improvements can be converted into monetary
units. Please indicate the basis for your calculation.
|
6)
|
What
level of confidence do you place on the above estimations?
|
|
|
|
(0%
= No Confidence, 100% = Certainty)
|
|
|
7)
|
Other
factors generally tend to affect performance as well. Please indicate the
|
|
|
|
percent
of the above improvements that is related directly to this programme.
|
|
8. Indicate the extent to which you think this
programme has influenced each of these measures in your work unit, department
or business unit:
|
|
No
Influence
|
Some
Influence
|
Moderate
Influence
|
Significant
Influence
|
Momentous
Influence
|
|
a.Productivity
|
|
|
|
|
|
|
b.
Customer Response time
|
|
|
|
|
|
|
c.
Cost Control
|
|
|
|
|
|
|
d.
Employee Satisfaction
|
|
|
|
|
|
|
e.
Customer Satisfaction
|
|
|
|
|
|
|
f.
Quality
|
|
|
|
|
|
|
g.
Other
|
|
|
|
|
|
9. What barriers, if any, have you encountered that
have prevented you from using skills or knowledge gained in this Programme?
10) What specific suggestions do you have for
improving the Programme?
Questionnaire
on ‘Excellence in Project Management’ Programme Evaluation
1.
Listed below are the objectives of the ‘Excellence in Project Management’
programme. After reflecting on the programme, please indicate the degree of
success in meeting the objectives:
|
As a result of this
Programme, participants will be able to:
|
Failed
|
Limited Success
|
Generally Successful
|
Completely Successful
|
|
|
|
|
|
|
|
a. Grasp the concepts
of Project Management in the current scenario
|
|
|
|
|
|
b. Grasp the latest trends in Project
Management for optimum utilization of resources available for the projects
|
|
|
|
|
|
c. Reduce the execution time by using
modern construction equipment
|
|
|
|
|
|
d. Identify key factors for improvement in
time, cost and quality of projects
|
|
|
|
|
2. Please rate on a scale of 1-5, the relevance of
each of the Programme elements to your job by indicating (1) not relevant and
(5) very relevant
|
|
1
|
2
|
3
|
4
|
5
|
|
a.
Interactive Activities
|
|
|
|
|
|
|
b.
Group Discussions
|
|
|
|
|
|
|
c.
Networking Opportunities
|
|
|
|
|
|
|
d.
Reading Materials / Video
|
|
|
|
|
|
|
e.
Programme Content
|
|
|
|
|
|
3. Please indicate the degree to which your use of
the following actions was enhanced as a result of your participation in the
training Programme:
|
|
No
Change
|
Little
Change
|
Some
Change
|
Significant
Change
|
Momentous
Change
|
No
Opportunity to use Skills
|
|
a.
Minimizing the non-contributing actions per project
|
|
|
|
|
|
|
|
b.
Arranging the site for maximum effectiveness
|
|
|
|
|
|
|
|
c.
Assigning appropriate work for involved employees
|
|
|
|
|
|
|
|
d.
Dealing with problems on site
|
|
|
|
|
|
|
|
e.
Keeping the project members focused
|
|
|
|
|
|
|
|
f.
Accomplishing project objectives
|
|
|
|
|
|
|
|
g.
Evaluating the project
|
|
|
|
|
|
|
|
h.
Implementing action plan
|
|
|
|
|
|
|
|
i.
Planning a follow-up activity
|
|
|
|
|
|
|
|
j.
Better handling the competition faced by your company
|
|
|
|
|
|
|
4. Please identify any specific accomplishments that
you can link to this training Programme)? What specific suggestions do you have
for improving the Programme? (on time schedules, project completion, response
times, etc)
5. What specific value in Indian Rupees can be
attributed to the above accomplishments/ improvements (use first year values
only)? While this is a difficult question, try to think of specific ways in
which the above improvements can be converted into monetary units. Please
indicate the basis for your calculation.
|
6)
|
What
level of confidence do you place on the above estimations?
|
|
|
|
(0%
= No Confidence, 100% = Certainty)
|
|
|
7)
|
Other
factors generally tend to affect performance as well. Please indicate the
|
|
|
|
percent
of the above improvements that is related directly to this programme.
|
|
8. Indicate the extent to which you think this
programme has influenced each of these measures in your work unit, department
or business unit:
|
|
No
Influence
|
Some
Influence
|
Moderate
Influence
|
Significant
Influence
|
Momentous
Influence
|
|
a.Productivity
|
|
|
|
|
|
|
b.
Customer Response time
|
|
|
|
|
|
|
c.
Cost Control
|
|
|
|
|
|
|
d.
Employee Satisfaction
|
|
|
|
|
|
|
e.
Customer Satisfaction
|
|
|
|
|
|
|
f.
Quality
|
|
|
|
|
|
|
g.
Other
|
|
|
|
|
|
9. What barriers, if any, have you encountered that
have prevented you from using skills or knowledge gained in this Programme?
10) What specific suggestions do you have for
improving the Programme?
