Evaluation Research

Key evaluation issues:  Evaluations can help determine whether programs are achieving their objectives and provide direction on how programs can be improved.  Many of the evaluations which SPR has conducted for Federal, Provincial and other agencies over the past 25+ years have enhanced or streamlined existing programs and provided improved tools for managers.  Evaluation studies conducted by SPR have examined key issues such as:

  • Measuring results using administrative data, macro and micro-economic studies, surveys and audits.  Many SPR evaluations have developed new indicators of results directly related to program goals;

  • Surveying client and stakeholder satisfaction through the use of client satisfaction surveys (for example, SPR's surveys of users of federal information clearinghouses);

  • Identifying economic and social impacts using experimental, longitudinal and "after-the-fact" designs, such as SPR's analysis of macro-economic factors affecting public use of Canada Pension Plan (Disability) in the 1980's and 1990's;

  • Evaluating cost-effectiveness in program funding, Federal-Provincial-Municipal cost-sharing agreements;

  • Evaluation of horizontal programs (interdepartmental initiatives), such as the Federal Contractors Program (HRDC), which includes nearly all Federal agencies; and

  • Recommending alternative program approaches and strategies for program delivery, and needs for programs and services.
Methodologies:  SPR evaluations have identified strategies to improve the cost-effectiveness and impact of programs in business and the human services, and have included such varied methodologies as:

  • Descriptive studies of programs:  examining formal and operational views of programs; reviewing program goals and objectives as seen by different stakeholders; or providing analyses of services;

  • Case studies:  providing an overview of specific cases, such as an occupational health and safety program in a particular industrial facility, or operations of a Native Friendship Centre;

  • Surveys of clients:  assessing client views of and satisfaction with programs, or determining client priorities for program improvements; and

  • Multi-variate analyses of program impacts:  comparing the impacts of programs (for example, through "quasi-experimental" designs), or assessing mediating impacts of participant characteristics (such as the way firm size or export experience mediate the impact of Federal export support programs).
Evaluations that get results:  Many of SPR's studies have significantly influenced programs and legislation, such as SPR's work in occupational health and safety in Ontario (1984-1996), and SPR’s work on the Canada Pension Plan (1989-1996).