Here I argue that quality standards are not about impact analysis and that they have damaged impact analysis. I was involved in the development of Quality Standards for AGCAS services. I was the Project Manager for this process between 1997 and 2001 (until the introduction of Matrix, which superseded our individual efforts).

I have two 'observations' to add to this theme:

1. Quality Standards are not about Impact Analysis

   After reviewing my personal involvement in the process of developing Quality Standards and that of colleagues in preparing for and going through quality assessments, I think we are dealing with very separate issues. Quality Standards seem to be primarily concerned with process and service provision. The focus is on defining the provision and then providing evidence that this is being provided. There seems to be little focus on the actual outcomes, the final results of information, advice and guidance. The client feedback is primarily proving the existence of and the satisfaction with, the services provided.

   Obviously Quality Standards have moved on. My initial experience was in trying to closely define what should be provided. This has now progressed so that the service defines for itself what its provision should be in relation to its particular client group, resource level etc. Similarly , emphasis has shifted from meeting a minimum ‘threshold’ of service provision to engaging in a continuous process of CQI. As part of this there is some expectation that some impact analysis should take place but it appears in practise that this is still interpreted more in terms of overall client satisfaction with the service than in seeking to establish the impact on the individual.

2. Quality Standards have damaged Impact Analysis

   Again my statement is purely on an anecdotal level and doesn’t reflect my own broadly positive view of Quality Initiatives. However, I would like to start a debate about some of the detrimental effects on impact analysis of the wide spread introduction of Quality Standards over the last decade. I haven’t been able to find any research on these areas – does anyone else know of anything? Some of the points I observe are:

   • Practitioners have become wary of measuring the effect of their work and view all such activity is seen from a defensive viewpoint
   • Evidence is seen as something to be manipulated i.e. if it went badly don’t evaluate it as it will affect our standing
   • Widespread suspicion of why data on Impact is wanted and how it will be used – in the service, by the institution, by government etc
   • Impact analysis is seen as a chore – to be done to ‘get the badge’ and then forgotten until the next cycle of Quality Assessment
   • It is seen as an imposed activity that gets in the way of the ‘real job’ using up time and other resources.
   • Real concern that the careers service will be ‘blamed’ for impacts that are the result of factors outside their control.