How do we know that our services are efficiently organised and that customers benefit from the services we provide?

In recent years there have been increased levels of investment in information, advice and guidance services for adults and young people. Not surprisingly, the guidance community is facing increasing pressure to demonstrate that these investments are paying off. How do we know that customers benefit from the services we provide? What targets do we, and our funding bodies, set, and how do we know that these are being met? What evidence do we have that shows that tax payers are getting value for money?

With this increasing pressure for accountability has come an increase in the number of assessment tools, performance criteria, quality standards and reporting requirements that individual practitioners and organisations are faced with. Many providers have already achieved quality standards such as Investors in People and The European Model for Business Excellence (EFQM), and others are being actively encouraged, some would say “pressurised”, to do so. The Learning and Skills Council for England (LSC) has an established Provider Performance Review system to “audit” the services it funds, including those provided by IAG Partnerships.

There is no clear consensus about what effective guidance means in terms of outputs, and about what really matters and what we should really be trying to measure. An array of quality standards, performance criteria, assessments tools, reporting requirements, and so on, can add to the confusion. In this climate of pressure and confusion there may be a temptation to go for the easy assessment option and count that which can be counted; for example, numbers of interventions, and amount of data correctly returned on time, instead of trying to assess what is at the heart of effective guidance, the impact on the lives of those we are trying to help.

We need time to think about what we mean by  “effective guidance”, and to reflect upon how it could be measured

It may be helpful to return to some basic thinking about what we actually mean by “effective guidance”. What do you, as a practitioner or a manager, think effective guidance looks like and can be assessed? How does your own practice, or organisation, measure up? The answers given by a practitioner may be very different to those given by a manager. In addition, the organisational context in which services are delivered – in the work place, school, college, prison - will result in a very different perspective.

To try and answer these questions it may help to take time out to look at a number of different theoretical models of assessment, from the UK and elsewhere, each reflecting the backgrounds of their authors and their particular focus on impact assessment. Having different models available to reflect upon is an important recognition of the diversity of values and culture of providers, and it is part of the process of trying to move towards a consensus about ways of defining and measuring positive impact.

Health Check Discussion 

Here`s a summary of some comments from practitioners, managers and researchers who have used the CeGS Health Check instrument. We plan to continue the dialogue so that we can collectively increase our knowledge and understanding of assessing and measuring the impact of career guidance.

1. As a manager, and a practitioner, working in a Student Services unit in a very large and busy FE college, I find myself in many one to one, and often fairly intense, situations with many different types of people. I might be counselling a student on a personal difficulty, one minute, and the next, I might be dealing with an unhappy and frustrated parent who has just learnt of their child`s absence from college. Often I find myself having to make sense of competing interests; for example, guiding a young person to the conclusion that college is not for them even though I`m very conscious that enrolments might be ominously low; or the tensions between parent and child. It`s not surprising that I`m not entirely sure how my efforts should be measured and against whose needs – those of the parent, child, the college? For me, the assessment model that made most sense, and seemed to be most relevant to my situation, was the EFQM model. It`s telling me that individual professionals don`t work in isolation, that we`re all part of a team and that only when all the parts of the organisation are clicking  together can we be really effective in meeting the different needs of the community we serve.

2. As a practitioner, working in a higher education setting, I am conscious that my own institutional agenda linked to student retention and employability issues require even more focus on added-value benefits of careers service work.  As yet, we have not developed a transparent framework simply because we have been so busy responding to immediate needs of students.  The health check framework provides a good starting point to discussion within our institution as well as within our professional association (AGCAS). I liked the 'economic model' because it gives us the big picture in which we can focus on specific aspects of our service and develop evidence on adde-value.

3. Coming from an egineering background and now working in a sector skills context, I'm used to working to ISO 9000 standards i.e. performance indictors, monitoring systems and success criteria. In my exerience, this drives achievement. I have looked at the four Health Check approaches and would recommend the ISO 9000 as a fifth strand.