Participants were asked to consider four key questions associated with assessing and measuring the impact of guidance. The main themes that emerged are summarised below.

Why is impact assessment important for the guidance profession? 

· To help identify good practice and support continuous professional development.

· To help maximise the benefits to clients and enhance the credibility and standing of the profession.

· To provide evidence to justify current levels of funding for the service and future investment.

· To provide key performance indicators and benchmarks to enable comparisons to be made between different approaches and delivery mechanisms.

What do we want and need to know, and what measures should be used to demonstrate the impact of guidance?

· As a profession we need to “develop a language” about what we do and how we would measure/define our success.

· Some commercial organisations offering advice to young people define “effectiveness” as simply the ability to generate repeat business.

· “Hard outcomes” – volume of business, rates of entry into employment etc – are usually required by “third parties”, but “soft outcomes” – confidence building, empowerment etc – are just as important.

· We don’t need to know the eventual outcomes for all clients, eg. a sample of follow-ups should be sufficient to tell us something significant and would be more cost-effective to manage.

· Precise measures would depend on the context and the rationale for funding, eg. in educational guidance retention and achievement would be important; in workforce guidance promotion, increased job satisfaction and productivity could be relevant.

 How can we know that our results are significant and are telling us something meaningful?

· With “hard outcomes”, a key issue is knowing whether a particular outcome is as a result of the guidance intervention or whether it would have happened anyway.

· It is important to have data from “control groups”, and to have access to other comparative data, to be confident that outcomes are linked to interventions.

· Data in relation to “soft outcomes” should be used to help “corroborate” results.

What are the significant factors that could support and/or limit our ability to assess and measure the impact of guidance?

· If practitioners are required to be involved themselves, there must be adequate time allocated to do this.

· If practitioners are to be involved, they need to be ‘willing researchers’ in order for the process to be successful and they need to be “comfortable” with research criteria and methodology.

· Using practitioners to collect data about their own effectiveness, or that of their peers, raises a number of ethical issues, including impartiality.

· Clarity around protocols and code of practice and ethics, including client permissions and data protection.