Dos and Don'ts when choosing between sources of LMI
Introduction:
As a practitioner, you already know the crucial importance of high quality, reliable labour market information (LMI) for effective advice and guidance. Hopefully, you are already making good use of the National Guidance Research Forum Future Trends section to access quality LMI. But of course whilst the coverage of the site is impressive, it will never be able to claim that it includes all existing labour market information.
It is particularly important to:
- look more widely than local authority/unitary authority level (at broader sub-regions and regions), and
- focus down at micro area levels (especially for certain disadvantaged groups) to get a more rounded picture
This means that if you are serious about seeking out your own LMI you will need to develop a critical eye to determine what's hot and what's not in the world of LMI!
Some hints on what to look for:
2. How was the LMI collected?
This seems obvious, but does the LMI seem to have been collected in a reasonable way? This covers everything from geographical spread, age range, ethnic background to type of questioning used in the research.
As those who use interviewing skills in their work will agree, it's easy to ask a 'leading question' i.e. one which tends to encourage a particular response - so if you are looking at LMI be sure that you trust the approach used in the gathering of the data.
As a minimum think about:
- How was the data collected?
- What questions were asked?
- How did they ask them?
- Who was asked?
3. Do the claims seem reasonable and proportionate to the research approach?
Does the research lead to broad generalisations or are they simply giving more modest illustrative case studies? This may call for a degree of critical thinking and common sense on your part.
Beware too of believing to quickly any claim that because two things are happening at the same time that they are necessarily linked. For example, research might show that people with bigger feet are much more knowledgeable about the world... but that may be more to do with the fact that most people with big feet are adults not because having small feet makes you ignorant!
Two things happening at the same time are not necessarily related, though statistics can be used to show that they are. This trick is used a lot by people wanting to show that a particular strategy is working (or that it isn't) so don't be fooled. Also, beware of numbers taken out of context. This is called 'cherry-picking', an instance in which the analysis only concentrates on such data that supports a foregone conclusion and ignores everything else. The bbc h2g2 website (link to http://www.bbc.co.uk/dna/h2g2/A1091350 )has lots of examples that can help you see through the haze of statistical tricks if you want to find out more.
As a minimum think about:
- Does the conclusion legitimately arise from the research?
- Could any other factor have produced these results?
- Has the research included enough respondents to justify the claims made?
- Is the research offering an illustrative case-study or snapshot that is of interest, or does it claim to make a more representative point?
4. When was the research carried out?
It's impossible to give any hard and fast rule in terms of when research might be considered to be past its use-by date.
It is important to acknowledge that different data sources are collected and updated at different times and so it is unwise to apply any universal rule or recommendation about whether information of a certain vintage is reliable or useful because of variations by source and topic.
For example, census information is collected only every 10 years, and then there is likely to be a considerable time lag in the analysis of that information, so between censuses data might look quite 'old' but still be the most recent available information source.
Labour Market Information is also quite complex to collect and scrutinise; reports that rely on data a few years old might actually be as up to date as possible. Equally, the labour market changes rapidly so contradictory information is out there. It wasn't so very long ago that a shortage of plumbers was predicted, now some say there is a glut. Still, date of collection is still worth scrutiny:
As a minimum think about:
- What date was the research carried out?
- What date was the report/ research published? (this may be a more accurate indication of currency)
- Have there been any pivotal factors between the date of the research and date of publication that might call into question the topicality of research findings? (e.g. new countries entering the European union may shift the available labour supply)
- Is there any more recent research in the public domain that either supports or contradicts the findings of this particular report?
5. Does the research echo with any other sources of information?
If research is truly innovative and original it may not echo with existing research, and that doesn't matter, it can be really exciting. But, if it does contain any big surprises you may want to cast your critical eye over it quite carefully before being too won over.
It can be helpful to get into the habit of always looking at more than one source of information for key data and noting similarities or differences. This doesn't necessarily mean that one set of data is wrong; the research may have been looking at slightly different things, or collected in a different way. However, noting and looking for explanations and clarifications where there are contradictions and similarities between data sets will help you gain more confidence in drawing your own conclusions about the LMI you are interested in.
As a minimum think about:
- Does the research have similar findings to other sources of data?
- If there are significant contradictions to 'accepted' wisdom are these explained?
- Is the research original, or is it based on recycling or reanalysing existing sources of data?
Critiquing the claims - examining a Construction Skills press release
- BBC NEWS | Education | Careers advice for girls 'biased'
-
(Local copy of) ConstructionSkills Press Release PDF, 3 April 2006
[ Download ]
(ConstructionSkillslaunchespositiveimage030406.pdf
-
87.56 Kb
)
Preview
- ConstructionSkills Press Release PDF, 3 April 2006, from their site
On the 3rd April 2006 Construction Skills launched a press release that claimed research shows young women feel let down by the careers advice they receive. But how credible is this research?
Gender stereotyping is clearly a cause for huge concern, but if the problem is incorrectly attributed then it can't be tackled effectively.
Based on this press release, do you feel the claim is justified, if not, what else would you like to know?
Add your thoughts and comments by replying using the links below.
2 comments.
- Latest comment:
- Career guidance: how bad can it get?
29-November-2006 14:27:56 by jenny bimrose
Last cached: 2008-05-07 09:55 AM