What are active ingredients?

Posted on 22 October 2020

Jonathan Haslam, Director, IEE

The idea of “active ingredients” comes from medicine. Look at the packaging of any drug and it will tell you what the active ingredients are, and how much of them there are. The active ingredient in a paracetamol tablet might be 500mg paracetamol, with the rest of the tablet being potato starch, maize starch, talc, colouring, and so on.

This idea is attractive when it comes to introducing an evidence-based approach in education. If we know what the “active ingredients” are of, let’s say, feedback, we know the bits that must be included for it to have an impact, and the bits that we can tweak for local conditions. So, from the Education Endowment Foundation’s Guide to Implementation:

Specify the active ingredients of the intervention clearly: know where to be ‘tight’ and where to be ‘loose’.

One stumbling block with this is figuring out the active ingredients of any particular approach. For example, the only mention of “active ingredients” on the EEF website is in material relating to implementation. To the best of my searching ability, “active ingredients” aren’t mentioned in any of the EEF’s 100+ trials. So if you want to figure out the active ingredients of an EEF-trialled approach for yourself, you’ll have to make an educated guess. It would clearly be helpful if the active ingredients were a regular feature in EEF reports.

One report that comes close, but leads us into further challenge, is Talk for Writing. In this report the evaluators looked at the evidence supporting all the various aspects of the programme.

there is increasing evidence indicating that the influence of oral language on writing is partly mediated through reading both in terms of word decoding and reading comprehension…. there is strong evidence that writing skills have a positive impact on reading both in terms of word reading performance (ages 6 to 10) and reading comprehension (ages 9 to 18)… We found no evidence in the research literature that daily repetition of texts supports generic oral language skills or writing skills of school age children.

(My emphasis) And so on. This is helpful in identifying those elements of the programme that already have supporting evidence. However, the developer has then compiled these elements into a programme. Which of these are the active ingredients of the programme? It’s not necessarily the ones with the best existing evidence.

The active ingredients of a particular programme are usually self-identified. So a teacher, researcher or programme developer might insist that certain activities are the the active ingredients. They might be right, or they might not. Similarly, if you read a research report and decide to implement the approach, you might decide for yourself the important elements. Again, you might be right or you might not. (This issue has cropped up in our innovation evaluations, when teachers have tried to replicate approaches that have been researched previously.)

There are also potential biases when selecting active ingredients. For example, a proposed active ingredient might be CPD on the approach. An organisation that relies for its living on running training courses might decide that CPD is an “active ingredient”, and essential when using an approach. A publisher selling materials on the approach might decide that it isn’t, allowing it, by coincidence, to sell (and teachers to implement) the approach more cost-effectively. (In fact, the reality is probably more complicated than that. I don’t think CPD is an active ingredient at all; it’s more like a delivery route (similar to injecting an antibiotic instead of taking it by tablet). CPD is likely to increase the effectiveness of the approach, but only because it supports the better transmission of the actual active ingredients.)

Unfortunately we rarely see, at the outset of a trial, a clear definition of what the active ingredients are thought to be. And even more rarely, in the results of the trial, information on which of those ingredients actually influenced outcomes.

One exception is this paper by Tashia Abry and colleagues, who used measures of intervention fidelity to identify active ingredients. In other words, slightly doing things the other way around. If you monitor which ingredients participants were actually using, and compare it with effectiveness, you can deduce which might be the active ones. As the paper notes, though, such applications are rare.

Until there is more information on which active ingredients are, actually, active, I think the usefulness of active ingredients in practice may be limited, and possibly misleading.

 

References
Putting Evidence to Work: A School’s Guide to Implementation, Education Endowment Foundation (2019)
Tashia Abry, Chris S. Hulleman, Sara E. Rimm-Kaufman, Using Indices of Fidelity to Intervention Core Components to Identify Program Active Ingredients, American Journal of Evaluation (2014)
Talk for Writing: Evaluation report and Executive summary, Education Endowment Foundation (2015)

Leave a reply

Your email address will not be published.