Critical appraisal

A critical appraisal is an essential part of evidence-based healthcare.

Critical appraisal's involve 3 stages.

1. Evidence cycle.
2. Methodology.
3. Results and conclusions.

Stages of a critical appraisal

Evidence cycle

The evidence cycle makes it clear that evidence should not just be found and then used, it also needs to be critically appraised. Numerous Randomised Controlled Trials (RCTs) are published every day, let alone other study types, and not all are of consistent quality.

Critical appraisal is important so that the good studies can be found amongst the many. These can then be used to confirm or change treatments and advice given to patients.

Critical appraisal generally seeks to answer 2 broad questions.

• Is this study likely to be accurate (validity)?
• Can we use its findings (relevance)?

Methodology

To answer these questions, we pose further questions. We start by looking at what the purpose of the article was, and then whether the chosen study methodology was appropriate for answering this question.

Following that the described methods need to be considered for anything that is unclear, anything that seems confused, potential for bias, or confounders that have not been mitigated against.

There will very rarely be a perfect paper where everything that could be explained has been clearly done so, but just because a paper is imperfect, it does not mean it should be ignored.

The point is to look for potential flaws in the research that mean either its validity cannot be trusted, or it is no longer relevant in our setting or situation.

Results and conclusions

Finally, the results and conclusions should be looked at to see what the results were, how precise they were and whether they were statistically significant.

Critical appraisal does not mean simply summarising a paper descriptively. It is about forming a judgement upon it. The discipline of critical appraisal is critical in sense of questioning the research, its results, and conclusions, not in the sense of having to be critical of a paper.

It is alright to agree with all the conclusions of the authors, and it is fine to disagree with them all, or to agree with some and disagree with others. However, is not okay to be led to their conclusions by them. The reader should form their own conclusions.

No particular expertise is required to critically appraise an article, though more subject specific knowledge is an advantage for more technical papers.

The more experience and confidence an appraiser has the quicker they should be able to form their own opinions on the study.

Example from practice by Stuart Glover

University Hospitals of Leicester NHS Libraries provide their critical appraisal training, called ‘Critical Reading Made Easy’, in 2 parts. In the morning session they cover the introduction to, and principles of, critical appraisal.

This covers, what it is, why it is important, an overview of the main study methodologies and some of the main statistical terms that papers use.

Throughout, the session is interspersed with games and activities which help to make a dry subject a little more fun and re-enforce the main learning points. These include:

• activities to design your own clinical study
• a quiz on which methodology best addresses different issues
• which treatment would you recommend based upon various statistics.

Examples used are never too technical and tend to be quite light-hearted.

The other part of the training is the afternoon practical session in which papers are given out to be read by participants and then discussed as a group. This session only take place if there is a group of at least 3 as the group discussion element is vital.

Again, papers chosen reflect the fact that the groups are mixed abilities and include people from different specialties and job roles. They also tend to be more light-hearted papers

Every time the session is run different papers are chosen, which keeps the sessions interesting for the tutors and it means that everyone can get more practice as participants can come back multiple times.

Aims of the session are to raise the confidence of participants and the course is therefore evaluated against that aim. Feedback collected since 2016 from over 200 attendees show that on average this course increases confidence by just over 95%. The course has also been described as ‘the most fun that critical appraisal can be’.

Resources