![[SIGN thistle header]](../../../images/page-header-template.jpg)
Notes on the use of Methodology Checklist 4: Case-control studies
The studies covered by this checklist are designed to answer questions of the type “What are the factors that caused this event?”, and involve comparison of individuals with an outcome with other individuals from the same population who do not have the outcome. These studies start after the outcome of an event, and can be used to assess multiple causes of a single event. They are generally used to assess the causes of a new problem, but may also be useful for the evaluation of population based interventions such as screening.
Section 1 identifies the study, the reviewer, the guideline for which the paper is being considered as evidence, and the key question(s) it is expected to address. The reviewer is asked to consider a series of aspects of cohort study design and to make a judgement as to how well the current study meets this criterion. Each relates to an aspect of methodology that research has shown makes a significant difference to the conclusions of a study.
Case-control studies need to be very carefully designed, and the complexity of their design is often not appreciated by investigators, leading to many poor quality studies being conducted. The questions in this checklist are designed to identify the main features that should be present in a well designed study. There are few criteria that should, alone and unsupported, lead to rejection of a study. However, a study that fails to address or report on more than one or two of the questions addressed below should almost certainly be rejected.
For each question in this section you should use one of the following to indicate how well it has been addressed in the study:
1.1 The study addresses an appropriate and clearly focused question
Unless a clear and well defined question is specified, it will be difficult to assess how well the study has met its objectives or how relevant it is to the question you are trying to answer on the basis of its conclusions.
1.2 The cases and controls are taken from comparable populations.
Study participants may be selected from the target population (all individuals to which the results of the study could be applied), the source population (a defined subset of the target population from which participants are selected), or from a pool of eligible subjects (a clearly defined and counted group selected from the source population. If the study does not include clear definitions of the source population it should be rejected.
1.3 The same exclusion criteria are used for both cases and controls
All selection and exclusion criteria should be applied equally to cases and controls. Failure to do so may introduce a significant degree of bias into the results of the study.
1.4 What percentage of each group (cases and controls) participated in the study?
Differences
between the eligible population and the participants are important, as
they may influence the validity of
the study. A participation
rate can
be calculated by dividing the number of study participants
by the number of eligible subjects. It is more useful
if calculated separately
for
cases and controls. If the participation rate is
low, or there is a large difference
between the two groups, the study results may well
be invalid due to differences between participants and non-participants.
In these
circumstances,
the
study should be downgraded, and rejected if the differences
are
very large.
1.5 Comparison is made between participants and non-participants
to establish their similarities or differences
Even if participation rates are comparable and acceptable, it is still possible that the participants selected to act as cases or controls may differ from other members of the source population in some significant way. A well conducted case-control study will look at samples of the non-participants among the source population to ensure that the participants are a truly representative sample.
1.6 Cases are clearly defined and differentiated from controls
The method of selection of cases is of critical importance to the validity of the study. Investigators have to be certain that cases are truly cases, but must balance this with the need to ensure that the cases admitted into the study are representative of the eligible population. The issues involved in case selection are complex, and should ideally be evaluated by someone with a good understanding of the design of case-control studies. If the study does not comment on how cases were selected, it is probably safest to reject it as a source of evidence.
1.7 It is clearly established that controls are non-cases
Just as it is important to be sure that cases are true cases, it is important to be sure that controls do not have the outcome under investigation. Control subjects should be chosen so that information on exposure status can be obtained or assessed in a similar way to that used for the selection of cases. If the methods of control selection are not described, the study should be rejected. If different methods of selection are used for cases and controls the study should be evaluated by someone with a good understanding of the design of case-control studies.
1.8 Measures will have been taken to prevent knowledge of primary exposure influencing case ascertainment
If there is a possibility that case ascertainment can be influenced by knowledge of exposure status, assessment of any association is likely to be biased. A well conducted study should take this into account in the design of the study.
1.9 Exposure status is measured in a standard, valid and reliable way
The primary outcome measures used should be clearly stated in the study. If the outcome measures are not stated, or the study bases its main conclusions on secondary outcomes, the study should be rejected. Where outcome measures require any degree of subjectivity, some evidence should be provided that the measures used are reliable and have been validated prior to their use in the study.
1.10 The main potential confounders are identified and taken into account in the design and analysis
Confounding is the distortion of a link between exposure and outcome by another factor that is associated with both exposure and outcome. The possible presence of confounding factors is one of the principal reasons why observational studies are not more highly rated as a source of evidence. The report of the study should indicate which potential confounders have been considered, and how they have been assessed or allowed for in the analysis. Clinical judgement should be applied to consider whether all likely confounders have been considered. If the measures used to address confounding are considered inadequate, the study should be downgraded or rejected, depending on how serious the risk of confounding is considered to be. A study that does not address the possibility of confounding should be rejected.
1.11 Confidence intervals are provided
Confidence
limits are the preferred method for indicating the precision
of statistical
results,
and can be
used to differentiate
between
an inconclusive study and
a study that shows no effect.
Studies
that report
a single
value with no assessment
of precision should be treated
with extreme
caution.
Section 2 relates to the overall assessment
of the paper. It starts by rating the methodological quality of the study,
based on your responses in Section 1 and using the following coding system:
| ++ | All or most of the
criteria have been fulfilled. Where they have not been fulfilled the conclusions of the study or review are thought very unlikely to alter. |
| + | Some of the criteria
have been fulfilled. Those criteria that have not been fulfilled or not adequately described are thought unlikely to alter the conclusions. |
| - | Few or no criteria
fulfilled The conclusions of the study are thought likely or very likely to alter. |
The code allocated here, coupled with the study type, will decide the level of evidence that this study provides.
The aim of the other questions in this section is to summarise your view of the quality of this study and its applicability to the patient group targeted by the guideline you are working on.
Section 3 asks you to summarise key points about the study that will be added to an evidence table at the next stage of the process. It is important that you complete this section as fully as possible, and include actual data from the study wherever relevant.