Guideline 61 - Supporting Material: Option appraisal for recommendations on use of transvaginal ultrasonography

Guideline 61: Investigation of post-menopausal bleeding - Supporting material

Option appraisal for recommendations on use of transvaginal ultrasonography

This appendix describes the background to some of the discussion by members of the guideline development group on how best to use available evidence in developing recommendations on the use of transvaginal ultrasonography.

Background

Post-menopausal bleeding represents one of the most common reasons for referral to gynaecological services, largely because of the need to detect or exclude endometrial carcinoma. The most rigorously evaluated investigation is transvaginal ultrasonography (TVUS) which measures endometrial thickness. This is diagnostically useful because for an individual woman, the greater the measured endometrial thickness, the higher the probability that cancer is present. Transvaginal ultrasonography is attractive as an initial investigation as it is non-invasive and well tolerated. Sufficient reassurance from a negative result can help avoid unnecessary and more invasive procedures seeking a tissue diagnosis. The guideline development group had to decide whether or not to recommend the use of this as a first-line test and, if so, what diagnostic threshold to recommend.

The first drafts of the PMB guideline drew on a meta-analysis by Smith-Bindman et al.¹ The Group concluded from this that a threshold of 3 mm or less endometrial thickness was sufficient (and safe enough) to exclude endometrial cancer in the majority of women (negative post-test probability of < 0.1%). A 5 mm threshold was proposed for women on sequential HRT (negative post-test probability of 0.1%). This was put out to peer review.

Feedback from external referees notified the Group of a paper in press by Chien et al.² Appraisal of this paper indicated that it represented a more methodologically rigorous review than Smith-Bindman et al. Furthermore, the authors of this paper pointed out that the 95% confidence intervals around the 3 mm threshold were relatively wide because fewer studies had assessed this threshold (although, logically, a 3 mm threshold will be better than higher thresholds at excluding cancer). In February 2002 the guideline was altered to recommend a 4 mm threshold for the majority of women (negative post-test probability of 0.8%). A 5 mm threshold would probably remain for women on sequential HRT.

Later in 2002, the Group was notified by Chien et al that the conclusions of their review were to change. None of the studies assessing a 4 mm threshold were judged to be of sufficient quality to include in a meta-anlaysis of likelihood ratios, i.e. their estimates of accuracy were likely to be biased in favour of TVUS. Subsequently the authors stated that high quality evidence (from 4 studies) existed only on the use of the 5 mm threshold. The paper does add the qualification that: ‘As the exclusion of endometrial cancer is very important, we would be wary of relying on the pooled estimates from only 4 studies, despite them being of good quality.’ Based on their pooling of these 4 studies, using a 5 mm threshold for women (not on sequential HRT) would result in a negative post-test probability of 1.7%. From previous discussions in the SIGN PMB Group, this probability is insufficient to justify use of TVUS as the sole method to exclude endometrial cancer.

Implications for using TVUS to exclude endometrial cancer

For women with PMB NOT on sequential HRT

TVUS threshold and version of Chien et al review	Negative likelihood ratio	Pre-test probability	Post-test probability
3 mm (new version; all studies)*	0.04	10%	0.4%
3 mm (new version; no high quality studies)	-	-	-
4 mm (new version; all studies)*	0.08	10%	0.8%
4 mm (new version; no high quality studies)	-	-	-
5 mm (new version; all studies)*	0.15	10%	1.6%
5 mm (new version; high quality studies only)	0.16	10%	1.7%

For women on sequential HRT

TVUS threshold and version of Chien et al review	Negative likelihood ratio	Pre-test probability	Post-test probability
4 mm (new version; all studies)*	0.08	1%	< 0.1%
4 mm (new version; no high quality studies)	-	-	-
5 mm (new version; all studies)*	0.15	1%	0.1%
5 mm (new version; high quality studies only)	0.16	1%	0.2%

OPTION A. Base recommendations only upon high quality evidence from 4 studies of 5 mm threshold.

For women not on sequential HRT, a finding of 5 mm or less endometrial thickness from TVUS may not be sufficient to exclude endometrial cancer by itself (negative post-test probability 1.7%). Therefore, TVUS, hysteroscopy or other investigations may represent appropriate first line investigations. The choice of procedure should be based upon womens’ preferences and available skills and resources
TVUS remains as the investigation of choice in excluding endometrial cancer in women on sequential HRT (negative post-test probability 0.2%)

Advantage

Based upon most rigorous evidence

Disadvantages

The 95% CI for this LR -ve is now wider than that for the 3 mm, with the upper limit being 0.42 versus 0.19 for 3 mm
That this threshold is not sensitive enough: even after a negative result the probability of cancer is 1.7% and this is likely to be unacceptable to clinicians and women, especially as the upper 95% CI is 4.4%

OPTION B. Use 4 mm threshold for majority of women:

For women not on sequential HRT, TVUS of 4 mm or less represents first line investigation of choice (post-test probability based upon poorer quality studies 0.5%)
TVUS remains as the investigation of choice in excluding endometrial cancer in women on sequential HRT (negative post-test probability 0.1-0.2%)

Advantage

Endometrial cancer can be excluded with more certainty than the 5 mm threshold

Disadvantages

The estimates of post-test probability will be based upon studies prone to bias, i.e.

Level IV. An independent, non-blind comparison with reference standard among an appropriate population of non-consecutive patients or confinement of population to a narrow spectrum

Level V. Verification of diagnosis by reference standard in < 90% of all study patients

Using results from Chien et al to estimate the best and worst case scenarios for the effects of bias; the negative post-test probability ranges from 1.0 to 1.2%. This is consistently (albeit crudely) with data from Lijmer et al³ suggesting (generally) that a lack of blinding is associated with a 30% relative increase (bias) in accuracy. Therefore, it is likely that a 4 mm threshold would not be sufficient to confidently exclude cancer as the negative post-test probability approaches or exceeds 1%.

OPTION C. Use of a 3 mm threshold for the majority of women:

For women not on sequential HRT, TVUS of 3 mm or less represents first line investigation of choice (post-test probability based upon poorer quality studies is 0.4%)
TVUS remains as the investigation of choice in excluding endometrial cancer in women on sequential HRT (negative post-test probability 0.1-0.2%)

Advantages

Makes safest use of TVUS - compared with 4 and 5 mm thresholds
For women not on sequential HRT, the estimate of negative post-test probability (0.4%) may be biased, but the true post-test probability must remain below that for 4mm or 5mm and is very likely to remain well below 1%
Approximate adjustments to the likelihood ratio for bias can still be made, with the best and worst cases for negative post-test probabilities ranging from 0.6% to 0.8%.

Disadvantages

Risk of residual bias is adjusting estimates of LR
3 mm not commonly used as a threshold; recommending this might represent the least acceptable option to the general body of professional opinion

Decision taken by the Group

The Group decided that a threshold of 3 mm should be recommended when TVUS is used in the investigation of women not on sequential HRT. A threshold of 5 mm is sufficient for women on sequential HRT.

Smith-Bindman R, Kerlikowske K, Feldstein VA, Subak L, Scheidler J, Segal M, et al. Endovaginal ultrasound to exclude endometrial cancer and other endometrial abnormalities. JAMA 1998;280:1510-7.
Chien PFW, Voit D, Clark TJ, Khan KS, Gupta JKl. Ultrasonographic endometrial thickness for diagnosing endometrial pathology in women with postmenopausal bleeding: A meta-analysis. Acta Obstet Gynecol Scand. In press 2002.
Lijmer JG, Mol BW, Heisterkamp S, Bonsel GJ, Prins MH, van der Meulen JH, et al. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA 1999;282:1061-6.