Inclusion/Exclusion Criteria

We included randomized, quasi-randomized and non-randomized controlled trials, cross-sectional observational studies examining self-collected verses clinician-collected samples for nucleic acid amplification test (NAAT) screening of gonorrhea and chlamydia. Studies were included if they: (a) used a clinician-collected specimen as their reference standard, (b) included both positive and negative test results and (c) used a NAAT platform of: polymerase chain reaction testing (PCR), strand displacement amplification (SDA), transcription-mediated amplification (TMA), or ligase chain reaction (LCR). We excluded studies if: (a) the time period between self- and clinician-collected samples was longer than three weeks (because of potential change in the disease status of the patient), (b) participants were known to be positive for disease before enrolment in the study, (c) any combined test result (e.g. urine and vaginal combined as self-collected), (d) more than 20 percent of included patients dropped out of the study, (e) authors did not reply to our request for additional data, (f) the study did not include a NAAT, (g) anatomical sites were not comparable (see Table 1 in our published protocol for comparable sites) [23], (h) more than one test was used as a the reference standard, (i) the self-collected and clinician-collected samples were not done on the same person, (j) results were not retested, i.e. a confirmatory test was performed. For budgetary reasons we also excluded non-English-based studies.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of studies comparing the accuracy of self-collected compared to clinician-collected samples (n = 21).

https://doi.org/10.1371/journal.pone.0132776.t001

NAATs

NAAT testing for the detection of chlamydia and gonorrhea has become the standard diagnostic method used in most laboratories. NAATs have high sensitivity and specificity for the detection of chlamydia and gonorrhea using genital tract samples or first-catch urine specimens [22]. The different NAAT platforms include: PCR (Abbott LCx and Roche Amplicor), SDA (Becton Dickinson ProbeTec ET), TMA (Aptima, GenProbe Aptima Combo 2 [AC2], and Aptima Neisseria gonorrhea assay), and LCR (Abbott LCx [LCx] assay) [22,26,27]. The LCx assay uses LCR amplification and was discontinued in 2002. Combination assays which test for chlamydia and gonorrhea include the AC2, Becton Dickinson ProbeTec chlamydia/ gonorrhea, Amplicor-Cobas, and Abbott Realtime chlamydia/ gonorrhea assays [26,28,29]. The GenProbe PACE 2, discontinued as of December 2012, and the Digene Corporation Hybrid Capture 2 are non-amplified probe tests [30].

Data Collection

The following data elements were extracted and if appropriate, used for subgroup analysis a priori: true positive (TP) false positive (FP); true negative (TN); false negative (FN); sensitivity; specificity; reference (gold) standard used to compare test characteristics; NAAT platform; assay type; specimen type (urine, vaginal, cervical, urethral, pharyngeal, or rectal); gender; mean age; sample size; HIV status; date of the study initiation; country location of study; location of self-collection (home, clinic, ect…); and prevalence. Data was extracted independently by two reviewers using a standardized pre-tested form. Any disagreements between the reviewers were resolved by a third party reviewer. Our main outcomes were the sensitivity and specificity of specific comparable anatomical sites. When comparing anatomical site for comparison, we took into account the fact that cervical samples may be more sensitive than urine or vaginal samples.

Quality Assessment

Validity was assessed using the Quality Assessment of Diagnostic Accuracy Studies (Quadas) checklist [31]. We adapted the 14 point Quadas checklist to a 12 point checklist. We removed two questions related to the description of the index and reference tests (as chlamydia and gonorrhea tests are well known and described in past literature) and one question related to blinding of the index test patients (as our inclusion criteria stipulated that both tests be conducted on the same patient). We then added one question: Were all included patients counted when reporting true positives, true negatives, false positive and false negatives, and were less than 20 percent of the patients missing from the analysis?

Data Analysis

TP, FP, TN, and FN from each study were extracted to construct two-by-two tables and test sensitivity and specificity were calculated. Meta-analyses were conducted according to the Cochrane Collaboration’s new recommended methodology for systematic reviews of Diagnostic Test Accuracy [32]. Several studies identified for inclusion in the review had multiple study units (arms) [33]. In these cases we included only one study arm per subgroup to avoid creating unit-of-analysis error. The summary estimates of sensitivity and specificity estimates and their 95% confidence intervals (CIs) were constructed using Stata 11’s metandi Hierarchical Summary Receiver Operating Characteristic (HSROC) model, based on the bivariate normal model for the logit transforms of sensitivity and specificity [25,34,35].

HSROC plots were not fitted if there were four studies or less, as this is not enough to fit the hierarchical models with all five parameters by maximum likelihood. When four studies or less were included, the Moses-Littenburg fixed effects method was used. The area under an ROC was used to pool studies using the fixed effect method providing a summary diagnostic estimate of test accuracy.

Summary estimates were calculated separately for each NAAT platform type (PCR, LCR, TMA, and SDA) or assay type, combination (combo)- or single-disease assay, and other a priori categories (prevalence, Quadas score, gender, sample size or device used for sample collection). Sensitivity analyses were conducted to investigate the effect of outliers on the summary estimates. If a disease category contained ten studies or less, meta-regression was not done. Forest plots were generated using RevMan 5.3 [36].

Search Results

A total of 5333 citations were identified (Fig 1). Thirteen references were retrieved from reference lists of identified papers. A total of 4831 titles and abstracts were excluded because they were not diagnostic studies. The remaining 202 citations were retrieved for more detailed evaluation. Of these, 74 were excluded because they were did not relate to the topic. Of the remaining 128 studies comparing self-collected verses clinician collected samples, 107 were excluded (see reasons in Fig 1). Thirty papers were excluded because we could not retrieve the required data from the study and the authors did not reply to our repeat requests for more information. The final number of studies included was 21 (14 conducted chlamydia testing only [37–50], 6 tested for both chlamydia and gonorrhea separately within the same study [17,51–55], and one tested for gonorrhea only [56]. In the14 chlamydia studies, 12,916 participants were included, and in the 6 gonorrhea studies, 6040 participants were included. Table 1 displays the location of the studies, the study designs, patient characteristics, the NAAT platform used and the Quadas score.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. PRISMA flowchart of identification and selection of studies.

https://doi.org/10.1371/journal.pone.0132776.g001

Chlamydia Results

Self-collected urine vs clinician-collected urethra samples: Chlamydia in men.

Six studies conducted with males compared self-collected urine samples vs clinician-collected urethra swabs (n = 2133) [37,45,47,48,52,55]. These studies were all cross-sectional designs. Sample sizes were relatively similar (range 250–550) except for one very small study of 50 participants [45]. According to the HSROC model, the pooled sensitivity was 0.88 (95% CI 0.83–0.93) and specificity was 0.99 (95% CI 0.94–0.99). All studies had male populations.

In subgroup analysis, one study used LCR as a NAAT platform [37] compared to the other five that used PCR. Two used an assay other than Amplicor-Cobas and used single testing assays [37,47]. Combo assays were used by four studies [45,48,52,55]. When the four combo assay studies were sub-grouped, the sensitivity was 0.90 (0.84–0.94) and specificity was 0.99 (95% CI 0.89–1.00), which was slightly more accurate (Table 2). When the one study with high prevalence (33.7%; [48] was taken out, the sensitivity was 0.89 (95% CI 0.82–0.93) and specificity was 0.99 (0.98–1.00). When Quadas scores were examined, the one study with a score of under 10 was filtered out [48], resulting in a sensitivity of 0.88 (95% CI 0.81–0.92) and a specificity of 0.98 (95% CI 0.93–0.99).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Pooled effect size of chlamydia studies using a HSROC random effects model (for sub-groups of 4 studies or more).

https://doi.org/10.1371/journal.pone.0132776.t002

Fig 2 displays the HSROC curves of the six chlamydia studies of urine self-sampling verses urethra clinician-sampling by type of assay (Amplicor, Amplicor-Cobas or LCx). Forest plots are found in Fig 3. Amplicor-Cobas had a higher sensitivity and specificity than Amplicor or LCx. The tests showed variation in their sensitivity, ranging from 87%-96% in studies using Amplicor-Cobas, 87% for study that used Amplicor, and 83% for LCx. Fig 3 displays the forest plots for chlamydia urine/urethra testing. The funnel plot and the Peter’s test (p = 0.17) indicated no publication bias.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. HSROC plots of chlamydia self-collection compared to clinician-collection (top row) and HSROC plots of chlamydia self-sampling by assay type (bottom row).

Top row: The circles represent individual studies and size of the circle is proportional to the number of patients included in the study. The red square is the summary estimate of sensitivity and specificity, and the red dotted ellipse around the spots respresents the 95% confidence interval around the summary estimate. The green dotted ellipse around the spots represents the prediction contours outlining the prediction region for the true sensitivity and specificity in a future study. Bottom row: Coloured lines represent the HSROC curve by assay type in Chlamydia studies.

https://doi.org/10.1371/journal.pone.0132776.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Forest plot of the accuracy of chlamydia self-collection compared to clinician collection.

https://doi.org/10.1371/journal.pone.0132776.g003

Gonorrhea Results

Self-collected urine vs clinician collected urethra samples: Gonorrhea in men.

Three male population studies (n = 1012) comparing self-collected urine samples to clinician collected urethra swabs [52,55,56]. The pooled sensitivity was 0.92 (95% CI 0.83–0.97) and specificity 0.99 (95% CI 0.98–1.00). All were conducted in a clinic setting. Sample sizes were similar and ranged from 125 to 389 participants. They also had similar prevalence (10.6%, 12.1%, and 13.8%, respectively). Quadas scores ranged from 10 to 11. The pooled sensitivity was 0.92 (95% CI 0.83–0.97), and the pooled specificity was 0.99 (95% CI 0.99–1.00), as depicted in the forest plots (Fig 4). Two studies used Amplicor –Cobas [55] [44, 48] and when these were pooled, the sensitivity fell to 0.90 (95% CI 0.79–0.96) and specificity was 0.99 (95% CI 0.99–1.00), and are depicted in the sROC plots (Fig 5). The figure indicates that LCx performed better than the combo assay Amplicor-Cobas studies. No publication bias was present.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Forest plot of gonorrhea self-sampling compared to clinician sampling.

https://doi.org/10.1371/journal.pone.0132776.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Fixed effects sROC plots of gonorrhea studies comparing self-collected urine to either clinician-collected urethra (left) or cervical (right) samples by PCR assay type.

Colored lines represent the sROC curve by assay type.

https://doi.org/10.1371/journal.pone.0132776.g005

Summary Statement

Fourteen included studies examined chlamydia only, 6 compared both gonorrhea and chlamydia separately in the same study, and one examined gonorrhea. The six chlamydia studies comparing self-collection by vaginal swab to a clinician-collected cervical swab, had the highest sensitivity (92%, 95% CI 87–95) and specificity (98%, 95% CI 97–99), compared to other specimen-types (urine/urethra or urine/cervix). Six studies compared urine self-samples to urethra clinician-collected samples in males and produced a sensitivity of 88% (95% CI 83–93) and a specificity of 99% (95% CI 0.94–0.99). Taking into account that urine samples may be less sensitive than cervical samples, eight chlamydia studies that compared urine self-collected verses clinician-collected cervical samples had a sensitivity of 87% (95% CI 81–91) and high specificity of 99% (95% CI 0.98–1.00). For gonorrhea testing, self-collected urine samples compared to clinician-collected urethra samples in males produced a sensitivity of 92% (95% CI 83–97), and specificity of 99% (95% CI 0.98–1.00).

Comparison with Other Recommendations

The current MMWR report [13] makes a statement recommending vaginal self-sampling based on two studies only [14,15], both of which we did not include in our results due to poor quality. We excluded the Knox study [14] because a confirmatory test was performed on positive results. We excluded the Masek study [15] as did not have comparative tests and did not perform the self- and clinician-collected samples within one hour of each other. Moreover, the Knox study [14] only had only 318 patients included and the Masek study [15] had swab sample (n = 500) which is small number compared to our meta-analysis. In the Masek study [15], chlamydia detection by self-collection, assayed by ProbeTec and Combo2, had sensitivities of 81% and 100%, with specificities of 100% and 100%, respectively. For gonorrhea detection by self-collection, ProbeTec and Combo2 had sensitivities of 80% and 100%, with specificities of 100% and 100%, respectively. For the Knox study [14], gonorrhea self-collected vaginal swabs had a sensitivity of 72% and specificity of 100%, and gonorrhea self-collected urine sensitivity was 31% and specificity was 100%. Knox et al [14] found that chlamydia self-collected swabs had sensitivity of 85% and specificity of 99%. Chlamydia self-collected urine sensitivity was 73% and specificity was 100%.

For urine self-sampling, the MMWR report [13] states: “First catch urine from women, while acceptable for screening, might detect up to 10% fewer infections when compared with vaginal and endocervical swab samples.” This statement is based on three studies [16–18]. The Falk et al study [18] was excluded from our review because it reported only positive results and the authors could not be contacted with further information. Falk et al [18] reported that the sensitivities calculated in all 171 chlamydia infected women were 88%. Michel et al [16] reported sensitivity of 85%. The Falk and Michel studies did not report specificity. We tried to include the Michel et al [16] study, but the authors did not reply to our request for data. We did include Schachter 2005 data [17] in our meta-analysis [46]. In comparison, our pooled meta-analytic results indicated a good 87% sensitivity in women who collected first catch urine samples.

Programmatic Considerations

Due to acceptable diagnostic accuracy in our meta-analysis, self-collected specimens could be introduced as a new testing modality in STI control programs for patients that are hard to reach, who refuse to go for clinician-based testing, or who decline an examination. A positive NAAT test for an STI should be followed by a confirmation NAAT. Expanding screening for STIs outside specialist clinical services to internet-based or mail-in home and community service testing can allow more people to get private and anonymous access to testing. If home-based testing involving self-sampling was recommended, sufficient support by STI nurses would have to be provided to patients to properly interpret their test results. Results in a research setting may not translate well in the real world, and thus the introduction of a new screening scenario should be carefully evaluated to the particular context and culture.

One limiting factor that may have impacted the accuracy of patient-collected samples in the included studies is the use (or lack of use) of guidelines on how to self-collect gonorrhea and chlamydia urine, vaginal, rectal and pharyngeal specimens, which may have affected the sensitivities of our pooled results. Some studies such as Freeman et al. [51] and Hook et al [53] gave instructions on self-collection with clear and precise instructions on how to collect a sample, whereas other studies such as Berwald et al. [39] and Keane et al. [43] gave women written instructions, and Puolakkainen et al. [48] did not describe any patient self-collection instructions. Few public health and research organizations have published guidelines about how patients can to properly collect a vaginal or rectal swab specimen [58], a urine sample [59], or a throat swab [60]. The New Zealand guidelines [10] recommend that a self-collected vaginal chlamydia swab may be taken in some circumstances (e.g. annual or opportunistic asymptomatic screening for chlamydia, or when a female declines a genital examination) and that a first catch urine (first 30ml stream) is not the specimen of choice as it has lower sensitivity than vaginal swabs, but is useful if the patient declines examination or to do a self-collected swab. The North Carolina Public Health Program [61] and the Wyoming Public Health Laboratory [12] have written guidelines with instructions for women about how to collect a sample for chlamydia detection. Their instructions state that tampons need to be inserted for at least an hour, and that two consecutive vaginal swabs tipped with Dacron renders better results than using a single one. A John Hopkins guideline [58] for self-collection of a vaginal swab goes through nine steps and stipulates the patient must wash their hands thoroughly, insert the swab one inch inside the vagina, rotate the swab for 15 seconds, remove and not let the swab touch anything, and place the swab in the transport tube. It is clear that public health guidelines are needed that would make the process of self-collection standardized. In addition, studies also did not routinely report on the quality of the samples taken, for example they did not compare the frequency of specimen rejection due to poor sample quality for self-collected specimens vs. clinician-collected rejected specimens.

Self-collected vaginal swabs were the most accurate testing modality in our meta-analysis, and they can be collected in the clinic, at home, in community settings and delivered to a laboratory or sent through the mail. Samples sent through the mail in Canada, through Canada Post, have to be pre-packaged according to their liquid and dry patient specimen guidelines [62]. Swabs can be sent through the mail dry and in a pre-packaged kit. Tampons or sanitary pads can also be used for gonorrhea or chlamydia self-collected testing, but these have not been approved in many jurisdictions [63].

Urine samples had acceptable sensitivity and specificity when collected by patients included in this systematic review for chlamydia/ gonorrhea in men and for chlamydia in women. Urine samples can be left at room temperature for several days, allowing for the possibility of mail-in home-based testing [63]. An alternative to urine collected in a cup has recently been put onto the market. The new sponge-based urine collection device (UriSWAB, Copan Diagnostics, CA, USA) collects urine specimens either by keeping the sponge-tipped applicator under urine stream until saturated, or by dipping the sponge directly into the urine until saturated [64]. In some countries, such as Australia, it is not legal to post liquid biological specimens in the mail; however, UriSwabs are able to be posted, creating opportunities for increased access to testing. STI control programs and clinics aiming to roll-out self-sampling in their populations should routinely evaluate the quality of self-samples taken [65,66].

Study Limitations

Among the study limitations, several covariates that could affect our summary estimates were not collected, namely: presence of symptoms in patients, and the number of tests done per patient. However, dichotomising patients into asymptomatic/symptomatic was very difficult as no included studies separated the 2x2 table data based on this criteria. When more than one test was done per patient, the first test score was taken as the included data in our analyses, as this was most likely to reduce verification bias. Verification bias occurs when the percentage of subjects receiving disease verification differs according to the test result. Studies that conducted a greater number of diagnostic tests per participant tend to identify more infections (whether correctly or incorrectly), which generally result in a lower calculated sensitivity for the test. We also avoided any possibility of disease re-infection by excluding studies when the time period between self- and clinician-collected sampling was longer than three weeks.

When possible, we grouped studies by prevalence (low, medium or high) to avoid underestimating sensitivity and specificity. However, this was only possible for two chlamydia outcomes (urine self-samples verses urethra clinician samples, and urine self-samples verses cervical clinician samples) and only by one prevalence grouping as there were not enough included studies to compare with other prevalence (low or high) categories. Therefore, we were not able to compared test accuracy by prevalence.

Our study used clinician-collected specimens as the gold reference standard which assumes 100% accuracy. However, 100% accuracy is not a clinical reality. In the physician-collected studies we referenced, sensitivity ranged from about 44% to 99% in gonorrhea NAAT testing depending on the specimen collected, and 33% to 100% in chlamydia NAAT testing depending on the specimen collected [26,67–71]. Another limitation of our meta-analysis was that we included only studies that were published in English, even though included studies were conducted worldwide. Potentially, non-English studies could have changed our reported summary estimates.

In addition, we did not collect unpublished literature, even though papers presented at conferences or done by private test manufacturers may also be applicable. Our justification for this was that unpublished literature would not have been peer-reviewed or generalizable. We also excluded 30 studies because the authors did not reply, which might have biased the results.

There were several gaps in the literature. No studies were included that conducted internet-based (or mail-in) self-collection testing or any studies located in rural areas, as they did not pass our quality assessment for inclusion. There were not enough studies found that conducted gonorrhea self-sampling compared to clinician-based sampling, and this meant that we unfortunately did not have enough studies to pool results from rectal/rectal, vaginal/cervix, or pharyngeal/ pharyngeal specimen gonorrhea groups. In each of the disease and specimen categories, we did not find publication bias which means that we probably found the majority of published studies that examined diagnostic accuracy of self-collected compared to physician-collected chlamydia and gonorrhea samples.

Elimination of Access Barriers

Uptake of STI testing remains a public health challenge, with particularly low testing rates among young men (age<25) and gay, bisexual and other men who have sex with men (MSM) [72–74]. Other studies and one recent systematic review have shown that self-collection of specimens for STI testing is acceptable among these marginalized populations [11,75–77]. The Odesanmi et al. [76] systematic review found that home-based screening significantly increased the uptake compared to clinic-based testing (RR 1.55, p<0.001). Self-sampling can eliminate access barriers to STI testing, such as stigma, shame, negative interactions with service providers, concerns about privacy and confidentiality, inaccessible hours, and travel time to clinics. Furthermore, Gilbert el al. [78]showed that self-collection can be feasibly used within existing community venue-based HIV surveillance systems for MSM.

Conclusion

The sensitivity and specificity of vaginal self-collected swabs compared to swabs collected by clinicians supports the use of vaginal swab self-collection for chlamydia and gonorrhoea testing. Urine samples for gonorrhea collected by men and women had comparably high sensitivity and specificity, so could be recommended as they can be left at room temperature for several days, allowing for the possibility of mail-in home-based testing. In populations that may not go for testing at all, do not have the option of clinical testing, or who refuse a clinical examination, self-collected screening would be a good alternative. We recommend that guidelines on how to self-collect gonorrhea and chlamydia urine, vaginal, rectal and pharyngeal specimens be published.

The evidence provided in our meta-analysis could help to introduce self-collected specimens as a new testing modality in STI control programs for patients that are hard to reach, who refuse to go for clinician-based testing, or who decline an examination. Expanding screening for STIs outside specialist clinical services to mail-in home-based and community service testing can allow more people to get private and anonymous access to testing.