Why a Safe but Challenging Environment Is Important

Medical Decision Making http://mdm. sagepub. com/ Do Patient Decision Aids Meet Effectiveness Criteria of the International Patient Decision Aid Standards Collaboration? A Systematic Review and Meta-analysis Annette M. O'Connor, Carol Bennett, Dawn Stacey, Michael J. Barry, Nananda F. Col, Karen B. Eden, Vikki Entwistle, Valerie Fiset, Margaret Holmes-Rovner, Sara Khangura, Hilary Llewellyn-Thomas and David Rovner Med Decis Making published online 14 September 2007 DOI: 10. 1177/0272989X07307319.   A more recent version of this article was published on – Oct 5, 2007 Published by: http://www. sagepublications. com On behalf of: Society for Medical Decision Making Additional services and information for Medical Decision Making can be found at: Email Alerts: http://mdm. sagepub. com/cgi/alerts Subscriptions: http://mdm. sagepub. com/subscriptions Reprints: http://www. sagepub. com/journalsReprints. nav Permissions: http://www. sagepub. com/journalsPermissions. nav Version of Re cord – Oct 5, 2007 ;; OnlineFirst Version of Record – Sep 14, 2007 What is This? Downloaded from mdm. sagepub. com by guest on July 22, 2012 Med Decis Making OnlineFirst, published on September 14, 2007 as doi:10. 1177/0272989X07307319 Do Patient Decision Aids Meet Effectiveness Criteria of the International Patient Decision Aid Standards Collaboration? A Systematic Review and Meta-analysis Annette M. O’Connor, RN, PhD, Carol Bennett, MSc, Dawn Stacey, RN, PhD, Michael J. Barry, MD, Nananda F. Col, MD, MPH, MPP, Karen B. Eden, PhD, Vikki Entwistle, PhD, Valerie Fiset, MScN, Margaret Holmes-Rovner, PhD, Sara Khangura, Hilary Llewellyn-Thomas, PhD, David Rovner, MD Objective. Related article: Explain the Post 16 Options 2017 To describe the extent to which patient decision aids (PtDAs) meet effectiveness standards of the International Patient Decision Aids Collaboration (IPDAS). Data sources. Five electronic databases (to July 2006) and personal contacts (to December 2006). Results. Among 55 randomized controlled trials, 38 (69%) used at least 1 measure that mapped onto an IPDAS effectiveness criterion. Measures of decision quality were knowledge scores (27 trials), accurate risk perceptions (12 trials), and value congruence with the chosen option (3 trials). PtDAs improved knowledge scores relative to usual care (weighted mean difference [WMD] = 15. %, 95% confidence interval [CI] = 11. 7 to 18. 7); detailed PtDAs were somewhat more effective than simpler PtDAs (WMD = 4. 6%, 95% CI = 3. 0 to 6. 2). PtDAs with probabilities improved accurate risk perceptions relative to those without probabilities (relative risk = 1. 6, 95% CI = 1. 4 to 1. 9). Relative to simpler PtDAs, detailed PtDAs improved value cong ruence with the chosen option. Only 2 of 6 IPDAS decision process criteria were measured: feeling informed (15 trials) and feeling clear about values (13 trials). PtDAs improved these process measures relative to usual care (feeling uninformed WMD = –8. , 95% CI = –11. 9 to –4. 8; unclear values WMD = –6. 3, 95% CI = –10. 0 to –2. 7). There was no difference in process measures when detailed and simple PtDAs were compared. Conclusions. PtDAs improve decision quality and the decision process’s measures of feeling informed and clear about values; however, the size of the effect varies across studies. Several IPDAS decision process measures have not been used. Future trials need to use a minimum data set of IPDAS evaluation measures. The degree of detail PtDAs require for positive effects on IPDAS criteria should be explored. Key words: decision support techniques; patient education; patient participation; randomized controlled trials. (Med Decis Making 2007;XX:xx–xx) Received 23 July 2007 from the Ottawa Health Research Institute, Canada (AMO, SK, CB); University of Ottawa, Canada (AMO, DS); Massachusetts General Hospital, Boston (MJB); Maine Medical Center, Portland, Maine (NFC); Oregon Health and Science University, Portland (KBE); Social Dimensions of Health Institute, Dundee, UK (VE); Algonquin College, Ottawa, Canada (VF); Michigan State University, East Lansing (MH-R, DR); and Dartmouth Medical School, Hanover, New Hampshire (HL-T). Financial support for this study was provided by a group grant of the Canadian Institutes of Health Research. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, and writing and publishing the report. Address correspondence to Annette M. O’Connor, RN, PhD, University of Ottawa, Ottawa Health Research Institute, 1053 Carling Avenue, ASB, Ottawa, Ontario, Canada K1Y 4E9; e-mail: [email  protected] ca. DOI: 10. 1177/0272989X07307319 P atient decision aids (PtDAs) are adjuncts to counseling that explain options, clarify personal values for the benefits versus harms, and guide patients in deliberation and communication. With the rapid proliferation of these tools, the International Patient Decision Aids Collaboration (IPDAS) has reached agreement on criteria for judging the quality of PtDAs. 1 IPDAS is a network of more than 100 researchers, practitioners, patients, and policy makers from 14 countries. These collaborators developed a checklist of criteria that payers, patients, practitioners, developers, and researchers can use to assess PtDAs they encounter. The criteria address 3 domains of quality: clinical content, the development process, and effectiveness. 554 †¢ MEDICAL DECISION MAKING/MON–MON 2007 Downloaded from mdm. sagepub. com by guest on July 22, 2012 Copyright 2007 by Society for Medical Decision Making.DECISION AIDS â€Å"EFFECTIVENESS CRITERIA OF DECISION AIDS†This article addresses the 3rd domain, the evaluation of PtDAs’ effectiveness in fostering a high-quality decision process and a high-quality choice. Over the past decade, there has been considerable debate about the definition of a good decision when there is no single best therapeutic action and choices depend on how patients value benefits versus harms. 2–6 To select criteria for decision quality, IPDAS participants were asked to identify â€Å"the things that you would nee d to observe in order to say that after using a patient decision aid, the way the decision was made was good and the choice that was made was good. IPDAS endorsed the following criteria for establishing that a decision aid is effective: †¢ Decision quality: The PtDA improves the match between the chosen option and the features that matter most to the informed patient. †¢ Decision processes leading to decision quality: The PtDA helps patients to recognize that a decision needs to be made, know options and their features, understand that values affect the decision, be clear about the option features that matter most, discuss values with their practitioner, and become involved in preferred ways. Our study objectives were 1) to describe the number and types of measures used in randomized controlled trials (RCTs) that correspond to IPDAS criteria for effectiveness and 2) to determine the extent to which RCTs of PtDAs meet these new IPDAS criteria for effectiveness. METHODS We have been updating the Cochrane Review of decision aids since the late 1990s. 7–9 This review differed from previous reviews by focusing on the new IPDAS criteria. Moreover, we used a new systematic review software, TrialStat SRS, to manage the search and data extraction; therefore, our search, screen, and data extraction were redone completely. Data sources included 1) electronic databases to July 2006 (MEDLINE, PsycINFO, CINAHL, and EMBASE), 2) Cochrane Controlled Trials Register (2006, issue 2), and 3) contact with known developers and evaluators to December 2006. The search strategy is described in the appendix. The search was not restricted on the basis of language. PtDAs were defined as interventions designed to help people make specific, deliberated choices among options (including the status quo) by providing information about the options and outcomes (e. . , benefits, DECISION AIDS: PAST, PRESENT, AND FUTURE harms) in sufficient detail that an individual could judge their value implicitly. Patient decision aids may also include information about the clinical condition, outcome probabilities tailored to personal risk factors, an explicit values clarification exercise (e. g. , a relevance chart, utility assessments of probable outcome states, a weigh scale), descriptions of others’ experiences, and guidance in the steps of decision making and communicating with others. This definition excludes interventions focused solely on lifestyle changes, hypothetical situations, clinical trial entry, or general advanced directives; education programs not geared to a specific decision; and interventions designed to promote adherence to a recommended option or to elicit passive informed consent. In the current review, we also excluded studies whose PtDAs were not available for inspection to catalogue their elements according to the new IPDAS domains. As a consequence, a few studies reported in the previous reviews were not included. We included published RCTs comparing 1) PtDAs to usual-care controls or 2) detailed PtDAs to simpler ones (which may not have the level of detail or may not contain all of the IPDAS elements). Participants were deciding about screening or treatment options for themselves, for a child, or for an incapacitated significant other. Two reviewers independently screened each study (CB, SK, DS, AMO, VF), extracted data (CB, SK), and assessed study quality (C. B. , S. K. ) using standardized forms, including the Jadad scale. 0 Inconsistencies were resolved by consensus. Trial results were described individually. Metaanalysis was used for decision quality and for decision process measures because these effects were expected to be independent of the type of decision. Meta-analysis was performed only on those outcomes with similar types of measures. Review Manager 4. 211 was used to estimate a weighted treatment effect (with 95% confidence intervals [CIs]), defined as weighted mean differences ( WMDs) for continuous measures and pooled relative risks (RRs) for dichotomous outcomes. The data used in each meta-analysis can be viewed in the online supplement available at http://mdm. sagepub. com/cgi/ content/full/Volume/Issue/Page#/DC1. All data were analyzed with a DerSimonian and Laird12 random effects model because of the diverse nature of the trials. Forest plots were used to assess and display potential heterogeneity, and funnel plots were used to explore publication bias. Because of statistically significant heterogeneity for most of the outcomes, we performed post hoc subanalyses to explore the potential causes of heterogeneity. Heterogeneity was explored according 555 Downloaded from mdm. sagepub. com by guest on July 22, 2012 O’CONNOR AND OTHERS to the following factors: type of decision (treatment versus screening), type of media of decision aid (video/ computer versus audio booklet/pamphlet), and a possible ceiling effect based on good usual-care scores (removal of studies with lower knowledge and realistic risk perception scores; removal of studies with higher decisional conflict scores for subscales feeling uninformed and unclear values). We analyzed the effects of removing the biggest outlier(s) defined by visual inspection of the forest plots. In addition, a post hoc analysis was performed to examine the effect of 1) excluding trials of low methodological quality and 2) excluding trials that were outliers and contributed to heterogeneity.RESULTSOf the 22,778 unique citations obtained in the review, we identified 1293 as relevant by title and then screened those abstracts (see Figure 1). Of these, 130 citations were retrieved for full-text review. Sixty-four studies were excluded for the following reasons: the study was not focused on making a choice (n = 33), the study was not an RCT (n = 14), the decision support intervention did not meet the definition of a PtDA (n = 8), the study involved a hypothetical situation (n = 6), and no outcome data were provided (n = 3). In all, 55 eligible trials (66 references) were found for duplicate data extraction and analysis. The 55 published RCTs evaluating individual PtDAs13–78 used 51 different PtDAs that focused on 23 different screening or treatment topics (see Table 1). Among the 51 different PtDAs, the elements most frequently included were information about the options and outcomes in sufficient detail to judge their value implicitly (100% by definition), information about the clinical condition (98%), outcome probabilities (84%), examples of others’ experiences (59%), explicit values clarification exercises (55%), and guidance in the steps of decision making (47%). Quality ratings in the trials ranged from 0/5 to 3/5. All studies lost 2 points because patients or practitioners could not be blinded to the intervention. As shown in Table 2, 38 of the 55 trials (69%) reported at least 1 outcome that could be mapped onto an IPDAS criterion for effectiveness; 33 (60%) measured some aspect of decision quality, and 15 (27%) measured a decision process leading to decision quality. Decision Quality As noted above, the definition of decision quality has 2 elements: the extent to which decisions are 556 †¢ MEDICAL DECISION MAKING/MON–MON 2007 informed and based on personal values. Trials used 3 measures corresponding to this definition: knowledge test results, accuracy of risk perceptions, and value congruence with chosen option. Knowledge. Twenty-seven of the 55 studies examined the effects of PtDAs on knowledge; 18 of these compared PtDAs to usual care, and 9 compared PtDAs with more or less detail. The studies’ knowledge tests were based on information contained in the PtDA, thereby establishing content validity. The proportion of accurate responses was transformed to a percentage scale ranging from 0% (no correct responses) to 100% (perfectly accurate responses). In the comparison of PtDAs to usual care15,16,18,26,28,29, 31,36,39,41,43,48,50,64,65,69,73,78 (Figure 2), PtDAs had higher average knowledge scores (WMD = 15. 2%, 95% CI = 11. 7, 18. 7). The 9 studies comparing detailed with simpler PtDAs22,24,30,35,54,60,61,63,66 (Figure 3) showed a smaller effect (WMD = 4. 6%, 95% CI = 3. 0, 6. 2). Accurate risk perceptions. Eleven of 55 studies examined the effects of including probabilities of PtDAs on the accuracy of patients’ perceived probabilities of outcomes. 4,28,41,43–45,54,63,73,74,77 Eight studies measured perceived probabilities as percentages,24,28,43–45,54,73,74 and 3 gauged probabilities in words. 41,63. 77 Perceived outcome probabilities were classified as accurate according to the percentage of individuals whose judgments corresponded to the scientific evidence about the chances of an outcome for similar people. In 4 of 5 studies that elicited perceived probabilities for multiple outcomes,24,44,54,60 the propo rtion of realistic expectations was averaged; in the remaining study,43 the most conservative result was chosen for meta-analysis. People who received a detailed PtDA with descriptions of outcomes and probabilities were more likely to have accurate risk perceptions than those who did not receive this information; the pooled RR of having accurate risk perceptions was 1. 6 (95% CI = 1. 4, 1. 9; Figure 4). The pooled relative risk for probabilities described in words was 1. 3 (95% CI = 1. 1, 1. 5). The pooled relative risk for probabilities described as numbers was 1. 8 (95% CI = 1. 4, 2. 3). Value congruence with chosen option. Four of 55 studies measured value congruence with the chosen option; however, Lerman and others41 did not calculate differences between interventions. The 3 trials comparing interventions were similar in that they 1) focused on the decision to take menopausal hormone replacement therapy (HRT) and 2) compared 2 active interventions. However, these trials used different measures of value (text continued on p 565) Downloaded from mdm. sagepub. com by guest on July 22, 2012 DECISION AIDS â€Å"EFFECTIVENESS CRITERIA OF DECISION AIDS† 2,778 unique citations identified for initial screening (screening based on review of the title) 1,293 potentially relevant citations identified and screened (based on review of the abastract) for retrieval 130 citations retrieved for full-text review 64 excluded: study not focused on making a choice (n = 33); study was not RCT (n = 14); decision support intervention did not meet the definition of a PtDA (n = 8); study involved a hypothetical situatio n (n = 6); no outcome data provided (n = 2); protocol only (n = 1) 5 eligible trials (66 references) for duplicate data extraction Data entry & RCT meta- analysis Figure 1 Flowchart of the procedural steps in the systematic review. RCT = randomized controlled trial; PtDA = patient decision aid. DECISION AIDS: PAST, PRESENT, AND FUTURE Downloaded from mdm. sagepub. com by guest on July 22, 2012 557 Table 1 Elements in DAs Characteristics of 55 Trials Included in the Systematic Review of Patient Decision Aids 558 Number of Enrollees in Intervention + Comparison: Options Considered Quality Rating (Jadad) Others’ Experiences Comparison of Most and Least Intensive Intervention Options and Clinical Outcomes Problem Explicit Outcome Values Probability Clarification Guidance in Steps of DM Source, Year, Location Auvinen and others13,14 2004, Finland 3/5 1/5 2/5 3/5 2/5 2/5 — X — X X X — — X X X — — X — X X — — X X — — X — X — — — X — — X X X — X — — — X — — — — — X — — — X — — — X — — — — — — — — X — — Barry and others15 1997, United States Bekker and others,16,17 2004, United Kingdom Bernstein and others18 1998, United States Pamphlet PtDA Standard care by clinical guideline Interactive videodisc PtDA Usual care Decision analysis plus consultation Usual care Video PtDA Usual care Clancy and others19 1988, United States 30 + 30 men: prostate cancer treatment 67 + 61 women: HRT 1/5 — X X X X X X X X X — X X X X X 103 + 100 men: prostate cancer treatment 104 + 123 men: benign prostate hypertrophy treatment 59 + 58 women: prenatal diagnostic screening for Down syndrome 65 + 53 patients: ischemic heart disease treatment 753 + 263 physicians: hepatitis B vaccine Davison and Degner20 1997, Canada Deschamps and others21 2004, Canada 3/5 — X X X — X — X — X — — — X — X — X — X — X — — — X — — — X — X Downloaded from mdm. sagepub. com by guest on July 22, 2012 Deyo and others22 2000; Phelan and others23 2001, United States 2/5 Dodin and others24 2001, Canada 50 + 47 adults: colon cancer screening 3/5 2/5 3/5 143 + 144 parents: infant polio vaccine schedules 112 + 114 men: PSA testing 190 + 203 patients: herniated disc or spinal stenosis treatment 52 + 49 women: HRT Pamphlet + decision analysis PtDA Usual care Written materials, PtDA, and audiotape of consultation Usual care Audiotape and booklet Pharmacist consultation Interactive videodisc PtDA Simple PtDA pamphlet Audiotape booklet PtDA Simple PtDA pamphlet Dolan and Frisina25 2002, United States Dunn and others26 1998, United States Frosch and others27 2003, United States — X — X X X X X X X — X — X X — — — — — — X — X X — — — — — Computer: analytic hierarchy process and pamphlet PtDA Usual care Video and pamphlet PtDA Usual care Video PtDA Internet presentation mirroring content of video continued) Gattellari and Ward28 2003, Australia 3/5 3/5 3/5 X X — X — X — X X — X X — X — X X — — X — — — — — X X — — — X — — X — — — X — X â⠂¬â€ X — X — X X X X X X — X — X — X — — — — — X — — — — — X — X — X — X — X — X — — — — — 140 + 140 men: PSA testing 86 + 50 women: breast cancer surgery 1/5 126 + 122 men: PSA testing Gattellari and Ward29 2005, Australia Goel and others30 2001, Canada Green and others31 2001, United States 3/5 CD-ROM PtDA plus counseling Genetic counseling Pamphlet PtDA Usual care Pamphlet PtDA General information leaflet Pamphlet PtDA General information leaflet Audiotape and booklet PtDA Simple PtDA pamphlet CD-ROM PtDA plus counseling Usual care Green and others32,33 2004, United States 0/5 2/5 Herrera and others34 1983, United States Hunter and others35 2005, Canada 3/5 3/5 2/5 3/5 Audiotape and booklet PtDA Usual care Audiotape and booklet Individual genetic counseling Decision board PtDA Usual care Video plus booklet PtDA Usual care Booklet PtDA Personal risk profile X — X — X — X — X X X — 2/5 X X X — X — X X X — X X X — X X 9 + 14 higher risk women: breast cancer genetic testing 106 + 105 higher risk women: breast cancer genetic testing 56 + 47 parent(s): circumcision of male newborns 116 + 126 women: prenatal diagnostic testing X — X — X X X — X X X — X X — — X — X — X — X — X — — — Downloaded from m dm. sagepub. com by guest on July 22, 2012 Johnson and others36 2006, United States Kennedy and others37 2002, United Kingdom Lalonde and others38 2006, Canada — — X — X — X — X — X — — — — — X — X — X — X — X — X — (continued) Laupacis and others39 2006, Canada 2/5 Legare and others40 2003, Canada 122 + 164 women: breast cancer genetic testing 100 + 101 women: prenatal diagnostic testing 1/5 32 + 35 patients: dental surgery 300 + 298 women: menorrhagia treatment 13 + 13 patients: cardiovascular health treatment 60 + 60 patients: preoperative autologous blood donation 97 + 87 women: HRT erman and others41 1997, United States Leung and others42 2004, China Audiotape booklet PtDA Simple pamphlet PtDA Discussion PtDA and counseling Usual care wait list control Interactive multimedia PtDA Video and pamphlet 559 Table 1 Elements in DAs continued) 560 Number of Enrollees in Intervention + Comparison: Options Considered Quality Rating (Jadad) Others’ Experiences Comparison of Most and Least Intensive Intervention Options and Clinical Outcomes Problem Explicit Outcome Values Probability Clarification Guidance in Steps of DM Source, Year, Location Man-Son-Hing and others43 1999, Canada — X X X X — — — 3/5 1/5 2/5 X X X X X X — X — X — X — X — X — X — X — X — X — X — — — X — X 3/5 Audiotape and booklet PtDA Usual care X X X X X — X — X — — — — — — X — X — X — — — — — — McAlister and others44 2005, Canada McBride and others45,46 2002, United States Miller and others47 2005, United States 139 + 148 aspirin users in atrial fibrillation trial: move to warfarin 219 + 215 patients: antithrombotic therapy 289 + 292 women: HRT 279 women: BRCA1 BRCA2 gene testing Montgomery and others48,49 2003, United Kingdom 52 + 55 + 51 + 59 adults: hypertension treatment 3/5 Downloaded from mdm. sagepub. com by guest on July 22, 2012 Morgan and others50 2000, Canada 3/5 3/5 — X — X — X — X X — X — X — X — X X — X — X — X — — — — — — — — — — X — — X — X — X — — — — — — — — — — X — Murray and others51 2001, United Kingdom 3/5 2/5 Murray and others52 2001, United Kingdom Myers and others53 2005, United States 121 + 121 men: PSA testing 120 + 120 patients: ischemic heart disease treatment 57 + 55 men: benign prostate hypertrophy treatment 102 + 102 women: HRT Audiotape and booklet PtDA Usual care Pamphlet PtDA Usual care Discussion PtDA and general information pamphlets General information pamphlets Decision analysis PtDA Video and booklet PtDA Decision analysis, video and booklet PtDA Standard care Interactive videodisc PtDA Usual care Interactive videodisc PtDA Usual care O’Connor and others54 1998, Canada 81 + 84 women: HRT 1/5 X X X X X — X — X — X — Interactive videodisc PtDA Usual care Discussion PtDA and general information pamphlet General information pamphlet Audiotape and booklet PtDA Simple PtDA pamphlet O’Connor and others55 1999, Canada 3/5 X X X X X X 1/5 3/5 0/5 — X — X — X — X — X — — — — — — — X — — X X X X X X — X X X X X — — — X 16 + 17 women: osteoporosis treatment 384 + 384 men: PSA testing 37 + 37 patients: dental orthognathic surgery 3/5 3/5 X X X — X — X — X — — — — — — — — X — X 101 +100 women: HRT Oakley and Walley56 2006, United Kingdom Partin and others57 2004, Canada Phillips and others58 1995, United States Pignone and others59 2000, United States Audiotape and booklet PtDA DA without explicit values clarification Audiotape and booklet PtDA Usual care Video PtDA Usual care Video imaging of facial reconstruction PtDA Usual care Video PtDA Usual care — — — X Rostom and others60 2002, Canada X X X X X X — X X X X X X X X X — X — — — 125 + 124 adults: colon cancer screening 25 + 26 women: HRT X X X X — — — — — — — — — — X X — — — — — 83 + 89 women: HRT 1/5 Computer PtDA with testing + feedback regarding knowledge Audiotape with booklet Lecture with personal decision exercise PtDA Simple PtDA pamphlet Booklet PtDA Simple PtDA pamphlet Booklet PtDA Usual care Rothert and others61 1997; Holmes-Rovner and others62 1999, United States Schapira63 2000, United States 1/5 2/5 Downloaded from mdm. sagepub. com by guest on July 22, 2012 Schwartz and others64 2001, United States 2/5 Booklet PtDA Usual care Shorten and others65 2005, Australia X — X — X — X — — — X — Street and others66 1995, United States 1/5 22 + 135 men: prostate cancer screening 191 + 190 Ashkenazi Jewish women: breast cancer genetic testing 85 + 84 pregnant women: birthing options after previous cesarean delivery 30 + 30 women: breast cancer surgery Interactive multimedia PtDA Simple PtDA X X X X — — — — X — X — (continued) 561 562 Table 1 Elements in DAs (continued) Source, Year, Locat ion Number of Enrollees in Intervention + Comparison: Options Considered Quality Rating (Jadad) Comparison of Most and Least Intensive Intervention Options and Clinical Outcomes Problem Explicit Outcome Values Probability Clarification Others’ Experiences Guidance in Steps of DM VanRoosmalen and others67,68 2004, the Netherlands X X — X — X — X — X — X — X — X X — — X — X — X — — X — — X — X — X — X — X — X — X X X X — — — — — — — — — — — — — X — 44 + 44 women with BRCA1/2 mutation: prophylactic surgery 3/5 X X X X — — X — — — — — — — X — — — — — X X — — — — — — — — — — — — — — Volk and others69,70 1999, United States 3/5 3/5 3/5 80 + 80 men: prostate cancer screening Vuorma and others71,72 2003, Finland Video and brochure PtDA with decision analysis Same video and brochure PtDA pamphlet Video with pam phlet PtDA Usual care Booklet PtDA Usual care Whelan and others73 2003, Canada 3/5 2/5 184 + 179 women: menorrhagia treatment 82 + 93 women: breast cancer chemotherapy Downloaded from mdm. sagepub. com by guest on July 22, 2012 Whelan and others74 2004, Canada Wolf and others75,76 1996, United States 1/5 2/5 Script PtDA Usual care Pamphlet PtDA Usual care 94 + 107 women: breast cancer surgery 103 + 102 men: prostate cancer screening Decision board PtDA and booklet Usual care with booklet Decision board PtDA Usual care Script PtDA Usual care Wolf and Schorling77 2000, United States Wong and others78 2006, United States 266 + 133 seniors: colon cancer screening 162 + 164 women: pregnancy termination Note: DM = decision making; PtDA = patient decision aid; HRT = hormone replacement therapy; PSA = prostate-specific antigen. Table 2 Cumulative Studies Still in 2007 Review Reporting Outcome in Each Cochrane Review Update Year % n/N Lead Author 15 50 18 Trials Measuring Outcomes That Map onto the International Patient Decision Aid Standards (IPDAS) Criteria Outcome Decision quality 2007 1999 2003 15 27 2/13 8/30 49 27/55 Knowledge scores 999 2003 54 57 7/13 17/30 Realistic expectations, accurate risk perceptions Barry, Morgan, Bernstein, Lerman,41 Rothert,61 O’Connor,54 Street66 As above plus Schwartz,64 Man-Son-Hing,43 Volk,69 Dunn,26 Green,31 Goel,30 Shapira,63 Rostom,60 Phelan,23 Dodin24 As above plus Bekker,16 Gattellari,28 Johnson,36 Whelan,73 Shorten,65 Montgomery,48 Gattellari,29 Laupacis,39 Wong,78 Hunter35 OConnor,54 Lerman41 As above plus Wolf,77 McB ride,45 Man-Son-Hing,43 Rostom,60 Shapira,63 Dodin24 As above plus Whelan,74 Whelan,73 McAlister,44 Gattellari28 Value congruence with chosen option Decisional Conflict Scale (DCS) 2007 1999 2003 2007 1999 2003 2007 80 2007 1999 2003 2007 1999 2003 57 15 30 27 15 33 24 15/55 2/13 10/30 13/55 2/13 9/30 17/30 Downloaded from mdm. sagepub. com by guest on July 22, 2012 22 0 10 5 15 30 12/55 0/13 3/30 3/55 2/13 9/30 Decision process leading to decision quality Feeling informed, subscale of the DCS Feeling clear about values, subscale of DCS O’Connor,55 Holmes-Rovner,62 Dodin24 As above OConnor,54 Morgan50 As above plus Murray,51 Murray,52 Dolan,25 Man-Son-Hing,43 Dodin,24 Goel,30 OConnor55 As above plus Montgomery,48 Shorten,65 Laupacis,39 Whelan,74 McAlister,44 Lalonde,38 Legare,40 Hunter35 O’Connor,54 Morgan50 As above plus Murray,51 Murray,52 Dolan,25 Man-Son-Hing,43 Dodin,24 Goel,30 OConnor55 As above plus Montgomery,48 Laupacis,39 McAlister,44 Wong,78 Bekker,16 Lalonde38 O’Connor,54 Morgan50 As above plus Murray,51 Murray,52 Dolan,25 Man-Son-Hing,43 Dodin,24 Goel,30 OConnor55 As above plus Montgomery,48 Laupacis,39 McAlister,44 Lalonde38 Note: Trials included in 1999 and 2003 but not in 2007 are Davison and others (measuring feeling informed, clear values); Maisels and others,81 Michie and others82 (measuring knowledge scores), and Thornton and others. 83 These authors were eliminated because we were unable to verify what was in their decision aid to meet the IPDAS definition of a decision aid. 563 O’CONNOR AND OTHERS Study or subcategory y Bekker 2004 Gattellari 2003 Johnson 2006 Whelan 2003 Schwartz 2001 Man-Son-Hing 1999 Morgan 2000 Shorten 2005 Montgomery 2003 Gattellari 2005 Laupacis 2006 Volk 1999 Lerman 1997 Barry 1997 Wong 2006 Bernstein 1998 Dunn 1998 Green 2001 N Decision Aid Mean (SD) 74. 00(14. 50) 50. 00(18. 40) 92. 60(11. 00) 80. 20(14. 40) 65. 71(14. 29) 75. 91(15. 72) 76. 00(32. 04) 75. 33(15. 00) 75. 00(17. 00) 57. 20(21. 30) 83. 00(19. 50) 48. 00(22. 40) 68. 90(19. 00) 75. 00(45. 00) 85. 00(26. 70) 83. 00(16. 00) 83. 67(23. 13) 95. 00(7. 00) N Usual Care Mean (SD) 71. 50(16. 00) 45. 00(15. 90) 85. 20(15. 60) 71. 70(13. 30) 57. 14(15. 71) 66. 46(16. 07) 62. 00(32. 04) 60. 53(17. 07) 60. 00(18. 00) 42. 20(16. 70) 67. 40(17. 00) 31. 00(18. 30) 49. 00(21. 70) 54. 00(45. 00) 60. 00(21. 70) 58. 00(16. 00) 55. 53(22. 80) 65. 00(21. 00) W MD (random) 95% CI W eight % 5. 68 6. 3 5. 49 6. 16 6. 41 6. 24 4. 61 6. 04 5. 43 6. 03 5. 32 5. 50 6. 00 3. 84 5. 81 5. 61 5. 83 3. 97 100. 00 W MD (random) 95% CI 2. 50 [-3. 31, 8. 31] 5. 00 [0. 39, 9. 61] 7. 40 [0. 98, 13. 82] 8. 50 [4. 37, 12. 63] 8. 57 [5. 55, 11. 59] 9. 45 [5. 68, 13. 22] 14. 00 [4. 81, 23. 19] 14. 80 [10. 23, 19. 37] 15. 00 [8. 39, 21. 61] 15. 00 [10. 40, 19. 60] 15. 60 [8. 64, 22. 56 ] 17. 00 [10. 61, 23. 39] 19. 90 [15. 17, 24. 63] 21. 00 [9. 25, 32. 75] 25. 00 [19. 60, 30. 40] 25. 00 [18. 95, 31. 05] 28. 14 [22. 83, 33. 45] 30. 00 [18. 71, 41. 29] 15. 22 [11. 71, 18. 73] 50 106 32 82 191 137 90 99 50 131 53 78 122 104 154 61 143 29 6 108 35 93 190 136 97 92 58 136 53 80 164 123 159 48 144 14 Total (95% CI) 1712 1786 Test for heterogeneity: ? 2 = 130. 32, df = 17 (P ; 0 . 00001), I? = 87. 0% Test for overall effect: Z = 8. 50 (P ; 0. 00001) -50 Favors Usual Care 0 50 Favors Decision Aid Figure 2 Effect of patient decision aids on patients’ mean scores on knowledge tests: decision aid versus usual care. WMD = weighted mean difference; CI = confidence interval. Study N Goel 2001 Rothert / H-Rovner O'Connor 1998-RCT Hunter 2005 Schapira 2000 Street 1995 Rostom 2002 Deyo / Phelan Dodin 2001 Total 77 83 81 116 122 30 25 41 52 627 Detailed DA Mean (SD) 81. 67(11. 1) 86. 79(11. 34) 75. 00(20. 00) 64. 53(19. 61) 83. 33(12. 78) 82. 60(11. 60) 93. 80(9. 00) 71. 76 (17. 06) 71. 04(15. 45) N Simple DA Mean (SD) 80. 00(12. 22) 83. 75(11. 54) 71. 00(21. 00) 60. 13(19. 00) 78. 33(15. 00) 76. 40(13. 80) 87. 10(11. 80) 62. 35(23. 53) 61. 20(17. 90) WMD (random) 95% CI Weight % 14. 31 21. 90 6. 62 10. 92 22. 45 6. 23 7. 85 3. 67 6. 06 100. 00 WMD (random) 95% CI 1. 67 [-2. 59, 5. 93] 3. 04 [-0. 40, 6. 48] 4. 00 [-2. 26, 10. 26] 4. 40 [-0. 47, 9. 27] 5. 00 [1. 60, 8. 40] 6. 20 [-0. 25, 12. 65] 6. 70 [0. 95, 12. 45] 9. 41 [1. 00, 17. 82] 9. 84 [3. 30, 16. 38] 4. 63 [3. 02, 6. 24] 48 87 84 126 135 30 26 49 49 634 Test for heterogeneity: ? 2 = 7. 18, df = 8 (P = 0. 52 ), I? = 0% Test for overall effect: Z = 5. 63 (P < 0. 00001) -50 Favours Simple 0 50 Favours Detailed Figure 3 Effect of patient decision aids (DAs) on patients’ mean scores on knowledge tests: detailed versus simple decision aids. WMD = weighted mean difference; CI = confidence interval. 564 †¢ MEDICAL DECISION MAKING/SEP–OCT 2007 Downloaded from mdm. sagepub. com by guest on July 22, 2012 DECISION AIDS â€Å"EFFECTIVENESS CRITERIA OF DECISION AIDS† Study or subcategory y Decision Aid n/N 90/122 189/266 73/94 109/265 82/122 33/52 58/81 47/82 70/187 88/139 57/106 1516 Usual Care n/N 108/164 72/133 62/107 82/274 62/135 21/49 39/84 34/92 27/165 40/148 11/108 1459 RR (random) 95% CI W eight % 11. 11 10. 83 10. 55 10. 01 10. 16 7. 66 9. 45 8. 54 7. 54 9. 04 5. 12 100. 00 RR (random) 95% CI 1. 12 [0. 96, 1. 31] 1. 31 [1. 10, 1. 56] 1. 34 [1. 10, 1. 63] 1. 37 [1. 09, 1. 73] 1. 46 [1. 17, 1. 83] 1. 48 [1. 01, 2. 17] 1. 54 [1. 18, 2. 02] 1. 55 [1. 12, 2. 15] 2. 29 [1. 55, 3. 38] 2. 34 [1. 75, 3. 14] 5. 28 [2. 93, 9. 50] 1. 61 [1. 35, 1. 92] Lerman 1997 Wolf 2000 Whelan 2004 McBride 2002 Schapira 2000 Dodin 2001 O'Connor 1998-RCT Whelan 2003 McAlister 2005 Man-Son-Hing 1999 Gattellari 2003 Total (95% CI) Total events: 896 (Decision Aid), 558 (Usual Care) Test for heterogeneity: ? 2 = 52. 06, df = 10 (P ; 0. 00001), I? = 80. 8% Test for overall effect: Z = 5. 34 (P ; 0. 00001) 0. 1 0. 2 0. 5 1 Favours Usual Care 2 5 10 Favours Decision Aid Figure 4 Effect of patient decision aids on the proportion of patients classified as having accurate risk perceptions. RR = relative risk; CI = confidence interval. congruence. Holmes-Rovner and others62 measured the correlation between the subjective expected value of hormones and women’s likelihood of taking HRT, converted here to the percentage of variance in likelihood explained by alues. Dodin and others24 measured the percentage of variance in decisions explained by values. O’Connor and others55 used logistic regression to estimate the percentage agreement between values and choice. PtDAs improved value congruence with the chosen option in 2 of 3 studies. In the trial by Dodin and others,24 24% of the variance in HRT decisions wa s explained by personal values when a detailed PtDA with explicit values clarification was used; in contrast, 14% of the variance in decisions was explained when a simpler PtDA was used (P = 0. 003). In the study by Holmes-Rovner and others,62 the percentage of variance in the likelihood of choosing HRT that was explained by women’s expected values was greater when a more detailed PtDA was used (13%–14%) than when a simpler PtDA was used (0. 09%–2%). O’Connor and others55 found that the addition of an explicit values clarification exercise in a PtDA did not improve agreement between values and the chosen option. However, in the subgroup of women who chose HRT, women who used the PtDA with explicit values clarification DECISION AIDS: PAST, PRESENT, AND FUTURE ad a trend toward better agreement (40%) than did those who used an identical PtDA without explicit values clarification (0%, P = 0. 06). Decision Processes Leading to Decision Quality There were no trials evaluating the extent to which PtDAs helped patients to recognize that a decision needs to be made, understand that values affect the decision, and discuss values with their practitioner. Althoug h 8 trials evaluated effects on patient participation, none focused on helping patients become involved in preferred ways. Some studies measured patients’ self-reports about feeling informed and clear about personal values. The measures used to evaluate these 2 criteria were 2 subscales of the Decisional Conflict Scale (DCS). The DCS is reliable, discriminates between those who make or delay decisions, is sensitive to change, and discriminates between different decision support interventions. 54,79 The scores are standardized to range from 0 (no decisional conflict) to 100 points (extreme decisional conflict). Scores of 25 or lower are associated with follow through with decisions, whereas scores that exceed 38 are associated with delay in decision making. 54 When PtDAs are compared with usual care, 565 Downloaded from mdm. agepub. com by guest on July 22, 2012 O’CONNOR AND OTHERS Study or subcategory y Decision Aid N Mean (SD) 22. 17(9. 47) 27. 56(10. 51) 16. 25(13. 75) 29. 93(17. 26) 15. 75(13. 00) 20. 00(21. 50) 15. 75(13. 25) 15. 00(12. 50) 21. 67(15. 83) 32. 50(15. 00) Usual Care N Mean (SD) 58 45 54 93 37 94 148 215 159 56 959 49. 14(25. 40) 38. 88(20. 02) 27. 25(15. 00) 38. 89(22. 53) 24. 50(21. 25) 27. 50(21. 50) 21. 00(14. 75) 20. 00(15. 00) 25. 83(19. 17) 31. 67(14. 17) WMD (random) 95% CI Weight % 8. 64 9. 09 10. 03 9. 73 7. 93 9. 28 11. 82 12. 25 11. 33 9. 90 100. 00 WMD (random) 95% CI -26. 97 [-34. 1, -19. 93] -11. 32 [-17. 83, -4. 81] -11. 00 [-16. 43, -5. 57] -8. 96 [-14. 73, -3. 19] -8. 75 [-16. 67, -0. 83] -7. 50 [-13. 79, -1. 21] -5. 25 [-8. 49, -2. 01] -5. 00 [-7. 60, -2. 40] -4. 16 [-8. 05, -0. 27] 0. 83 [-4. 74, 6. 40] -8. 35 [-11. 89, -4. 80] 02 Uninformed Subscale Montgomery 2003 50 Murray BPH 2001 52 Laupacis 2006 54 Murray HRT 2001 93 Dolan 2002 41 Morgan 2000 86 Man-Son-Hing 1999 139 McAlister 2005 219 Wong 2006 154 Bekker 2004 50 Subtotal (95% CI) 938 Test for heterogeneity: 48. 12, df = 9 (P ; 0. 00001), I? = 81. 3% Test for overall effect: Z = 4. 61 (P ; 0. 0001) -50 0 Favours Decision Aid 50 Favours Usual Care ?2 = Figure 5 Effect of patient decision aids on patients’ scores on the Uninformed subscale of the Decisional Conflict Scale: d ecision aid versus usual care. WMD = weighted mean difference; CI = confidence interval. Study or subcategory y Detailed DA N Mean (SD) 22. 50(17. 50) 17. 50(12. 50) 20. 75(10. 75) 22. 50(17. 50) 38. 25(12. 00) Simple DA N Mean (SD) 84 49 45 100 12 27. 50(20. 00) 22. 25(14. 75) 24. 00(16. 00) 20. 00(17. 50) 31. 25(10. 75) W MD (random) 95% CI W eight % 20. 52 21. 72 22. 00 23. 42 12. 34 100. 00 W MD (random) 95% CI -5. 0 [-10. 73, 0. 73] -4. 75 [-10. 10, 0. 60] -3. 25 [-8. 51, 2. 01] 2. 50 [-2. 34, 7. 34] 7. 00 [-2. 12, 16. 12] -1. 32 [-5. 27, 2. 62] 02 Uninformed Subscale O'Connor 1998-RCT 81 Dodin 2001 52 Goel 2001 76 O'Connor Wells 1999 101 Lalonde 2006 12 Subtotal (95% CI) 322 290 Test for heterogeneity: ? 2 = 9. 24, df = 4 (P = 0. 06), I? = 56. 7% Test for overall effect: Z = 0. 66 (P = 0. 51) -50 0 Favours Detailed DA 50 Favours Simple DA Figure 6 Effect of patient decision aids on patients’ scores on the Uninformed subscale of the Decisional Conflict Scale: detailed ve rsus simple decision aid (DA). WMD = weighted mean difference; CI = confidence interval. a negative score indicates a reduction in decisional conflict, which is in favor of the PtDA. In our review, 15 trials used the DCS subscale for feeling informed and 13 trials used the DCS subscale for feeling clear about values. Because this DCS subscale measures self-reported comfort with knowledge and not actual knowledge, we elected to consider it a process measure and to reserve the gold standard of objective knowledge tests in assessing decision quality. The WMD in feeling uninformed about options, benefits, and harms was –8. (95% CI = –11. 9 to –4. 8) in the 10 trials16,25,39,43,44,48,50–52,78 that compared the PtDAs to usual care (Figure 5). The 5 trials that compared detailed with simpler PtDAs24,30,38,54,55 had a WMD in feeling uninformed of –1. 3 (95% CI = –5. 3 to 2. 6; Figure 6). Eight trials comparing PtDA to usual care25,39,43,44,48,50–52 had a WMD of –6. 3 (95% CI = –10. 0, –2. 7) for feeling clear about values (Figure 7). Five trials compared detailed to simpler PtDAs. 24,30,38,54,55 For these trials, the WMD in feeling clear about values was –1. 1 (95% CI = –4. 8 to 2. ; Figure 8). 566 †¢ MEDICAL DECISION MAKING/MON–MON 2007 Downloaded from mdm. sagepub. com by guest on July 22, 2012 DECISION AIDS â€Å"EFFECTIVENESS CRITERIA OF DECISION AIDS† Study or sub-category Decision Aid N Mean (SD) 50 54 41 82 53 139 219 86 724 28. 50(12. 50) 18. 75(16. 50) 19. 75(15. 75) 37. 50(15. 00) 35. 38(12. 33) 16. 25(12. 50) 15. 00(12. 50) 30. 00(3. 25) Usual Care N Mean (SD) 58 55 37 84 45 148 215 94 736 51. 29(25. 73) 30. 00(17. 00) 29. 25(24. 00) 42. 85(16. 57) 40. 56(16. 44) 19. 00(14. 75) 17. 50(15. 00) 30. 00(3. 25) WMD (random) 95% CI Weight % 9. 8 11. 11 8. 15 12. 88 11. 64 14. 75 15. 30 16. 40 100. 00 WMD (random) 95% CI -22. 79 [-30. 26, -15. 32] -11. 25 [-17. 54, -4. 96] -9. 50 [-18. 61, -0. 39] -5. 35 [-10. 16, -0. 54] -5. 18 [-11. 02, 0. 66] -2. 75 [-5. 91, 0. 41] -2. 50 [-5. 10, 0. 10] 0. 00 [-0. 95, 0. 95] -6. 33 [-9. 98, -2. 69] 03 Unclear Values Subscale Montgomery 2003 Laupacis 2006 Dolan 2002 Murray HRT 2001 Murray BPH 2001 Man-Son-Hing 1999 McAlister 2005 Morgan 2000 Subtotal (95% CI) Test for heterogeneity: 57. 71, df = 7 (P ; 0. 0 0001), I? = 87. 9% Test for overall effect: Z = 3. 40 (P = 0. 007) -50 Favours Decision Aid 0 50 Favours Usual Care ?2 = Figure 7 Effect of patient decision aids on patients’ scores on the Unclear Values subscale of the Decisional Conflict Scale: decision aid versus usual care. WMD = weighted mean difference; CI = confidence interval. Study or sub-category y Detailed DA N Mean (SD) 81 77 52 12 97 25. 00(17. 50) 24. 00(12. 50) 25. 00(13. 75) 39. 50(10. 75) 22. 50(15. 00) N 84 45 49 12 100 Simple DA Mean (SD) 32. 50(17. 50) 25. 75(15. 75) 24. 75(13. 50) 37. 50(13. 00) 20. 00(15. 00) W MD (random) 95% CI W eight % 21. 23 21. 09 21. 32 10. 94 25. 42 100. 0 W MD (random) 95% CI -7. 50 [-12. 84, -2. 16] -1. 75 [-7. 13, 3. 63] 0. 25 [-5. 07, 5. 57] 2. 00 [-7. 54, 11. 54] 2. 50 [-1. 69, 6. 69] -1. 05 [-4. 81, 2. 70] O'Connor 1998-RCT Goel 2001 Dodin 2001 Lalonde 2006 O'Connor Wells 1999 Subtotal (95% CI) 319 290 Test for heterogeneity: ? 2 = 9. 02, df = 4 (P = 0. 06 ), I? = 55. 7% Test for overall effect: Z = 0. 55 (P = 0. 58) -50 0 Favours Detailed DA 50 Favours Simple DA Figure 8 Effect of patient decision aids on patients’ scores on the Unclear Values subscale of the Decisional Conflict Scale: detailed versus simple decision aid (DA). WMD = weighted mean difference; CI = confidence interval. Post hoc Analysis Effects of study quality. To examine the effect of possible bias from including trials of low methodological quality, the 13 trials15,21,31,34. 41,45,54,56,58,61,63,66,77 with Jadad scores of 0 or 1 were excluded from the analysis. Overall, the results remained the same. There was a significant improvement in knowledge scores for the comparison of PtDAs to usual-care controls (WMD = 14. 0%, 95% CI = 2. 4, 8. 6) and for the comparison of detailed to simpler PtDAs (WMD = 5. 5%, 95% CI = 2. 4, 8. 6). The proportion of patients having accurate risk perceptions was greater for patients receiving PtDAs with information on outcome probabilities (RR = 2. 0, 95% CI = 1. 4, 2. 8). Publication bias. There were too few studies to explore potential publication bias for all of the outcomes, with the exception of knowledge for the comparison of PtDAs to usual care. The funnel plot for this outcome (Figure 9) points to the absence of smaller negative studies. DECISION AIDS: PAST, PRESENT, AND FUTURE Downloaded from mdm. sagepub. com by guest on July 22, 2012 567 O’CONNOR AND OTHERS Comparison: 01 Decision Aids versus Usual Care Outcome: 0 07 Knowledge: Decision Aids vs Usual Care SE(WMD) 2 4 6 8 -100 -50 0 50 100 WMD (fixed) Figure 9 Funnel plot of all 18 randomized controlled trials comparing patient decision aids to usual care (knowledge). WMD = weighted mean difference. Heterogeneity. There was statistically significant heterogeneity when PtDAs were compared with usual care for 4 outcomes: knowledge test scores, realistic risk perceptions, feeling uninformed, and feeling unclear regarding personal values (Table 3). It should be noted that the heterogeneity of the effect was not in the direction but in the size. When we explored the potential factors contributing to heterogeneity (Table 3), we found that none of the factors eliminated heterogeneity for the outcomes of knowledge scores. When grouped into treatment and screening decisions, the WMD for knowledge scores was slightly higher for the treatment group (16. 6% v. 13. 1%), but there was still statistically significant heterogeneity. For the outcomes of accurate risk perceptions, heterogeneity was not significant when we removed 3 studies with lower accurate risk perception scores in the usual-care control group (P = 0. ). 28,43,44 For the outcome of feeling uninformed, heterogeneity was no longer significant with 1) removal of 3 studies with higher uninformed scores in the usual-care control group (P = 0. 11), 2) inclusion of only audio booklet/ pamphlet decision aids (P = 0. 06), and 3) removal of an outlier48 (P = 0. 06). None of the factors eliminated heterogeneity for the outcomes of unclear values scores. DISCUSSION The majority o f trials report on at least 1 IPDAS effectiveness measure, predominately knowledge test scores. Of those reporting IPDAS measures, we found that PtDAs were superior to usual practices in 568 †¢ MEDICAL DECISION MAKING/SEP–OCT 2007 meeting the new IPDAS standards 1) for decision quality and 2) for 2 process measures (feeling informed and feeling clear about personal values). Detailed PtDAs had superior effects over simpler PtDAs on value congruence with the chosen option and on accurate risk perceptions but not on knowledge test scores or on self-reports about feeling informed and feeling clear about values. We also identified the gaps in the use of measures of effectiveness endorsed by IPDAS, notably, value congruence with the chosen option and most of the decision process measures. There are some study limitations. Study quality ratings of all trials included in the review were low because they all lost 2 points for lack of blinding. Although not an a priori exclusion criterion for this review, in the future, we may consider using study quality ratings for the selection of included trials. The conclusions of this review are limited by 1) inadequate power to detect important differences in effectiveness in subgroups and 2) the wide variability in the decision contexts, the elements within the PtDAs, the type of comparison interventions, the targeted outcomes, and the evaluation procedures. This article focuses solely on measures of effectiveness, not harms. The small number of studies for most outcomes did not allow for analysis of publication bias because of the failure to publish negative studies. Moreover, there may have been publication bias because of failure to report all negative findings in a published study. Lastly, several of the outcomes demonstrated statistically significant heterogeneity. It reflects differences across clinically diverse studies; therefore, the pooled effect size and CI should be interpreted as a range across conditions, which may not be applicable to a specific condition. There are several implications for future research. Studies are needed to evaluate the effects of PtDAs on congruence between values and chosen options. Moreover, the methods for quantifying value congruence should be explored. The IPDAS decision processes criteria leading to decision quality should also be measured. It would be helpful to develop a standardized approach to measurement. With the addition of more trials to the database, it may be possible to tease out the reason for heterogeneity of results, including variability in 1) study quality, 2) comparison intervention, 3) elements within PtDAs, 4) decision type, and 5) format of decision aid (e. g. , video, Internet, booklet). The degree of detail in PtDAs that is required for positive effects on IPDAS criteria should also be explored. Downloaded from mdm. sagepub. com by guest on July 22, 2012 Table 3 Exploration of Potential Factors Affecting Heterogeneity Outcome Overall Effect Treatment Decision Screening Decision Video/Computer Decision Aid Audio/Pamphlet Decision Aid Baseline Risk in Usual-Care Groupa Removal of Outliers Knowledge 1. 6 (1. 4, 1. 9) –3. 5 (–12. 9, 5. 8) 1. 6 (1. 1, 2. 3) No data 15. 2 (11. 7, 18. 7) 16. 6 (12. 0, 21. 2) 13. 1 ( 7. 7, 18. 5) 21. 4 (16. 5, 26. 2) 11. 9 (8. 3, 15. 6) 1. 6 (1. 4, 1. 9) 15. 6 (11. 3, 19. 9) 1. 3 (1. 2,1. 5)* 1. 6 (1. 4, 1. 9) 17. 316,28,36 (13. 7, 20. 9) 1. 528 (1. 3, 1. 7) –8. 4 (–11. 9, –4. 8) –9. 4 (–13. 3 –5. 5) 12. 6 (–19. 5, –5. 8) –4. 9 (–7. 6, –2. 3)*** –5. 4 (–7. 7, –3. 2)** –6. 248 (–8. 4, –4. 1)*** –8. 0 (–15. 1, –1. 0) –4. 5 (–8. 4, –0. 6) –3. 6 (–6. 8, –0. 5) –4. 0 r48 (–6. 7, –1. 3) Downloaded from mdm. s agepub. com by guest on July 22, 2012 Accurate risk perceptions Uninformed Subscale of the Decisional Conflict Scale Unclear values subscale of the Decisional Conflict Scale –6. 0 (–9. 8, –2. 3) Insufficient data –6. 3 (–10. 0, –2. 7) Note: Values are presented as the weighted mean treatment effect (95% confidence interval). Chi-square heterogeneity test P value