Articulate Presenter version 4 Pro,build106d0 E:\Intro 2006\2.Conaway\2.Conaway-09-26-06.ppt 1/18/2000 12:20:36 PM 2 slide1.swf bgsm1.swf Experimental Design of Clinical Studies: Role of the Biostatistician   Mark Conaway, PhD Division of Biostatistics and Epidemiology 40.098 false 30 slide2.swf Biostatistical Resources (PLAN AHEAD) For GCRC protocol development: GCRC Biostatistics Core JT Patrie (jpatrie@virginia.edu or 4-8576) DG Boyd (dgb6r@virginia.edu or 4-5232) RD Abbott (rda3e@virginia.edu or 4-1687) For all other needs: Division of Biostatistics and Epidemiology 15 faculty in biostatistics or epidemiology (PhD or MD) 4 support staff in biostatistics (MS, 2 pursuing a PhD) For mentoring: MTPCI (School of Medicine and the GCRC) Department of Public Health Sciences Formal course offerings Seminars MS/MPH degree programs 101.094 false 30 slide3.swf Typical Biostatistical Needs For research planning (a must for GCRC and NIH funding): - Grant preparation and protocol development - Sample size and efficient experimental design For general statistical analyses, data interpretation, and report writing (a must for publication – and to improve the future prospect for NIH funding) 94.093 false 30 slide4.swf bgsm1.swf Focus today on sample size issues For planning: How many subjects need to be recruited into a study to “adequately” answer an important question? How much will an experiment cost? For critiquing other studies: Is the sample size sufficient to justify the conclusions of the study? 155.089 false 30 slide5.swf bgsm1.swf Example: phase II clinical trial in oncology Based on proportion of patients who “respond” to a therapy “response” defined as an objective measure of disease improvement Response rate current standard: assumed to be 15% (= p0) a new therapy: response rate = p (unknown) 203.102 false 30 slide6.swf bgsm1.swf Example Recruit a sample of 100 patients Count the number of responders (X) Based on number of responders, either decide Treatment is no better than standard Treatment has promise and is worth studying further (i.e, response rate P > P0) 70.087 false 30 slide7.swf Possible Results Note: We must fall into one of these cells. a and d are correct decisions. B = declare the treatment is worthless when it has promise. (usually called the ‘type II error’) C = declare the treatment has promise when it’s worthless. (usually called the ‘type I error’) 195.083 false 30 slide8.swf Design Goal Design an experiment so that the probabilities of falling into cells B and C are small. Why? If cells B and C unlikely, the chance of making a correct decision will be high (cells a and d). 37.094 false 30 slide9.swf bgsm1.swf Return to example 100 subjects. Let’s say we will declare the treatment worthy of further study if 17 or more patients respond. Even if the treatment is no better than the standard, we might see 17 or more responders out of 100 Question: what’s the chance this will happen? 105.091 false 30 slide10.swf bgsm1.swf Formula Chance that one person responds is 0.15 Chance of observing K responders out of 100 Chance that 17 or more: Prob(Obs 17) + Prob(obs 18) +....=P(Obs 100) 71.08 false 30 slide11.swf bgsm1.swf Graph Prob of 17 or more responders even though treatment is no better than standard is equal to 0.33 136.098 false 30 slide12.swf bgsm1.swf Next question What if the treatment really is promising? What is the probability that this rule will tell us that the treatment is no better than the standard? 100.102 false 30 slide13.swf bgsm1.swf Depends Answer depends on how effective the treatment really is Suppose the new therapy has a response rate of 22% What’s the probability you’ll see 16 or fewer responders? 252.082 false 30 slide14.swf bgsm1.swf Answer is 9% 35.083 false 30 slide15.swf bgsm1.swf Design so far With n = 100, a rule of ’17 or more’ *Computed assuming response rate is 22% 69.094 false 30 slide16.swf bgsm1.swf Another try? Probably of “C” (type I error) is too high (0.33) Maybe we should use a different decision rule: require 24 or more responders 98.09 false 30 slide17.swf bgsm1.swf Graph for 24 or more. Prob of 24 or more responders even though treatment is no better than standard is equal to 0.01 59.089 false 30 slide18.swf bgsm1.swf Proposal Decision rule looks better Enroll 100 patients; if 24 or more respond, declare the treatment worthy of further study Has only a small chance of making the error of declaring a treatment worthy if, in fact, the treatment is no better than the standard 37.094 false 30 slide19.swf bgsm1.swf But... Type II error much greater... 103.08 false 30 slide20.swf bgsm1.swf But... Prob of 23 or fewer responders even though treatment is better than standard is equal to 0.63 Not acceptable! 21.081 false 30 slide21.swf bgsm1.swf Summary of choices Declare treatment worthy if 17 or more if 24 or more 28.082 false 30 slide22.swf bgsm1.swf What to do? By convention, limit the type I error to 5% type I error is the probability of declaring the treatment worthy of further study when it is no better than the standard For a given n, choose the rule that gives a type I error probability of about 5% 69.094 false 30 slide23.swf bgsm1.swf Example, 22 or more If choose rule: 22 or more to declare treatment worthy of further study.. 129.098 false 30 slide24.swf bgsm1.swf Is this good enough? Can we design the study so that probability of type I error and type II error are sufficiently small? 80.092 false 30 slide25.swf bgsm1.swf Can this be done? Identify all sample sizes N, and identify all rules that meet the requirements Then choose the smallest N 108.095 false 30 slide26.swf bgsm1.swf Examples Note: All these computed assuming p = 0.15 (no better) and p = 0.22 (worthy) 134.087 false 30 slide27.swf bgsm1.swf Design If we enroll N = 188 patients and declare treatment worthwhile if 37 or more respond, less than 5% chance of declaring treatment worthy when it is in fact not worthwhile less than 20% chance of not declaring treatment worthwhile when it is 172.095 false 30 slide28.swf bgsm1.swf Cautions ALL of these calculations are based on chosen values : response rate for standard (15%) response rate for “worthwhile” treatment (22%) Sample size for an adequate design will change if definition of a worthwhile therapy changes 58.097 false 30 slide29.swf bgsm1.swf Example if change definition of worthwhile to p = 0.25 Note: All these computed assuming p = 0.15 (no better) and p = 0.25 (worthy) 71.08 false 30 slide30.swf bgsm1.swf Sample size calculations Only as good as the assumptions that go into them Should be done to avoid the consequences of: pursuing therapies that are no better than the standard missing potentially useful therapies 52.089 false 30 slide31.swf bgsm1.swf Other benefits Studies often have multiple endpoints Sample size usually driven by having sufficient power for primary endpoint Sample size/ power calculations force investigators to choose a primary outcome be specific in what the study is trying to accomplish quantify an endpoint that will shed light on the scientific question 87.093 false 30 slide32.swf bgsm1.swf Other benefits, continued Sample size/ power calculations force investigators to examine similar studies or work with their own pilot data demonstrate understanding/ technical skill in dealing with the data that will be generated as part of the study 57.104 false 30 slide33.swf bgsm1.swf Other issues Power is not the only determinant of the quality of a study Pilot studies Mechanisms to be worked out 107.102 false 30 slide34.swf bgsm1.swf Sample size calculations Increasing sample size is but one of a number of ways to limit error Increased ‘power’ can also be gained through experimental design choices 62.093 false 30