Probability and Statistics in Engineering - Fourth Edition

Probability and Statistics in Engineering Probability and Statistics in Engineering Fourth Edition William W Hines Prqjessor Emeritus School of Industrial and Systerns Engineenng Georgia fnstitge of Technology Douglas C Montgomery Professor of Engineering and Statistics Department of Industrial Engineenng Arizona State University David M Goldsman Professor School of Industrial and Syserns Engineering Georgia Institute of Technology Connie VL Borror Senior Lecroirer Department of lrdustrlal Engireering Arizona State University WILEY John Wiley Sons Inc Acquisitions Edior Wayne Anderson Associate Editor Jenny Welter Marketi1g Manager Kathedne Hepburn Seior Production Editor valerie A mrgas Senior Designer Dawn Stamey Cover Image Alfredo PasiekalPhoto Researchers Production Management Services Argosy Publishing This book was set in 1012 Tunes Roman by Argosy Publishing and printed and bound by Hamilon Printing The cover was printed by Phoenix Color This book is printed on acidfree paper Copyright 2oo3 John WIley Sons me All rights reserved No p31t of this publication may be rcproduced stored in a retrieval system or transmitted in any form or by any means electronic mechanical photocopying recording scanning or otherwise except as permitted under Sec tions 107 or 108 of the 1976 United States CopyrightAct witbouteither the poor written permission of the Pub lisher or auhorization through payntent of the appropriate pereopy fee to the Copyright Clearance Center 222 Rosewood Drive Danvers MA OHi23 978 750S400 fux 973 7504470 Requests to the Publisher for per mission should be nddressed to the PermissionS Dartment John Wuey Sons Inc 111 River Street Hoboken 1 07030 201748OCQ mx 2017486088 Email PERMJffiQWLEYCOM 10 order books or fur custOtrer service please call1800CALL vmEY 2255945 Library of Congress CaJaloging in PublicaJian Data Probabilit and statistics in engineering fWl1Jjaro W Hines et ll 4th cd pem Includes bibliographical referenceS 1 EngineeringStatistica1 methods L Hines Wtlliun W TA340 Jj55 2002 5192de21 2002026703 ISBN 0471240877 cloth acjdfree paper Printed in le United States of Aclerica 1098765432 PREFACE to the 4th Edition This book is written for a first course in applied probability and statistics for undergradu ate smdents in engineering physical sciences and management science curricula We have found that the text can be used effectivelY as a twosemester sophomore or juniorlevel course sequence as well as a onesemester refresher course in probability and statistics for firstyear graduate students The text has undergone a major overhaul for ille fourth edition especially wiill regard to many of the statistics chapters The idea has been to make the book more accessible to a wide audience by including more motivational examples rea1world applications and use ful computer exercises With the aim of making the course material easier to learn and eas ier to teach we have also provided a convenient set of cOurse notes available on the Web site wwwwileycomico11egehines For instructors adopting the text the complete solu tions are also available ou a passwordprotected portion of this Web site Structurally speaking we start ille book off with prObability illeory Chapter I and progress through random variables Chapter 2 functions of random variables Cbapter3 joint random variables Chapter 4 discrete and continuous distributions Chapters 5 and 6 and normal distribution Chapter 7 Then we introduce statistics and data description techniques Chapter 8 The statistics chapters follow the Same rough outline as in the pre vious edition namely sampling distributions Chapter 9 parameter estimation Chapter 10 hypothesi testing Chapter 11 single and multifactor design of experiments Chap ters 12 and 13 and simple and multiple regression ChapterS 14 and 15 Subsequent spe cialtopics chapters include nonparametric statistics Chapter 16 qUality control and reliability engineering Chapter 17 and stochastic processes and queueing theory Chap ter 18 Frnally there is an entirely new Chapter on statistical techniques for computer sim ulation Chapter 19perhaps ille first of its kind in this type of statistics text The chapters that have seen the most substantial evolution are Chapters 814 The dis cussion in Chapter 8 on descriptive data analysis is greatly enhanced over that of the previ ous editions We also expanded the discussion on different types of interval estimationin Chapter 10 In addition an emphasis has been placed On reallife computer data analysis examples Throughout ille book we incorporated other SlIUctural changes In all chapters we included new examples and exercises including numerous computerflbased exercises A few words on Chapters 18 and 19 Stochastic processes and queueing theory arise naturally out of probability and wefeelillat Chapter 18 serves as a good introduction to the subjectnonnally taught in operations research management science and certain engi neering disciplines Queueing theory bas garnered a great deal of use in such diverse fields as telecommunications manufacturing and production planning Computer simulation the topic of Chapter 19 is perhaps the most widely used tool in operations research and man agement science as well as in a number of physical sciences Simulation ma7ies all the tools of probability and statistics and is used in everyJring from financial analysLs to factory control and planning Our text provides what amounts to a simulation mL1icourse covering the areas of Monte Carlo experimentation random number and variate generation and simulation output data analysis v vi Preface We are grateful to the following individuals for their help during the process of com pleting the current revision of the text Christos Alexopoulos Georgia fustitute of Tech nology Michael Cararnanis Boston University David R Clark Kettering University J N Hool Auburn University Johu S Ramberg University of Arizona and Edward J Williarus Gniversity of lichigan Dearborn served as reviewers and provided a great deal of valuable feedback Beamz Valdes Argosy Publishing did a worderful job super vising the typesetting and page proofing of the text and Jennifer Welter at Wiley provided great leadership at every tum Everyone was certainly a pleasure to work with Of course we thank our families for their infinite patierce and support throughout the endeavor Hines Montgomery Goldsman and Borror Contents 1 An Introduction to 1 1 rntroduction 12 A Review of Sets 2 13 Expeiments ald Saruple Spaces 5 14 Events 8 15 Probability Definition and Assignment 16 FInite Sample Spaces and Enumeration 161 Tree Diagram 14 1 162 MultiplicatioD Principle 14 163 Permutations 15 164 Combinations 16 8 14 165 Permutations of Like Objects 19 17 CODditiODal Probability 20 18 Patitions Total Probaility and Bayes Theorem 25 19 Summay 28 110 Exies 28 2 OneDimensional Random Variables 33 21 Introduction 33 22 The Distribution Function 36 23 Discrete Random Variables 38 24 Continuous Random Variables 41 25 Some Characteristics ofPitnOUtiODS 44 26 Chebyshev Ioequality 48 27 SuInmary 49 28 Exercises 50 3 Functions of One Random Variable and Exption 52 31 Introduction 52 32 Equivalent Events 52 33 Functions of a Discrete Random Variable 54 34 Continuous Functions of a Continuous Random Variable 55 35 Expectation 58 36 Approximations to EHXl and I1HX 62 37 The MomentGene1Iting Function 65 38 Sumnay 67 39 Eercises 68 4 Joint Probability Distributions 71 41 Introduction 71 42 Joint Distribution for TwoDimensional Random Variables 72 43 Marginal DistrlbutionS 75 44 Conditional Distributions 79 45 Conditional Expectation 82 46 Regression of the Mean 85 47 Independence of Random Variables 86 48 Covariance and Correlation 87 49 The Distribution Function for T WJ Dimensional Random Variables 91 4 1 0 Funetions of Two Random Variables 92 411 Joint Distributions of Dimension n 2 94 412 Linear Combinations 96 413 omenGenerating Functions and Linear Combinations 99 414 The Law ofL1le Numbers 99 415 SlIll1llUy 101 416 Exercises 101 5 Some Important Discrete Distributions 106 51 Introduction 106 5w2 Bernoulli Trials a1d the Bernoulli Disributior 106 53 The Binomial Disrioutiol 108 531 Mean and Variance of the Binomial Distribution 109 532 The Cumulative Binomial Distribution ltO 533 An Application of the Binomial Distributon 111 54 The Geometric Distribution 112 541 Mean and Variance of the Geometric Distribution 113 55 The Pascal Distribution 115 551 Mear ad Variance 0 the Pascal DistricriolJ 115 56 The Multinomial DistnlJotioa 116 57 The Hypergeometric Distribution 117 571 Mean ard Variance ofthc Hyper geometric Distribution 1 I8 viii Contents 58 rre Poisson Distribution 118 581 Development from a Poisson Process 118 5S2 Development of the Pojsson Distribution from the Biroroial 120 583 Mean and Vaiance of the Poisson Distribution 120 59 Some Approximations 510 Generation of Realizations 122 123 5l Summary 123 512 Exercises 123 6 Some Important Continuous Distributions 128 61 Introduction 128 62 The Unifonn Distribution 128 621 Mean and Variance of the Uniform Lnsbibution 129 63 The Exponential Disbibution 130 631 The Relationship of the Exponential Distribution to the Poisson Distribution 131 632 Mean and Variance of the Exponential Distribution 131 633 Memoryles Property of the ExponcLtial Distribution 133 64 The Gamma Distribution 134 641 The Gamma Function 134 642 Definition of the Gamma Distribution 134 643 Relationship Between the Gamma Distribution and the Exponential Distribution 135 644 Mean and Variance of thc Gamma Distribution 135 65 The Weibull Distribution 137 651 Mean and Variance of the Vleibull Distribution 137 66 Generation of Realizations 138 67 Summary 139 68 Exercises 139 7 The Normal Distribution 143 71 Introduction 143 72 The Normal Distribution 143 721 Properties of the Normal Distribution 143 722 Mean and Variance of the Normal Distribution 144 723 The Normal Cumulative Distribution 145 724 The Standard Normal Distribution 145 7M25 ProblemSolving Procedure 146 73 The Reproductive Property of the Normal Distribution 150 74 The Central Limit Theorem 152 75 The Nonnalipproximation to the Binomia Distribution 155 76 The Lognormal Distribution 157 761 Density Function 158 762 Mean and Varianee of the Lognormal Distribution 158 763 Properties of the Lognormal bution 159 77 The Bivariate Normal Distdbution 160 78 Generation of Normal Realizaions 164 79 Summary 165 710 Exercises 165 S Introduction to Statistics and Data Description 169 81 The Field of Statistics 169 82 Data 173 83 Graphical Presentation of Data 173 831 Numerical Data Dot Plots and Scatter Plots 173 832 Numerical Data The Frequency Distribution and Histogram 175 833 The StemandLeafPlot 178 834 The Box Plot 179 835 ThePaoto Chart 181 836 Tine Plots 183 84 NUlIlerical Description of Data 183 841 Measures of Central Tendency 183 842 Measures of Dispersion 186 843 Other Measures for One Variable 189 844 Measuring Awciation 190 845 Grouped Data 191 85 Summary 192 86 Exercises 193 9 Random Samples and Sampling Distributions 198 91 Random Samples 198 911 Simple Random Sampling from a Finite Universe 199 9l2 Stratified Random Sampling of a Finite Universe 200 92 Statistics and Sampling Distributions 201 921 Sampling Distributions 202 922 Finitc Populations and Enumerative Studies 204 93 The ChiSquare Distribution 205 94 The t Distribution 208 95 The F Distribution 211 96 Summary 214 97 Exercises 214 r Parameter Estimation 216 101 Point Estimation 216 1011 Properties of Estimators 1012 The Method of Maximum 217 Likelihood 221 1013 The Metbod of Moments 1014 BaycsianInferencc 226 1015 Applications to Estimation 1016 Precision of Estimation The StandardError 230 102 SingleSample Confidence Interval Estimation 232 224 227 1021 Confidence Llterval on the Mean of a Normal Distribution Variance Known 233 1022 Confidence Interval on the Mean of a Nonnal Distribution Variance Unknown 236 1023 Confidence Interval on the Variance of a Nonull Distribution 238 102A Confidence Interval on a Poportion 239 103 TwoSampl Confidence Interval Eson 242 1031 Confidence Interval on the Difference between Means of Two Normal Distr1outions Variances Known 242 1032 Confidence Interval on the Difference between Means ofIWo Normal Distnoutious Vaiatices Unknown 244 1033 Confiderce Interval an IJ J4 for Plrired Observations 247 1034 Confidence Interval 01 the Ratio ofVariauces of1vo Normal IAstributions 248 1035 Confidence bterval on the Difference between Two Proporions 250 104 Approximate Confidence Intervals in Maximum Likelihood Estitrtation 251 105 Simultaneous Confidence Intervals 252 106 Bayesian Confidence Intervals 252 107 Bootstrap Confidence Intervals 253 Contents ix 108 Othe Interval EstiJnation Procedures 255 1081 Pediction IneIYals 255 1082 Tolerance Irtervals 257 09 Sunnruy 258 1010 Exercises 260 11 Tests of Hypotheses 266 111 Introduction 266 lLl Statistkal Hypotheses 266 1112 Type and Type II Errors 267 112 l13 OneSided and TwoSided Hypotheses 269 Tests of Hypotheses on a Single Sample 271 11 21 Tests of Hypotheses on the Mean of a Normal Distribution Variance KnoVitn 271 1122 Tests of Hypotheses on the Mean of a Kormal Disttfoution Varianee Unkaon 27 1123 Tests of Hypotheses on the Variance of a Normal Distribution 281 1124 Tests of Hypotheses 01 a Proportion 23 1l3 Tests of Hypotheses on Two Samples 286 1131 Tests of Hypotheses on the Means of Two Normal Distributions Variances KnOT 26 1132 TesLS of Hypotheses on the Means of1vo Normal Distributions Variances Unknovro 288 1133 The Paired tTest 292 11 34 Tests for the Equality ofIwo Variances 294 11 35 Tests of Hypotheses on 1vo Proportions 296 114 Testing for Goodrless of Fit 300 115 Contingency Table Tests 307 116 Sample Computer Output 309 117 Summary 312 118 Exercises 312 12 Design and Analysis of Single Factor Experiments The Analysis of Variance 321 121 The Completely Randomized SingleFactor Experiment 321 121 AD Example 321 1212 The Analysis of Variance 323 1213 Estimation of the Model Parameters 327 X Contents 1214 Residual Analysis and Model Checking 330 1215 An linbaLmcedDeign 33 122 Tests on Individual Treaonen Means 332 1221 Orthogonal Contrasts 332 1222 fJkeys Test 335 123 The RandomEffects Model 337 124 The Randomized Block Design 341 1241 Design and Statistical Analysis 341 1242 Tests on Individual Treatment MealS 344 1243 Residual Analysis and Model Checking 345 125 Deterrrnng Sample Size in SilgleFactor Experiments 347 126 Sample Cornputer Ouut 348 127 Summary 350 128 Exercises 350 13 Design of Experiments with Several Factors 353 131 Examples of Experimental Design Applications 353 132 Factorial Experiuents 355 133 TvioFactor Factorial Experiments 359 1331 Statistical Analysis of the Fixed Effects Model 359 1332 Model Adequacy Checking 364 1333 One Observation per Cell 364 1334 The RandomEffects Model 366 1335 The Mixed Model 134 General Factorial Experiments 135 The 2 FaCorial Design 373 1351 The2Design 373 367 369 1352 The 2 Design for k 3 ractors 379 1353 A Single Replicate of the 2 Design 386 136 Confounding in the 2 Design 137 Fractional Replication of the 2k Design 394 390 1371 The OneHalf Fracuon of the 2 Design 394 13M72 Smaller Fractions The 2k p Fractional Factorial 400 138 Sample Computer Output 4D3 139 SlUlllllary 404 1310 Exercises 404 14 Simple Linear Regression and Correlation 409 141 Simple Linear Regression 409 142 Hypotesis Tesing in Simple Linear Regression 414 143 lnterval Estimation in Simple Linear Regression 417 144 Prediction of New Observations 420 145 Measuring the Adequacy of the Regression Model 421 1451 Residual Analysis 421 1452 The LackofFit Test 422 1453 The Coefficient of Determination 426 146 Transformations to a Straight Line 426 14w7 Correlation 427 148 SampleComputerOuut 43 149 Summary 431 1410 Exercises 432 15 Multiple Regressiol 437 15 1 Multiple Regression Models 437 152 Estimation of the Parameters 438 153 Confidence Intervals in Multiple Linear Regression 444 154 Prediction of New Observations 446 15 5 HypOthesis Tting in Multiple Linear Regression 447 1551 Test for Significance of Regression 447 1552 Tests on Individual Regression Coefficients 450 156 MeasuresofMcdelAdequacy 452 1561 The Coefficient of Multiple Detennination 453 1562 ResidualAnalysis 454 157 Polynomial Regression 456 158 Indicator Variables 458 159 The Correlation Matrix 461 1510 Problems in Multiple Regression 464 151O Multicollinearity 464 15102 Influential Observations m Regression 470 15103 Autocorrel1tion 472 1511 Selection of Varab1es in Multiple RegressioQ 474 15111 The ModelBuililing Problem 474 15112 Computational Procedures for 1512 SUlIlIrulJ 1513 Exercises Variable Selection 474 486 46 16 161 162 Nonparametric Statistics 491 Introduction 491 The Sign Test 491 1621 A Description of the Sign Test 491 1622 The Sign Test for Paired Samples 494 1623 Type II Error i3 for the Sign Tet 494 1624 Conparisou of the Sgl Test and the tTest 496 163 The Wilcoxon Signed Rark Test 496 1631 A Description of the Test 496 1632 A LargeSample Approrimaton t97 1633 Paired Observations 498 163A Comparison with the Test 499 164 The Wilcoxon Rmk Sum Test 99 1641 A Description of the Test 1642 A LargeSample ApproxJmztion 501 1643 Compaison with the Test 501 499 165 Nonparnmetric Methods in the Analysis of Variance 501 1651 11le KruskalWallis Test 1652 The Rank Transformation 166 Summary 5l4 167 Exercises 505 11 Statistical Quality Control and 501 5l4 ReliabilityJJIlIi 507 171 Quality Improvement and Statistics 507 172 Statistical Qality Con1 508 173 Statistical Process Control 508 1731 Ilttoduction to Control Char 509 1732 Control Charts for Measurements 510 1733 Control Cbw for Individual Measurements 518 1734 Control Charts for Attribotes 520 1735 CUSLM and EWMA Control Charts 525 1736 Average Run Length 532 1737 Other SPC ProblemSolving Tools 535 174 Reliability Engineering 537 1741 Basic Reliability Definitions 538 Contents xi 1742 The Exponential TimetoFaiJure Model 541 1743 Simple Serial Systems 542 1744 Simple Active Redundancy 544 1745 Standby Redundancy 545 1746 Life Testing 547 1747 Reliability Estimation with a Known TirretoFailure Distribution 548 1748 EsEmation with the Exponential TuneoFailure Distribution 548 1749 Demonstration and Acceptance Testing 551 175 Summary 551 176 Exercises 551 18 Stochastic Procsses and Queueing 555 181 Introduction 555 182 DiscreteTIme Markov Chains 183 Classification of States and Clams 184 ContinuousTime Markov Chains 18 The BirthDeath Ptocess in Queueing 564 555 557 561 186 Considerations in Queueing Models 567 187 Basic SingleServer Model1A1th Consrant Rates 568 188 Single Server with Limited Queue Length 570 189 Multiple Servers vith an U Queue 572 1810 Other Queueing Models 573 1811 S1JJl1IIlaIY 573 1812 Exercises 573 19 Computer Simulation 516 191 Yfotivatioral Examples 576 192 Generation ofRandon Variables 530 92 onerating Uniform 01 Random Variables 581 1922 Generating Nonuniform Random Variables 582 193 Output Analysis 586 1931 Terminating Simulation Analysis 587 1932 Initialization Problems 589 1933 Steady State Simulation Analysis 590 194 Comparison of Systems 591 1941 Classical Confidence Ixtervals 592 xii Contents 1942 Common Random Numbers 593 1943 Antithetic Rmldom Numbers 593 1944 Selecting the Best Syst 593 195 SlllIllDZIY 593 196 Exercises 594 Appendlx 597 Table I Cumllative Poisson Distribution 598 Table II Cumulative Standard Nonna Distribution 601 Table ill Percentage Points of the y Distribution 603 Table IV Percentage Points of the Table V CbartVI CbartlI t Distribution 604 Percentage Pomts of the F DiJtribution 605 Operating CliaraJteristic Curves 610 Operating Characteristic Ccrves for the FXdElfects ModelAnalysiJ of Vuiance 619 Chart VIII Operating Characteristic Curves for the RandomEffects Model Analysis of VarianCe 623 Table IX Citical Values for the WUcoxon Two Sample Test 627 Table X Critical Values for the Sign Test 629 Table XI Critical Values for the Wdcoxon Signed Rank Test 630 Table XTI Percentage Points of the Studentized Range Statistic 631 Table xm Factors for QualityControl Cbarts 633 Table XIV k Values for OneSided and TwoSided Tolerance Intervals 634 Table XV Rmldom Numbers 636 References 637 Answers to Selected Exercises 639 Index 649 Chapter 1 An Introduction to Probability 11 INTRODUCTION Since professionals working in engineering and applied science are often engaged in both the analysis and the design of systems where system component characteristics are nonde tetnlinistic the understamling and utilization of probability is essential to the description design and analyis of such systems Examples reflecting probabilistic behavior are abun dant and in fact true deterministic behavior is rare To illusrrate consder the description of a variety of product quality or perfonnance measurements the operational lifespan of mechanical andor electronic systems the pattern of equipment failures the occurrence of natural phenomena such as sun spots ortomados particiecouots from a radioactive source travel times in delivery operations vehicle accident counts during a given day on a section of freeway or customer waiting times in line at a branch bank The termprobability has come to be widely used in everyday life to quantify thc degree of belief in an event of interest There are abundant examples such as the statements that there is a 02 probability of rain showers and the probability that brand X personal com puter will survive 10000 hours of operation without repair is 075 In tltis chaprer we intro duce the basic structure elementary concepts and methods to support precise and unambiguous statements like those above The formal study of probability theory apparently originated in the seventeenth and eighteenth centuries in France and was motivated by the study of games of chance WIth lit tle formal mathematical understructure people viewed the field with some kepticism however this view began to change in the nineteenth century when a probabilistic model description was developed for the behavior of molecules in a liquid This became known as Brownian motionr since it was Robert BrOvn an English botanist who first observed the phenomenon in 1827 In 1905 Albert Einstein explained Brownian motion under the hypothesis that particles are subject to the continual bombardment of molecules of the sur rounding medium These results greatly stimulated interest in probability as did the emer gence of the telephone system in the latter part of the nineteenth and early twentieth centuries Since a physical connecting system was necessary to allow for the interconnec tion of individual telephones with call1engths and interdemand intervals displaying large variation a strong motivation emerged for developing probabilistic models to describe this systems behavior Although applications like these were rapidly expanding in the early twentieth cenrury it is generally thought that it was not until the 19305 that a rigorous mathematical structure for probability emerged This chapter presents basic concepts leading to and including a definition of probability as well as some results and methods useful for problem solution The emphasis throughout Chapters 17 is to encourage an understanding and appreciation of the subject with applications to a variety of problems in engineering and science The reader should recognize that there is a large rich field of mathematics related to probabil ity hat is beyond the scope of this book 1 2 Chapter 1 An Introduction to Probability Indeed our objectives in presenting the basic probability topics considered in the cur rent chapter are threefold First these concepts enhance and enrich our basic understanding of the world in which we live Second many of the examples and exercises deal with the use of probability concepts to model the behavior of realworld systems Finally the prob ability topics developed in Chapters 17 provide a foundation for the statistical methods presented in Chapters 816 and beyond These statistical methods deal with the analysis and interpretation of data drawing inference about populations based on a sample of units selected from them and with the design and analysis of experiments and experimental data A sound understanding of such methods will greatly enhance the professional capa bility of individuals working in the dataintensive areas commonly encountered in this twentyfirst century 12 A REVIEW OF SETS To present the basic concepts of probability theory we will use some ideas from the theory of sets A set is an aggregate or collection of objects Sets are usually designated by capital letters A B C and so on The members of the set A are called the elements of A In gen eral when x is an element of A we write x E A and if x is not an element of A we write x A In specifying membership we may resort either to enumeration or to a defining prop ertj These ideas are illustrated in the following examples Braces are used to denote a set and the colon within the braces is shorthand for the t such that The set whose elements are the integers 5 6 7 8 is a finite set with four elements We could denote this by A5678 Note that 5 E A and 9 A are both true If we write V a e i 0 u we have defined the set of vowels in the English alphabet We may use a defining property and write this using a symbol as V is a vowel in the English alphabet If we say thatA is the set of all real numbers between 0 and 1 inclusive we might also denote A by a defining property as A xx E R Ox 1 where R is thc set of all real numbers The setB 3 3 is the same set as B x x E R x 9 where R is again the set of real numbers 12 A Review of Scts 3 In the real plane we can consider points x y that lie on a given lineA Thus the condition for inclu sion for A requires x y to satisfy ax by c so that A x y x E R y E R axbyc where R is the set of real numbers The universal set is the set of all objeets under consideration and it is generally denoted by u Another special set is the null set or empty set usually denoted by 0 To mus trate tbls concept consider a set AXXE Rxl The universal set here is R the set of real numbers Obviously set A is empty since there are no real numbers having the defining property 1 We should point out that the set 000 If two sets are considered say A and B we call A a subset of B denoted A c B if each element in A is also an element of B The sets A and B are said to be equal A B if and only if A c B and B c A As direct consequences of this we may show the following 1 For any set A 0 c A 2 For a given U A considered in the context of U satisfies the relation A c U 3 For a given set A A c A a rejJexive relation 4 If A c B and Bee then A ceCa transitive relation An interesting consequence of set equality is that the order of element listing is imma terial To illustrate let A a b c and B c a b Obviously A B by our definition Furthore when defining properties are used the sets may be equal althougb the defin ing properties are outwardly different As an example of the second consequence we let4 xx E R where x is an even prime number andS x x 3 5 Since the integer 2 is the only even prime A B We now considet some operations on sets Let A and B be any subsets of the universal set U Then the following hold 1 The complemernof4 with tespectto U is the set made up of the elements of Uthat do not belong ro A We denote this complementary set as A That is AXXE uxc A 2 The intersection of A and B is the set of elements that belong to both A and R We denote the intersection as A j R In other words AnB xxEAandxEB We should also note that A n B is a set and we could give this set some designator such as C 3 The union of A and B is the set of elements that belong to at least one of the etsA and B 1f D represents the union then DA u B x x E A or x E B Corbothl These operations are illustrated in the following examples 4 Chapter 1 An Introduction to Probability Eiel Let Ube the set of leiters ttl the alphabet that is U is a letter of the English alphabet and let A 0 is a yowell and B is one ofthe1etrers a b c As a consequence of the definitions ElleI1 A the set of consonants B t et g x y zl AuB a b c e io U A0Bal If the unlversalsetis defined as U 1 2 34 5 6 7 and threesubsetsA l 2 3B 24 6 C 1 3 5 arc defined then we see immediaely from the deficitions that A 456 II B U5 71C A c B 1 2 3 4 6 AUC12357 BvCU A82I o4nCl3 80C0 The Venn diagram can be used to illustrate certain set operations A rectangle is drawn to represent the universal set U A ssetA of U is represented by the region within a cir de drawn inside the rectangle Then A will be represented by the area of the rectangle out side of the circle as illustrated in Fig II Using this notation the intersection and union are illustrated in Fig 12 The operations of intersection and union may be extended in a straightforward manner to accommodate any finite number of sets In the case of three sets say A B and C A u B u C has the property that Au B u C A u B u C which obviously holds since both sides have identical members Similarly we see that A n B n C A n B 0 C A n B C Some important laws obeyed by sets relative to the operations previously defined are listed below Identity laws Au0A AuUU AnUA An00 De Morgans law Au B A n B A 0 B A u B Associative iaws A u B u C A u B u c A nCB n CA nB n C Distributive laws A U B n C A u B n A u C A n B u C A n B u A n C The reader is asked in Exercise 1 2 to illustrate some of these statements with Venn dia grams Fonnal proofs are usually more lengthy u FJgUr 11 Asetln Venn diagram 13 Experiments and Sample Spaces 5 u u aJ bJ Figure 12 The interseetion and union of two sets in a Venn diagram a The intersection shaded b The union shaded In the case of more than three sets we usc a subscript to generalize Thus if n is a pos itive integer andE j E2 En are given sets thenE j nE2 n n En is the set of elements belonging to all of the sets and E I U E2 U U En is the set of elements that belong to at least one of the given sets If A and E are sets then the set of all ordered pairs Ca b such that a E A and bEE is called the Cartesian product set of A and B The usual notation is A x B We thus have AxBa baE AandbE B Let r be a positive integer greater than 1 and let A j Ar represent sets Then the Carte sian product set is given by Al xA 2x xA r ai az ar ajE AJorj 1 2 r Frequently the number of elements in a set is of some importance and we denote by nA the number of elements in set A If the number is finite we say we have afinite set Should the set be infinite such that the elements can be put into a onetoone correspon dence with the natural numbers then the set is called a denumerably infinite set The non denumerable set contains an infinite number of elements that cannot be enumerated For example if a b then the set A x E R a x b is a nondenumerable set A set of particular interest is called the power set The elements of this set are the sub sets of a set A and common notation is O 1 t For example if A I 2 3 then O I 2 3 p 2 13 23 123 13 EXPERIMENTS AND SAMPLE SPACES Probability theory has been motivated by reallife situations where an experiment is per formed and the experimenter obser es an outcome Furthermore the outcome may not be predicted with certainty Such experiments are called random experiments The concept of a random experiment is considered mathematically to be a primitive notion and is thus not otherwise defined however we can note that random experiments have some common characteristics First while we cannot predict a particular outcome with certainty we can describe the set opossible outcomes Second from a conceptual point of view the exper iment is one that could be repeated under conditions that remain unchanged with the out comes appearing in a haphazard manner however as the number of repetitions increases certain patterns in the frequency of outcome occurrence emerge We will often consider idealized experiments For example we may rule out the out come of a coin toss when the coin lands on edge This is more for convenience than out of 6 Chapter 1 An Introduction to Probability necessity The set of possible outcomes is called the sample space and these outcomes define the particular idealized experiment The symbols and t are used to represent the random experiment and the associated saITlplespace Following the terminology employed in the review of sets and set operations we will classify sample spaces and thus random experiments A discrete sample space is one in which there is a finite number of outcomes or a countably aenumerably infinite number of outcomes Likewise a continuous sample space has nondenumerable uncount able outcomes These might be real numbers on an interval or real pairs contained in the product of intervals where measurements are made on two variables following an experiment To illustrate random experiments with an associated sample space we consider several examples Jiiii Tse coin and observe the up face ff Hn Note that this set is finite 2 Toss a true coin three times and observe the sequence of heads and tails ff HHH HHI HIli HIT THH THT TTH TIT 3 Toss a true coin three times and observe the total number of heads ff CO 12 3 t4 Toss a pair of dice and observe the up faces ff4 I 1 I 2 1 3 1 4 I 5 I 6 2 1 2 2 2 3 24 2 5 2 6 3 1 3 2 3 3 34 3 5 3 6 41 4 2 4 3 4 4 4 5 4 6 5 1 5 2 5 3 5 4 5 5 5 6 61 6 2 6 3 6 4 6 5 6 6 1g5 An automobile door is assembled with a large number of spot welds After assembly each weld is inspected and the total number of defectives is counted i5 0 12 X whereX the total number of welds in the door 13 Experiments and Sample Spaces 7 G A cathodeay rube is manufactured put on life test ard aged to failure The elapsed time in hoUts at faihte is recorded Ef UE R 0 This set is uncountable tj7 A monitor records the emission count from a 7adioactive source in one minute Ef O 1 2 Tnis set is cocntably infinite 8 Two key solder joints On a printed circuit board are inspected with a probe as well asvisu ally and each joint is classified as gqod G or defective D requiring rework or scrap Ef GG GDDGDD eg9 In a particular chemical plant the volume produced per day for a particular product ranges between a minimum value fl and a maximum value c which corresponds to capacity A day is randomly selected and the amount produced is observed Ef XXE R bxc iO An extrusion plant is engaged il making up an order for pieces 20 feet long Inasmuch as the trim eration creaes scrap at both ends the extruded bar rrus exceed 20 feet Because of costs involved the amount of scrap is critical A bar is extruded trimmed and iinished and the totalleagth of Smtp is measured flO rxeRxO 1 In a missile launch the three components of velocity are mentioned from the ground as a function of time At 1 minute after launch these are printed for a control unit 9 l v VY v Vx vJ1 v arc real numbers i 11 In the preceding example the veocity components are continlously recorded for 5 minutes 912 The space lS complicated here as we have all possible realizations of the functions vt and vtt for 0 S tS 5 minutes to consider 8 Chapter 1 An Introduction to Probability All these examples have the characteristics required of random experiments With the exception of Example 119 the description of the sample space is straightforward and although repetition is not considered ideally we could repeat the experiments To illustrate the phenomena of random occurrence consider Example 18 Obviously if l is repeated indefinitely we obtain a sequence of heads and tails A pattern emerges as we continue the experiment Notice that since the coin is true we should obtain heads approximately one half of the time In recognizing the idealization in the model we simply agree on a theo retical possible set of outcomes In I we ruled out the possibility of having the coin land on edge and in 6 where we recorded the elapsed time to failure the idealized sample space consisted of all nonnegative real numbers 14 EVENTS An event say A is associated with the sample space of the experiment The sample space is considered to be the universal set so that event A is simply a subset of t Note that both 0 and t are subsets of t As a general rule a capital letter will be used to denote an event For a finite sample space we note that the set of all subsets is the power set and more gener ally we require that if A c t then A c t and if AI A1 is a sequence of mutually exclu sive events in t as defIned below then Ul Ai c g The following events relate to experiments i I t1 i 10 described in the preceding section These are provided for illus tration only many other events could have been described for each case A 2 A iSJA lA 5 A 6 A is A iSs A is A 10 A The coin toss yields a head H All the coin tosses give the same face HHH TIT The total number of heads is two 2 The sum of the up faces is seven I 6 2 5 3 4 4 3 5 2 6 I The number of defective welds does not exeeed 5 0 12 34 5 The time to failure is greater than 1000 hours t t 1000 The count is exactly two 2 Neither weld is bad GG The volume produced is between a b and c x x E R b axc The scrap does not exeeed one foot x x E R 0 x I Since an event is a set the set operations defined for events and the laws and proper ties of Section 12 hold If the intersections for all combinations of two or more events among kevents considered are empty then the kevents are said to be mutually exclusive or disjoint If there are two events A and B then they are mutually exclusive if A n B 0 With k 3 we would requireAJ nA 0Aj nA 0A nA 0 andA j nA nAJ 0 and this case is illustrated in Fig13 We emphasize that these multiple events are asso ciated with one experiment 15 PROBABILITY DEFINITION AND ASSIGNMENT An axiomatic approach is taken to define probability as a set function where the elements of the domain are sets and the elements of the range are real numbers between 0 and 1 If event A is an element in the domain of this function we use customary functional notation peA to designate the corresponding element in the range 1 5 Probability Definition and Assignment 9 Figun 13 Three mutually exelusive events Definition If an experiment has sample ce ff and an event A is defined on ff then peA is a real number called the probability of event A or the probability of A and the function P has the following properties 1 00 PAO 1 for each event A of ff 2 PCI 1 3 For any finite number k of mutually exclusive events defined on fJ k 1 k pi UA 1 lpAJ 11 J ICd 4 If Ai A2 AJ is a denumerable sequence of mutually exclusive events defined on 1 then PQAPAJ Note that the properties of the definition given do not tell the experimenter how to assign probabilities however they do restrict the way in which the assignment may be accomplished In practice probability is assigned on the basis of 1 estimates obtained from previous expeJience or prior observations 2 an analytical consideration of experi mental conditions or 3 assumption To iliustrate the assigmnent of probability based on experience we consider the repe tition of the experiment and the relative frequency of the occurrence of the event of interest This notion of relative frequency has intuitive appeal and it involves the conceptual repetition of an experiment and a counting of both the number of repetitions and the num ber of times the event in question occurs More precisely is repeated m times and vo events are denoted A and B We let mA and mB be the number of times A and B occur in the m repetitions Definition The value fA mA 1m is called the relative frequency of event A It has the following properties 1 0 A I 2 IA 0 if and only if A never oeeurs and I I if and only if A occurs on every repetition 3 If A and B are mutually exclusive events thenl 8 1 la 10 Chapter 1 An Introduction to Probability As rn becomes large fA tends to stabilize That is as the number of repetitions of the experiment increases the relative frequency of event A will vary less and less from repetition to repetition The concept of relative frequency and the tendency toward stability lead to one method for assigning probability If an experiment has sample space g and an event A is defined and if the relative frequency fA approaches some number PA as the number of repeti tions increases then the number PA is ascribed to A as its probability that is as rn 7 00 11 Inpractice something less than infinite replication must obviously be accepted As an example consider a simple cointossing experiment in which we sequentially toss a fair coin and observe the outcomeeither heads H Or tails 7arising from each trial If the observational process is cODsidered as a random experiment such that for a particular rep etition the sample space is S H T we define the event A H where this event is defined before the observations are made Suppose that after m 100 repetitions of we observe mA 43 resulting in a relative frequency off A 043 a seemingly low value Now suppose that we instead conduet m 10000 repetitions of and this time we observe rnA 4924 so that the relative frequency is in this case fA 04924 Since we now have at our disposal 10000 observations instead of 100 everyone should be more comfortable in assigning the updated relative frequency of 04924 to peA The stability of fA as m gets large is only an intuitive noion at this point we will be able to be more precise later A method for computing the probability of an event A is as follows Suppose the sam ple space has a finite number n of elements ei and the probability assigned to an outcome iSPiPE whereE e and i 1 2 n while PI P2 Pn 1 PA LPi 12 iCi GA This is a statement that the probability of event A is the sum of the probabilities asso ciated with the outcomes making up event A and this result is simply a consequence of the definition of probability The practitioner is still faced with assigning probabilities to the outcomes ej It is noted that the sample space will not be finite if for example the elements ei of g are countably infinite in number In this case we note that PiO i 1 2 However equation 12 may be used without modification If the sample space is finite and has n equally likely outcomes so thatp P2 p lin then PtA nA n 13 and nA outcomes are contained in A Counting methods useful in determining n and nA will be presented in Section 16 15 Probability DefInition and Assignment 11 Suppose the coin in Example 19 is biased so that the outcomes of the sample space g HHH Hm HTH HIT THH THT TTH TIT haveprobabilitiesPJ fpz f P3 17 P4 P5frP61 4 8 P7TiPS 21 where e1 HHH ez HHI etclfwe let event A be the event that all tosses YIeld the I 8 I same face then PA 27 27 3 Suppose that in Example 114 we have prior knowledge that e2 211 Pi iIl 0 iI otherwise where Pi is the probabili that the monitor will record a count outcome of iI during a Iminute inter vallfwe consider event A as the event containing the outcomes 0 and 1 then A O I and PA PI P2 e2 22 32 0406 Consider Example 19 where a true coin is tossed three times and consider event A where all coins show the same faco By equation 13 since there are eight total outcomes and hvo are favorable to event A The coin was assumed to be true so all eight possible outcomes are equally likely v Assume that the dice in Example 111 are true and consider an event A where the sum of the up faces is 7 Using the resulrs of equation 13 we note that there are 36 outoomes of which six are favorable to the event in questiOI so that PA i Note that Examples 122 and 123 are extremely simple in two respects the sample space is of a highly restricted type and the counting process is easy Combinatorial methods fre quently become necessary as the counting becomes more involved Basic counting methods are reviewed in Section 16 Some important theorems regarding probability follow Theorem 11 If 0 is the empty set then P0 0 Proof Note thatg g u 0 andg and 0 are mutually exclusive ThenP9 P9 P0 from property 4 therefore P0 0 Theorem 12 PA 1 PA 12 Chapte 1 An Introduction to Probability Proof Note that gA J A and A and A are mutually exclusive ThenP9 PA PA from property 4 but from property 2 P9 1 therefore PA 1 PCA Theorem 13 PA V B PCA PB peA n B Proof Since A vB A v B nA where A and B nA are mutually exclusive andB A n B V B n A where CA n B and B n A are mutually exclusive then peA v B PA PCB n A and PCB PeA n B PB nA Subtracting peA v B PCB PA peA n B and thus PA v B PA PB PA n B The Venn diagntm shown in Fig 14 is helpful in following the argument of the proof for Theorem 13 We see that the double counting of the hatched region in the expression PCA PCB is corrected by subtraction of PA n B Theorem 14 PCA v B v C PAPB PC PAnB pA n C PCB n C PCA n B n C Proof We may write A v B v C CA v B v C and use Theorem 13 since A v B is 3D event The reader is asked to provide the details in Exercise 132 Theorem 15 k k k PAt vA2 v vA 2pA 2pA nAj 2PAi nAj nA il i2 r3 Proof Refer to Exercise 133 Theorem 16 f A c B then PCA PCB Proof fA cB thenB A v A n B andPB PA PA nB PA since peA B O CD L Figure 14 VeM diagram for two events 15 Probability Definition and Assignment 13 piIb1 If A and B are mutually exclusive events and if it is known that peA 020 while PCB 030 we can evaluate several prbiliti 1 peA I peA 080 2 pS I PCB 070 3 peA u B PA PCB 02 03 05 4 PAnBO 5 peA nBA uB by De Morgans law V 1PAuB I peA PCB 05 Suppose events A and B are not mutually exclusive and we know that PA 020 PCB 030 and peA n B 010 Then evaluating the silities as before we obtain 1 peA 1 peA 080 2 pS I PCB 070 3 peA u B PA PCB peA n B 02 03 01 04 4 peA n B 01 5 peA n B peA u B I peA PCB peA n B 06 Rtfii Suppose that in a certain city 75 of the residents jog 1 20 like ice cream 1 and 40 enjoy music M Further suppose that 15 jog and like icc cream 30 jog and enjoy music 10 like ice cream and music and 5 do all three types of activities We can consolidate all of this information in the simple Venn diagram in Fig 15 by starting from the last piece of data PJ n In M 005 and working our way out of the center 1 Find the probability that a random resident will engage in at least one of the three activities By Theorem 11 Pl u lu M pel pel PM Pl n l Pl n M pel n M Pl n I n M 075 020 040 015 030 010 005 085 This answer is also immediate by adding up the components of the Venn diagram 2 Find the probability that a resident engages in precisely one type of activity By the Venn dia gram we see that the desired probability is Pl nl n M Pj n In M pJ n I n M 035 0 005 040 M Figure 15 Venn diagram for Example 126 14 Chapter 1 An Inrroduction to Probability 16 FINITE SAMPLE SPACES AND ENUMERATION Experiments that give rise to a finite sample space have already been discussed and the methods for assigning probabilities to events associated with such experiments have been presented We can use equations 11 12 and 13 and deal either with equally likely or with not equally likely outcomes In some situations we will have to resort to the relative frequency concept and successive trials experimentation to estimate probabilities as indi cated in equation 11 with some finite m In this section however we deal with equally likely outcomes and equation 13 Note that this equation represents a special case of equa tion 12 where P P2 p lIn In order to assign probabilities peA nAn we must be able to detennine both n the munber of outcomes and nA the number of outcomes favorable to event A If there are n outcomes in g then there are 2 possible subsets that are the elements of the power set 0 IA The requirement for the n outcomes to be equally likely is an important one and there will be numerous applications where the experiment will specify that one or more items is are selected at random from a population group of N items without replacement If n represents the sample size n N and the selection is random then each possible selection sample is equally likely It will soon be seen that there are NInN n such samples so the probabillty of getting a particular sample must be nN nNL It should be carefully noted that One sample differs from another if one or more item appears in one sample and not the other The population items must thus be identifiable In order to illus trate suppose a population has four chips N 4 labeled a b c and d The sample size is to be two n 2 The possible results of the selection disregarding order are elements of J ab ac ad bc bd cd If the sampling process is random the probability of obtain ing each possible sample is i The mechanics of selecting random samples vary a great deal and devices such as pseudorandom number generators random number tables and icosahedron dice are frequently used as will be discussed at a later point It bccomes obvious that we need enumeration methods for evaluating n and nA for experiments yielding equally likely outcomes the following sections 161 through 165 review basic enumeration techniques and results useful for this purpose 161 Tree Diagram In simple experiments a tree diagram may be useful in the enumeration of the sample space Consider Example 19 where a true coin is tossed three times The set of possible outcomes could be found by taking all the patha in the tree diagram shown in Fig 16 It should be noted that there are 2 outcomes to each trial 3 trials and 23 8 outcomes HHH HHT HTH HIT THH THY TTH TIT 162 Multiplication Principle If sets AI A2 Akhave respectively nl nk elements then there are n1 nk ways to select an element first from AI then from and finally fromAk In the special case where nl nk n there are nk possible selections This was the situation encountered in the cointossing experiment of Example 19 Suppose we consider some compound experiment t consisting of k experiments cgl cgz egk If the sample spaces g I g 2 g k contain n1 nk outcomes respectively then there are n1 nk outcomes to eg In addition if the n outcomes of g are equally likely forj 1 2 k then the n1 n n outcomes ofl are equally likely First toss Second toss Third toss 16 Finite Sample Spaces and Enumeration 15 Figure 16 A tree diagram for tossing a true coin three times Suppose we toss a true coin and cast a true die Since the coin and the die are true the two outcomes to if l SOl H T are equally likely and the six outcomes to 2 S02 123456 J are equally likely Since n l 2 and TIz 6 there are 12 outcomes to the total experiment and all the outcomes are equally likely Beeause of the simplicity of the experiment in this case a tree diagram permits an easy and complete enumeration Refer to Fig 17 A manufacturing process is operated with very little inprocess inspection When items are com pleted they are transported to an inspection area and four characteristics are inspected each by a dif ferent inspector The first inspector rates a characteristic according to one of four ratings The second inspector uses three ratings and the third and fourth inspectors use two ratings each Each inspector marks the rating on the item identification tag There would be a total of 4 3 2 2 48 ways in which the item may be marked 163 Pennutations A permutation is an ordered arrangement of distinct objects One permutation differs from another if the order of arrangement differs or if the content differs To illustrate suppose we T 4 5 6 go H 1 H 2 H 3 H 4 H 5 H 6 T t T 2 T 3 T 4 T 5 T 6 Figure 17 The tree diagram for Example 127 16 Chapter I An Introduction to Probability again consider four distinct chips labeled a b c and d Suppose we wish to consider all per mutations of these chips taken one at a time These would be a b c d If we are to consider all pemlUtations taken two at a time these would be ab be ba cb ac bd ca db ad cd da de Note that permutations ab and ba differ because of a difference in order of the objects while pennutations ac and ab differ because of content differences In order to generalize we consider the case where there are n distinct objects from which we plan to select per mutations of r objects r n The number of such pennutations p is given by P nnln2n3nrl n nr This is a result of the fact that there are n ways to select the first object n 1 ways to select the second n r 1 ways to select the rth and the application of the multi plication principle Note that P n and OJ 1 i1ipjiir A major league baseball team typically has 25 players A lineup consists of nine of these players in a particular order Thus there are P 741 x lOll possible lineups 164 Combinations A combination is an arrangement of distinct objects where one combination differs from another only if the content of the arrangement differs Here order does not matter In the case of the four lettered chips a b c d the combinations of the chips taken two at a time are ab ac ad be bd cd We are interested in determining the number of combinations when there are n distinct objects to be selected r at a time Since the number of permutations was the number of ways to select r objects from the n and then permute the r objects we note that 16 Finite Sample Spaces and Enumeration 17 14 where represents the number of combinations It follows that n I j plljr n r rlnr1 15 In the illustration witll the four chips where r 2 the reader may readily verify thatY 12 and 6 as we found by complete enumeration For present purposes e is defined where n and r are integers such that 0 rS n how ever the terms may be generally defined for real n and any nonnegative integer r In this case we write n nln 2 n r I I r r The reader will recall the binomial theorem n f n aby L jab rooO r 06 The nmbers J are thus called binomial coefficients Returning briefly to the definition of dam sampling from a finite population with out replacement there were N objects with n to be se1ected There are thus e different sam pIes If the sampling process is random eacb possible outcome has probability l of being the one selected Two identities that are often helpful in problem solutions are 17 and 18 To develop the result shown in equation 17 we note that and to develop the relt shown in equation 18 we expand the righthand side and collect terms To verify that a finite collection of n elements has 2 subsets as indicated earlier we see that 2 l iln nln n rG r 01 I n from equation 16 The right side of this relationship gives the total number of subsets since Gl is the number of subsets with 0 elements is the number vth one element d tl is the number with n elements 18 Chapter 1 An Introduction 0 Probability A production lot of size 100 is known to be 5 defective ArandoDl sample of 10 items is selected without replacement In order to determine the probability that there will be no defectives in the sam ple we resort to counting both the number of possible samples aod The number of samples favorable to eentA where eYent A is taken to mean that there are nO defectivc The number of possible sam pIes is l l The number favorable to A is Il 1 SO that 595 51 951 P 010 O510l85 A 100 100 05375 llo J 1090 To generalize the preceding example we consider the case where the population has N items of which D belong to some class of interest such as defective A random sample of size n is selected without replacemenL If il denotes the event of obtaining exactly r items from the class of interest in the sample then PA fDrN D rJrj N n r 0 12 min n D Problems of this type are often referred to as hypergeometric sampling problems EDi An NBA basketball team typically has 12 players A starting team consists of five of these players in no particular order Thus there are 2 792 possible starting teams Finjiji One obvious application of counting methods lies in calculating probabilities for poker bands Befoe proeeeding we remind the reader of some staldard erminology The rank of a particu1 cd drawn from a standard 52card deck can be2 3 Q KA while the possLble suUs are 0 v In poker we dravo five cards at random from a deck The number of possble hands is G 2598960 1 We fitst calculate the probability of obtainirg ViO pairs for example Av A 3v 30 O We proceed as follows a Select tv ranks eg A 3 We can do this ways b Select two suits for first pair eg There are W ways e Select two suits for second pair eg 0 There are W ways d Select remaining card to complete the hand There are 44 ways Thus the number of ways to select two pairs is and So 00475 16 Finite Sample Spaces and Enumeration 19 2 Here we calculate the probability of obtaining a full house onc parr one threeofakind for exampleAYAoo 3Y 3 3 a Select two ordered ranks eg A 3 There are PJJ ways Indeed the ranks must be ordered sinee three As two 3 s differs from two As three 3s b Select two suits for the parr eg Y There areways e Select three suits for the threcofakind eg Y ways Thus the number of ways to select a full house is nfull house 1312aJ 3744 and so 3744 Pfull house 000144 2598960 3 Finally we calculate the probability of a flush all five cards from same suit How many ways can we obtain this event a Select a suit There are ways b Select five cards from that suit There are 3 ways Then we have PflllSh 5148 000198 2598960 165 Pennutations of Like Objects In the event that there are k distinct classes of objects and the objects within the classes are not distinct the following result is obtained where n is the number in the first class n2 is the number in the second class nk is the number in the kth class and n J n2 nk n n 110 Consider the word TENNESSEE The number of ways ro arrange the letters in this word is n 93780 nTnEnNnS 1422 where we use the obvious notation Problems of this type are often referred to as multinomial sam pling problems The counting methods presented in this section are primarily to support probability assignment where there is a finite number of equally likely outcomes It is important to remember that this is a special case of the more general types of problems encountered in probability applications 20 Chapter I An Introduction to Probabilirj 17 CONDITIONAL PROBABILITY As noted in Section 14 an event is associated with a sample space and the event is repre sentod by a subset off The probabilities discussed in Section 15 all relate to the entire sample space We bave used the symboll4 to denote the probability of these events bow eve we could have used the symbol PAI read as the probability of AgLesple Eace In this section we consideiilie probability of events where the event is con4l tioned On some subset of the sample space Some illustrations of tllisidea should be helpful Consider a group of 100 persons of whom 40 are college graduates 20 are se1femployed and 10 are both college graduates and selfemployed Let B represent the set of college graduates and A represent the set of selfemployed so thatA r1 B is the set of college graduates who are selfemployed From the group of 100 one person is to be randomly seJected Each person is given a number from I to 100 and 100 chips with the same numbers are agitated with one being selected by a blindfolded outsider Then PA 02 PB OA and PA n B 0 I if the entire sample space is considered As noted it may be more instructive to write PAI PBIf and PA n BIi in such a case ow suppose the following event is considered self employed giver that the person is a college graduate AlB Obviously the sample space is reduced in that only college graduates are considered Fig 18 The probability PAIB is thus given by nAnB AnB pAiB pAnBl 025 B nBn PB 04 The reduced sample space consists of the set of all subsets of i that belong to B Of course A n B satisfies the condition As a second illustfation consider the case wbere a sample of size 2 is randomly selected from a lot of size 10 It is known that the lot has seven good and three bad items LetA be the event that thefirst item selected is good and B be the event that the second item selected is good If the items are selected without replacement that is the first item is not replaced before the second item is selected then and PtA 2 10 If the first item is replaced before the second item is selected the conditional probability PBIA PB and the events A and B resulting from the two seleetion experiments comprising are said to be jrdependent A formal definltion of conditiooal probability a b Figure 18 Conditional probability a Initial sample space b Reduced sample space 17 Conditoal Probability 21 PAIB will be given later and independence will be discussed in detail The following examples will help to develop some intuitive feeling for conditionallfObability Exampiel34 Recall Example 111 where two dice are tossed and asse that each die isJroe The 36 possbe Out comes were enumerated If we consider two events B d dz dz d where d is tlle value of the up face of the first die a1d dzis the value of the up face of the second die then PiA t PCB f peA n B 3 PBIA and PAIB eft The probabilities were obtained from a direct consideration of the sample space and the counting of outcomes Note that anke3 pAIB peA n B PCB and PBIA PAnB peA In World Vrar II an early operations research effort in England was directed at establishing search pat terns for Uboats by patrol flights or sorties Fo a time there was a tendency to corcertratc the flights on hshore areas as it had been believed Lim more sightilgs took place inshore The research group studied 1000 sortie recor witll the following result the data are fictitious Inshore Sighting 80 No sighting 820 Total sorties 900 Let St There was a sighting 31 There was no sighting B Inshore ortie Bz Offshore sortie WI sel immediately that Offshnre 20 80 100 p SdB 9 00889 psdB 22020 100 which iIldicates a search strategy counter to prior practice Definition Total 100 900 1000 We may define the conditional probability of event A given event B as pA1B PAnB PB if PB 0 9 r I j I I 111 This definition results from the intuitive notion presented in the preceding discussion The conditional probability PI satisfies the properties required of probabilities That is 22 Chapter 1 An Introduction to Probability 1 0 PAIB a 2 NflB 1 3 PAiIB PAIB if A n Aj 0 for i j 4 PQAiIB J PAIB for Ab A2 A3 a denumerable sequence of disjoint events In practice we may solve problems by using equation 111 and calculating PA n B and PB with respect to the original sample space as was illustrated in Example 135 or by considering the probability of A with respect to the reduced sample space B as was illus trated in Example 134 A restatement of equation 111 leads to what is often called the multiplication rule that is PA n B PB PAIB PB 0 and PA n B PA PBIA PA O H2 The second statement is an obvious consequence of equation 111 with the conditioning on event A rather than event B It should be noted that if A and B are mutually exclusive as indicated in Fig 19 then A n B 0 so that PAIB 0 and PBIA o In the other extreme if B cA as shown in Fig 110 then PAIB 1 In the first case A and B cannot occur simultaneously so knowledge of the occurrence of B tells us that A does not occur In the second case if B occurs A must occur On the other hand there are many cases where the events are totally unrelated and knowledge of the occurrence of one has no bearing on and yields no information about the other Consider for example the experiment where a true coin is tossed twice EventA is the event that the first toss results in a heads and event B is the event that the second toss results in a heads Note that PA t since the coin is true and PBA t since the coin is true and it has no memory The occurrence of event A did not in any way affect the occurrence of B and if we wanted to find the probability of A and B occurring that is PA n B we find that PAnB PApBlA L 224 We may observe that if we had no knowledge about the occurrence or nonoccurrence of A we have PB PBIA as in this example 00 Figure 19 Mutually exclusive events Figure 110 Event B as a subset of A 17 Conditional Probability 23 Infonnally speaking two events are considered to be independent if the probability of the occurrence of one is not affected by the occurrence or nonoccurrence of the other This leads to the following definition Definition A and B are independent if and only if PtA n B PtA PB 113 An immediate consequence of this definition is that if A and B axe independent events then from equation 112 PAIB PtA and PBIA PB 114 The following theorem is sometimes useful The proof is given here only for the first part Theorem 17 If A and B are independent events then the following holds 1 A and B axe independent events 2 A and B axe independent events 3 A and B axe independent eventS Proof part 1 PtA n B PtA PBIA PtA 1 PBIA PA 1 PB PtA PCB I Jy i In practice there axe many situations where it may not be easy to determine whether two events axe independent however there are numerous other cases where the require ments may be either justified or approximated from a physical consideration of the exper iment A sampling experiment will serve to illustrate r ei Suppose a random sample of size 2 is to be selected from a lot of size I O and it is known that 98 of the I 00 items are good The sample is taken in such a manner that the first item is observed and replced before the second item is selected If we let A First item observed is good B Second item observed is good and if we want to determine the probability that both items are good then 98 98 PAnB PA PB 09604 100 100 If the sample is taken without replacement so that the first item is not replaced before the seeond item is selected then 98 97 PAnB PA PA 09602 100 99 24 Chapter 1 An Introduction to Probability The results are obviously very dose and one coromon practice is to assume the events independent when the sampling fraction sample sizepopulation size is small say less than 01 EX3lIiieji The field of reliability engineering has developed rapidly since the early 1960s One type of problem encountered is that of estimating system reliability given subsystem reliabilitics Reliability is defined here as the probability of proper functioning for a staed period of time Consider the structure of a simple serial system shown in Fig 111 The system functions if and only if both subsystems fullC tion If the subsystems surVive independectly then where Rl and R2 are the reliabilities for subsystems 1 and 2 respectively For exampte if Rj 090 and R 080 then R 072 Example 137 illustrates the need to generalize the concept of independence to more than two events Suppose the system consisted of three subsystems or perhaps 20 subsys terns What conditions would be required in order to allow the analyst to obtain an estimate of system reliability by obtaining the product of the subJstem reliabilities Definition The k events AI Az A are mutually independent if and only if the probability of the intersection of any 2 3 k of these sets is the product of their respective probabilities Stated more precisely we require that for r 2 3 k P nAt n nAt P n4 peA rrpI jt In the case of serial system liability calculations where mutual independence may reasonably be assumed the system reliability is a product of subsystem reliabilities 115 In the foregoing definition there are 2 k 1 conditions to be satisfied Consider three events A B and C These are independent if and only if P n B peA PCB PC4 n C P4 PC PCB n C PCB P C and peA n B n C peA PCB P C The fol lowing example illustrates a case where events are pairwise independent but not mutually independent System Figure 111 A simple serial system 18 Partitions Total Probability and Bayes Theorem 25 0Pi Suppose the sample space with equally likely outcomes for a particular experiment is as follows 9 CO o 0 01 1 Cl 01 01 OJ Let Ao First digit is zero B J Second digit is one Co Third digitjs zero C J Third digit is one A J First digit is one Bo Second digit is zero It follows that and it is easily seen that However we note that ptA nBj PA p BJ ptA ncj PAP Cj PB n CJ PBP Cj i 01 j 01 1 pAo nBo n Co pAoPBoP Co 4 PCAo n Bo n Cl 0 PAo PBO PCC1 and there are other triplets to which this violation could be extended The concept of independent experiments is introduced to complete this section If we con sider two experiments denoted I and 2 and let Al and A2 be arbitrary events defined on the respective sample spaces11 and12 of the two experiments then the following defini tion can be given Definition If peAl n A peAl peA then 1 and 2 are said to be independent experiments 18 PARTITIONS TOTAL PROBABILITY AND BAYES THEOREM A partition of the sample space may be defined as follows Definition When the experiment is performed one and only one of the events B occurs if we have a partition of 1 26 Chapter 1 All IJlIroduetion to Probability A particular binary word consists of five bits bl b2 b1 b4 Os where blOllt 1 2 3 4 5 Au experiment consists of tranSmitting a word and it follows that there are 32 possible words If the events are Bi CO 0 0 0 0 0000 OJ O 0 0 10 0 0 0 1 I 0 0 100 0 0 10 I 0 0 1 10 0 0 1 1 1 E 0 1 0 0 0 01001 0 1 0 1 0 0 10 1 11 01 1001 01 1011 01110 011 11 E4 I 0 0 0 0 I 0 0 0 1 100 I 0 I 0 0 I I 1 0100 1 01 0 1 101 10 1 0 I 1 1 E I 1 0 0 0 1100 I I 10 10 1 10 1 1 11100 11101 1 1 1 1 OJ B 1 I 1 1 1 In general if k events B i 1 2 kl form a partition and A is an arbitrary event with respect to then we may vnite A A nBl u A nBu u A nB so that PtA PtA n Bl PA n B peA n B since the events A n B are pairwise Illutually exclusive See Fig 112 for k 4 It does not matter thetA n B 0 for some or all of the i since P0 0 Using the results of equation 112 we can state the folloving theorem Theorem 18 If B B B represents a partition of and A is an arbitrary event on ff then the total probability of A is given by PtA PB peA I Bl peA 1Jz PB PtA i Bl LPEPA I Bil i1 The result of Theorem 18 also known as the law of total probability is very useful as there are numerous practical imations in wbicb PA cannot be computed directly However 18 Partitions Total Probabiliry and Bayes Theorem 27 with the infonnation that Bi has occurred it is possible to evaluate PAIB and thus deter mine PA when the values PB are obtained Another important result of the total probability law is known as Bayes theorem Theorem 19 If B1 B2 Bic constitute a partition of the sample space j and A is an arbitrary event on Ef then for r I 2 k 116 Proof I pBnA P BA PA pBPIAIB k 2 p BJ p AlB il The numerator is a result of equation 112 and the denominator is a result of Theorem 18 Three facilities supply microprocessors to a manufacturer of telemetry equipment All are supposedly made to the same specifications However the manufacturer has for several years tested the micro processors and records indieate the following information Supplying Facility 2 3 Fraction Defective 0Q2 001 003 Fraction Supplied 015 080 005 The manufacturcr has stopped testing because of the costs involved and it may be reasonably assumed that the fractions that are defective and the inventory mix are the same as during the period of record keeping The director of manufacturing randomly selects a microprocessor takes it to the test department and finds that it is defective If we letA be the event that an item is defective and Bi be the event that the item came from faciliry i i 1 2 3 then we can evalnate PCBA Suppose for instance that we are interested in determining PB31A Then pspAIs p SiA p pAIB ps pAis p s pAIs 005003 3 015002 080001 005003 25 28 Chapter 1 An Introduction to Probability 19 SUMMARY This chapter has introduced the concept of random experiments the sample space and events and has presented a formal definition of probability This was followed by methods to assign probability to events Theorems 11 to 16 provide results for dealing with the probability of special events Finite sample spaces with their special properties were dis cussed and enumeration methods were reviewed for use in assigning probability to events in the case of equally likely experimental outcomes Conditional probability was defined and illustrated along with the concept of independent events In addition we considered partitions of the sample space total probability and Bayes theorem The concepts pre sented in this ehapter form an important background for the rest of the book 110 EXERCISES 11 Television sets are given a final inspection fol lowing assembly Three types of defects are identi fied criticaFmajo and minbr defects and are coded A B and C respectively by a mailorder house Data are analyzed with the following results Sets having only critical defects 2 Sets having only major defects 5 Sets having only minor defects 7 Sets having only criticS and major defectsO 3 Sets having only critieal and minor dcfcctsU Q 4 Sets having only major and minor defects V 3 Sets having all three types of defects 1 a What fraction of the sets has no lefects O1 b Sets with either critical defects or major defects or both get a complete rework What fraction falls in this category 0 1 1 2 illustrate the following properties by shadings or colors on Venn diagrams a Associative laws A u B u C A u B u C A nCB n CAnBn C b Distriburive laws A u B n C A u B n A u C A n B u C A n B u A n C e IfAcBthenAnBA d IfAcBthenAuBB e IfAnB0thenAcB I IfAcBandBcCthenAcC 13 Consider a universal set consisting of the inte gers 1 through 10 9r U 123456789 1O LetA 2 3 4 B 3 4 5 and C 5 6 7 By enumeration list the membership of the following sets a A n B bAuB eAnE d A n B n C e An B u C 14 A flexible circuit is selected at random from a production run of 1000 circuits Manufacturing defects are classified into three different types labeled A B and C Type A defects occur 2 of the time type B defects occur 1 of the time and type C occur 15 of thc time Furthermore it is known that 05 have both type A andB defects 06 have bothA and C defects and 04 have B and C defects while 02 have all three defects What is the probability that the flcxible circuit selected has at least One of the three types of defects 15 In a humanfactors laboratory the reaetion timcs of human subjects are measured as the elapsed time from the instant a position number is displayed on a digital display until the subject presscs a button located at the position indicated Two SUbjects are involved and times are measured in seconds for each subject t 1 t2 What is the sample space for this exper iment Present the following events as subsets and mark them on a diagram tl tzfl 015 max t1 t2J 015 Itr1 006 16 During a 24hour period a computer is to be aecessed at time X used for somc processing and exited at time Y2 X Take X and Y to be measured in hours on the time line with the beginning of the 24 hour period as the origin The cxperiment is to observe XandY a Describe the sample space ff b Sketch the following eVents in the X Yplane i The time of use is 1 hour or less li The access is before tl and the exit is after t2 where a s tl t2 24 iii The time of use is ess than 20 of the period 17 Diodes from a batch aoe tested one at a time and marked either defective or nondefective This is con tinued until either tv defective items are found or five items have been ted Describe the sample space for this experiment 1MK A set has four elements A a b c d Describe the power se O J lA 19 Describe the sample space for each of the fol lowng experiments a A lot of 120 battery lids for pacemaker cells 1S knovU to contain a number of defectives because of a problem with JJe barrier materia applied to the glassedin feedthrough Three lids are ran domly selected without replacement and are carefully inspected following a cut doVD b A pallet of 10 castings is known to contain one defective and nine good Ullits Four castfugs are randomly selected without eplacement and inspected 110 The production manager of a certain company is interested in testing a fInished product which is available in lots of size 50 She would like to rework a lot if she can be reasonably sure that 10 of the items are defective She decides to select a random sample of 10 iters wilhom replacement and rework the lot if it contains one or more defecriveiems Does this pro cedure seem reasonable 111 A trucking firm has a conblictto ship a load of goods from city W to city Z There are no direct routes connecting W to Z but there are six roads from W to X and five roads from X to Z How many rotal routes are there to he considered 112 A state has one million registered vehicles and is considering using license plates with six symhols where the fIst three are letters and the last three are digit Is this scheme feasihle 113The maLagcI of a small plant wishes to deter mine the number of waS he caa assign workers to the first shift He has 15 workers who can scrve as opera tors of the production equipment eight who can serve as maintenance personnel and fotI who can be super visors If the shift requires six operators two mainte nance personnel and one supervisor how many ways can the first shift be nal1led 1 14 A production lot has 100 units of which 2Q are knotD to be defective A nuldom sample of four units is selected hithout replacemenL What is the 110 Exercises 29 probability that the sample will contain no more than two cefective units sk inspecting incoming lots of merchandise the Jollowing inspection rule is used where lhe lots con tain 300 ulllts A random sample of 10 items is selected Ifthereis no more than one defective item in the samp1e the lot is accepted Otherwise it is returned to the vendor If the fraction defective in the original lot is p determine the probability of accept ing the lot as a function of p 116 In a plastics plant 12 pipes empty different chemicals into 2 mixing vaL Each pipe has a fiveposi tion gauge lhat measures the rate of flow into the lat One day while expcr1menting with various mixtures a solution is obtained that emits a poisorous gas The settings on the gauges were not recorded What is the prObability of obtaining this same solution when ran domlyexperimenting again 117 Eight equally skilled men and WOmen are appying for two jobs Because the two new empoy ees must work closely together their personalitis should be compatible To achieve this the personnel manager has administered a test and must compare the scores for each possibility How wany compar isons must the ager make 118 By accident a chemist combined two labora tory substances that yielded a desirable product Unfortunately her assistant did not record the names of the ingredients There are forty substances avail2ble in the lab If the twO in question must be located by successie triakmderror experiments what is the maximum number of tests that might be made 1 19 Suppose in the previous problem a known Cat alyst iiS used in JJe first accidenta reaction Because of this the order in wxch the ingredients axe mixed is important Wllat is JJe maximum number of tests that might be made 120 A conpany plans to build five additioual ware houses at new locations Ten locations are cnder con sideration Hov many total possible choices are there for the set of five locations 1 21 Washing maclllnes can have five kinds of major rud five kinds of minor defects In how many ways can one oajor ane one rrtinor defect occur In how many ways can two major and two minor defects occur 1 Z2 Consider the diagra11 at the top of the oext page of aJ electronic system which shows the probabilities of the system conponents operating properly The entire system operates if assembly m and at least one of the coopOoents in each of assemblies I and IT 30 Chapter 1 An Introductiou to Probability II operates Assume that the components of each assem bly operate indepeudently and that the assemblies operate independently What is the probability that the entire system operates 123 How is the probability of system operation affected if in the foregoing problem the probability of successful operation for the component in assembly ill changes from 099 to 09 124 Consider the seriesparallel assembly shown below The values Rj i I 2 3 4 5 are the reliabil ities for the five components shown that is Rj prob ability that unit i will function properly The components operate and fail in a mutually inde pendent manner and the assembly fails only when the path fromA to B is broken Express the assembly reli ability as a function of R1 R2 R R4 and R5 125 A political prisoner is to be exiled to either Siberia or the Urals The prObabilities of being sent to these places are 06 and 04 respectively It is also known that if a resigent 9f Siberia is sejected at ran dom the probability is 05that hewill be wearing a fur coat whereas the probability i 07 that a resident of thc Urals will be wearing one Upon arriving in exile the first person the prisoner sees is not wearing a fur coat What is the probability he is in Siberia 126 A braking device designed to prevent automo bile skids may be broken down into three series sub systems that operate independently an electronics system a hydraulic system and a mechanical activa Or On a particular braking the reliabilitics of these A III units arc approximately 0995 0993 d 0994 respectively Estimate the system reliability 127 Two balls are drawn from an urn containing m balls numbered from I to m The first ball is kept if it is numbered I and returned to the urn otherWise What is the probability that the second ball drawn is num bered 2 128 Two digits are selected at random from the dig its 1 through 9 and the selection is without replace ment the same digit cannot be picked On both selections If the sum of the two digits is even find the probability that both digits are odd 129 At a certain university 20 of the men and 1 of the WOmen are over 6 feet tall Furthermore 40 of the students are Women If a student is randomly picked and is observed to be over 6 feet tall what is the probability that the student is a woman 130 At a maehine center there are four automatic screw machincs An analysis of past inspection records yields the following data Perccnt Percent Defectives Machine Production Produced 15 4 2 30 3 3 20 5 4 35 2 8 Machines 2 and 4 are newer and more production has been assigned to them than to machines 1 and 3 Assume that the current inventory mix reflects the production perceltages indicated a If a screw is randomly picked from inventory what is the probability Llat it will Ie defective b If a screw is picked and found to be defective what is the probability that it was produced by machine 3 131 A point is selected at random inside a cirele What is the probability that the point is closer to the center than to the circumference 1 32 Complete the details of the proof for Theorem 1 4 in the text 133 Prove Theorem 1 5 134 Prove the second aud third parts of Tleorem 17 135 Suppose there are n people in a room If a list is made of all their birthdays the specific month and day of the month what is the probability that tvo or more persons havethe same birhday Assume there are 365 days in the year and that each day is equally likely ro occur for any persons birthday Let B be the event that two or more persons have the same binh day FindPB andPB for n 10 20 21 22 23 24 25 30 4 50 and 60 136Jn a certain dice game players continue to throw tw dice until they either win or lose The player Nins on the first throw if the sum of the two uptumed faces is either 7 or 11 an9 loses if the sum is 2 3 0 12 Otherwise the sum of the faces becomes the players pOint The player continues to throw llttil the first succeeding throw on which he makes his A B c x HO Exercises 31 point in which case he wins or until he throws a 7 in which case he loses lhat is Lle probability that he player with the dice w eventally win the game 137 The industrial engineering depa1ment of the XYZ Company is performing a work sampling study on eight technicians The engineer wishes to randomize the order in which he visits the techni cians work areas In how many ways may he arrange these visit 1 38 A hiker leaves poin A shown in the figure below choosing at random one path froi AB AG AD aldAE At each subsequenljunction she chooses another path at rmdom What is the probability trat she arrives at poirt X1 I 139 Three primers do work for the publications office of Georgia Thch The publications office does not negotiate a contraCt penalty for late work and the data below reflect a large amount of experience vith these pmters Printer i 1 2 3 Fraction of Contracts Held Printer i 02 03 05 Fraction of Deliveries More tha1 One Month Late 02 05 03 A department obscrves that its recruiting bookIe is more th2n a month late Wha is the probability tlat the contract is heM by printer 3 32 Chapter I An Inrroduction to Probability 140 Following aircraft accidents a detailed invesLiga tion is conducted The probability that an accident due to structural failure is correctly identified is 09 and the probability that an accident that is not due to structural failure being identified incorrectly as due to structural failure is 02 If 25 of all aircraft accidents are due to structural failures find the proba bility that an aircraft accident is due to structural failure given that it has been diagnosed as due to structural failure Chapter 2 OneDimensional Random Variables 21 DlTRODUCTIO The objectives of this chapter are to introduce the concept of random variables to define and illustrate probabilitydisoributions and cumulative disoribution functions and to present useful characterizatiollsfor vaiables When describing the sample space of a random experiment it is not necessary to spec ify that an individual outcome be a number In several examples we observe this such as in Example 19 where a true coin is tossed tlrree times and the sample space is 1 HHB HIfT HnI HIT THH THT 7TH TlTj or ExampJe 115 where probes of solder joints yield a sample space 1 GG GD DG DD In most experimental situations however we are interested in numerical outcomes For example in the illustration involving coin tossing we might assign some real number x to every element of the sample space In general we want to assign a real number x to every out come e of the sample space if A functional notation will be used initially so that x Xe whCreX is the function The domain of X is f and the numbers in the range are real numbers The functi0n Xis called arandom varible Figure 21 illustrates the nature of this fu1cnon Definition Ift is an experiment having sample space and X is aunction that assigns a real num ber Xe to every outcome e E if tbenXe is called a random variable pI Consider the cointossing eXperiment discussed in the preceding paragraphs If X is me fnrnbcr of heads showtng thee XHHlI 3 XHlflJ 2 XliTH 2 XHT 1 XTHH 2 XTH1 1 XTm I and XTTI 0 The ta1ge spaceRx xx 0 t 2 3 in t1is example see Fig 22 Rx e Xel e b Figure21 The concept of a random variabk al The sample space ofil bR The range space of X 33 34 Chapter 2 OneDimensional Random Variables Of Rx TIT TTH 0 THT HIT 1 HHT HTH THH 2 HHH 3 X Figure 22 The number of heads in three coin tosses The reader should recall that for all functions and for every element in the domain there is exactly one value in the range In the case of the random variable for every outcome e E g there corresponds exactly one value Xe It should be noted that different values of e may lead to the same x as was the case where XITH I XTHTJ I and XHTl I in the preceding example Where the outcome in g is already the numerical characteristic desired thenXe e the identity function Example 113 in which a cathode ray tube was aged to failure is a good example Recall that g t OJ If X is the time to failure then Xt t Some authors call this type of sample space a numericalvalued phenomenon The range space Rx is made up of all possible values of X and in subsequent work it will not be necessary to indicate the functional nature of X Here we are concerned with events that are associated with Rx and the random variable X will induce probabilities onto these events If we return again to the cointossing experiment for illustration and assume the coin to be true there are eighlqually likely outcomes HHH HHT liTH HIT THH THY TTH and ITT each having probability t Now suppo i the event exacJly two heads and as previously we let X represent the number of hadssee Fig 22 The event I 3 that X 2 rehltes Rxno3jhoeverpxX 2 peA 8 since A HHT liTH THHris the equivalent event in g and probabilitywas definedon events in the sample space The random variable X induced the probability of f to the event X 2 Note that parentheses will be used to denote an event in the range of the random variable and in gen eral We will write PxX x In order to generalize this notion consider the following definition Definition If g is the sample space of an experiment and a random variable X with range space Rx is defined on g and furthermore if event A is an event in g while event B is an event inR then B ar equivalent events if AeE gXeEBj Figure 23 illustrates this concept More simply if event A in g consists of all outcomes in g for whichXe E B then A and B are equivalent events Whenever A occurs B occurs and whenever B occurs A occurs Note that A and B are associated with different spaces 21 Introduction 3S RX B x Figure 23 Equivalent events Definition If A is an event in the sample space and B is an event in the range space Rx of the random variable X then we define the probability of B as PxB PA where A leE 9XeE B With this definition we may assign probabilities to even in Rx in terms of probabili ties defined on events in 1 and we will suppress the function X so that P xX 2 f in the familiar cointossing example means that there is an event A HHT HTH THH e Xe 2 in the sample space with probabilityt In subsequent work we will not deal with the nature of the function X since we are interested in the values of the rangc space and their associated probabilities Vihile outcomes in the sample space may not be real num bers it is noted again that all elements of the range of X are real numbers An alternative but similar approach makes use of the inverse of the function X We would simply define XB as so that Table 21 Equivalent EVents Some Events in Ry Y2 Yd Y4 Y5 Y6 Y7 Y8 Y9 Y 10 Y 11 Y12 XB e E 9 Xe E B P xB PXlB P Equivalent Events in 9 I I I 2 2 I l 3 2 2 3 I l 4 2 3 3 2 4 I l 5 2 4 3 3 4 2 5 I I 6 2 5 3 4 4 3 5 2 6 I 2 6 3 5 4 4 5 3 6 2 3 6 4 5 5 4 6 3 4 6 5 5 6 4 5 6 6 5 66 Probability j6 2 j6 J j6 36 36 j6 j6 j6 J j6 j6 36 Chapter 2 OneDimensional Random Variables The following examples illustrate the sample spacerange space relationship and the concern with the range space rather than the sample space is evident since numerical results are of interest ElI22 Consider the tossing of two true dice as described in Example 111 The sample space was described in Chapter 1 Suppose we define a random variable Yas the sum of the up faces Then Ry 2 3 4567 8 9 10 11 12 and the probabilities arc 316 ft 6 6 16 3 6 k fc respee tively Table 21 shows equivalent events The reader will recall that there are 36 outcomes which since the dice are true are equally likely One hundred cardiac pacemakers were placed on life test in a saline solution held as close to body temperature as possible The test is functional with pacemaker output monitored by a system pro viding for output signal conversion to digital form for comparison against a design standard The test was initiated on July 1 1997 Vlhen a pacer output varies from the standard by as much as 10 this is considered a failure and the computer records the date and the time of day d t If X is the random variable time to failure then d t d date t time and Rx x x a The random vari able X is the total number of elapsed time units since the module went on test We will deal directly with X and its probability law This concept will be discussed in the following sections 22 THE DISTRIBUTION FUNCTION As a convention we will use a lowercase of the same letter to denote a particular value of a random variable Thus X x eX yx eX x are events in the range space of the random variable X where x is a real number The probability of the event ex x may be expressed as a function of x as Fxx Pix x 21 This function Fx is calledthe distribution junction or cwnulative function or cumulative dis tribution function CDF of the random variable X In the case of the cointossing experiment the random variable X assumed four values 0 1 2 3 with probabilities t t t t We can state F Jx as follows Fx 0 t 8 4 8 7 8 1 A graphical representation is as shown in Fig 24 xO 0x1 tx2 x3 Fx 1 78 0 48 0 18 o 2 3 22 The Distribution Function 37 x Figure 24 A distribution function for the number of heads in tossing three true coins fipj Again recall Example 1 13 when a cathode ray tube is aged to failure Now f t t O and if we let X represent the elapsed time in hours to failure then the event X x is in the range space of X A mathematical model that assigns probability to Jfx is FXx 0 le1r x 0 xo where is a positive number called the failureJH cfailureslhour The use of this exponential model in practice depends on certain assumptions abot the failure process These assumptions will be presented in more derail later A graphical representation of the cumulative distribution for the time to failure for the CRT is shown in Fig 2 5 A eustomer enters a bank where there is a common waiting line for all tellers with the individual at the head of the line going to the first teller that becomes available Thus as the customer enters the waiting time prior to moving to the teller is assigned to the random variable X If there is no one in line at the time of arrival and a teller is free the waiting time is zero but if others are waiting or all tellers are busy then the waiting time will assume some positive value Although the mathematical form of F X depends on assumptions about this service system a general graphical representation is as illustrated in Fig 2 6 Cumulative distribution functions have the following properties which follow directly from the definition 1 0 FXx 1 ooxoo Fx o x Figure 2 5 Distribution function of time to failure for CRT 38 Chapter 2 OneDimensioDal Random Variables L Probability of no wat oLox Figure 26 Waitingtime distribution function 2 limxXx I limJ xtl O 3 The function is nondecreasing That is if Xl S x then F Xt F X 4 The function is continuous from the rigbl That is for all X and Ii 0 pm Fxx Ii Fxx 0 00 In reviewing the last three examples we note that in Example 24 the values of x for which there is an increase in F x were integers and where x is not an integer then F jx bas the value that it had at the nearest integer X to the left In this case Fxx bas a saltus or jump at the values 0 I 2 and 3 and proceeds from 0 to I in a series of such jumps Exam ple 25 illustrates a different situation wbere F xt proceeds smoothly from 0 to 1 and is continuous everywhere but not differentiable at x O Finally Example 26 illustrates a sit uation wbere there is a saltus at X 0 and for x O F x is continuous Using a simplified ann of results from the zbesgue decomposition thect it is noted that we can represent F Xx as the sum 6 two component functions say GP and HP or F P Gxx Hxx 22 where GXx is eontinuous and Hix is a rightband continuous step function with jumps coinciding with those of FP andHX Olf Gxt 0 for all x then X is called adis crete random variable and if Hxx 0 then X is called a continuous random variable Where neither situation bolds X is called a mixed type random variable and although this was illustrated in Example 26 this uXt will concentrate on purely discrete and continuous random variables since most of the engineering and management applications of statistics and probability in this book relare either to counting or to simple measurement 23 DISCRETE RAIDOM VARIABLES Although discrete random variables may result from a variety of experimental situations in engineering and applied science they are often associated with counting If X is a discrete random variable then F Xx will bave at most a countably infinite number of jumps and Rx x Xk Suppose hat the number of worldng days in a particular year is 250 and that the records of employ ees are marked for each day they are absent from work An experiment consists of randomly selecting 23 Discrete Random Variables 39 a record to observe the days marked absent The random variable X is defined as the number of days absent so that Rx O 12 250 Ibis is an example of a discrete random variable with a finite number of possible values A Geiger counter is connected to a gas tube in such a way that it will record the background radiation count for a selected time interval 0 tJ The random variable of interest is the count If X denotes the random variable then Rx O 1 2 k and we have at least conceptually a countably infi nite range space outcomes ean be placed in a onetoone correspondence with the natural numbers so that the random variable is discrete Definition If X is a discrete random variable we associate a numbepXx PxX xilwith each out come Xi in Rx for i 1 2 n where the numbers PTxi satisfy the following I px 0 for all i 2 2 Pxx I We note immediately that I px Fix FXxH v 23 and Fxx pxx Xi 2pxx 24 xSx The function Px is called the probability function or probability mass fnction or prob ability lall of the random variable and fucollection of pairs xi px0i2 ris called the probability distribution of X The functionpx is usually presented in either tabu Jfr grap91 or matktjcCl fIm as illustrated in the following examples jmf2 For the cointossing experiment of Example 19 where X the number of heads the probability dis tribution is given in both tabular and graphical form in Fig 27 It will be recalled tha O Tabular Presentation x o 1 2 3 pix 18 38 38 18 18 o Graphical Presentation 38 38 18 2 3 Figure 27 Probability distribution for cointossing expcriment x 40 Chapter 2 OneDimensional Random Variables Suppose we have a random variable X with a probability distribution given by the relationship Pxxi JpXIPx xO l n 0 otherwise 25 where n is a positive integer and 0 p 1 This relatiouship is known as the binomial distribution and it will be studied in more detail later Although it would be possible to dipiay this model graphical or tabular form forpartieular n andp by evaluating pf for x 0 12 n this is seldom done in Recall the earlier discussion of random sampling from a finite population ithout replacement Sup pose there are N objects of which D are defective A random sample of size It is selected Vlithout replacement and if we let X represent the number of defectives in the sample then DYNDI Pxx It X 0 1 2 min n D 0 otherwise 26 This distributiol is knovIl as the hypergeometric distribution In a particular case suppose N 100 items D 5 items and n 4 then 0 otherwise In the event that either tabular Or graphical presentation is desired this would be as shown in Fig 28 hOVfever Uuess there is some special reason to use these forms we will use the mathematical relationship Tabuar Presentation Graphica Present9tion x px 50 0B051 0 0805 i m 5 CO 0178 2 5 O 0014 px 3 m 95C0 Q003 0178 4 9 0000 0Q14 0003 0 1 2 3 4 x Figure 28 Some hypergeometric probabilities N 100 D 5 r 4 24 Continuous Random Variables 41 jir1 In Example 28 where the Geiger countel was prepared fol deteetirg the background radiation count we might use the following relationship which bas been experimentally sbown to be appropriate pix etx x 0 12 i 0 0 oiliewise 27 This is called the fzjQJLJi1JibUtiO1 and at a later point it will be derived analytically The param eter is the mean ratem hitsper unit time and x is rhe number of rhese hits These examples have illustrated some discrete probability distrihutions and alternate means of presenting the pabs XiPXi i 12 In ater sections a number of proba bility distributions will be developed each from a set of postulates motivated from consid erations ofredlorzdphino A general graphical presentation of the discrete distribution from Example 212 is given in Fig 29 This geometric interpretation is often useful in developing an intuitive feeling for discrete distributions There is a close analogy to mechanics if e consider the probability distribution as amass of one unit distributed over the real line in amounts Pxxi at points XI i 12 n Also utilizing equation we note the following useful re ult where b2 a 28 24 CONTINUOUS RANDOM VARIABLES Recall from Section 22 that where Hxx O X is called continuous Then Fxx is contin uous Fix has derivative fb dldx Fx for all x with the exception of possibly a countable number of values and fxx is piecewise contiuuous Under these conditions the range space Rx will consist of one or more intervals An interesting difference from the case of discrete random variables is that for i5 0 We define the probability density function f Jx as d fxx dx Fxx Pxx XX2X3X4XJnxn Figure 29 Geometric interpretation of a probability distribttion 29 HO 42 Chllprer 2 OneDimemional Random Variables and it follows that 211 We also note the close correspondence in this form with equation 24 with an integral replacing a summation symbol and the following properties off 1 fO forallxE Rx 2 J fxrdx L 3 fAx is piecewise continuous 4 fAx 0 if x is nOt in the range Rx These concepts are illustrated in Fig 210 This definition ofa density function stipu lates a function fx defined on Rx such that 212 where e is an outcome in the sample space Vve are concerned only with Rx and lx It is important to realize thatfxx does not represent the probability of anything and that only when the function is integrated between two points does it yield a probability Some comments about equation 29 may be useful as this result may be counterintu itive If we allow X to assume all values in some interval then PxX xo 0 is not equiv alent to saying the event X xo in Rx is impossible Recall that if A 0 then P jA 0 however although PiX xol 0 the fact that the setA x X xo is not empty clearly indicates that the converse is not true All immediate result of this is that Pxa X b Pxa X b Pxa X b Pxa X b where X is continuous all given by Fxb Fxa iNl3ii The time to failure of the cathode ray tube described in Example 113 has the follOwing probability density function O otherwise where A 0 is a eonstant knovU as ilie faiure rate This probability density function is called the exponentiol density and experimental evidence has indicated that it is appropriate to describe the time x x Figure 210 HJTOthetical probability density function 24 Contimous Random Vaiables 43 to failure a realworld occurrence for some types of components In this example suppose we want find PfIOO hours This is equivalent to stating P 100 T and PT T100 r kdt h etoo We might again employ the cOncept of conditional probability and determine P T21OOIT 99 the probability the tube lives at least 100 hours given that it has lived beyond 99 hours From our earlier wok PlT lOOT 99 PAT 100 and T 99 PPT 99 10 e 9t e e A random variable X has the triangular probability density fuuction given below and shown graJrl cally in Fig 211 jx x 2x 0 Thefollowing are calculated for illustration 1 plXfOI2 xd1 2 I 8 3 f 1 11 2 2 pxX 0 xd 2x 2 0 1 1 x1 7 2 2XZJ g 3 PX 3 1 4 P X 25 O 5 Px X xdHf 2x 1 3 r 4 2 Jj4 J 15 3 27 32 8 Ox 1 1 Sx2 otherwise I n o 2 x Figure 211 Triangular density function 44 Chapter 2 OneDimensional Random Variables In describing probability density functions a mathematical model is usually employed A graphical or geometric presentation may also be useful The area under the density func tion corresponds to probability and the total area is one Again the student familiar with mechanics might consider the probability of one to be distributed over the real line accord ing to Ix In Fig 212 the intervals a b and b c are of the same length however the probability associated with a b is greater lrT 25 SOME CHARACTERISTICS OF DISTRIBUTIONS While a discrete distribution is completely specified by the pairs Xl Pxex i I 2 n and a probability density function is likewise specified by exxex x E Rx itis often convenient to work with some descriptive characteristics of the random variable In this sec tion we introduce two widely used descriptive measures as well as a general expression for other similar measures The first of these is the first moment about the origin TIris is called the mean of the random variable and is denoted by the Greek letter J1 where for discrete X for continuous X 213 This measure provides an indication of central tendency in the random variable Returning to the cointossing experiment where X represents the number of heads and the probabil ity distribution is as shoNIl in Fig 27 the calculation of J1 yields f 1 XPXXI0mlJ2HJ3J 1 as indicated in Fig 213 In this particular example because of symmetry the value J1 eould have been easily detenninedfrom inspection Note that the mean value in this example cannot be obtained as the output from a single trial Er11 In Example 214 a density Ix was defined as a Figure 212 A density function AN X 2 0 b c OX I Ix 2 otherwise x 25 Some Characteristics QfDistribunons 45 316 38 118 18 o 2 3 x 32 Figure 213 Calculation of the mean The mean is determined by JJ fxXdtfX2Xdx fOdxftOdx4 another result that we could have determined via symmetry Another measure describes the spread or dispersion of the prebability associated with elements in RJ This measure is called the variance denoted by cr and is defined as i t follows for discrete X for continuous X 214 This is the second moment about the mean and it corresponds to the moment of inertiain mechanics Consider Fig 214 where two hypothetical discrete distributions are shown in graphical fonn ote that the mean is one in both cases The variance for the discrete ran dom variable shovn in Fig 214a is 0 01 11 i21 i and the variance of the discrete random variable shown in Fig 214b is 0 11 101 1 11 5 5 I 1 21 531 5 2 WIDch is four times as great as the variance of the random variable shown in Fig 214a If the units on the random variable are say feet fuen the units of the mean are the same but the units on the variance would be feet squared Another measure of dispersion called the standard deviation is defined as the positive square root of the variance and denoted owhere j2 215 It is noted that the units of jare the same as those of the random variable and a small value for jindicates little dispersion whereas a large value indicates greater dispersion 46 Cbllpter 2 OneDimensiorull Random Variables PxIA 12 114 114 o 2 x 1 o 2 3 x alu and 00 05 Figure 214 Some hypothetical distributions An alternate form of equation 214 is obtained by algebraic manipulation as for discrete X 1 2 x f xxdx1 for continuous X 216 This simply indicates that the second moment about the mean is equal to the second moment about the origin less the square of the mean The reader familiar with engineering mechanics will recognize that the development leading to equation 216 is of the same nature as that leading to the theorem of moments in mechanics Using the alternate form 2 2 1 2 3 2 3 2 1J 3 3 f lO 81 82 83 8 2 4 which is only slightly easier 2 Binomial distributionExample JalO From equation 213 we may show that jJ P and or Jtx n 111 pnz1np2 I rl x I wbich after some algebta simplifies nplp 3 JExpoen1iaJ dismbutionEuvnpie 213 Consider the density functionftx where Nx2e x 0 0 otherwise Tnen using integration by parts o and 4 Another density is exCx where 25 Some Characteristics of Distributions 47 x gx 16re4 xlO 0 otherwise 9xX o x rnen and Note that the mean is the same foe the densities in parts 3 and 4 with part 3 having a variance twice that of part 4 In the development of the mean and variance we used the terminology mean of the random variable and variance of the random variable Some authors use the terminology mean of the distribution and variance of the distribution Either terminology is accept able Also where several random variables are being considered it is often convenient to use subscript on If and Y for example Ifx and Yx In addition to the mean and variance other mOments are also frequently used to describe distributions That is the moments of a distribution describe that distribution measure its properties and in certain circumstances specify it 1oments about the origin are called ongin moments and are denoted J4 for the kth origin moment where 48 Chapter 2 OneDimensional Random Vaiables for discrete X fxxdx for continuous X k 012 217 Moments about the mean are called central moments and are noted Ilk where P IXpkpXx for discrete X for continuous X 218 Note that the mean jJ jJ and the ariance is 0 2 i12 Central moments may be expressed ill terms of origin moments by fue relationship ik ILkIHljpJpj kOI2 219 jO J1 26 CHEBYSHEVS ThEQUALITY In earlier sections of this chapter it was pointed out that a small variance 0 2 indicates that large deviations from the mean Jll are improbable Chebyshevs inequality gives us a way of understanding how fue variance measures the probability of deviations about IL Thwrem 21 Let X be a random variable discrete or continuous and let k be some positive number Then 120 Since it follows that 1 JPK 1 JW 1 f x IL fxxdx x IL fxxdx JtJK Now x ILl K if and only if b JLi VK fuerefore Jlfi r J KfxxdxJ mKfXxdx ftTvK KPx X IL fK pxx p fK and so that if k YKcr then The proof for discrete X is quite similar An alternate form of this inequality or is often useful PxlXIlIkcr 1 1 PxllkcrXIlkcr II 27 Summary 49 221 The usefulness of Chebyshevs inequality stems from the fact that so little knowledge about the distribution of X is required Only J1 and 02 must be knOVll However Cheby shevs inequality is a weak statement and this detracts from its usefulness For example pxix 111 cr 1 which we knew before we started lithe precise form oflxx or Pxx is known then a more powerful statement can be made From an analysis of company records a materials control manager estimates that the mean and stan dard deviation of the lead time required in ordering a small valve are 8 days and 15 days respee tively He does not know the distribution of lead time but he is willing to assume the estimates of the mean and standard deviation to be absolutely correct The manager would like to determine a time interval such that the probability is at least that the order will be received during that time That is 1 8 1 k 9 so that k 3 and j1 ka gives 8 315 or 35 days to 125 days It is noted that this interval may very well be too large to be of any value to the manager in which case he may elect to learn more about the distribution of lead times 27 SUMMARY TItis chapter has introduced the idea of random variables In most engineering and man agement applications these are either discrete or continuous however Section 22 illus trates a more general case A vast majority of the discrete variables to be considered in this book result from counting processes whereas the continuous variables are employed to model a variety of measurements The mean and variance as measures of central tendency and dispersion and as characterizations of random variables were presented along with more general moments of higher order The Chebyshev inequality is presented as a bound ing probability that a random variable lies between Ji ka and Ji ka i 50 Ch2pter 2 OneDimensional Randoa Variables 28 EXERCISES 21 A fiveard poker hand may contain from zero to f aces If X is the random variable denoting the number of aces enumerate the nmge space of X Vlhat are the probabilities associated with each possible vitue of Xl 22 A car rental agency has either 0 1 2 3 4 or 5 th bahil I I J ears returned each U4y Wi pro lUes 6 6 J 12 t and h respectively Fmd the mean and the vari ance of the number of cars returned random variable X has the probability density function cex Find the proper value of c assuming 0 X ZI Find the mean and the variance of X 24t The cumulative distribution function that a tele vision tube will fail in hours is 1 et where c is a parameter dependent on the manufacturer and t 2 0 Find the probability density function of T the life of the tuhe Consider the three functions given below Deter mine which functions are distribution functions CDFs 07J a Fodx 1 h GJx e 0 e HJxe 1 OX Osxoo xO xSO xO x012 0 otherwise Find the probability that he will have suits left over at the seasons end 2 3 J zlO A ra1dom variable X has a CDF of the form 1 V1 Fxx 12 x012 0 x a Find the probability function for X h Find P odO X 8 2Ml1 Consider the following probability density fLction fxfxa Ox2 k4x 2x4 0 o1heJ1lf1se a Fmd the value of k for which f is a probbility demity function b Find the mean and variance of X c Find the C1JlJulative distributionfuncdon 212 Rework the above problem except let the prob ability density function be defined as fxlo Oxa 26 Refer to Exercise 25 FInd the probability den k2a x ax 2a sity function corresponding to the functions given if 0 otherwise they are distritaltion functions al The manager of a job shop does not know the lJWbich of the followillg functions are screte pr05ability distribution of the time required to com probability distributioDs plete an order Howeler from past pcdormance she 1 has been able to estimate the mean and variance as 4 va Px x 3 xO deys and 2 deys respectively Fmd an interval such 2 xl 0 1l 0 othenvise h 512 151 Pxx x J3 j ld x 012345 0 otherwise 2 8 The demandfof a product isl 0 1 2 per day th b biliti J J 2 3 1 Ad d W1 pro a es SIO5lDrespectlvey eman of 1 implies a Ul1it is returned Find the expected demand and the variaxce Sketch the distribution nCDF f r manager 0 a men s clothing store 1 con Iled over the inventory of suits which is currently 30 all sizes The number of suits sold from nOw to te end of the seasor is distributed as that the probability is at least 015 that an order is fin ished during that time 214 The continuous random variable Thas the prob ability density funcnonj ki for I 5 O Find 1hc following a The appropriate value of k h The mean and variancc of T c The cumulative distribution function 215 A discrete random variable X has probability functionpXx where plx k12t 0 a Fmd k xI23 otherwise h Fmd the mean and variance of X c Fmd the cumulative distribttion ctionFxfx 216 The discrete random variable N N 0 1 has probabilities of OCC1iTCnce of kf1 0 r 1 Find the appropriate value of k 211Ihe postf service requires on the average 2 days to deliver a letter across town The variance is estimated to be 04 dayyz If a business executive wants at least 99 of his letters deLvered on time how early should he mail them 218 Two different real estate developers A and B On pacels of land being offered for sale The proba bility distributions of selling prices per parcel are shoWTI in the following ab1e Price Sl000 1050 Sl100 1150 1200 1350 A B 02 01 03 01 01 03 03 03 005 005 01 01 Assuming that A and B are operating independently corrqJute the foCowilg a The expected selling price of A and of B b The expeeted selling price of A given that the B selling price is 1150 c The probability marA and B both have the same selling price 219 Show that the probability function for the sum of values obtained Ll tossing two dice may be written as xI PxX 36 x23 6 13x 36 x78 12 22 Fbd the mean and variance of the rando1 vari able whose probability furction is defined in the pre vious problem 28 Exercises 51 221 A continuous random fanabIe X has a density function 2x fx9 o x3 O otherwise a Deveop the CDF for X V Il1 Find the of X and the VlIIWnce of X lc Find 14 jr rd F1nda value m such that PIX lm PxX 5m This is called the median of X 5ff2 222 Suppose X takes on the values 5 and 5 with probabilities t Plot the quantity PJX 11 kr as a ftmction of k for k 0 On the same set of axes plot the same probability determined by Chebyshevs inequality lmLnd the cumulative distribction function asso dated with X If x 2 fxxr exp 2 2t I t020 O otherwise 224 Find the cumulative distribution function asso ciated with 225 Coasider tie probability density unctioa fy k sin y 0 y S 7If2 What is the appropriate ruue of k Find the mean of the distribution 226 Show tba central momenS can be expressed in ter1s of origin moments by eqaUon 219 Hint See Chapter 3 of Kendall and Stuart 1963 Chapter 3 Functions of One Random Variable and Expectation 31 INTRODUCTION Engineers and management scientists are frequently interested in the behavior of some function say H of a random variable X For example suppose the circular crosssectional area of a copper wire is of interest The relationship Y 1tX24 where X is the diameter gives the crosssectional area Since X is a random variable Y also is a random variable and we would expect to be able to determine the probability distribution of Y HX if the dis tribution of X is known The first portion of this chapter will be concerned With problems of this type This is followed by the concept of expectation a notion employed extensively throughout the remaining chapters of tlris book Approximations are developed for the mean and variance of functions of random variables and the momentgenerating function a mathematical device for producing moments and describing diytributions is presented with some example illustrations 32 EQUIVALENT EVENTS 52 Before presenting some specific methods used in determining the probability distribution of a function of a random variable the concepts involved should be more precisely formulated Consider an experiment with sample space g The random variable X is defined On g assigning values to the outcomes e in g Xe x where the values x are in the range space Rx of X Now if Y HX is defined so that the values y Hc in Ry the range space of Y are real then Y is a random variable since for every outcome e E g a value y of the random variable Yis determined that is y HlXe This notion is iliustrated in Fig 31 If C is an event associated With Ry and B is an event in Rx then B and C are equivalent eventsiftbey occur together that is ifB XE RxHx E C In addition if A is an event asso ciated With g and furthermore A and B are equivalent thenA and C are equivalent events s RX Ry x f y Hx Figure 31 A function of a random variable 32 Equivalent Events 53 Definition If X is a random variable defined on ff having range space Rx and if H is a realvalued function so that Y HX is a random lariable with range space R y then for any event C c R we define 31 It is noted that these probabilities relate to probabilities in the sample space We could write PCPee 9 HXe E C However equation 31 indicates the method to be used in problem solutiou We find the event Bin Rx that is equivalent to event C in Ry then we find the probability of event B fl In the case of the crosssectional area Yof a vire suppose we know tmt the diamete of a Vvire has density function LOOO x L0Il5 othervise Let Y 1t14X be the crosssectional area of the wire and suppose we want to find py 101ri4 The equivalent event is deteIpJined PrY 10Jri4 P xri4X 101ri4j pxiX1 JiJiI The event x E Rxlxl JJij1 is in the range space Rxmd sinceJxx 0 for all x l0 we calculate r W OI PxliXi101 PxILOXvLOI 200dx 1000 09975 rj1YlJ In the case of the Geigercouater experiment of Example 212 we used the distribution given in equa tion 27 px MYX x 0 12 O orlerwise Recall that A where A 0 represents the Clean hit rate and l is the tirJe interval for which the counteI is operated Now suppose we wish to find P Y5 5 where Y2X2 Proceeding as in the previous example PxO Pxlj e MO 1 O le M 1 eIMJ The event XE Rx xst ism the rnngespace ofX and we have thefuoctlonpx to work with in that space 54 Chapter 3 Functions of One Random Variable and Expectation 33 FIDlCTIONS OF A DISCRETE RANDOM VARIABLE Suppose that both X and Yare discrete random variables and letxix21 xij represent the values of X such that Hx Yi for some set of index values D U j 1 2 s The probability distribution for Y is denoted EiY given by I i PYYi pyY y LpxtxdI 32 jfJQ L For example in Fig 3 A where s 4 the probability of yls phD Pxx pxxJ Plx pxJ In the special case where H is such that for each y there is exactly one x then ph Pxx where y Hx To illustrate these concepts consider the following examples In the cointossing experiment where X represented the number of heads recall that X assmned four values 0 12 3 ith probabilities t t t t If Y 2X I then the possible values of Y ae 1 1 3 5 and p 1 t pyl 1 py3 1 pyS In this case H is such that for each y there is exactly one x i21i1 V Xis as in the previollS example however suppose now that Y 21 so that the possible values of Yae 0 1 2 as indiCllted in Fig 33 In this case Rx Xi I I Xi i I Xl I Xil Figure 32 Probabilities in Ry H pyO Px2t 4 pyl Pxl px3S 1 pA2 PxOg Ry 34 ConrIuous Fuerioas of a Coatinuous Random Variable S5 Ax yHxx21 Ay rrl101 111 8 Figure 33 An example rmccon H In the event that X is continuous but Y is discrete the formulation for pfy is where the event B is the event that is equivalent to tlre eveot Y y in R yo iXIl Suppose X has the exponential probability density function given by N kr X 0 Furthermore if and 0 otherwise yo forXS l for X 1 34 CONTIlilJOUS FUNCTIONS OF A CONTINUOUS ROMVARIABLE 33 lfX is a continuousrandom variable with probability density functionf and His also con tinuous tben Y HX is a continuous random variable The probability density function for the random variable Ywill be denotedfyand it may be found by performing three steps 1 Obtain the CDFof Y Fy PY y by finding the eventBinRx which isequiv alent to the ovem Y y in Ry 2 Differentiate Fy with respect to y to obtain the probability density functionfty 3 Find the range space of the new random variable 56 Chapter 3 Functions of One Random Variable and Expectation xie Suppose that fue random variable X has the following density function fxx xJg 0 x A 0 omewise If y HX is therendom variable for wbich the density ilis desired and H 0 8 as shovmin Fig 34 then we proceed according to the steps given above f y RY 1 h r 32 4 3 If xO 8 and ifx4 16 so then we have y 1 trY 32 4 gy16 0 otbTNise Y Hx 1 I I Hex 2x8 16 i Q FiJlUl 34 The function Hx 2x S y Hx HX x 22 Figure 35 The function Hex x 2 34 Continuous Functions of a Continuous Random Variable 57 li Consider the random variable X defined in Example 36 and suppose YHX X 2 as sbown in Fig 35 Proceeding as in Example 36 we find the following 1 Fh PYy PxX 21 y pxJY X 2 JY Px2JY H2JY 25 x x2 Y tY8dx16 c 20 I 4 4JY i44JY Y JY 2 2 fyy Fyy Ir 4y 3 If x 2 Y O and if x 0 or x 4 then y 4 Howeverfy is not defined for y 0 therefore I fyY4JY Oy4 0 othenvise In Example 36 the event in Rx equivalent to y y in Ry was X y 812 and in Example 37 the event in Rx equivalent to Y y in Ry was 2 JY X 2 JY In the first example the funetion H is a strictly increasing function of x while in the second example this is not the case Theorem 31 If X is a continuous random variable with probability density functionx that satisfiesX 0 for a x b and if y Hx is a continuous strictly increasing or strictly decreasing function of x then the random variable Y HX has density function lfXrII 34 with XHly expressed in terms ofy IfHitiiicreasmg theny 0 if Ha y Hb and if H is decreasing theny 0 if Hb y Ha Proof Given only for H increasing A similar argument holds for H decreasing Fy pY y PHX y pXXWly FxW1y dFxx dx y y Fy y dy by the chain rule xx dx wherexWy dy 58 Chapter 3 Functions of One Ramlom Vble aru Expectation In Example 36 we hall Ux xl8 Ox 4 O otherwise andFJx 2x 8 which is a strictly increasing function Using equation 34 tyy tXXidxi y8 dy 16 Z since x y 82 RO 8 and R4 16 therefore ry c 8y16 32 4 0 otherwise 35 EXPECTATION If X is a random variable and Y HX is a function of X bell the expected value of HX is defined as follows 35 EHX r Hx fxxdx for X conti 36 In the case where X is continuous we restrict H so that Y HX is a continuous random variable In reality we ought to regard equations 35 and 36 as theorems not definitions and these results have come to be known as the law of the unconscious statistician The mean and variance presented earlier are special applications of equations 35 and 36 If HX X we see that EHX EX 1 37 Therefore the expected value of be random variable X is just lhe mean p If HX X J1 then EHX EX 11 d 38 Thus lhe variance of the random variable X may be defmed in terms of expectation Since the variance is utilized extensively it is customary to introduce a variance operator V that is defined in terms of the expected value operator E VHX EHX EHX Again in the case where HX X VeX EX EXil EX EX which is the vanante of X denoted dl 39 HO The origin moments and central moments discussed in the previous chapter may also be expressed using be expected value operator as J1 EXl 311 35 Expectation 59 and ilk EX EXj There is a special linear function H that should be considered at this point Suppose that HX aX b where a and b are constants Then for discrete X we have c I EaX bl Iax b Pxx a 2xiPxxb 2Pxx i faEXbJ 312 and the same result is obtained for continuous X namely Using equation 39 axbfxxdx a xfxxdxbfxxdx aEXb VaX b EaX b EaX b EaX b aEX bJ aE X EX 2 313 tivX 314 In the further special case where H X b a constant the reader may readily verify that Eb b 315 and Vb O 316 These results show that a linear shift of size b only affects the expected value not the variance Suppose X is a random variable such that EX 3 and VeX 5 In addition let HX 2X 7 Then EIHX E2X 7 2EX 7 1 and VHX V2X 7 4VX 20 The next examples give additional illustrations of calculations involving expectation and variance Suppose a contractor is about to bid on ajob requiring X days to complete where X is a random vari able denoting the number of days for job completion Her profit P depends on X that is P HX The probability distribution of X x px is as follows 60 Chapter 3 Functions of One Random Variable and Expectation x Pxxl 3 8 4 5 ii 5 8 otherwise O Using the notion of expected value we calculate the mean and variance of X as and 1 5 2 33 EX 345 8 8 8 8 VX32452J332 23 8 8 8 8 64 If the function HUG is given as x Hx 3 10000 4 2500 5 7000 then the expected value of HX is E HX 10000 m 2500 7000 i 106250 and the contractor would view this as the average profit that she would obtain if she bid this job many many times actually an infinite number of times where H remained the same and the random vari able X behaved according to the probability function Px The variance of P HX can readily be calculated as V HX IOOOO 2 25002 7000 J 106252 888 2753 10 A wellkn simple inventory problem is the newsboy problem described as follows A newsboy buys papers for 15 cents each and sells them for 25 cents each and he cannot return unsold papers Daily demand has the following distribution and each days demand is independent of the previous days demand Number of customers x 23 24 25 26 27 28 29 30 Probability pIx 001 004 010 010 025 025 015 010 If the newsboy stocks too many papers he suffers a loss attributable to the excess supply If he stocks too few papers he loses profit because of the exeess demand It seems reasonable for the newsboy to 35 Expecation 61 stock some numberbf tapers so as to ninirnize the expected loss If we let s represent the number of papers stocKed Xtbe daLy demand andLX s the newsboys loss for a pa1icular stock level then the loss is simply LX s OlOX s 015sX ifXs ifXs and for a given stock level s the expected loss is 3ll ELXs 2015sx pxCx 2OIOxs Pxx o rl and the ELX sI is evaluated for some different values of s For 326 ELX 26 01526 23001 26 24004 26 25010 Fors27 26 260101 01027 26025 28 26025 29 26015 30 26010 01915 ELX 27 015i27 23001 27 24004 07 25010 27 26010 27 27025 01028 27025 29 27015 30 27010 0 154 For 28 BILX 28J 01528 23001 28 24004 28 25010 28 26010 28 27025 28 280251 01029 28015 30 28010 01790 Thus the newsboys policy should be to stock 27 papers if he desires to minimize his expected loss Consider the redundant system sbam in the diagram below At least one of the units must function the redundancy is standby meaning that the second unit does not operate until the first falls sMtching is perfect and the system is nonmaintained It can be shown that det certain conditions when the time to failure for each of the units of this system has an exponential distribution then the time ttl failure for the system has the following probability den sity function 62 Chapter 3 Functions of One Random Variable and Expectation fx Jxe x 0 J 0 0 otherwise where A is the failure rate parameter of the component exponential models The mean time to fail ure MTTF for this system is Hence the redundancy doubles the expected life The terms mean time to failure and expected life are synonymous 36 APPROXIMATIONS TO EHX AND VHX In cases where HeX is very complicated the evaluation of the expectation and variance may be difficult Often approximations to EHX and VHX may be obtained by uti lizing a Taylor series expansion This technique is sometimes called the delta method To estimate the mean we expand the function H to three terms where the expansion is about x IL If Y HX then Y HIl X IlHIl X Il WIl R 2 where R is the remainder We use equations 312 through 316 to perform EY EHIl EHIlX Il EWIlX Il ER HIl WIlVX ER 2 HIl W1l a 2 Using only the first two terms and grouping the third into the remainder so that Y HCl X Il HCl RI where then an approximation for the variance of Y is determined as Vy VHCl VX Il HCl VRI 0 veX HCl HCl a 317 318 If the variance of X 02 is large and the mean J1 is small there may be a rather large error in this approximation The surface tension of a liquid is represented by T dynecentimeter and under certain eonditions Too 21 0005X12 where X is the liquid temperature in degrees centigrade If X has probability den sity functionjx where then and 36 Aproximations to EHX and VHX 63 fxx 30oox x 10 0 otherwise ET 210oo5x12 3OO0x dx to VT r 4ID005x 3000x4dxET In order to determine these values it is necessary to evaluate r10OO5x ax Jo X4 Since the evaluation is difficult we use the approximations given by equations 317 and 313 Note that Since HX 21 OOOSX then HX 00121 0005Xz HX 00000121 O005X Thus HlS 2IO00515lt 182 HlS 0012 li15 O Using equations 317 and 318 ET H15tW15cr 182 and V1 HI5 d 001275 0Q108 An alternative approach to this sort of approximation utilizes digital simulation and statistical methods to estimate Ey and Vl Simulation will be discussed in genem in Chapter 19 but the essence of this approacb is as follows 64 Chapter 3 Functions of One Random Variable and Expeetation 1 Produce n independent realizations of the random variable X where X has proba bility distribution Px orfx Call iliese realizations x x x The notion of inde pendence will be discussed further in Chapter 4 2 Use Xi to compute independent realizations of Y namely y Hx Y2 HxJ y H 3 Estimate ErY and Vl from ilie yYz y values For example the natural esti matorior Ey is the sample mean y 2 yn Such statistical estimation prob lems will be rreated in detail in Chapters 9 and 10 As a preview of this approach we give a taste of some of the details FIrst how do we generate the n realizations of X that are subsequently used to obtain Y1 h Y 11 The roOst important technique called the imerse transform method relie on a remarkable result Theorem 32 Suppuse that X is a random variable with CDF F x Then the random variable FX has a uoiforrn distribution on 0 1 that is it has probability density function Julu 1 0 OS uS I otherwise 319 Proof Suppose that X is a continuous random variable The discrete case is similar Since X is continuous its CDF Fxx has a unique inverse Thus the CDF of the random variable F xX is PFiX S u PX S Fiu FFiu u Taking ilie derivative wiili respect to u we obtain ilie probability density function of F X d duPFxXuI which matches equation 319 and completes ilie proof We are now in a position to describe the inverse transform method for generating random variables According to Theorem 32 ilie random variable FX has a uoiforrn distribution on 01 Suppose iliatwe can somehow generate such a uoiforrn OIJ random variate U The inverse transform meiliod proceeds by setting FX U and solving for X via the equaJion X FiU 320 We can ilien generare independent realizations of Y by Y HXi HFi Ui i 12 n where U U2 U are independent realizations of the uoiforrn 01 dis tribution We defer the question of generating U until Chapter 19 when we discuss com puter simulation techniques Suffice it for now to say iliat iliere are a variety of methods available for this task the most widely used acmally being a pseudouniform generation meiliodone iliat generates numbers that appear to be independent and uniform on 0 I but that are actually calculated from a dererntinistic algorithm Suppose that U is unifOITn on 01 We will show how to use the inverse transfonn method to generate aD exponential random variate with parameter that is thc continuous distribution 3 7 lce MomentGenerating Funetion 65 having probability density functionfxCx Aetr for x 2 0 lne CDF of the exponential random vaiable is lnerefore by the inverse rransform theorem we can Set and solve for X X F U IlnlU where we obtain the inverse after a bit of algebra In other words 1A In1 1 yields au expo nential random vaiable with parameter A We remark that it is not always possible to find Pi in closed form so the usefulress of equation 320 is sometimes limited Luckily many other random variate generation schemes are available and taken together they span all of the coonly used probability distributions 37 THE MOvIENTGEJIoERATING FTNCTION It is often convenient to utilize a special function in finding the moments of a probability dis tribution TIlls special function called the momentgereratingfonction is definedas follows Definition Given a random variable X the momentgenerating funetionMJt of its probability distri bution is the expected value of ex Expressed mathematically Mft Eex Le Pxx discreteX r e Ix xdx continuous X 321 322 323 For certain probability distributions the momentgenerating funetion may not exist for all real values of t However for the probability distributions treated in this book the momentgenerating function always exists Expanding etX as a power series in t we obtain On taking expectations we see that 2 r IX 2 t t Mx t Be hE X tElX IEIX 2 r so that 324 66 Chapter 3 Functims of One Random Variable and Expectation Thus we see that when 41yjt is written as a power series in r the coefficient of fir in the expansion is the 7th moment about the origin One procedure then for using the moment generating function would be the following 1 Find M xit analytically for the particular distribution 2 Expand Mt as a power series in t and obtain the coefficient of fir as the 7th origin moment The main difficulty in using this procedure is the expansion of Mit as a power series in t If we are only interested in the first few momenTS of the dislribution then the process of derenninlng these moments is usually made easier by noting that the rth derivative of Mxil with respect to t evaluated at t 0 is just d Mxt1 0 EflXe Jl dtT t tO 325 assuming we can interchange the operations of differentiation and expectation So a sec ond procedure for using the momentgenerating function is the following 1 Determine M analytically for fue particular distribution 2 Find Jl Mxtllo Momentgenerating functions have many interesting and useful properties Perhaps fue most important of these properties is that the momentgenerating function is unique when it exists so that if we know the momentgenerating function We may be able to determine the form of the distribution In cases where the momentgenerating funetion does not exist we may utilize tbe characteristic function Ct which is defined to be the expectation of fiX where i H There are several advantages to using the characteristic function rather than the moment generating function but the principal One is that Cxt always exists for all t However for simplicitYj we will use only the momentgenerating function imRftIj Suppose that X has a binomial distrfbutifm that is It Pxx l p Ip X 0 x 012 t otherwise where 0 p l and n is a pOSitive integer The momentgenerating fimction MJt is This last summation is reognize as the binomial expansion ofpe Ip so that Mtl pe 1pl Taking the derivatives we obtain and Mt npel p npe pe ll Thus and 14 MtI npl p npl The second cenrral moment may be oblained using cr J4 il np1p ipE Assume X to have the following gamma distribution ab bl fxtx e O5xaObO reb O otherwise whererb r eyldy isrhegammajunction The mOlllelltgencratiJlg fmctlon is which if ve let y xa t becomes Since the integral on the right Is just rb we obtain MxtaJ 1 at a fort a Now using the power series expansion for we find which gives the moments 38 SUMlHRY 1r aJ t bbl MxIhbl a 21 a u b 1 a and 001 1i a 38 Summary 67 This chapter first introduced methods for determining the probability distribution of a ran dom variable that arises as a function of another random variable with known distribution That is where YHX and either Xis discrete with knowndistributionpx or X is COIl 68 Chapter 3 Functions of One Random Variable and Expectation tinuous with known density xx methods were presented for obtaiIDng the probability dis tribution of Y The expected value operator was introduced in general terms for EHXl and it was shown that EXl J1 the mean and EX J1l ri the variance The variance operator V was given as VXl EX EXl Approximations were developed for EHXl and VlHK that are useful when exacr methods prove difficult We also showed how to use the inverse transform method to generate realizations of random variables and to estimate their means The momentgenerating function was presented and illustrated for the moments fl of a probability distribution It was noted that EX 1 39 EXERCISES 3 1 A robot positions 10 units t a chuck for machin v iog as the chuck is indexed If the robot positions the unit improperly the unit falls away and the chuek position rClllains open thus resulting in a cyele that produces fewer than 10 units A study of the robots past performance indicates that if X number of open positions pxxO6 03 01 00 xo x 1 x 2 otherwise If the loss due to empty positions is given by Y 20 find the following a piy b EY and VCy 32 The content of magnesium in an alloy is a random variable given by the following probability density function 0 otherwise The profit obtained from this alloy is P 10 2X Ca Find the probability distribution of P b That is thc expected profit 33 A manufacturerof color television sets offers a 1 year warracty of free replacement if the pictue tube fails He estimates the time to failure T to be a ran dom variable with the following probability distribu non in units of years f e4 0 T 4 t O otherwise a Vlhat percentage of the sets will he have to service b If the profit per sale is 200 and thc replacement of a picture tube eosts 200 tind the expected profit of the business 34 A contractor is going to bid a project and the number of days X required for completion fonows the probability distribution given as pix 01 x 10 03 x 11 04 x12 01 x13 01 x14 0 otherwise The contractors profit is Y 200012 X a Find the probability distribution of Y b Find EX VX Ey and VY 35 Assume thaI a continuous random variable X has probability density function fxx 2xerr 0 XO otherwise Find the probability distribution of Z X 36 In developing a random digit generator an impor tant property sought is rhat each digit D follows the folloving discrete unifonn distribution 1 pJd 10 d0L23 9 O otherwise a Find ED and VD b If y l D 45 J where l J is the greatest integer round down function find Pr0 Y Vl 37 The percentage of a certain additive in gasoline determines the wholesale price If A is a random vari able representing the percentage then 0 A 1 If the percentage of A is less than 070 the gasoline is low test and sells for 92 eetltS per gallon If the percentage of A is greater than or equal to 070 the gasoline is bighwtest and sells for 98 cents per gallon Find the expected reenue per gallon wberefAa 1 Oa S 1 otherwiseka O 38 The probability function of the random variable X f I vexp x x Be 0 othenvise is knOtVtl as the twoparameter exponential distrfbu non Find the momentgenerating function of X Eval uate EX and VX using the momentgenerating function 39 A random variable X has the following probabil ity density function flx a O XO othelWise al Develop the density function for Y 2 X b Develop the density for V Xn cJ Develop the density for U In X 39 Exexcises 69 Evaluate EA and VA by using the approximations derived in this chap 313 Suppose that X bas the uniform probability density function 1 5x 2 otherwise aJ Flnd the probability density funetion of Y HX wbere Hx 4 x b Find the probability density function of Y HX where H e 314 Suppose that X has the exponertial probability density function x 0 otherwise Find the probability density function of r FIX where 3 HX tlx I 310 A twosided rotating antenna receives signals usedcar salesman finds that he sells either 1 The rotanonal position angle of the antenna is denoted 2345 or 6 caS per week with equal probabilIty X and It may be assumed that this position at the tlme ind the momentgenerallng function of X a IUgnal is received is a ra1dom variable with the den busin the momentgereating function fincCEX sity below Actually the randomness lies in the signal and VgJ I I fxx 2 0 otherwise The signal can be received if Y Yo where Y tan X For instance Yo 1 corresponds to t X and X 3 Find the density function for Y 311 The demand for antifreeze ma season is consid ered to be a uniform random variable X with density fxx lO 10 x S 2 x 10 e 0 otherwise wbere X is measured in liters If the manufacturer tlakes a 50 eent profit an eacb liter shc sells in the fall of the year and if she must carry any excess aver to the next year at a cost of 25 OOnt per liter find the optimum stock level for a particular full season 312 The acidity of a certain product measured on an arbitrary scale is given by the relationship A 3 0050 where G is the amount of one of the constituentS bav ing probability distribution fcgt Og4 0 otherwise 316 1t X be a random variable with probability density function flx aieb x 0 O othetVise a Evaluate the constant a b Suppose a new function Y lax is of interest Find an approximate value for EeY and for VY 317 Assume that Y has the exponential probability denSity function yo otherwise Find the approximate values of EX and VX where 31S The concentration of reactant in a chemical process is a random variable having probability distribution fk 6r1 r 0 05r51 otherwise The profit associated with the final product is P 100 300R Find the expected value of P Vtbat is the probability stribution of P 70 Chapter 3 Functions of One Random Variable and Expectation 319 The repair time in hours for a certain e1ectron iafuy controlled milling machine follows the density function f wr2I 0 70 othenvise Determine the momentgenerating function for X and use this fu1crion to evaluate EX and VX 3 20 The ctQSsecrional diameter of a piece of bar stock is circular with diameter X It is known that EXJ 2 em and VX 25 x 1 cm2 A cutoff tOOl cuts wafers that are exactly 1 Cm thick and this is con stal Fmd the expected volume of a wafer 2VI a random variable X has momentgenerating 1 Tunction Mxt prove that the random variable Y aX b has momentgenerating function eib Mx at 322 Consider the beta distribution probability density vJJ1Ction Ix x kl XIxbl 0 a Evaluate the constant k b Find the me ffmd the variance OxslaObO othCItlse f3 The probability distribution of a random vari able X is given by pIx 112 x 0 14 t 1 118 x2 I i1 118 73 0 otherwise fJ Determine the mean and vadane of X from the momentgenerating function 1 1 l If Y X 2 find the CDFfor 1 324 The third moment about the mean is related to the asymmetry or skewness of the distribution and is defined as J EX J1 Show that fJJ p 3u2jl 2PJ Show that for a symmetric distribution fJJ 0 325 LetJbe a probability density function for which the rth order moment1 exists Prove that all moments of order less than r also exist 326 A set of COllStaIlts k called cumuIants may be used instead of moments to characterize a probability distribution If 1Jr is the momentgenerating func rion of a random variable X then the cumulants are defined by the generating function vW log MI Tuus the rth cumulant is given by k dIf xCl1 r dt IrO Find the cumulants of the f1Orrra1 distributlon whose densty function is 1 lixulq f exp 1 i J av2r 2 J 327 Using the inverse transform method produce 20 realizations of the variable X described by Pxx in Exercise 323 328 Using the inverse transform method produce 10 realizations of the random variable Tin Exercise 33 Chapter 4 Joint Probability Distributions r 141 INTRODUCTIO In many situations we must deal with two or more random variables simultaneously For example we might select fabricated sheet steel specimens and measure shear strength and 2lddiameter 01 sP9lilelds Thus both weld lierustrllWllQ weld dilletere the ran jlbluf mtre5t Or we may select people from a certain population and mcasure their height and weigbL The objective filii chapter is to formulate joint probability distributions for two or more random variables and to present methods for obta1Ung both marginal and coruItiwnal distributions Conditional tas wtlllpe regre91tpfm1 We also present a definition of irulependence for random variables and covariance and corre lation are defined Functions of two or more random variables are pSentecfd a specIal f linear combinations is presented with its corresponding momentgenerating func tion Finally the law of large numbers is discussed Definition If if is the sample space associated with an experiment and Xl X1 Xk are functions each assigning a rea number Xe Ie Xie to every outcome e we call Xl X XJ a kdimensional rtJJdom vector see Fig 41 The range space of the random vector Xl X Xl is the set of all possible values of the random vector This may be represented as Rx XX1Y xXe where Rxxxx xx XlXz xJXI E Rxt2 E Rx Xk E Rx This is the Cartesian product of the range space sets for the componentS In the case where k 2 that is where we have a twodimensional random vector as in the earlier illustrations RXI xX is a subset of the Euclidean plane e R rRx 71 72 Chapter 4 Joint Probability Distributions 42 JOINT DISTRIBUTION FOR TWODIMENSIONAL J RAtIDOM VARLffiLES In most of our considerations here we will be concerned with twodimensional random vectors Sometimes the equivalent term twodimensional random variables will be used If the possible values of X xJ are either finite or countably infinite in number then XI Xz will be a twodimensional discrete random vector The possible values of XI Xz are x xljJ i I 2 i 11 2 If the possible values of XI Xl are some uncountable set in the Euclidean plane then Xl XJ ill be a nvodimensional continuous random vector For example if a S b and c s d we would have RXI xXz xl xJ a 5 Xl b c 5 Xi d It is also possible for one component to be discrete and the other continuous however here we consider only the case where both are discrete or both are continuous Consider the case where weld shear srength a1d weld diameter are measlred If we let Xl represent diameter in inches andX2 represent strength In pounds and if we know 0 x 025 inch while 0 x S 2000 poUlJds then me range space for X X is the ser x x 0 XI 025 0 z 2000 This space is shoWD graphically in Fig 42 1E1 A small pump is inspered for four qualitycoltrol characteristics Each charaCteristic is classified as good mino defect not affecting operation or IIJajor defect affecting operation A pump is to be selected and the defects counted IfXt the number of minor defects andX2 the number of major defects we know that Xl 0 1 2 3 4 and Xz O I 4 X I because only four characteristics are inspected The range space for X X is thus O 0 0 I 0 2 0 3 04 1 0 I IJ 12 13 2 0 2 1 2 2 30 3 1 4 0 These possible outcomes are shown in Fig 43 X2 pounds 2000 o 2 o 2 4 X Figure 42 The range space of Xl XJ where Xl is weld diareeter andX2 is shear strength Figure 43 The range space of Xl Xi where Xl is the nUlllOc of minor defects andX2 is the number of major defects The range space is indicated by heavy dots 42 10im Distribution for TwoDimensional Random Variables 73 In presenting the joint distributions in the definition that follows and througbout the remaining sections of this ctapter where no ambiguity is introduced we shall simplify the notation by omitting the subscript on the symbols used to specify these joint distributions Thus if X X XJPxx x px xJ andxx xJ x x Definition Bivariate probability functions are as follows o iscrete ci To each outcome x Xzl of X Xl we asociate a number pxxJ PX x and X x where and 41 The values x xl pX Xz for all i j make up the probability distribution of X XJ 3 ContimOHS I Xl Xi is a continuous random vector with range space R in the Euclidean plane tlienf the joint density fwtction has the follOVing properties for all x x E R and A probability statement is then of the form Pal XI1a X b tJxXzdxjdx 0z 1 see Fig 44 Figore44 A bivariate dentyfunction were peal X oJ az SXz 07 is given by the shaded volume 74 Chapter 4 Joint Probability Distributions It should again be noted thatjx x does not represent the probability of anything and the convention thatjx x 0 for Xl R will be employed so that the second prop erty may be ritten In the case where X XJ is discrete we migbt present the probability distribution of Xl X2 in tabular graphical or mathematical fonn in the case where Xl XJ is continu ous we usually employ a mathematical relationship to present the probability distribution however a graphical presentation may occasionally be helpful llf A hypothetical probability distribution is showt in both tabular and graphical form in Fig 45 for the random variables defined In Example 42 il In the casc of the weld diameters represented by Xl and tensile strength represcnted by Xl we might have a uniform distribution as by o I 1 2 3 LJ I 0 130 130 2 I 330 I 130 1 130 i 130 3130 i 430 I 2 130 230 lI3Oi I 3 130 I 330 i i i 4 330 I I I a pix xzJ 330 X 130 lI3O 330 430 0 30 2 3 4 x b Figure 45 Tabular and graphical presentation of a bivariate probability distribution a Tabulated values are px1 xJ b Graphical presentatiQIl of discrete bivariate distribution Uf 43 Maginal Distributions 75 OX 02502 2000 0 otherY1Se The range spare was shOViD in Fig 2 and if we add another dimension to display graphically y fix l0 then the distribution would appear as in Fig 46ln the univariate case area corresponded to probability in the bivariate case volume under the surface represents the probability For example suppose we wish to find PO 1 02 100 X2 200 Tnis probability would befound by integratingjx x over Ihe region OJ X 02 1005 X 200 That is rlOO r 1 J100 JO1 500 G MARGDlAL DISTRIBUTIOKS 1 Having defined the bivariate probability distribution sometimes called the joint probabil ity distribution or in the continuous case the joint density a natural question arises as to the distribution of Xl Of X2 alone These distributions are called marginal distributions In the discrete case the marginal distribution of Xl is r ph LPXx 1 for all x I r J 42 43 JfR1f In Example 42 we considered the joint discrete distribution shov in Fig 45 The marginal distri butions are shown in Fig 47 We see that Xl PlX is a univariate distribution a11d it is the distribu tiQn of XI the number of mL1OC defects alone Likewise l1 p is a unl1irlate distribution and it is the distribJlion the number of major defects alone If X XJ is a continuous randnm vector the marginal distribution of X is 44 2000 Fgnrc 46 A bivariate uniform deosity 76 Chapter 4 Joint Probability Distributions Ni 0 1 2 i 3 4 px I 1 o I 1130 liS 2130 8130 130 8130 I 1 illS 130 330 i 430 SIS I 1 2 1130 2S 3130 630 I 3 ill30 330 41S I i 4 I 3130 31S0 I lPX 7S 1713 830 730 1130 1 a PI 7130 La 130 4 X PX lCL ai30 9130 1 6130 L 1 4130 j330 o 1 2 3 4 2 o 1 2 3 b 0 Figure 47 Marginal distributions for discrete XI XJ a Marginal distributionstabular fann b Mar ginal dislribution xpx el Marginal distribution x px and the marginal distribution of X is 45 The function is the probability density function for Xl alone and the fUllctionh is the den sity function for X2 alone l6 In Example 44 the joint donalty 01 LX X was given by tx 5 OXI 025OSx 2000 0 Qilierwise The marginal dislributions of X and X are otherwise and otherwise These are shown graphically in Fig 48 43 Marginal Disubutions 77 flX LL 12000 o 025 Xl o 2000 z a b Figure 48 Marginal distributions for bivariate uniform vector Xl Xzl a Marginaliistribution of X b Marginal distribution of X Sometimes the maroJnals do not come out so nicely This is the case for example when the range space of Xl X is not rectangular ElLfs Suppose that the jOint density of X Xii is given by Then the marginal of X is and the mginal of Xz is x x 6x d 0 othelVlise tXI fxxdx f6xt dx2 x forOx 1 z 6x1 dt1 3xi forOx2 1 TIle expected values and variances of X and X are determined from the marginal dis ttibutions exactly as in the univariate case Where X X2 is discrete we have EXUl 2lPh LLxlPhx XI xl Xl 46 LLxpXIXu 47 x 78 Chapter 4 Joint Probability Distributions and similarly 48 9 In Example 45 and Fig 47 marginal distributions for XI and were given Working with the ntar gjnal distribution of Xl shown in Fig 4lb we may calculate 7 7 S 7 1 g EX 1 01234 V 1 30 30 30 30 30 5 and 2 I 7 7 3 2 7 2 1 J 18J VXdOj 0 12 3 4 1 L 30 30 30 30 30 L5 103 75 The mean and variance of X2 could also be detennined using the maIginal distribution of Equations 6 tlJrough 9 show that the mean and variance of X and X respectively may be determined from the marginal distribetions or directly from the joint distributioIL In practice if the marginal distribution has already been determined it is usually easier to make use of it In the case where Xl X21 is continuous then EXI ill XlfiXldxl XfXIXdxdx HO 1 vXIa XlIll fixIJdx 11 and EX 12 xdxdx xfx xz dxldx 412 S 2 VXaz X2 u fzxdx r xifzxlJdx JLi 413 44 Conditional Distributions 79 Again in equations 410 through 4I3 observe that we may use either the marginal densi ties or the joint density in the calculations l4l 1L Example 44 the joint density ofwe1d diameters XJ and shear strength Xz was given as 1 fxx SIlO 0 025 0 x s2000 0 otherolfse and the marginal densities for Xl and X2 ere given in Exampe 46 as OxlO25 V and 1 A 2000 0 2000 O otherwise WoddJg with L1e arginal densities the mean and V3 iance of XI are thus and rg CONDITIONAL DISTRIBUTIONS When dealing with twojointly distributed random variables it may be of interest tcfind the stributionOf one of these variables given a particular value of thoilir That ISI we may istribution OLl giveJ2 s For example what is the disttltruin of a persons weightgvenfhat he is a particularneight This prObability distribution would be ealled the conditional distribution of X given that X Suppose that the random vector X XJ is discrete From the definition of conditional probability it is easily seen that the conditional probability distributions are and where px 0 and PlX O PXlX2 px 414 415 It should be noted that there axe as many eonditional distributions of X2 for given Xl as there axe values Xl With PI ex t 0 and there are as many conditional distributions of Xl for given X2 as there are values X2 with Pl0 O 80 Chapter 4 Joint Probability Distnbutions iIt1j Consider the COUlting of minor and major defects of the small pumps in Example 42 and Fig 47 There will be five conditional distributions ofXZ one for each value of Xl They are shown in Fig 49 The distribution PXJrJxJ fot XI Ois shown in Fig 49a Figure 49b shows the disaibution Px xz Other conditional distnbutions could likewise be determined for XI 23 and 4 respectively The distribution of Xl given that Xl 3 is otherwise If Xl X21 is a continuous random vector the conditiooal densities are 416 and 417 jamIj Suppose the joint density of Xl X2 is the functionfpresented here and shown in fig 410 x Quotient x x rx xx 1 3 0 pOO pO ISO 7130 7 o pOl pO 1130 1 mo7 a otherwise 2 p02 pO 2 pO 3 pO 1130 730 1 3 4 pO4 P1O 4 Pi x pl x2fp1 Quotient p10 p1 130 1 7130 7 p11 p1 110 1 7130 7 b Figure 49 Some examples of conditional distributions p12 pCl 2130 2 7130 7 p13 p1 33 3 7130 1 p14 p1 o 736 0 44 Conditional DistributioIlS 81 fx xl x Figure 410 A biva ate density unction The matginal densities are dz These are detennined as otherwise OxzS2 othervrise The margina densities are shown in Fig 41l The conditional ensities may be determined using equations 416 anP 417 as and 1 2 xtxz x lx x i 2xx 3 O 3x x 1x2 otheIWse otherwise Note that for fxxJ there are an infinite number of these conditional densities one for each value o Xl L TWo of thesefxJ2Xz andfxjlxz are shown in Fig 412A1so forfxxzxl there are an 2 o Figure 411 The matginal deusities for Example 411 82 Chapter 4 Joint Probability Distributions Figure 412 Two conditional densities iX1jltl and fXl from Example 411 o x infinite nUlloer of these conditional densities one for each va1ue 0 S Xz 2 Three of these are shovn in Fig 4 13 45 CONDITIONAL EXPECTATION In this section we are interested in such questions as determining a person s expected weight given that he is a particular height More generally we want to find the expected value of Xl given iIiformation about X2 If Xl X2 is a discrete random vector the condi tional expectations are EXx XtPxlx xt 418 x and EXIX1J x2PXlz X2 419 x Note that there will be an EXf for each value of x The value of each EXf will depend on the value x which is in turn govemed by the probability function Similarly there will be as many values of EXIx as there are values X and the value of EXIx will depend on the value X determined by the probability function L5 Conditional Expectation 83 pIt2 Consider the probability distribution of the discrete random vector XI Xz where X represerts the number of orders for a large turbine in July and X2 represents the number of orders in August The joint distributon as wee as the marginal distributions are given in FIg 414 We corsider the tJree conditional distributions PXIO PX2j and PXzf and the conditional expected values of each 2 3 O Otherw1Se I 1 PXjlX2J 10 5 10 3 10 1 JO 0 Xl 1 1 4 1 4 0 0 Xz 1 Xl 3 otherwise 1 Ex12oO 1 2 4 123CO75 If Xl X is a continuous random vectQr the conditional expectations are and 420 421 and in each case there will be an infinite number of values that the expected value may take In equation 420 there will be one value of EXixl for each value X and in equation 421 there will be one value of EXIx for each value x 1iiRil3 In Example 411 we considered a joint density I where X1 0 005 005 010 025 2 010 015 3 005 005 pix 03 05 J 2 Xx XjXuXj 3 0 2 P2XiJ OO 02 005 04 005 03 000 01 02 OxS1Oxz S2 otherwise Figure 414 Joint and margiaal distrib 1ion of X X Values in body of table are px X 84 Chapter 4 Joint Probability Distributions The conditional densities were and Then using equation 421 the Exix is dete as I I 3x x EXx x2 dxz I 2 3x 1 4 9x3 It should be noted that this t a function of Xl For the two conditional densities shovn in Fig 4 12 where Xl t and Xl 1 the corresponding expected values are ECXJi i andEX211 Since EXixll is a function of x and x is a realization of the random variable XI EX2iXl is a random variable and we may consider the expected valne of EX2iX that is EEXIXl The inneroperatoris the expectation of X2 given X xl and the outer expec tation is with respect to the marginal density of X This suggests the following double expectation result Theorem 41 EEXiXl EX f12 422 and 423 Proof Suppose that Xl and Xz are continuous random vMiables he discrete case is sim ilar Since the random iable EXiX is a function of X the law of the unconscious stat istician see Section 35 says that xfxxdxdx2 x2xdx2 EX2 which is equation 422 Equation 423 is derived similarlY and the proofis complete Consider yet again the joint probability density function from Example 4 I fJlt2t 12 0 othervise In that example we derived expressions for the marginal densities fl a1dfX We also derived EXJx 9x 49x 3 in ExampJc 413 Thus 9 Note that this is also EXiJ since EX Exu S x G u J The variance operator may be applied to conditional distributions exactly as in the uni variate C3Se REGRESSION OF THE MEAN It has been observed previously that EXix is a value of the random variable EXiX for a particular Xl x and it is a function of Xl The graph of this function is called the regres sion of X on X Alternatively the function EXix would be called the regression of Xl on X This is demonstrated in Fig 415 j3f5l In Example 413 we found Ep for the bivariate density of Example 411 that is XX2 13 OX1 1Ox2 2 0 otherwise a b Figure 415 Some regression curves a Regression of X2 on X b RegrSsion of Xj on X2 86 Chapter 4 Joint Probability Distributions The result was In a like manner we may find V Regression will be discussed further in Chapters 14 and 15 47 INDEPENDENCE OF RANDOMVARRBLES The notions of independence and independent random variables are very useful and impor tant statistical concepts In Chapter 1 the idea of independent events was introduced and a fonnal definition oftbis concept was presented We are now concerned with defining iru1e pendent random variables Vlhen the ontcome of one variable say Xt does not influence the outcome of Xl and vice versal we say the random variables Xi and XL are independent Definition 1 If Xl KJ is a discrete random vector then we say that Xl and X are independent if and only if 424 for all x and x 2 If X KJ is a continuous random vector then we say that Xl and X1 are independ ent if and only if 425 for all Xl and x Utilizing this definition and the properties of conditional probability distributions we may extend the concept of independence to a theorem Theorem 42 1 Let Xl Xzl be a discretc random vector Then Pxx px and pxX PIX for all Xl and X if and only if Xl andX are independent 2 Let Xl KJ be a continuous random vector Then IXIx hex 48 Covariance and Correlation 87 and Ixx x for all x and if and only if X and X are independent Proof We consider here only the continuous case We see that 11 hx for all x and x if and only if t x if and only if ftx xJ I xfx for all Xl and x if and only if Xl and X are independent and we are done Note that the requirement for the joint distribution to be factorable into the respective marginal distributions is somewhat similar to the requirement that for indepeudent events the probability of the intersection of events equals the product of the event probabilities A city trrulSit service receives calls from brokendown buses and a VTeeker crew must haul the buses in for service The joint distribution of the number of calls received on Mondays and Tueseays is given in Fig 416 along nirh the marginal distributions The variable Xl represents the number of calls on Mondays and X2 represents the number of calls on Tuesdays A quick inspection uill show thatX andXl are independent since the joint probabilities are the product of the apprOpriate Ull probabilities 48 COVARIACEAND CORRELATlOK Wehavecnotedtfutt iX 11 and VeX 0 are the mean and variance of X They may be detem1ined from the marginal distribution of Xl In a s1miJar manner 1 and a are the mean and variance of Xz Two measures used in describing the degree of association between Xl and Xl are the covariance of Xl X2 and the correlation coefficitmt Xl 0 2 3 4 px 0 002 004 006 004 004 02 002 004 006 004 004 02 2 001 002 003 002 002 01 3 004 008 012 008 008 04 4 001 002 003 002 002 01 PX 01 02 03 02 02 Figure 416 Joint probabilities for wrecker calls 88 Chapter 4 Joint Probability DistributioDS Definition If Xl X2 is a twodimensiona1 random variable the cJVariance j denoted Ci2 is CovX X EX EXX EX 426 and thelation COeffi denoted p is 1 p CoVX2 l i c 0 I 427 JVX1 rVX2 1 I j The cQvariance is measured in the units of Xl times the units of X2 The correlation coefficient is a dimensionless quantity that measures the linear association between two random variables By performing the multiplication operations in equation 426 before dis tributing the outside ex lue operator across the resulting quantities we obtain an alternate form for tb covariance follows 428 Theorem 43 ik If X andX2 are independent then p o Proof We again prove the result for the continuous case If XI and X1 are independent EX1 X r lX fx1xdxdx lAxxtzXzdxJdxz r XJjxJdx xf xdx J EX1 EX Thus CovX X 0 from equation 428 aod p 0 from equation 427 A similar argument would be used for X X discrete The converse of the theorem is not necessarily true and we may have p 0 withom the variables being independent If p 0 the random variables are said to be uncorrelated Theorem 44 The value of p will be on the interval 1 1 that is pl Proof Consider the function Q defined below aod illustrated in Fig 417 Qt EiX EX tX EX EX EX 2tEX EXX EX EX EX Since QJ 0 the discriminant of Qt must be 0 so that ZEX EXX EX 4EX EXEX EX O It follows that so and 48 Covariance and correlation 89 t Ylllre 417 The quadJatic QI 4CovX X 4VX VX 0 COVX1X2 1 VXVX Ipl 429 A correlation of p 1 indicates a hlgh positive correlation such as one might find between the stock prices of Ford and General Motors two related cOtopanies On the other baud a high negative correlation p I might exist between snowfall and tem perature li4i tLW Rocall Example 47 which examined a continuous random voctor rXj X2 with the joint probability density function The marginal of Xl is and the marginal of Xl is OX1 X1 1 otherwise fxl 6x x forO 0 otherwise fx 3s for 0 x O ohciwise These facts yield the following results EX fXXldxl 12 I r EX 10 6x1xlldx1 30 VXlEXJEX 20 90 Chapter 4 loint Probability Distribotions Further we have This implies that and then EX J3xidx 34 xl 3xIdx 35 VX ExiEX 039 CovX X EXXJ EXEX lf40 CovXX p 0179 VXJ VX A continuous random vector XI Xz has density functionjas given below fx x 1 0 XIOxl Qtbenvisc This function is shown in Fig 418 and The marginal densities are IX Ix lxl 0 X f0r0xl fOI1 Xl O otherwise Figure 418 A joint density from Example 418 49 The Distribution Function for Two Dimensional m Variables 91 SinceftxJ xzP x1 fieXJ the variables are not independent If we calculate the covariance We obtain and thus p 0 so the variables are uncorTelated although they are not independent Finally it is nored that if X is related to XI linearly that is X A BXI then rf L lfB O then pl and if B 0 pl Thus as we obServed earlier the correlation coef ficient is a measure of linear association between two random variables 49 THE DISTRIBUTION FUNCTION FOR rwODIMENSIONAI RANDOM VARIABLES The distribution function of the random vecror XI X is F where Fxl x PXI Xl X x This is the probability over the shaded region in Fig 419 If XI XJ is discrete then Fxxl L LPV2 11Xjfls and if XI Xl is continuous then ii11 Suppose XI and Xl have the following density jxp XJ 24xJXz x O X 0 Xl Xz 1 0 0 otherwise 430 431 432 Looking at the Euclidea1 plane shown in Fig 420 we see several cases that must be considered 1 XI sO Fx x O L x s 0 F x O Figure 419 Domain of inteon or sum mation for Fx1 x 92 CluIpter 4 Joint Probability Distributions SOxjlandXl Figure420 The domain of F Example 419 FXX2 J ri24tlt2dtldtz 6xfxJ FXtX2 Itr24tltzdtzdt1 6x8x3xi The function F has properties analogous to those discussed in the onedimensional case We note that when X I and Xl are continuous if the derivatives exist 410 FUNCTIONS OF TWO RANDOM VARIABLES Often we will be interested in functions of several random variables however at present this section will concentrate on functions of two random variables say Y HX1X Since X X1e andX Xe we see that Y BXe Xe clearly depends on the aUEome of the original experiment and thus Y is a random variable with range space Ry The problem of finding the distribution of Y is somewhat more involved than in the case of functions of one variable however if Xl X2 fs discrete e procedure is straight forward if Xand X take on a relatively small number of values 410 Functions of Two Random Variables 93 3Q If Xl represents the number of defective units produce by machine No1 in 1 hour andXz represents the number of defective units produced by machine No2 in the same hour then the joint distn1mtion might be presented as in Fig 421 Furthermore suppose the random variable Y HX X where Hx x lx I follows that Ry 0 1 2 3 4 5 6 78 9j In order to determine say PY 0 pO we note that 0 lfand only if X 0 andX 0 thereforepO 0D2 We Dote that Y 1 if and only if Xl 0 and X1 1 therefOle pyl 006 We also note that Y 2 lfand onlyifeitberX 0 X 2orX IX 0 sop2OJO 003 013 Using similar logic we obtain the r1 of the distribution as follows YI PyY 0 0D2 006 2 013 3 011 4 019 5 015 6 021 7 0D7 8 005 9 om otherwise In the case where the random vector is continuous with joint density functionftx xJ and Hx x is continuous then Y HXp X is continuous onedimensional random variable The general procedure for the determination of the density function of Y is out lined below 1 We are given Y HiX XJ 2 Introduce a second random variable Z HXl X The function H is selected for convenience but we want to be able to solve y HlXI and z flzx1 Xz for XI and x in ternis of y and z 3 Find 1 GY zj and x Gy z 4 Find the folloving partial derivatives we assume they exist and are continuous ox J ax o dy oz dy az Xi 2 3 px 002 003 004 001 O 006 009 012 003 03 2 010 015 020 005 05 3 002 003 OM 001 01 PX 02 03 04 01 Figure 421 Joint distrihltion of defectives produced au two macbiDes pr 94 Ctwpter 4 loint Probability Distributions 5 The joint density of Y 2J denoted ty z is found as follows ty zfG1y z Gy zll iJY zl 433 where Jy z called the Jacobian of the tnuJsformation is given by the determinant IOxtfay oxtaz Jy axzldy axoI 434 6 The density of Y say gy is then found as gyy lyzdz 435 1iapzI Consider the continuous raxdom eCtor Xl Xi Oith the folling density jXl4eUYlj xjOxO O otherwise Suppose we are interested in the distributioJ of Y XjlJJ We will let y xlx2 and ehoose z Xl 1 sothatxyzllyxzIlyItfoowsthat Therefore and Thus and and and y aXlrJZY ly 1 dXaZ ly ly z zy Z 1 1y lyIy ly IGy z Giy ill 4e1yjtdlY1 4eZ z ily 4e I ly J yO otherwise 411 JOINT DISTRmunoNs OF DIMENSION n 2 Should we have three or more random variables the random vector will be denoted Xl Xl Xli and extensions will follow from the twodimensional case We will assume 411 loint Distributions of Dimension n 2 9S that the variables are continuous however the results may readily be extended to the dis crete case by substituting the appropriate summation operations for integrals Vle assume the existence of a joint density I such that 436 and Thus 437 The marginal densities are determined as follows fiL r Jxtzxdx dx 12 JJV2 xnldx dxdxl Ix Lxxxdx1dx2dxl The mtegration is over all variables having a subscript different from the one for which the marginal density is required Definition The variables Xl x XJ are independent random variables if and only if for all xpx xJ 1x x x IIX x Ixn 438 The expected value of say XI is 111 X Jr XI lxx xndxldx2 dx and the variance is 439 440 We recognize these as the mean and variance respectively of the marginal distribution of Xl 11 the twodimensional case considered earlier geometric interpretations were instruc tive however in dealing with rzdimensional random vectors the range space is the Euc1id ean nspace and graphical presentations are thus not possible The marginal distributions are howeverj in one dimension and the conditional distribution for one variable given val ues for the other variables is in one dimension The conditional distribution of Xi given va1 ues x x xJ is denoted 441 and the expeered value of XI for given Xz x is EXX2 X x J Xl IxI xdxl 442 96 Chapter 4 Joint Probability Distributions The hypothetical graph of EXJx x x as a function of the vector tXt x x is called the regression of x on X x3 X 412 LINEAR COMBINATIONS The consideration of general functions of random variables say XlXz X is beyond the scope of this text However there is Olle particular function of the form Y HX X where 443 that is of interest The Q are real constants for i 0 I 2 n This is called a linear com binan011 of the variables X X21 1 XI A special situation occurs when ao 0 and a 1 all lin which case we have asumYXl X2 Xn 5li2ifuItJ6 Four resistors are connected in series as shown in Fig 422 Each resistor has a resistance that is a ran dom variable The resistance of the assembly may be denoted Y where YX1 Xz X3 Xlj g23 Two parts are to be assembled as shovll in Fig 423 The clearance can be expressed as YX Xl or Y lX 1X2 Of eourse a negative clearance would mean interference This is a1i1ear com bination i11 0c 0 a 1 t and al 1 A sample of 1 items is randomly selected from the output of a process that manufactures a small shaft used in electric fan motors and the diameters are to be measured with a alue called he sample mean calculated as 1 XioXXX The valueX lois a linear combination withacO and at az oww x x x Figure 422 Resistors in series Figur 423 A simple assembly 412 Linear Combnations 97 Let us next consider how to determine the mean and variance of linear combinations Consider the sum of two random variables YX x The mean of Yor 1 E is given as However the variance calculation is not so obvious or V EY E2 EY E EX X EX X EX 2XX X EX EXl EX 2EXX EX EXll 2EX EX EX EX EX2 EX EX 2EXX EX EX VeX VX 2CovX X 2 2 7y 0 02 2a2 These results generalize to any linear combination Y 0X azX cx as follows n EY Co L E X 11 ECY ao IaIJii 11 where EX 1 and n n n VY LaVX L2CJOijCovXXJ hi 11 j1 li or a iaYG7 iiaiajvij jl 1 jl j 444 445 446 447 448 449 If the variables are independent the expression for the variance of Yig greatly simpli fied as all the covariance terms are zero In this situation the variance of Yis simply n VY La vx 450 i1 or 98 Chapter 4 J omt Probability Distributions In Exal1ple 422 four resistors were connected in series so that Y Xj Xl X X4 was the Ieist ance of the assembly where Xl was the resistance of the first resistor and so on The mean and van a1ce of Y m terms of the me3llS and variances of the components may be easily calcJlated If the resistors are selected randomly for the assembly it is reasonable to assume that Xlt Xl X3 and are independent We have said nothlng yet about the distribution of Y however given the mean and variance of XI Xz X3 and X4 we may readily calculate the mean and the variance of Y smce the variabes are independent pe2ijf ill Example 423 where two components were to be assexnbled suppose the jobt distribution of XXJ is N x 8e 0 Sinceflxl xJ can be easily factored as XI OXzO otherwise flx xJ 2 4e itx fx XI andXz are independent Furthermore EXj PI and EX0 J12 We may calculate the vari ances xe 1 r 2 d 1 1 o 2 4 and V In Example 424 we might expect me random variables Xi Xi X1C to he indndent because of the random SalIlpling process Furthermore the distribution for each variable X is identical This is shon in Fig 4241n me earlier exarple the linear combination of interest 38 the sample mean It follONS that 1 I 1 1 X E XItEX EX E x 10 10 10 1 1 1 wt 1Ot 101 JL r I I t I 4l4 The Law of Large Numben 99 t1 j X1 Figure 424 Some identical distributions Furthermore 413 MOMENTGENERATING FlNCTIONS AlolJ LINEAR COMBINATIONS In the case where Y aX it is easy to show that Mt Mxlat For sums of independent randam variables Y X X 1 1 Xn MrMxtlMxt MxCt 451 452 This property has considerable use in statistics If the linear combination is of the general fonn Y 0 aX a and the variables X X are independent then Mt eMxatt Mxar Mxat Linear combinatiot15 are to be of particular significance in later chapters and we will dis cuss them again at greater length r 41 THE LAW OF LARGE NUMBERS A special case arises in dealing with sums of independent random variables where each variable may take only two values 0 and 1 CODSider the following formulation An exper iment i consists of n independent experiments trials jj I 2 rt There are only two outcomes success lSI and failure F to each trial so that the sample space 9 S Fl The PS p and PF Ipq remain constant for j I 2 n We let if the jth trial results in failure if the j1h trial results in success 100 Chapter 4 Jomt Probability Distributions and YXXX Thus Y represents the number of successes in n trials and YIn is an approximation or esti mamr for the unknYlIl probability p For convenience we will let p YIn Note that this value corresponds to the lenni used in the relative frequency definition of Chapter 1 The law of large numbers states that or equivalently To indicate the proof we note that EYn EX nO q 1 p np and Vy nVXj nO q 1 p P2 npJ p 8ince p Yin we have Ep EY p n and Using Cnebyshevs inequality so if then we obtain equation 453 Thus for arbitrary e 0 as n 7 10 Equation 453 may be rewritten with an obvious notation as plp pi j 1 a 453 454 455 456 457 We mayuow fix both E and ain equation 457 and determine the value of n required to sat isfy the probability statement as plp nz a 458 I 416 Exercises 101 pIe A manufacturing process operates so that there is a probability p that each item produeed is defective and p is uumoWll A random sample of n items is to be selected to estimate p The estimator to be used ispYIn where and t 0 if the jth item is good Xj ifthejth item is defective YXIXzX It is desired that the probability be at least 095 that the error 1ft pi not exceed OOL In order to determine the required value of n we Dote that 001 and a 005 however p is unknown Equa tion 458 indicates that n plrpi 001 005 Since p is unknQVn the worst possible case must be assumed note thatp1 p is maximum when p This yields 0505 11 2 50 ODD 001 OOS a very large number indeed Example 428 demonstrates why the law of large numbers sometimes requires large sample sizes The requirements of 001 and a 005 to give a probability of 095 of the departure 1ft pI being less than 001 seem reasonable however the resulting sample size is very large In order to resolve problems of this nature we must know the distribution of the random variables involved ft in this case The next three chapters will consider in detail a number of the more frequently encountered distributions 415 SlJMMARY This chapter has presented a number of topics related to jointly distributed random variables and functions of jointly distributed variables The examples presented illustrated these top ics and the exercises that follow will allow the student to reinforce these concepts A great many situations encountered in engineering science and management involve situations where several related random variables sinultanoously bear on the response being observed The approach presented in this chapter provides the structure for dealing with several aspects of such problems 416 EXERCISES 1 A refrigerator manufacturer subjects his firished prodUcts to a final inspection Of interest are two cat egories of defects scratches or flaws iD the porcelain futish and mechanical defectS Ibe number of each type of defect is a random variable The results of 1rupecting 50 refrigerators are shOWl11n the following table where X represents die occuence of finish defects and Yrepresents the occummce of mechanical defects a Find the margmal distributions of X and Y b Find the probability distribution of mechanical defects given that there are no finish defects 102 Chapter 4 Joint Probability Distributions IX o 1 I 2 I 3 4 i 5 r 0 115 4i50 i 2150 1 1150 1150 1150 L 1 6150 250 I 150 150 i 1 2 4i50 350 I 2150 1 150 I 3 3150 I 150 I I i 4 1150 I 1 i c Find the probability distribution of finish defects given that there are no mechauical defects 42 An inventory manager has accumulated records of demand for her companys product over the last 100 days The random variable X represents the num ber of orders received per day and the random variable Y represents the number of unlt per order Her data are shown in he table at e bottom of this page a Find the marginal distributions of X and f b Find all conditional distributions for Y given X 43 Let XI and X2 be the scores on a general intelli gence test and an occupational preference test respec tively The probability density function of the rdlldom variables Xl X2J is given by k fxx 1000 0 x 100 0 x S 10 O otherwise a Find the appropriate value of k fb Find the lll3rginal densities oX and x c Find an expression for the cumulative distribution function Fx x 44 Consider a situation in which the surface tension and acidity of a chemical product are measut These variables are coded such that surface tension is meas ured on a scale 0 SXI 2 and acidity is measured on a scale 2 sXi S 4 The probability density function of XxJ is NX k6 xx 0 x 2 2x 4 0 otherwise a Find the appropriae value of k fb Calculate he probability that X 1 X 3 c Calculate the probability that Xl Xl 4 d Find the probability that X 15 e Find the marginal densities of both X andX 45 Consider the density function fwxyz16wxyz OSwxyzOl 0 otherwise a Compute the probability that WO t and Y i b Compute the probability that X Stand Z t e Find the marginal density of1 46 Suppose the joint densitY of X Y is 1 fxYg6x y 05x225y4 0 Find the conditional densities fx andfly 47 For the data in Exercise 42 find the expected number of units per order given that there are three orders per day 48 Consider the probability distribution of the dis crete random vector Xl XJ where Xl represents the number of orde S for aspirin in August at the neigh borhood drugstore and X2 represents the number of orders in September The joint distribution is shown in the table on the next page a Find he marginal disoeution b Find the expected sales in September given that sa1es in August were either 51 52 53 54 or 55 49 Assume that X and Xi are coded scores On two intelligence testS and the probability density function of Xl Xz is given by 2 Jx x ixx 0 x 10 x 1 0 otheJ1se Find the expected value of the score on test No2 given the score on test o 1 Also find the expected i j 2 31456t i 71 1 8191 i 1 110110051100 31100 21100 111100 110011110011100 1100 r4 i 2 181100 51100 31100 21100 1100 1100 i 1100 I I i 3 181100 510021100 1I10011100 I I S1 52 53 54 55 S1 006 005 005 0Q1 OQ1 52 001 005 001 0Q1 I om 53 005 010 010 005 OOS 54 005 002 001 001 003 ss 005 006 005 001 i 003 value of the score on test No1 given the score or test No2 410 Let jxlXz 4x1Xzer1 H t X xz 0 O otherwise a Fmd the marginal distributions of Xl andXz b Fmd ce conditional probability diStribJtiODS of X andXz c Find expressions for the conditional expectatiollS of X and X 411 Assume that X Y is a coutiJuous rmdom vec tor and that X and Yare independcntsucb thatftx y gxhy Define now random variable ZXY Show cat the probability density function of Z tz z is given by Hint Let Z XY and T X and fuJd the Jacobian for the transformation to the joint probability density function of Z and T say rz t Then integrate ret t with respect to 4U Use the result of the previous problem to find the probability density function of the area of a re tangle A SIS where the sides are of random length Specifically the sides are independent random vari abIes sJcb that and o I hsils 0 OSIl otherwise 0s4 otheIllise Some care must be taken in determining the 1iIlits of integration because the variable of integration cannot asSll1C negative values 413 Assume that pc Y is a continuous random vec tor and that X and Yare independent sucb thatfx y gxhy Define a new random variable Z XIY Show that the probability density function of Z tfz is given by 416 Exercises 103 guzhuludu Hint LetZlY and U Y and find the Jacobian for tlle transforrration to the joint probability density function ofZmd U say rz u Then irtegrate rz u with respect to u 414 Suppose we have a simple electrical circuit in which Obms law V IR holds We vish to find the probability distribution of resisance given that the probability clistribuciollS of voltage 11 and current l are ktiOWll to be gy e VO 0 othervrise Vi iO 0 othervrise Use the results of the previous problem and assume that V and I are independent random variables 415 Demand for a certain product is a randOtn vari able baving a mean of 20 units per day and a variance of9 We define the lead time to be the time that elapses between the placement of an order and its arrival The lead time for the product is fixed at 4 days Fmd the expected value and thl variance of lead time demand assuming demands to be independotly distributed 416 Prove the discrete case of Theorem 42 417 Let XI and Xl be random variables such thatX2 ABX Show that p 1 and thatpl ifBO whileplifBO 4 18 Let X and Xl be random variables sucb that Xz A BXt Show that me momentgenerating function for Xzis Mx iIMxBt 419 Let X and Xz be distributed according to jxx 2 OXLX 0 otherwise Find the correlation coefficient betweeu Xl and Xz 420 Let XI and Xl be random variables Viith correlaw tion coefficient Px1Xz Suppose we de5ne twOlCVI ran dam variables UA BX and v C DX whereA S C and D ae constants Show that PIlY BDIjsDilPxx 421 Consider the data shown lr Ewercise 41 Axe X and Y independent Caculate the correlation coefficient 422 A couple vishes to sell their bouse The miniw mum price that they are willirg to accept is a random variable say X where SI S X S 2 A population of btyers is interested in the house Let y where PI Y S P denote the mallirmm price they are willing to 104 Chapter 4 Joint Probability Distributions pay Yis also a random variable Assume that the joint disribution of XI isir y a Under what citcumstances will a sale take place b Write an expression for the probability of a sale tlkiJg place c Write an expression for the expected prke of the transaction 423 Let X y be UIllformly distributed over the semicircle in the following diagram Thusf y 21 if x y is in the semicircle a Find the marginal distributions of X and Y b Find the conditional probability distributions c Find the conditional expectations y 424 Let X and Y he independerit random variables Prove that EeXiY EX and that EeYjxl El 4D25 Show that in the discrete case EExjll EX EEYiXJJ El 4ffl26 Consider the two independent random Uiab1es Sand D whose probability densities are fss 3 l gn d 20 0 lOs40 otherwise WSd30 otherwise FiJd the probability distribution of the new random variable 427 If Iyxy 0 find the following a EXiYl b EX e EI WSD OxlOyl otherlse 42 For the bivariate distribution klx y fxy OxOy 1xlTy 0 otherwise a Evaluate the conitant k 0 Fl1d the marginal distribution of X 429 For the bivariate distribution fIx k 0y20n2 lxyr 0 a Evaluate the constant k b Fi1d Fx y otherwise 430 The manager of a small bank vvisbes to deter mine the proportion of the time a particular teller is busy He decides to observe the teller at n randomly spaced mwrvals The estimator of the degree of gain ful employment is to be YIn where 0 if on the ith observation the teller is idle Xi 1 if on the ith observation the teller is busy and y LIXi It is desired to estimatepPXi 1 so that the error of the estireate does not exceed 005 with probability 095 Determine the necessary value ofn 431 Given the following joint distributions deter mine whether X and Yare independent gx y4yeefl xOyO b 1 y 3iy 0 x y L e 1y 6lxy x0120 432 Ler y t hxhfylhz x 2 0 Y 0 z 0 Detennine the probability th2t a point drawn at ran dom will bve a coordinate x I z thaI docs not sat isfy either xy z or x y z 433 Suppose that X and Y are random variables denoting the fraction of a day that a request for mer chandlse occurs and the receipt of a shipment oceurs respectively The joint prObability density function is lxyl OxlOSyl 0 otherve a What is the probability that both the request for merchanilise and the receipt of an order occur during the first half of the day b What is the probability that a request for merchan dise occurs after its eceipt Before its eceipt 434 Suppose that in Prohlem 433 the merchandise is highly perishable and must be requested during the r tctay interval after it arrives hat is the probability that merchandise will not spoil 4 35 Let X be a continuous random variable With probability density function jx Find a general expression tOt the new random variable Z where a ZabX 0 Z IIX e ZlnX Cd Zi 416 Exercises 105 Chapter 5 Some Important Discrete Distributions 51 INTRODUCTION In this chapter we present several discrete probability distributions developing their ana lytical form from certain basic assumptions about rea1world phenomena We also present some examples of their application The distributions presented have found extensive appli cation in engineering operations research and management science Four of the distribu tions the binomial the geomerric the Pascal and the negative binomial stem from a random process made up of sequential Bernoulli trials The hypergeometric distribution the multlnomial distribution and the Poisson distribution will also be presented in this chapter When we are dealing with one random variable and no ambiguity is introduced the symbol for the random variable will once again be omitted in the specification of the prob ability distributions and cumulative distribution function thus pxx px and F Ix Fx This practice wili be continued throughout the rext 52 BERNOULLI TRIALS MlJ THE BERNOULLI DISTRIBUTION 106 There are many problems in which the experiment consists of n trials or subexperiments Here we are concerned with an individual trial that has as its two possible outcomes suc cess S1 orailure F For each trial We thus have the following 1 Perform an experiment the jth and observe the outcome 9 IS F For convenience we ill define a random variable X 1 if results in S and X 0 if results in F see Fig 51 The nBernoulli trials ogl 2 r are called a Bernoulli process if the trials are inde pendent each trial has only two possible outcomes say S or F and the probability of suc cess remains constant from trial to trial That is and Xjl j 12 Xj 0 jI2 n 51 otherwise 52 Bernoulli Trials and thc Bernoulli Distribution 107 0i rlXt hlXjSS FFj I 0 X F LV Figure 51 A Bernoulli trial For one trial the distribution giveD in equation 51 and Fig 52 is ealled the Bernoulli distribution The mean and variance are and VeX 0 q I pp plp pq The momentgenerating function may be shown to be MxPlqpe pY2 52 53 Suppose we consider a toallufactUling process in wbich a stIta11 steel part is produced by an auto matic machine Furthermore each part in a production nm of 1000 parts may be classified as defec live or good when inspected We can think of the production of a part as a single trial that results in success say a defective or failure a good item 1 we have reasOD to believe that ffie machine is just as likely to produce a defecthe on one run as on another and if the production of a defec tive on one run is neither more nor less likely because of the results on the previous runs then it would be quite reasonable to assume that the production run is a Bernoulli process vith 1000 trials The probability p of a defective being produced on one trial is called process CNerage fraction defective Note thatin the preceding example the assumption of a Bernoulli process is a matheff matical idealization of the actual realworld situation Effects of tool wear machine adjust ment and instrumentation difficulties were ignored The real world was approximated by a model that did not consider all factors but nevertheless the approximation is good enough for useful results to be obtained q p o Xl Figure 52 The Bernoulli distribution 108 Chapter Some Important Discrete Distributions We are going to be primarily concerned with a series ofBemoulli trials In this case the experiment is denoted 11 S 11 S are independent Bernoulli trialsj I 2 n The sample space is Xl XI Xi S or F i I n Suppose all experiment consists of three Bernoulli trials and the probability of success is p on each trial see Fig 53 The random variable X is given by X Ll Xj The distribution of X can be determined as follows Q PFFF q q qcl 1 PFFSPFSFj PSFFj 3p 2 PFSSPSFSPSSF3pq 3 PSSS p 53 THE BINOMIAL DISTRIBUTION The random variable X that denotes the number of successes in n Bernoulli trials has a binomial distribution given by px where lnl X X P xp Ip xOI2 0 ofuerwise 54 Example 52 illustrates a binomial distribution wifu n 3 The parameters of the binomial distribution are n andp where n is a positive integer and 0 p S 1 A simple derivation is outllned below Let pX Px successes in n trials S Rx P X 0 I I FFS I I FSP j I SFF I FSS I SFS 2 i SSF I i SSS e Figure S3 Three Bernoulli trials 53 The Binomial Distribution 109 The probability of the particular outcome in with Ss for the first x trials and Fs for the last n x trials is pssss Ff7F J pr where q 1 p due to the independence of the trials There are n comes having exactly x Ss and n x Fs therefore x px qn xOI2 n 0 otherwise Since q 1 p this last expression is the binomial distributiun 531 Mean and Variance of the Binomial Distribution The mean of the binomial distribution may be determined as n I EX n n Lx p q 0 xn x n1 l n npLx p q 1 x I n x and letting y x 1 so that EX np Using a similar approach we can find so that EXXl xxlnpr 0 x n x 1 2 n2 2 n nn PL p q 2x2nx n2 t nn 1p2 L n pyqny2 yoyny2 nn 1p2 VeX EX2 EX EXX 1 EX EX nn lp2 np np npq n out xnx 55 56 116 Chapter 5 Some Important Discrete Distributions An easier approach to find the mean and variance is to consider X as a sum of n inde pendent Bernoulli random variables each with meanp and variance pq so that X Xl X rXfj Then EX p p p np and VeX pq pq pqnpq The momentgenerating function for the binomial distribution is MIJ pi qr 57 i A production proess represented schematically by Fig 54 produces thousands of parts per day On the average 1 of the parts are defeetive and this avernge does not vary with time Every hour arm dom sample of 100 parts is selected from a oonveyor and severnl characteristics are observed and measured on each part however the inspector classifies the part as either good or defective If we consider the sampling as n 100 Berooulli trials with P 001 the total number of defectives in the sample X would have a binomial distribution 100 pX x jeOOlO99OO xO12 100 0 otherwise Suppose the inspector has instructions to stop the process if the sample has more than two defectives ThenPX 2 1 PX 2 and we maycalcuJate 100 PX52 1 OOIO99OO rO X 099lJ 100001099 4950001 09998 x 092 Thus the probability of the inspector stopping the process is approximatdy 1092 008 The mean number of defectives that would be found is EX np 100001 1 and he variance is VeX npq 099 532 The Cumulative Binomial Distribution The cumulative binomial distribution Or the distribution function F is Conveyor I I Warehouse I I I Hourly n 100 samples J 58 Figure 54 A sampling situation with attrib ute measurement 53 The Binomial Distribution 111 The function is readily calculated by such packages as Excel and Minitab For example suppose that n 10p 06 and we are interested in calculating F6 PX 6 Then the Excel function call BINOMDIST61005TRUE gives the result F6 06177 In Minitab all we need do is go to CalcIProbability DistributionslBinomial and click on Cumulative probability to obtain the same result 533 An Application of the Binomial Distribution Another random variable first noted in the law of large numbers is frequently of interest It is the proportion of successes and is denoted by pXn 59 where X has a binomial distribution with parameters n and p The mean variance and momentgenerating function are Ep lEX lnp p n n 510 Vp r VX r npq q 511 MtMxper q 512 In order to evaluate say Pp 5 Po where Po is some number bernreen 0 and 1 we note that Since npo is possibly not an integer 513 where l J indicates the greatest integer contained in function From a flow of product on a conveyor belt between production operations J and J 1 a random sam ple of 200 units is taken every 2 hours see Fig 55 Past experience has indicated that if the unit is not properly degreased the painting operation will not be successful and furthermore on the aver age 5 of the units are not properly degreased The manufacturing manager has grown accustomed to accepting the 5 but he strongly feels that 6 is bad performance and 7 is totally unacceptable He decides to plot the fraction defective in the samples that is p If the process average stays at 5 he would know that Ep 005 Knowing enough about probability to understand that p will vary he asks the qualitycontrol department to determine the Pj 007 j p 005 This is done as follows Operation J Operation J 1 Operation J 2 degreasing painting packaging n 200 every two hours Figure 55 Sequential production operations 112 Chapter 5 Some Importaot Discrete Distributions pi1 Pp 007jpO05 I Pfi O07jp 005 I PX 200O07jp 005 1f2001o05k095 k I 0922 0078 An indusmal engineer is concerned about the excessive avoidable delay time that one machine operator seems to have The engineer considers two actiities as avoidable delay time and not avoidable delay time She identifies a tin1edependent variable as follows Xt 1 0 avoidable delay otherwise A pMticular realization of Xt for 2 days 0 minutes is shown in Fig 56 Rather thanbave a time study technician contlnlously analyze this operation the engineer electS to USe work sallpling randomly selects rt points ou the 960minute span and estimates the fraction of time the avoidable delay category rusts She lets XI 1 if XCr 1 at the time of the ith obser vation and X 0 if X 0 at the time of the ith observation The statistic Ix P is to be evaluated Of course Pis anmdom variable having a mean equal to p variance equal to pqlrt and a standard deiation equal to The procedure oudined is not necessarily the best way to go about such a study but it does illusrrate one utilization of the rmdom variable P In summary analysts must be sure that the phenomenon they are studying may be rea sonably considered to be a series ofBemoulli trialsin order to use the binomial distribution to describe X the number of successes in n trials It is often useful to visualize the graphi cal presentation of the binomial distribution as shown in Fig 57 The values Pix increase to a point and then decrease More precisely Pix Pix I for x n Ip andpx px 1 for x n Iplf n Ip is an integer say m thenpm pm I 54 TIlE GEOMETRIC DISTRIBUTION The geometric distribution is also related to a sequence of Bernoulli trials except that the number of trials is not fixed and in fact the random variable of interest denoted X is defined to be the number of trials required to achieve the first success The sample space XC llLDDlL o 480 min 960 min t Figure 56 A realization of XI Example 55 54 Te Geometric Distribucon 113 OO2c3nL2nC1nLx Figure 57 The binomial distribution and range space for X are illustrated in Fig 58 The range space for X is Rx 123 and the distribution of X is given by x I 2 othenvise It is easy to veciiy that this is a probability distribution since Lpql p fq p1J1 xt kO lq and pX 0 for ali x 541 Mean and Variance of the Geometric Distribution The mean and variance of the geometric distribution are easily found as follows or d q 1 l P dql1q p x 6 FFFFFSt FFFFFiS li 7 I 0 Figure 58 Sample space and range space for X 514 515 114 Chapter 5 Some Important Discrete Distributions or after some algebra The momentgenerating function is pc Mxr Iqe 516 517 A certain experiLent is to be performed until a successful result is obtained The as are irdepend ent and the cost of perfonni1g the experiment is 25000 however if a failure results it costs 5000 to set up for the next trial The experimenter would like to detennine the expected COS of the proj ect If X is the number of trials required to obtain a successful experiment then the cost function would be Then CX 25OOOX 5000X I 30OOOX 5000 ECX 30000 EX E5000 r 130 000 I 5000 L P If the probability of success on a single trial is say 025 then the ECX 300001025 55000 SI15000 Ihis mayor may not be acceptable to the expermcnter It should also be recognized that it is possible to contime indefinitely without having a successful experiment Suppose that the exper imenter has a maximum of 500000 He may wish to find the probability that the experimental work would cost more thaa this amount that is P CX 500000 P30000X 5000 5500000 plX1 30000 PX 16833 IPX 16 6 1 2O250751 OO The experimenter may Dot be at aU willing to run the risk probability 001 of spending the available 500000 without getting a successful run The geometric distribution decreases that is px px 1 for x 2 3 This is shown graphically in Fig 59 An interesting and useful property of the geometric distribution is that it has no mem ory that is PXxsps PXx 518 The geometric distribution is the only discrete distribution having this memoryless properry 55 The Pascal Dostribution 115 PXI ili1 c i 2 3 4 5 x Figure 59 The geometlc distributiQr LetX denote the number of tosses efa fair die until we oberve a 6 Suppose we have already tossed the rlie five times without seeing a 6 The probalility that more than tvlO adCiticnal tosses will Ge requlrOO is PX 7 I X 5PX2 IPX2 1Lpx x 1 5 11 1 I j 6 25 36 55 THE PASCAL DISTRIBUTION 5S1 The Pascal distribution also has its basis in Bernoulli trials It is a logical extension of the geometric distribution In this case the random variable X denotes the trial on which the rth success occurs where r is an integer The Frobability mass function of X is px Ip q XI rI xrrlr2 0 othervtise 519 The term p q arises from the probability associated with exactly one outcome in if that has x rl Fs failures and r Ss successes In order for this outcome to occur there must be r 1 successes in the xI repetitions before the last outcome which is always success There are thus arrangements satisfying this condition a1d therefore the distribution is as abown in equation 519 The development thus fur has been for integer values of r If we have arbitrary r 0 and 0 p 1 the distribution of equation 5l9ls known as the negative binomial distribution Mean and Variance nf the Pascal Distribution If X has a Pascal distributiont as illustrated in Fig 510 the mean variance and moment generating function are J1 rip 520 116 Chapter 5 Some Irnportalt Discrete Distributions I Lj 11 r 34567 x Figure 510 An example of the Pascal distribution and tlciIhpie 8 crrqp pi Y Mxt I 1 qe 521 522 The president of a large corporation makes decisions by throwing darts at a board The center section is larked yes and represents a success The probability of his hitting a lyes is 06 and this pro ability remains constant from throw to throw The president continues to throw until he has three tits We cenoteX as the number of the trial on which he experienceshe third hit The mean is 306 5 mearirig that on the average it Will tale five throws The presidents decision rule is simple Ifhe gets three hits on or before the fifth throw be decides in favor of the question The probability that be will decide in favor is herefore PX 5 p3 p p5 21J r 3 3 l4 l2 06 04 l2 06 0 2fo6 04 06826 56 THE MULTlNOllfiAL DISTRIBUTION An important and useful higherdimensional random variable has a distribution kuown as the multinomial distribution Assume an experiment with sample space ff is partitioned into k mutually exclusive events say B 11 B2 BIr rYe consider n independent repetitions of I and let Pi PB be constant from trial to trial for i 1 2 Ie If k 2 we IDwe Bernoulli triaJs as described earlier The random vector X x XJ has the following distribution where Xj is the number of times Bf occurs in the n repetitions ofi i 1 2 1k ll r n llx Xl P XIX PI P2 Pk XlX2XkJ 523 for Xl 0 12 n i 12 k and where LX n il k It should be noted that X x X are not independent random variables since L x It for any n repetitions i It turns out that the mean and variance of Xi a particular component are 57 The Hypergeometric Distribution 117 EX Pi 524 and VXi npl p 525 1lli9 Mechanical peucils are manuactured by a process involving a large amount of labor in the assembly operations This is bighly repetitive work and L1Ceutive pay is invoved Final inspection has revealed that 85 of the product is good 10 is defective but may be reworked and 5 is defective and must be scrapped These percentages remain constant oyer time A random sampe of 20 items is seleced and if we ler then X number of good iteI11S A number of defective but reworkable items X number of items to be scrapped pxpxx 20 085 010 005 Zt1 2x3 Suppose we want to evaluate t probability function for Xl 18 2 andx 0 we must have Xl l xJ 20 ben ZOi Pilg 2 Oi O8SJ010O05 I 18 1110 r 0102 57 THE HYPERGEOMETRIC DISTRIBUTION In an earlier section an example presented the hypergeometric distribution We wiL now formally develop this distribution and further illustrate its application Suppose there is some finite population withN items Some number D D slv of the items fall into a class of interest The particular class will of course depend on the situation under consideration It might be defectivesvs nondefectives in the case of a production lot or persons With blue eyes vs not blue eyed in a classroom with N students A random sample of size n is selected without replacement and the random variable of interest X is the number of items in the sample that belong to the class of interest The distribution of X is DN DJ xJnx p x J x 0 I 2 rninn D 0 otherwise 526 The hypergeometrics probability mass function is available in many popular software pack ages For instance suppose that N 20 D 8 n 4 andt 1 Then the Excel function HYPGEOMDISTx 11 DN HYPGEOMDIST14 8 20 03633 118 Chapter5 Some Importart Discrete Distributions 571 Mean and Variance of the Hypergeometric Distribution The mean and variance of the hypergeometric distribution are Exnrl and DJ D NnJ Xln N 1 N NI 527 528 In a receiving inspeetion department lots of a pump shaft are periodically received The lots contain 100 units and the following acceptance sampling plan is used A random sample of 10 units is selected wihout replacement The lot is accepted if the sample ba1 no more thai one defective Sup pose a lot is received that is plOO percent defective What is the probability that it will be accepted lOOpUIOOl pJI x 1 lOx Paccept lot PXl 01 lO 10 100PX1001PJlOOPI00l p 0 l 10 1 9 eOO O Ob iously he probability of accepting he lot is a function of the lot quality p If p 005 then p accept lot 0923 58 TIlE POISSON DISTRIBLTION One of the most useful discrete distributions is the Poisson distribution The Poisson dis tribution may be deeloped in NO ways and both are instructive insofar as they indicate the cixcumstances where this random variable may be expected to apply in practice The first development involves the definition of a Poisson process The second development shows the Poisson distribution to be a limiting form of the binomial distribution 581 Development from a Poisson Process In defining the Poisson process we initially consider a collection of arbitrary timeoriented occurrences often called arrivals or births see Fig 511 The random variable of interest say X is the number of arrivals that occur on the interval 0 t The range space Rx O 1 2 In developing the distribution of X it is neeess3I to make some assump tions the plausibility of which is supported by considerable empirical evidence 58 The Poisson Distribution 119 o Figure 511 The time axis The first assumption is that the number of arrivals during nonoverlapping time inter vals are independent random variables Second we make the assumption that there exists a positive quantity A such that for any small time interval I1t the following postulates are satisfied 1 The probability that exactly one arrival will occur in an interval of width I1t is approximately A ill The approximation is in the sense that the probability is A ru oru where the function oruru 0 as ru O 2 The probability that exactly zero arrivals will occur in the interval is approximately 1 1 ru Again this is in the sense that it is equal to 1 1 ru oru and 02ruru 0 as ru O 3 The probability that two or more arrivals occur in the interval is equal to a quan tity oM where oruru 0 as ru O The parameter A is sometimes called the mean arrival rate or mean occurrence rate In the development to follow we let px PX x pit xO 12 529 We fix time at t and obtain port ru 1 1 ru port so that and 530 For x 0 pt ru 1 ru pt 1 1 ru pt so that and lim ptruPt Px t I Pxl t A Px t 0 ill 531 Summarizing we have a system of differential equations 532a and x 1 2 532b 120 Chapter 5 Some ImpOrtant Discete Distributions The solution to these equations is pir AtYelx xo 12 533 Thus for fixed we let c At and obtain the Poisson distribution as pX xO12 0 otherwise 534 Note that this distribution was developed as a consequence of certain assumptions thus when the assumptions hold or approximately hold the Poisson distribution is an appropriate modeL There are many realworld phenomena for which the Poisson model is appropriate 582 Development of the Poisson Distribution from the Binomial To show how the Poisson distribution may also be developed as a limiting form of the bino mial distribution with c np we return to the binomial distribution pX n rfI I I P P X n x x 0 12 n If we let np c so that P cn and I p 1 cn n en and if we then replace terms involving p with the corresponding terms involving c we obtain px nIn 2 n x I r eJ nc1 l l Xl n n C IV 2 f xIl cf c 1111 1 11 x n n n J n n 535 In lettingn andp Oin such a way that op cremainsfixed the terms I 1 J 1 1 c C tl tl I all approach 1 as does 1 T Now we know that 1 j tf as n II Thus the limiting fonn ofequatlon 535 is pix elx e whichs the Poisson distribution 583 Mean and Variance of the Poisson Distribution The mean of the Poisson distribution is c and the variance is also C as seen below C 536 Similarly so that VX EX EX c 537 58 The Poisson Distribution 121 The momentgenerating function is MJ e 538 The utility of this generating function is illustrated in the proof of the following theorem Theorem 51 If Xl X2 j Xk are independently distributed random variables each having a Poisson dis tribution with parameter c i I 2 k and Y Xl X2 Xkj then Yhas a Poisson distribution with parameter Proof The momentgenerating function of Xi is Mxt el and since Mt Mxt Mxt Mxt then Mlt ecc eJCl which is recognized as the momentgenerating function of a Poisson random variable with parameter C cj Cz c This reproductive property of the Poisson distribution is highly useful Simply it states that sums of independent Poisson random variables are distributed according to the Poisson Hstribution A brief tabulation for the Poisson distribution is given in Table I of the Appendix Most statistical software packages automatically calculate Poisson probabilities Suppose a retailer determines tha the number of orders for a certain home appliance in a particular period has a Poisson distribution with parameter c She would like to determine the stock level K for the beginning of the period so that there will be a probability of at least 095 of supplYng all cus tomers who order the appliance dwing the od She does lot wish to backorder merchandise or reupply me warehouse duing the period If X represents the number of orders the dealer wishes to determine K such that PX Kl 095 or PXKl 005 so that Iec X Ix 005 xX The solution may be determined directly from tables of the Poisson distribution and is obviously a fuuction of c The attainable sealitivjty for e1ectromc amplifiers and app is limited by noise or spontaLeous current fluctuations In vacuum tibes one noise source is shot noise due to the random emission of 122 Chapter Some Important Discrete Distributions electrons from the heated cathode Assume that the potential difference between the anode and cath ode is large enough to ensure that all electrons Imitted by the cathode hav high velocityhigh enQugh to preclude spare charge accumulation of electrons between the cathode and anode Under these conditions and cefining an arrival to be an emission of an electrode from the cathode Daven port and Root 1958 showed that the number of electcons X emitted from the cathode in time t has a Poisson distribution given by p x AtYetyx 0 X 01 2 otherwise The parameter 1 is the mean rate of emission of electrons from the cathode 59 SOME APPROXIMATIONS It is often useful to approximate one distribution using another particularly when the approximation is easier to manipulate The two approximations considered in this section are as follows 1 The binomial approximation to the hypergeometric distribution 2 The Poisson approximation to the binomial distribution For the hypergeometric distribution if the sampling fraction nIN is small say less than 01 then the binomial distribution with parameters p DIN and n provides a good approx imation The smaller the ratio nli the better the approximation Ppl5jl A prodction lot of 200 units has eight defectives A random sample of 10 units is selected and we want to find the probability that the sample will contain exacdy one defective The true probability is pXl Gf 2 J 0288 SiDcerNO05 is small we letp O4 3ld use the binomial approximation 10 pI I r04O96 0277 In the case of the Poisson approximation to the binomial we indicated earlier that for large n and small p the approximation is satisfactory In utilizing this approximation we let c np In general p should be less than 01 in order to apply the approximation The smallerp and the larger n the better the approximation pIi The probability that a particular rivet in the wing surface of a new aircraft is defective is 0001 There are 4000 rivets in the wing Vhat is the probability that not more than six defective rivets ill be inst2lJed PX6 4000IOOOlYO999 l x Using the Poisson approximation and PX6 244x 0889 m 510 GENERATION OF RELIZATlONS Sehemes exist for using random numbers as is described in Section 36 to generate rea1 izations of most common random variables With Bernoulli trials we might first generate a value u as the ith realization of a urn fonn 01 random variable U where f I 0 and independence among the sequence Ui 1S maintained Then if u p we let XI 1 and if p X O Thus if Y 2 Xi Y will follow a binomial ilistribution with parameters nand p and this entire process might be repeated to produce a series of values of Y that 8 realizations from the binomial ilistribution with parameters n and p Similarly we could produce geometrie variates by sequentially generating values Ii and counting the number of trials until u s p At the point tlJis conilinon is met the trial number is assigned to the random variable X and the entire preeess is repeated to produce a series of realizations of a geometrie random variable Also a similar seheme may be used for Pascal random variable realizations where we proceed testing until tlJis conilition has been satisfied r times at which point the trial number is assigned to X and once again the entire process is repeated to obtain subsequent realizations Realizations from a Poisson ilistribution with parameter At c may be obtained by employing a technique based on the socalled acceptancerejection method The approach is to sequentially generate values ui as described above until the product ul uk eC is obtained at which pomt we assign X t k and once again this process is repeated to obtain a sequence of realizations See Chapter 19 for more details on the generation of discrete random variables 511 SUMMARY The distributions presented in this chapter have wide use in engineering scientific and management applications The selection of a specific discrete distribution ill depend on how well the assumptions underlying the distribution are met by the phenomenon to be modeled The distributions presented here were selected because of their wide applicability A sllIl1Jl1JlI of these ilistributions is presented in Table 51 SU EXERCISES 5 1 An experiment consists of four independent Bernoulli trials with probability of success p on each trial The random ariable X is the number of suc cesses Enumerate the probability distribution of X 52 Six independent space missions to the moon are planned The estimated probability of success on each mission is 095 What is the probability that at least five of the planned missions will be successful Tble 51 Summary or Discrete Distributions Distribution Parameters Bernoulli Opl ninonuro 1112 Opl Geometric Op I Pascal Op 1 Neg binomial r I 2 r 0 Hypergeomelric N12 n 12 N DI2 N Poisson cO px q O Probability Function px xO1 otherwise px Xqx xOI2 1I 0 oilienvic px pq x011 otherwise x I px rl prqxr xr r1r2 f O otherwjse px GFJ x 012 rninD 0 otherwise XlI pXf C C xO2 O otherwie Mean Variance P pq r P llpq f lip qlP rIp rqlp pl eN flJIlDI N N NI C c I Momcnt Generating Futiction pe q pel 1 q pel I qe r pe 1 qel See Kendall and Stuart 1963 erI Q g i f f if S3 The XiZ Company has plalDed sales presenta tions to a dozen important customers Tne probability of receiving an order as a result of such a prentation is estirnated to be 05 What is the probability of receiving fOUI Or more orders as the result of the meet ings 54 A stockbroker calls her 20 most important cus tomerS every morning If the probability is one in three of making a transaction as the result of such a call what arc the chances of her hatdlilg 10 or more transactions 55 A production process that manufactures transis tors operates on the average at 2 fraction defective Ever 2 hours a random sample of size 50 is taken frOIll the proeess If the sample contaits more than tVO defectives the process must be stopped Deter Iilire the probability that the process will be stopped by the sampling scheme 56 Find the mean and variance of the binocrial dis tribution using the momentgenerating function see equation 57 57 A production process manufactoriDg tumindica tor dash lights is known to produce lights that are 1 defective Assume this value remains unChanged and assunle a sample of 100 such lights is randomly selected FindPp 003 where P is the sample frac tion defective 58 Suppose a random sample of size 200 is tacen frorr a process that is 007 fraction defective llat is the pobabilit that p will exceed the LUe fraction defective by one standard deviation By two standard deviations By three standard deviations 59 Five cruise missiles have beea bci1t by an aero space company The probability of a successful firing is on anyone test 095 Assuming independent fu ings what is the probability that thefust failure occurs on the fifth firing 510 A real estate agent estimates his probability of selling a house to be 010 He has to see fOUI clients today If he is successfd on the first three calls what is the probability that his fourth call is unsuccessful 511 Suppose five independent identical laboratory expe1mcnts are to be undertaken Each experiment is extremely sensitive to en ironmental conCitions and there is only a probability p that it will be completed successfully Plot as a functiol of p the probability hat thefifth experiment 15 the first failure Fird math ematically the value of p that maximizes the probabil ity of the fifth trial being the first unsuccessful eperiment 512 Exercises 125 512 The X1Z Compa1Y plans to isit potential cus tomers until a substantial sale is made Each sales presentation costs 1000 It costs 4000 to travel to the next customer and set up a new preseatation a What is the expected cost of making a sale if the probability of lMking a sale after any presentation is O1O b If he expeeled profit from each sale is 15000 should the trips be undertaken c If the budget for advertising is only 100000 what is the probability that this sum will be spent Vlithout getting an order 511 l1nd the mean and variance of the geoJlletric distribution usiog the momeutge1erating function 514 A snbmarines probability of sinking an enemy ship with anyone of its torpedoes is 08 If the firings ae irdepeode1t determine the probability of a sinking within the first two firings Vithin the Erst three 515 In Atlarta the probability that a thunderstorm will occur on any day during the spring is 005 Assuming independence what is the probability that the first thunderstorm occurs on April 25 Assume spring begins on March 21 516 A potential customer enters an automobile deal ership every hour The probability of a salesperson concluding a transaction is OJO She is detennined to keep working until she has sold th ee cars Nhat is the probability that she will have to work exactly 8 hours More than 8 hours 517 A persorncl manager is interviewing potential employees in ordct to fill twO jobs The probability of an interviewee having the necessary qualifications ad accepting an offer is 08 VIhat is the probability hat exactly four people must be inteniewed hat is the probability that fewer than four people must be inter viewed s18 Show that the momentgenerating function of the Pascal tandom variable is as given by equation 522 Use it to detennine the mean and variance of the Pascal distnoution 5 19 The probability that an experiment has a suc cessful outcome is 080 The experiment is to be repeated until five successful outcomes have occurret What is the expe1ed numbe of repetitions required WJat is the variance 520 A military comander wishes to destroy an enemy bridge Each fligflt of planes he sends out has a probability of 08 of scoring a direct hit on the bridge Ir takes four direct hits to completely destroy the bridge If he can mount seven assaults before the 126 Chapter 5 Some Important Discrete Distributions bridge becomes tactically unimportaJt what is the probability that the bridge will be dcstrOyed 521 Three companies X Y and Z have probabilities of obtaining an order for a particular type of mer chandise of 04 03 and 03 respectively Three orders are to be awarded independently What is the probability that one company receives all the orders 522 Four companies are interviewing five college students for positions after graduation Assuming all five receive offers from each company and assuming the probabilities of the companies hiring a new employee ate equal what is the probability that one company gets all of the new employees one of diem 523 We are interested in the weight of bags of feed Specifically we need to know if any of me four events below has occurred TJ X 10 TlOX 11 Tll X 115 T4 115 X pT1 02 pT 02 pT 02 pT 04 If 10 bags are selected at random what is the proba bility of four being less than or equal to 10 pounds one being greater than 10 but less than or equal to 11 pound ald two bcing greater than 115 pounds 524 In Problem 523 what is the probability that ill 10 bags weigh more than 115 pounds What is the probability that five bags weigh more than 115 pounds and the remaining five weigh less than 10 pounds 525 A lot of 25 color television ubes is subjected to an acceptance testing procedure The procedure con sists of drawing five tubes at random withourreplacew ment and testing them If two or fewer tubes fail the remaining ones are accepted Otherwise the lot is rejected Assume the lot contains four defective tubes a What is the exact probability of lot aeceptance b Ybat is the probability of lot acceptance com puted from the binomial distribution with p s 526 Suppose that in Exercise 525 the lot size had been 100 Would the binomial approxilnation be satis factory in this case 527 A purchaser receives small lots N 25 of a highprecision device She wishes to reject the lot 95 of the time iiit contains as many as seven defec tives Suppose she decides that the presence of one defective in the sample is sufficient to cause rejection How large should bet sample size be 5 28 Show that the momentgenerating function of the Poisson random variable is as give by equation 538 529 The number of antomobiles passing through a pa1icular intersection per hour is estimated to be 25 Find the probability that fewer than 0 vehicles pass through during any 1 hour interval Asume that the number of vehicles follows a Poisson distribution 530 Calls arrive ata telephone switchboard such that the number of calls per hour follows a Poisson distri bution with a mean of 10 The current equipment can hatdle up to 20 calls Vithout becoming overloaded What is the probability of such an overload occurring 531 The number of red blood cells per square unit vlsible under a microscope follows a Poisson distribu tion with a mean of 4 Find the probability that more than five such blood cells are visible to the observer 532 LetX be the number of vehicles passing through an intersection during a length of time t The random variable Xis Poisson distributed with a parameter t Suppose an autollatic counter has been installed to count the number of passing vehicles However this counter is not functioning properly and each passing vehicle has a probability p ofnot beitg counted Let Yt be the number of vehicles counted during t Fmd the probability distribution of 5 33 A large insurance company has discovered that 02 of the US population is injured as a result of a pacuJar tpe of accidentnlis company has lOOO policyholders carrying coverage against such an acci denL What is the probability that three or fewer claims will be filed against those policies next year Five o Qore claims 534 Maintenance crews arrive at a tool crib request ing a particular spare part according to a Poisson dis tibution with parameter 1 2 Three of these spare parts are nOIIlally kept on hand If more than three orders occur the crews must journey a consideabte distance to central stores a On a given day what is the probability that such a journey must be made b Wbat is the expected demand per day for spare partS c How many spare parts must be carried if the tool crib is to service all incoming crews 90 of the time Cd hat is the expected number of crews serviced daily at the tool crib e Wbatis the expected number of crews makiLg the journey to central stores 535 A loom experiences one yam breakage approx imately every 10 hours A panicular style of cloth is being produced that will take 25 hours on this loomlf tiuee or more breaks are required to tender the prod uct unsatisfactory find the probability that this style of cIoll is finished wih acceptable quality 536 The mmber of people boarCing a bus at each stOP follows a Poisson disnibution with parameter A TIle bus company is surveying its usages for scbedul ing purposes and bas installed an automatic counter on each bus However if more than 10 people board at any one sop the counter cannot record the excess and merely registers 10 If X is the number of riders recorded find the probability distribution of X 537 Amathemadcs textbook has 200 pages on which typographica errors in the equations could occur If there are in fact five euors randomly dispersed among these 200 pages what is the probability that a andom sample of 50 pages will contain at least one error How large must the random sample be to assure that at least three ertOS will be found with 90 probability 538 The probability of a vehicle baving an accident at a particular intersection is 00001 Suppose that 10000 vehicles per day travel thvugh this intersection tat is the probability of no accidents occurring What is the probability of two or more accidents 5 39 If the probability of being involved in an auto accident is 001 during any year wbat is de probabil ity of baving two Or Dare accidents during any 10 yem driving period 540 Suppose that the number of aecidents to employees working on highexplosive sbells over a period of time say 5 weeks is taken to follow a Pois son discrihtion With parameter A 2 512 Exercises 127 a Find the probabilities of 1 2 3 4 or 5 accidents b The Poisson distribution has been freey aPPlied in the area of industrial acciderts However ir fre quetly provides a poor fit to actual hismrical data Why rniglt this be t1le Hint See Kendall and Stuart 1963 pp 128130 541 ese either yo favorite conputer language or the random integers in Table XV in he Appendix and scale them by mUltiplying by 1 to ge uniform 01 random nunberrealizations to do the following a Produce five realizations of a binomial random variable with n Sp 05 b Produce ten realizations of a geometric distribu tion With p OA c Produce five realizations of a Poisson random variable with c 015 54 If Y XI3 a11 X follows a geometric discribution with a wean of 6 use uniform lOl random number realizations and produce five realizations of Y 543 Vith Exercise 542 above use your computer to do the following a Produce 500 realizations of Y b Calculatei 1 Ysoo the mean from this szmple 544 Prove the meworyless propety of the geometric distribution r G 1 i Chapter 6 Some Important Continuous Distributions 61 INTRODUCTION We will now study several important continuous probability distributions They are the uni form exponential gamma and Weibull distributions In Chapter 7 the normal distribution and several other probability distributions closely related to it will be presented The nor mal distribution is perhaps the most important of all continuous distributions The reason for postponing its study is that the normal distribution is important enough to warrant a sep arate chapter It has been noted that the range space for a continuous random variable X consists of an interval or a set of intervals This was illustrated in an earlier chapter and it was observed that an idealization i involved For example if we are measuring the time to failure for an electronic component or the time to process an order through an information system the measurement devices used are such that there are only a finite number of possible out comes however we vill idealize and assume that time may take any value on some inter val Once again we will simplify the notation where no ambiguity is introduced and we let fx fxx and Fx F x 62 THE UNIFORM DISTRIBlJT10N 128 The uniform density function is defined as fx f pa 61 where a and 3 are real COnStants with a 3 The density function is shown in Fig 6L Since a uniformly distributed random variable has a probability density function that is fxt 1 pa L L P x Figur1l 61 A uniform density 62 Toe Uuiforrr Distribution 129 constant over some inteval of definition the constant must be the reciprocal of the length of the interval in order to satisfy the requirement that fix dx1 A uniformly distributed random variable represents the continuous analog to equally likely outcomes in the sense that for any subinterval a b where a a b f3 the PaX b depends only on the length b a PaXb t ba af3a 0 The statement that we choose a point at random on a f3J simply means that the value cho sen say Y is uniformly distributed on a f3 621 Mean and Variance of the Unifonn Distribution The mean and variance of the uniform distribution are 62 which is obvious by symmetry and VXJP xdx f3 a af3a 2 f3a 12 63 The momenrgenerating function Mit is found as follows 64 For a uniformly distributed random variable the distribution function Fx PX x is given by equation 65 and its graph is shown in Fig 62 Fx 0 x a J x axf3 f3a f30 1 55 FXt o a f3 X Figure 62 Distnoution function for the uniform random variable 130 Chapter 6 Some Important Continuous Distributions Et6J A point is chosen at random on the interval 0 10 Suppose we wish to find the probability that the point lies between t and t The density of the random yariableX isfx 70 OSx5 10 andfxO otherwise Hence pet s X s t fa i6 Numbers of the form NNN are rounded off to the neaSt integer The roundoff procedure is such that if the decimal part is less than 05 the round is down by simply dropping the decimll part however if the decimal part is greater than 05 the round is UP that is the new number is livlVNJ I 1 whcre L J is the greatest integer contaixed in function Ii the decimal part is exactly 05 a coin is tossed to determine which way to rlUnd The roundoff errorx is defined as the difference between the number before roundiDg and the number after rounding These errors are comroonly distributed according to the uniform distribution on the interval 15 05 That is jxl I l5 X 05 O otherwise KaiIl1i3 Rw One of the special features of many simulation languages is asiruple automatic procedure for using the unifonn distrbution The user declares a mean and modifier eg 500 100 The compiler immediately creates l routine to produce realizations of a random variable X uniformly distributed on 400 600 In the special case where a 0 f3 1 the uniform variable is said to be uniform on 0 IJ and a symbol U is often used to describe this special variable Using the results from equations 62 and 63 we note that EU and VU lZ Ii U Uz U is a sequenee of such variables where the variables are mutually independent the values VI UJ Uk arc called random numhers and a realization u ut is properly called a random um ber realization however in COmmon usage the term random numbers is often given to the realizations 63 THE EXPONEiiTIAL DISTRIBUTION The exponential distribution has density function Jx Ie x 0 0 66 where the parameter A is a real positive constanL A graph of the exponential density is shown in Fig 63 fxlf o x Figure 63 The expouetttial density function 63 T1e Exponential Distribution 131 631 The Relationship of the Exponential Distribution to tbe Poisson Distribution 632 The exponential distribution is closely related to the Poisson distribution and an explana tion of this relationship should help he reader develop an undentanding of the kinds of sit uations for which the exponential density is appropriate In developing the Poisson distribution from the Poisson postulates and the Poisson process we fixed time at SOme value t and we developed the distribution of the number of occurrences in the interval 0 r We denoted this tandom variable X and the distribution was px ettlx 0 x 0 12 othenvise 67 Now consider prO which is the probability of no occurrences on 0 tJ This is given by pO e 68 Recall thet we originally fixed time at t Another interpretation of prO e is that this is the probability that the time to the Drst occurrence is greater than t Considering this time as a random variable T we note that pO peT I e t O 69 If we now let time vary and consider the random variable T as the time to occurrence then 0 C 610 lI And since jtt FI we see that the density is eJ j ft no V X J 0 otherwise yi 611 This is the exponential density of equation 66 Thus the relationship between the exponential and Poisson distributions may be stated as follows i the number of occur rences has a Poisson distribution as shown in equation 67 then the time between succes sive occurrences has an exponential distribution as shown in equation 611 For example if the number of orders for a certain item received pel week has a Poisson distribution then the time betvreen orders would have an exponential distribution One variable is discrete the count and the othe tiroe is continuous In oreer to verify thatfis a density function we note thatx for all x and l 15 o k dx e Q 1l V V Mean and Variance of the Elonential DistributiQY y The mean and variance of the exponential distribution are 612 and VX J x2kdxl2 x2eI zJ xedxl 1 613 132 Chapter 6 Some lIUportant Continuous Distnoutions Fxt Fig 64 Distribution fulletion for the x expoaential The standard deviation is 1A and thus the mean and the standard deviation axe equal The momentgenerating function is provided t yl Mxti 11 j 614 The cumulative distribution function F can be obtained by integrating equation 66 as follows XO J krdrlelx xo 615 Figure 64 depicts the distribution function of equation 615 An electronic component is known to have a useful life represented by an exponential density Xith failure rate of 10 failures per hour ie 1 105 The mean time to fwtLe EX is thus 10 hours Suppose we want to determine the fraction of such components that would fail before the mean life or expectedlife This result holds for arty value of J greater than zero In our example 63212 of the iteI1S would fail before 10 hours see Fig 65 fix fx AeAx X2 0 0 ot1erwfse A Figure 65 The mean of an exponential dlstribution 63 The Exponential Distribucon 133 Suppose a designer IS to make a decision between two manufacturing processes for the rnaufacture of a certain component Process4 cos C dollars per unit to manufactcre a component Pocess B costs k C dollars per unit to manufacrure a component where k 1 Corponents Jave an exponen tial time to failure density with a failure rate of 200 failures per hour for process A while compo nents from process B have a failure rate of 3001 failures per hour The mean lives are rhus 200 hours and 300 hou respectively for the tWo processes Because of a warrarty clause 1 a componen lasts for fewer than 400 hours the IDOllufacCter roust pay a penalty of K dollars Let X be be time to fail ure of each component Thus the component costs 34 and ifX400 UX 400 UX24OO ifX4oo The expected costs are and ECA CK 1 400 200ledxC J200 ledx C Kenoc 40l CeXlfJl 1 0J L 400 CKleCej CKIe2 E Co kC K f 3oole300 dx kCJ3OO1e13 dx kC K1eif3kCe4l3 kCKllej Therefore if k 1KlC2 e then E CA ECfj and itis likely that 1e designer would seect process B 633 Memoryless Property of the Exponential Distribution T4e exponential distribution has an interesting and unique memoryless property for con tinuous variables that is PXxs P XxsIXx PXx so that PX six x PXs 616 For example if a cathode ray tube has an exponential time to failure distribution and at time x it is observed to be still functioning then the remaining life bas the same exponential fail ure distribution as the tube had at time zero 134 Chapter 6 Some Important ConriDuous Distributions 64 THE GAMl1A DISTRIBlJTION 64l Tbe Gamma Function A function used in the definition of a gamma distribution is the gamma function defined by fornO 617 J important recursive relationship that may easily be shown on integrating equation 6 17 by parts is rn n Ifn I 618 If It is a positive integer then rn n I 619 since reI J e dx L Thus the gamma function is a generalization of the factorial The reader is asked in Exercise 617 to verify that 1 r r I J xlf2exdxfii 2 0 642 Definition of the Gamma Distribution 620 With the use of the gamma funtion We are now able to introduce the gamma probability density function as 0 otherwise 621 The parameters are r 0 and A O The parameter r is usually called the shape parameter and lis called the scale parameter Figure 66 shows several gamma distributions for 1 1 and various r It should be noted thatjx 0 for alb and J jxdx1 Tledx 0 fr 11 vleYdyI T r I fr 0 rr The cumulative distribution function CDP of the gamma distribution is analytically intractable but is readily obtained from software packages such as Excel and Mintab Jn par ticulat the Excel function GA1MADISTx r fA TRUE gives the CDF Fx For exam fx Figure 6k6 Gamma distribution for t L r 64 The Gamma DistributioJ 135 ple Gk11LWIST15 55 42 TRUE returns a value of F15 02126 fur the case r 55 142 0238 The Excel function GAMMAIlV gives the inverse of the CDP 643 Relationship Between the Gamma Distribution and the Exponential Distribution There is a close relationship between the exponential distribution and the gamma distribu tion Namely if r 1 the gamma distribution reduces to the exponential distribution 1bis fol lows from the general definition that if the random variable X is the sum of r independent exponentially distributed random vanables each with parameter then X has a gamJrtl density with parameters r and A That is to say if XXl x X whee has probability density function gx M 0 XO othervrise 622 and where the Xj are mutually independent then X has 11e density given in equation 621 b many appliations of the gamma distribution that we will consider r will be a positive integer and we may use this knowledge to good advantage in developing the distribution function Some authors refer to the special case in which r is a positive integer as the Erlang distribution 644 Mean and Variance of the Gamma Distribution We may show that the mean and variance of the gamma dstribution are EX rI 623 and VX rl 624 Equations 623 and 624 represent the mean and variance regardless of whether or not Tis an integer bow ever wben T is an integer and the interpretation given in equation 622 is made it is obvious that EX 2EXj rJjr j and VX 2 VXr12 r2 jl from a direct application of the expected value and variance operators to the sum of inde pendent random variables The momentgenerating function for the gamma distribution is 625 136 Chapter 6 Some Important Continuous Distributions RecalUng that the momentgenerating function for the exponential distribution was 1 II ir thls result is expected since MXX Xt UMXjt1IHT The distribution function F is Fxlr rrtYlendt 0 0 so If r is a positive integer then equation 627 may be integrated by partS giving I Fx 1 Le Ax k XO U 626 627 628 which is the sum of Poisson terms with mean Ax Thus tables of the cumulative Poisson may be used to evaluate the distribution function of the gamma i2 A redundant system operates as shownm Fig 67 Inirially unit 1 is On line whlleuuit 2 and unit 3 arc on standby When uniT 1 fails r the decision switch OS switches unit 2 on until it fails and then unit 3 is switehed on The decision switch i assumed to be perfect so that the system life X may be represented as the sum of the subsystem lives X Xl l2 XJ If the subsystem lives are independent of one another and if the subsystems each have a lifexj I 2 3 harng density gx 1l00eI1l00 X 0 then X will have a garrma density with r3 and lOOl That is 001 001 f x OOlx e xO 2 0 otherwise The probability that the system Hill operate at least hours is denoted Rx and isalled the reliabil ity ftmction Here RxlFxl e01oOlXk eOl1 OOlx O01x 2 Fignnl67 A standby redundant syst 65 The Weibull Distribution 137 pXZ For a garnIa distribution with t t and r v2 where v is a positive integer the chisquare distri lJution with v degrees of freedom results f 1 2 I 0 x 2 vtZ ry2 x It x 0 otherwise This distribution will be discussed further in Chapter 8 65 THE WEffiI1LL DISTRIBUTION The Weibull distribution has been widely applied to many random phenomena The princi pal utility of the Weibull distribution is that it affords an excellent approximation 0 the probability law of many random variables One important area of application has been as a model for time to failure in electrical and mechanical components and systems This is dis cussed in Chapter 17 The density function is fx xrr exp xrP xr O otherwise 629 Its parameters are y r 00 the location parameter 0 0 the scale parameter and f3 0 the shape parameter By appropriate selection of these parameters this density func tion will closely approximate many observational phenomena Figure 68 shows some Weibull densities for r 0 8 1 and f3 1 2 3 4 Note that when y 0 and f3 1 the Weibull distribution reduces to an exponential density with 18 Although the exponential distribution is 3 special case of both the gamma and Weibull distributioust the gamma and Vleibull in general are noninterchangeable 651 Mean and Variance of the Weibull Distribution The mean and variance of the Weibull distribution can be shown to be 630 08 20 x Figure 6s Weibull densities for rO 0 1 and j3 1 2 3 4 138 Chapter 6 Some Imporant Continuous Distributions and The distribution function has the relatively simple fonn r xrPl Fxlexp J j 631 632 The Weibull CDF Fx is conveniently provided by software packages sueh as Excel and Minitab For the case 0 the Excel function VlEIBlTLL x 3 y TRIJE retnms Fz i The tiruetofailure distribution for eleCtronic subassemblies is known to have a Weibul density with yO pt and 0 100 The fraction expected to survive to say 400 hours is fuus 1 F400 e 140000 01353 The same result could have been obtainedin Excel via the function call 1WEIBUu 400 112 100 TRUE The mea1 time to failure is EX 0 1002 200 hours XC6 Berrettoni 14 presented a number of applications of the Weibull distribution The following ae examples of natural processes having a probability law closely approximated by the Weibull distrl bution The randol variable is denoted X in the exampes 1 Corrosion resistance of magnesium alloy plates X Corrosion weight loss of 10 mgcmlday when magnesium alloy plates are immersed in an inhibited aqueous 20 solution ofMgBr2 2 Return goods classified according to number of weks after shipment X Length of period 10 weeks until a customer returns the defective product after shipment 3 Number of downtimes per shift X Number of downtimes per sblft times 101 occurring in a CoIltinuoUS automatic and Cm plicated assembly line 4 Leakage failure in drycell batteries X Age years when leakage starts 5 Reliability of capacitors X Life hours of 33uF 50V solid tantalum capacitors operating at an ambient tempera ture of 125C where the rated catalogue voltage is 33 V 66 GENERATION OF REALIZATIONS Suppose for now that U1 U are independent uniform 0 1 random variables We ill show how to use these uniionns to generate other random variables 68 Exenises 139 If we desire to prodce realizations ofa uniform random variable on a ill this is sim ply accomplished using X a u3 a i I 2 633 If we seek realizations of an exponential random variable with parameter A the inverse transform merlwd yields I xi TInUi i 12 634 Similarly using the same method realizations of a Weibull random variable with P2ame ters 11 3 5 are obtained usIng 635 The generation of gamma variable realizations usually employs a technique known as the acceptancerejection mefrod and a variety of these methods bave been used If we wish to produce realizations from a gamma variable with parameters r 1 and 0 one approach suggested by Cheng 1977 is as follows Step 1 Let a 2r 1112 and b 2rIn 4 ila Step 2 Generate u1 u as unifonn 0 1 random number realizations Step 3 Letyrulua step 4a If y b Inuu tejeet y and return to Step 2 Step 4b If Y b InCuu assign x yt For more details on these and other randomvariate generation techniques see Cbapter 19 67 SUMMARY This chapter bas presented four widely used density functions for continuous random vari abIes The uniform exponential gamma and Weibull distributions were presented along with underlying assumptions and example applications Table 61 presents a summary of these distributions 68 EXERCISES 61 A point is chosen at random on the line segrrent Od Whatjs the probability 1Jatitlies between t and 3 147 Between 24ad 3t1 62 The opening price of a particular stock is uni fonn1y distributed on the interval 35 44J What is the probability that on any given day the opening price is less than 401 Between 40 and 427 63 The random variable X is uniformly distributed on the interval 0 21 find the distribution of the ran dom variable Y 5 2X 4 A real estate broker charges a fixed fee of 50 plus a 6 cqmmission on the landoiillers profit If this profit is distributed between SO and 2000 find the probability distribution of the brokers total fees 65 lise the momentgenerating function for the uni form density as given by equation 64 10 generate the mean and VJriance 66 Let X be uniformly distributed and symmetric about zero with lariance 1 Find the appropriate vli ues for a and fl 6 7 Show how the uniform density function can be used to generate variates from the empirical probabil it distribution desctibed below Table 61 Summary of Continuous Diisiolls rrI Density Punctionfx Unifonn Density I 1 fix fJa axfJ Pnrl1meters afJ nxpouential Gamma Weibull f3a bO 00 bO 00 r 110 thO 1 O otherwise fxM 00 fix rrAreu XO o otherwise PxrPU r XrP fix 8 0 exp 0 xr 0 otherwise Mean Varianee a fl12 fJ a 112 1 t II I rll rl 2 I I rr pl 8H 1 i 1J 3 Q l Moment i Generating i Funetion e etC If a g g 1llAl t1 a llltr y 2 3 4 ply 03 02 04 OJ Hint Apply the inverse transform method 68 The random variable X is unifonnly distdbuted oyer the interval 0 4 Wbatis the probability thel the roots of 4Xy X 1 0 are real 69 Verify that the momentgenerating function of the eneDtia1 dislribution is as given by equation 614 Use it to generate the mean atd variance lO The engine and drive train of a new car is guar w anteed for 1 year The mean life of an engine and drive train is estimated to be 3 years and the time to fdilurc has an exponential densiy The realized profit on a new car is 1000 Including costs of parts and labor the dealer must pay 250 to repair each failure What is the expected profit per car 611 For the data in Exercise 6 10 what percentage of C3IS will experience failure ir the engine and drive train dug the fiSt 6 months of use 612 Let the length of time a machine will operate be an eponentially distributed random variable with probability density functionjt ee t O Suppose an operator for this machine must be hired for a pre determined and fixed length of time say Y She is paid d dollars per time period during this interval The net profit from operating this machine exclusive of labor costs is r dollars per time period that it is operating Fmd the value of Y that maximizes the expected total profit obtained tCi time to failure of a television tube is estt IDa to be exponentially distributed vith a mean of 3 years A company offers insurance on these tubes for the fSt year of usage On what percentage of poli cies will they have to pay a claim fl4Jrs there an exponential density that satisfies the fcm5wing condition PX 2 tpH 3 If so fuld the value of L 615 Two manufacturing processes are under consid eration The petumt cost for process I is C while for process IT it is 3C Products from both processes have exponential timetofailure densities with mean rates of 2S failures per nour and 35 faiures per our from I and II respectively If a product fails before 15 hours it must be replaeed at a cost of Z dollars Vlhich process would you recommend 68 Exercises 141 616 A transistor has an exponential timetofailme distribudon with a mean time to failure of 20000 hours The transistor has already lasted 20000 nours in a particular application Vihat is the probability that the transistor fails by 30000 hours 617Sllowthatrt 5 618 Prove the gamma function properties given by equations 618 and 619 619Aferry boa wJl take its customers across a rivet when 10 cars are aboord Experience shows that cars arrive at the ferry boat independently and at a mean rate of seven per hour Fmd the probability that the time betVeen consecutive trips will be at least 1 hour 6Ut A box of candy contains 24 bars The time beNeen demands for these candy bars is exponen tially distibuted with a mean of 10 minutes lflhat is the probability that a box of candy bars opened at 800 AM will ce empty by noon 621 Use the momentgenerating function of the gamma distribution as given by equation 625 to find the mean and variance 622 The life of an electronic system is Y Xl Xz X3 the surn of the subsystem component lives The subsystems are independent each having expo nential failure densities with a mean time between failures of 4 hours Wnat is the probability hat the system wffi opet1te for at least 24 homs 623 The replhment time for a certaic product is known to be gamma distributed ith a mean of40 and a variance of 400 FInd the probability that an order is received wirhin the first 20 days after it is ordered Within the first 60 days 624 Suppose a gamma distributed random variable is defined over the interval u x with density function fx r xuyi eJt x 2uO r O 0 otherwise Find the mean of this threeparameter gamma disoibution 625 Tne beta probability distribution is defined by f rr I I X rr X Ix OxIOrO 0 otherwise al Graph the distributiou fOLt 1 r L b Grapll the distribution for I I r L e Graph the distribution for 1 r L d Gmpll the distribution for 1 r L e Gmpll the distribution for r 142 Chapter 6 SOUle Important Continuous Distributions 626 Show that when i r 1 the beta distribution reduces 0 the unifoffi1 distribucion 627Show that when A2 r 1 Oft 1 r2he beta distribution reduces to a triargular probability distribution Graph the density fu1ction 628 Show that ift r 2 the beta distribution reduces to a parabolic probability distribution Graph the density functioD 629 Find the mca1 and variance of the beta distnbution 6 36 Flnd the mean and variance of the Weibull distribution 6 31 The dameter of steel shafts is Wcihull distrib uted with parameters r 10 inches j3 2 and S 05 Find the probability that a randomly selected shaft Nill not exceed 15 inches in diameter 632 The time to failure of a cetair transistor is known to be Weibull distributed with parameters r 0 f3 and j 400 Find the fraction expected to survive 600 hours 633 The time to leakage failure in a certain type of drycell battery is expected to have a Wcibull distribu tion wit panuneters r 0 13 t and 0400 What is the probability that a battery ill survive beyoJid 800 hours of use 634 Graph the Weibull distribution with r 08 1 and f3 1 2 3 and 4 635 The time to failure density for a small e01lputer system has a Wetoci1 density vith r o fj i and 0 200 a Vlhat fraction of these units will survive to 1000 hours b What is the mean time to failure 636 A manufacturer of a commercial television mou itor guarantees the picure tube for 1 year 8760 hours The monitors are used in airport terminals for flight schedules and they a1 in continuous use with power on The mean life of11e tuhes is 20000 hours and they follow an exponential timetofailure density It costs the manufacturer 300 to make sell and deliver a ClOnitof that will be sold for 400 It costs 150 to replace a failed tube including materials and labor The manufacnuer has no replacement obliga tion beyond the first replacement V1hat is the malU faeturers expected profit 6 37 The lead time for ordcts of diodes from a certain manufacturer is known to have a gamma distribution with a mean of 20 days and a standard deviation of 10 days Determine the probability of receiving an order within 15 days of lhe placement date 638 Use random numbers generated frOUl your fav6ritc conputer 1anguage or from scaling the ran dom integers in Table XV of the Appendix by multi plying by 1O and do the following a Produce 10 realizations of a lariable that is uni forn 0 10 20 b Produce five realizations of an exponential ran dom variable with a parameter of 2 x laso c Produce five realizations of a gamma variable withr2andi4 d Produce 10 realizations of a Weibull variable lith rOf3 lfl0100 639 Use the random number generation schemes suggested in Exercise 638 and do the following a Produce 10 realizations of Y 2X where X fo1 lows an exponcutial distribution witha mean of 10 b Produce 10 realizations of Y fX IZ where X is gamma with r 2 i 4 and is uniform on 0 1 Chapter 7 The Normal Distributio 71 INTRODUCTION In this chapter we consider the normal distribution This distribution is very important in both the theory and application of statistics We also discuss the lognormal and bivariate normal distributions The uormal distribution was first studied in the eighteenth century when the patterns in errors of measurement were observed to follow a symmetrical bellshaped distribution It was first presented in mathematical form in 1733 by DeMoivre who derived it as a lim iting fonn of the binomial distribution The disuibutiou was also known to Laplace uO later than 1775 Through historical error it bas been attributed to Gauss whose first published reference to it appeared in 1809 and the tenn Gaussian distribution is frequently employed Various attempts were made during the eighteenth and nineteenth centuries to establish this distribution as the underlying probability law for all continuous random variables thus the name normal came to be applied 72 THE NORlVlAL DISTRIBUTION The normal distribution is in many respects the cornerstone of statistics A random variable X is said to have a normal disttibution with mean J1 00 J1 XI and variance f1 0 if it has the density function fx 7 e IXl Jl2rc XtX 71 The distribution is illustrated grapbically in Fig 71 The normal distribution is used so extensively that the shorthand notation X NJl 0 is often employed to indice that the random variable X is normally distributed with meanl and variance cr 721 Properties of the Normal Distribution The normal distribution has several important properties 1 I fxleb I required of all density functions 2 fx20 for all xj 3 limfx 0 and limfx O 4 fJl x fJl x The density is symmetric about J1 5 The maximum value off occurs at x l 6 The points of inflection off are at x l cr 72 143 144 Chapter 7 The Normal Distribution Xii L J I x Figure 71 The normal disrribution Property 1 may be demoIlStrated as foJows Let y x Jlf in equatio 71 and deoote the integral I That is 1 21 e ly dy 2 Our proof that i x dx 1 will consist of showing thatI 1 and then inferring that 1 1 sincemust be everywhere positive Defining a second normally distributed variable Z we have 1 J e 1f2 dy I e lI dz 12 2 1 Joo 1 1j2y e dydz 21 On changing to polar coordinates with the transformation of variables y r sin e and z r cos 91 the integral becomes completing the proof 722 Mean and Variance of the Normal Distribution The mean of the normal distribution may be detennined easily Since EX 1 e lJ2xI dx J2fC and if we let z x Jllf we obtain EX 1 Jlfzezidz 2 JlJ 2 e12dz fI 1 zedz J2fC 2n Since the integrand of the first integral is that ofa normal density withJl 0 and I 1 the value of the first integral is one The second integral has value zero that is 72 The Normal Distribution 145 and thus EX 11 00 1 In retrospect this result makes sense via a symmetry argument To find the variance we must evaluate and letting 1 x 1lICF we obtain so that vX 0 73 74 In summary the mean and variance of the normal density given in equation 71 areJ and 0 respectively The momentgenerating jimcrion for the normal distribution can be sholI1 to be r po Mit exp t1 tJ 75 For the development of equation 75 see Exercise 710 723 The Normal Cumulative Distribution Function The distribution function F is J 1 uII F xPXx e duo T 2tr 76 It is impossible to evaluate this integral without resorting to numerical methods and even then the evaluation would have to be accomplished for eacb pair p 0 However a sim ple transformation of variables z x 110 allows the evaluation to be independent of 1 and 1 That is 77 724 The Standard Nonnal Distributiou The probability density function in equation 77 above I lIz OQ z 00 146 Qmptel 7 The Normal Distribution is that of a normal distribution with mean 0 and variance I that is Z NO 1 and we say that Z has a standard nonnal distribution A graph of the probability density function is shown in Fig 72 The corresponding distribution function is 4 where I Iz f2d 121 78 and this function has been well tabulated A table of the integral in equation HI has been provided in Table II of the Appendix In fact many software packages such as Excel and Minitab provide functions to evaluate lz For instance the Excel call NOR1SDISTl does just this task As an example we find that NOR1SDISTl 96 09750 The Excel function NOR1SlNV returns the inverse CDE For example NOR1SNVO975 1960 The functions NORMDISTx J1 J TRLE and NORMIlN give the CDF and inverse CDFof the Np i distribution 725 ProblemSolving Procedure The procedure for solving practical problems involving the evaluation of cumulative normal probabilities is actually very simple For example suppose that X NlOO 4 and we wish to find theprobability that X is less than or equal to 104 that is PX 104Fl04 Since the standard normal random variable is XIl Z J we can standardize the point of interest x 104 to obtain x u 104100 z 2 J 2 Now the piobability that the standard normal random variable Z is less than or equal to 2 is equal to the probability that the original nonnal random variable X is less than or equal to 104 Expressed mathematically or Fl04 r2 z u1 Fagure 72 The standard nomtal distributkm 72 Tne Normal Distribution 147 Appendix Table II contains cumulative standard nonna probabilities for various values of z From this table we can read 42 09772 Note that in the relationship z x 11cr the variable z measures the departure of x from the mean 11 in standard deviation cr units For instance in the case just considered F104 42 which indicates that 104 is Mo standard deviations 1 2 above the mean In general x J1 OZ In solving probleIPs we sometimes need to use the symmetry prop erty of P in addition to the tables It is helpful to make a sketch if there is any confusion in determining exactly which probabilities are required since the area under the curve and over the interval of interest is the probability that the random variable will lie on the interval The breaking strength in newtons of a synthetic fabric is denoted X and i is distributed as N800 44 The pwxhaser of the fabric requires the fabic to have a strength of at least 772 ntA fabric sam ple is randomly selected and tested To find pX 772 we first calculate PX 772lplXJ1 772801 J I Pz 4233001 Hence the desired probability PX 772 equals 099 Figure 73 shows the calculated probability reaive to both X and Z We have chosen to work with the randOm variable Z because its distribution function is tabulated iti Tle time required to repair an automatic loading machine in a complex foodpackaging operation of a production process is X minutes Studies have shon that tle approximationX N120 16 is quite goOd A sketch is shown Fig 74 If the process is dOu for more than 125 minates ail equipment y 772 I BOO x Figure 73 PX 772 where X N300 144 4 I 120 125 x Figure4 peXl25 whereXNl20 16 cr1 z 148 Chapter 7 The Normal Distribution must be cleaned with the loss of all product in process The total cost of product loss and c1eaning associated with the long downtime is 10000 In order to deterJIline the probability of this occurring we proceed as follows PX 125 1 z 125121 pZ 125 4 1 1125 108944 01056 Thus given a breakdovlI of the packaging machine the expected cost is EC 0105610000 CRI 08944CRj where C is the total costand CRI is the repair cost Simplified Ee CR 1056 SuP pose the management can reduce the mean of the service time distribution to 115 minutes by adding more maintelancc personneL The new Cost for repair will be CRl CRI however px 125 1 Z 125115 Z 25 4 125 109938 00062 so that the new expected cost would be Cftz 62 and ODe would logically make the decision to add to the maintenance crew if or CR C 994 It is asstlOCcd that the frequency of breakdons remains unchanged The pitch diameter of the thread on afitting is nonnally distributed with a mean of 04008 cmanda stan dJtrd deviation of 00004 CllL The designspecitiClltiOns are O4000OOOlO em This is illustruedillFig 75 Notice that the proccs is openrting with the mC3J1 not equal to the nominal specifications We desirc to detmrrine what fraction of product is itbin tolerance Using the approach employed previously 03990 Lower spec limit 10399 SX 0401 p0399004008 SZ 04010 04008 00004 00004 04010 Upper spec limit p45SZs 05 105145 06915 00000 06915 Figure 75 Distribution of thread pitch diameters 72 The Normal Distribution 149 As process engineers study tbe result of such calculations they decide to replace a worn cutting tool and adjust the machine producing the fittings so that the new mean falls dirwJy at the nominal value of 04000 Then P03990 X S 04lO Pi 03990 OA Z5 04010 041 00004 00004 P25 Z S25 I25t 25 0993800062 09876 We see that with the adjust1ntnts 9876 of the fittings will be ithin tolerance The distribution of adjusted maclrine pitch diameters is shown in Fig 76 The preious example illustratesa concept important in quality engineering Operating a process at the nominal level is generally superior to operating the process at some other leveL if there are twosided specification limits Another type of problem involving the use of tables of the nonna distribution sometimes arises Sup pose for example that X N50 4 Furthermore suppose we want to determine a value of X say x such that PX x 0025 Then J X50 PXxZ2 0025 or so that reading the normal table backward we obtain 50 L96I10975 2 x50 2L96 5392 There are several symmetric mrervals that arise frequently Their probabilities are PjJ LOOofXu 1000 06826 PjJ L6450X Ii 16450 090 PjJ 1960 X JI 1960 095 PjJ 2570f Xf JI 257 d 099 PjJ 3IoX Ii 300d 09978 79 Figure 7 Distribution of adjusted lIUlchine pHd x diameters 150 Chapter 7 The Normal Distribution 73 THE REPRODUCTIVE PROPERTY 01 THE NORMAL DISTRIBUTION Suppose we have n independent normal random yariables Xl X2 X where Xi Nip 07 for i 1 2 n It was shown eadier that if fX x X then Ef 1y L 711 il and n vfaiLor 11 Using momentgenerating functions we see that 712 Therefore 713 which is the momentgenerating function of a normally distributed random yariable with mean Jl Jlz 1 and variance a tT 0 Therefore by the uniqueness prop erty of the momentgenerating function we see that Y is normal with mean jJy and yari auee c 1riz An assembly consists of three linkage components as shoVn in Fig 77 The properties X s and X are given below with means in centimeters and variance in square centimeters XI N12 002 X N24 003 X NlS 04 Links are dUbY different machine andopemtors so we have reason to assume thatXX2 and XJ axe ent Suppose we want to detere P53 S Y S 542 Since Y XI X2 XJ Y is distributednomuilly withmemuy 12 24 18 54lmd variance rr a cr ajO02 003 OJ14 009 Thus lI 0667 I 0667 0748 0252 0496 73 The Reprodu4ive Property of the Normal DkLribution 151 0 0 fx v3i Figure 77 A linkage assembly These results can be generalized to linear combinations of indepencent norma variables Linear combinations of the form 714 were presented earlier and we found that JLy ao Again if X X X are indepndent and rtilIIIj A shaft is to be assembled into a bearing as shown in Fig 78 The clearance is Y X X Suppose and Then and so hat r x N1500 00016 x NI480 00009 Jly oJ aJ1 11500 11480 002 a0 100016 100009 00025 Vlhen the parts arc assenbled there Will be interference if Y 0 so i I I Pinterference Py 0 f Z OO2 l 000 104 03445 This indicates that 3446 of all aisemblies attempted would meet with failure If the designer feels that the nominal cleace fly 002 is as arge as it can be roadc for the assembly then the only way to reduce the 3446 figure is to reduce the variance of the distributions In many cases this can be accOIrl LPlisbcd by the overhaul of production equipment better training of production operators and so on x Figure 78 An assembly 152 Chapter 7 The Nonna1 Distribution 74 THE CENTRAL LIMIT THEOREM If a random variable Y is the sum of n independent random variables that satisfy certain general conditions then for sufficiently large n Y is approimately normally distributed We state this as a theoremthe most important theorem in all of probability and statistics Theorem 71 Central Limit Theorem If Xl X X is a sequence of n independent random variables withEX J1 and VeX both finite and Y Xl X X then under some general conditions n Y 1 k Z I 715 I 12 il I l il bas an approximate NO distribution as n approaches infinity If F is the distribution function of Zit then 716 The Hgeneral conditions mentioned in the theorem are informally summarized as fol lows The terms Xi taken individually contribute anegIigible amount to the variance of the SUlli and it is not likely that a single term makes a large contribution to the sum The proof of this theorem as well as a rigorous discussion of the necessary assuinp ions is beyond the scope of this presentation There are however sevenU observations that should be made The fact that Y is approimately normally distributed when the Xi terms may have essentially any distribution is the basic underlying reason for the importance of the nonnal distribution In numerous applications the random variable being considered may be represented as the sum of n independent random variables t some of whicb may be measurement error some due to pbysical considerations and so on and thus the normal distribution provides a good approximation A special case of the central limit theorem arises when each of the components has the same distribvtion Theorem 72 If Xl Xz Xn is a sequence of n independent identically distributed random variables withEX J1 and VeX 1 and YX X2 X then Z Y fI 1 has an approimate NOl distribution in the same sense as equation 7 16 717 Under the restriction that Ml exists for realI a straightforward proof may be pre sented for this form of the central limit theorem Many matheIlJltical statistics texts pres ent such a proof The question immediately encountered in practice is the following How large must 11 be to get reasonable results using the normal distribution to approximate the distribution of Y This is not an easy question to answer since the answer depends on the characteristics of the distribution of the Xl terms as leU as the meaning of lIreasonable resultsn From a 74 The Ceutral Limit Theorem 153 practical standpoint some very crude rules of thumb can be given where the distribution of the X terms falls mto one of three arbitrarily selected groups as follaws 1 Well bebavedThe distribution of Xi does not radically depart from the nonnal dis tribution There is a bellshaped density that is nearly symmetric For this case practitioners in quality control and other areas of application have found that n should be at least 4 That is n 4 2 Reasonably bebavedThe distribution of X has no prominent mode and it appears much as a uniform density In this case n 12 is a commonly used rule 3 TIl behavedThe distribution has most of its measure in the tails as in Fig 79 In this case it is most difficult to say however in many practical applications n 100 should be satisfactory Wpe1 Small parts are packaged 250 to the crate Part weights are independent random variables Iith a mean of 05 pound and a standard dCiatiou of OlQ pound Twentj crates are loaded to a pallet Suppose we wish to fiDd the probability that the parts Q a pallet 1ill exceed 2510 pounds b weight Neglect both pallet and crate weight Let YX j X2 XSIX represent the total weight of the parts so that and Then Jy 500005 2500 a 5000001 50 Oy 50 7071 PY25lOPZ 25102500 7071 11141 008 Note that we did not know the distribution of the individual part weights tx al Figure 79 illbehaved distributions b 154 Chapter 7 1M Normal Distribution Table 7 1 Activity Mean Times and Variances in Weeks and Weeks Activity 2 3 4 5 6 7 S Mean Variance Mean Varianee 27 10 9 31 12 32 L3 10 42 08 46 LO 11 36 16 21 12 12 05 02 36 08 13 21 06 52 21 14 l5 07 71 19 15 12 OA 15 05 16 28 07 hlIzJ In a construction project a network of major activities has been eonstucted to serve as the basis for plalUing and scheduling On a critical path there are 16 activities The means and variances are given in Table 71 The activity times may be considered independent and the project time is the sum of the activ ity times on the critical path that is YX1 Xz XIS where fiO the project time and XI is the time for the ith activity Although the distributions of the XI arc unknown the distributions are fairly well behaved The contractor would like to know a the expected completion time and b a project time corresponding to a probability of 090 of having the project completed Calculating Jly and we obtain J1 49 weeks er 16 weeks2 The expected completion time for the project is thus 49 weeks In detennining the tfrne Yo such that the probability is 09 of having the project completed by thaI time Fig 710 may be helpful We may calculate PlY y 090 or so that and y49 12824 54128 weeks t Oy 4 weeks Yo Y Figure 710 Distribution of project times L 75 The Norma1 Approximation to the Binomial Distribution 155 75 THE NORMAL APPROXIMATION TO THE BINOMIAL DISTRIBUTION Iu Chapter 5 the binomial approximation to the hypergemnetric distnDution was presented as was the Poisson approximation to the binomial distribution In this section we consider the normal approximation to the binomial distribution Since the binomial is a discrete probability distribution this may seem to go against intuition however a limiting process is involved keeping p of the binomial distribution fixed and letting n 7 The approxi mation is known as the DeMoivreLaplace approximation Vle recall the binomial distribution as px I n pxq x xOI2 n x n x 0 otherwise Stirlings approximation to n is The error 718 as n 7 IXI Using Stirlings formula to approximate the terms involving n in the binomial model we eventually find that Ilirgen 720 sO that 721 This result makes sense in ligbt of the central1intit theorem and the fact that X is the sum of iudependent Bemotillr1rial ES1tbt EX np and VX npq Thus the quantity X npljnpq approximately llJlj aNOI distnDutiolL IT p is close to t and n 10 the approximation is fairly good however for other values of p the value of n must be larger In general experience indicates that the approximation is fairly good as long as np 5 for 1 1 P S 2 or when nq 5 whenp 2 r Ir 05 iEl j IC J IJ20 V In sampm a production process that produces items of which 20oaare defte a random sample of itms is selected each hour of each production shift The r of defectives in a sam ple is denotiX To find say PX 15 we may use the nonnal approximation as follows IV J PXS1S j Z 1512002 J l IOOO208 PZSl25 125 01056 Since the bbomial distribution is discrete llld the normal distribution is continuous it is common practice to use a halfinJerva1 correction or cortinuity correction In fact this irt1fecessity b1 ca1cu latiug PX x The usual procedure is to go a halfunit on either side of th4Qs1epending on the interval of interest Several cases are shown in Table 72 EY 156 Chapter 7 The Normal Distributiol Table 72 Continuity Corrections QuaIltity Desired from Binomial Distribution pexx PX x PX xl PX x I Pxx paXsb rlHffitj Vlith Continuity CorrectiOl In Terms of the Distribution FUDctloa l Using the data from Example 79 where we had n 100 and p 02 We evaluate PX 15 PX IS peX 18 PX 22 aIld P18 X 21 0 PjP145 X 5155 115520114520 4 4 1Ll25 11375 0046 Vz PX S 15 I2520j 030 4 vPX IS PX 17iI752 0266 4 1Xi22II2154200354 JI PlSX21P1X20 I 205 20 ldI8520 4 4 05500354 0196 c 76 11e Lognormal Distribution 157 As discussed in Chapter 5 the random variable p XI where X has a binomial distribution with parameters p and n is often of mteTst Interestin this quantity stems primarily from sampling applications where a randQID sample of n observations is made with each obser vation classified success or failure and where X Is the number of successes in the sample The quantity p is simply the sample fraction of successes Recall that we showed that Ej p and vp In addition to the DeMoivreLaplace approximation nore that the quantity z pp jpqn 723 has an approximate NO I distribution This result has proved useful in many applications including those in the areas of quality control work measuement reliability cmgineering and economics The results are ouch more useful than those from the law of large numbers Instead of timing the activity of a maintenance mechanic over the period of a week to determine the fraction of his time spent in an activity classification called secondary but necessary a technician elects to use a work sampling st1dy randomly picking 400 time point over the week taking a flash observation at each and classifying the activity of the maintenance mecbrutic The value X will rep resent the number of times tbe mechanic was involved in a secondary but necessary activity ardp Xl400 If tbe true fraction of time that he is involved iT this activity is 02 we determine he prob ahility tharp the estimatcd fraction falls beteen 015 and 025 That is pOlS p s 03 Ir 025O2 JII oo21 JO1i400 016400 125 125 09876 76 THE LOGNORMAL DISTRIBUTION The lognormal distribution is the distribution of a random variable whose logarithm follows the normal distribution Some practitioners hold that the lognormal distribution is as fun damental as the normal distribution It arises from the combination of random terms by a multiplicative process The lognormal distribution has been applied in a wide variety of fields including the physical sciences life sciences social ciences and engineering In engineering applica tions the lognormal distribution has been used to describe time to fail in reliability engineering and time to repair in maintainability engineering 158 Chapter 7 The ormal Distribution 761 Density Function We consider a random variable X with range space Rx x 0 x oo where Y 1n X is normally distributed with mean f1yand variance 0 that is EY l1y and Vy 0 The density function of X is x 1 12InJlyCTy t e xO xOyJZ1 0 oilieIWise 724 The lognormal distribution is shown in Fig 7 II Notice that in general the distnbution is skewed with a long tail to the right The Excel funetions LOGNORMDIST and LOGINV provide the CDF and inverse CDF respectively of the lognoIlDlll distribution 762 Mean and Variance of the Lognormal Distribution The mean and variance of the lognormal distribution are E X I1x ePrlI2r and 725 726 In some applications of the lognormal distribution it is important to know the values of the median and the mode The median which is the value x such that PX Xl 05 is xeUl 727 The mode is the value of x for whichx is maximum and for the 10gnoIlDlll distribution the mode is 728 Figure 711 shows the relative location of the mean median and mode for the lognormal distribution Since the distribution has a right skew generally we will find thai mode median mean x Median Flgnre 711 The lognormal disuibution 76 The Lognormal Distribution 159 763 Properties of the Lognormal Distribution bile the normal distribution has additive reproductive properties the lognormal distribu tien has multiplicative reproductive properties Some of the more important properties are as follows 1 If X has a lognormal distribution wiili parametersIr and a and if a b and d are constants such thatb 1 then W bXC has a lognormal dis trihution wiili parameters d ally and arJy 2 If X and X are independent lognormal variables wiili parameters fly rJ and jJY2 a respectively then W Xl X1 has a lognormal distribution with parame ters jJy JLr a1 OJ 3 If Xl t Xz XI is a sequence of n independent lognormal variates with parameters iYI qj 1 2 n respectively and a I is a sequence of constants while b e is a single constant ilien ilie product WbTIX jl has a lognormal distribution with parameters and 4 If Xl Xz Xt are independent lognormal variates each with the same parameters fly ilien ilie geometric mean IT Xi jl has a lognormal distribution wiili parametersi and aln iiiiczr The random variable Y In X has a NO O 4 distribution sO X has a lognormal distribution with a mean and variance of EX e IC ll14 ell 162754 and VeX e1104e4 1 e4e4 1 5359Se respectively The mode and median are mode i 40343 and median eo 22026 In order to determiDe a specilic probability say PX S 1000 we use the transormPL1X In 1000 pY In 1000 PYSIn 1000 PI ZS In 1000lOi 2 1155 O0611 160 Chapter 7 The Normal Distribution Jt1 Suppose YInXN41 Y InX N3 05 Y InX N2 04 Y InX Nl 001 and futherrnore suppose XI Xz X and X4 are independent randon variables The random vaiable W defined as follows represents a critical perfornance 3riable 01 a telemetry system W eL5rxx1x7X1 By reproductive property 3 W wu have a lognormal distribucon with parameters 151 254023072311 166 and 25 1 02 05 07 Q4 31 001 6562 respectiely That is to say In W N166 6562 If the specifications on Ware say 20000600QOO we could determine the probability mat W would fall vithiIJ specifications as follows p 20000 W 60010 Plln20000 In w 1n60QIO 1 11n 60010 166Il In 20000 166 6526 6526 1129012621 0098500044 00941 77 THE BIVARIATE NORMAL DISTRIBLTION Up to this poiIt all of the continuous random variables have been of one dimension A erj important twodimensional probability law that is a generalization of the onedimensional normal probability law is called the bivariate normal distribution If Xl X2 is a bivariate nonn random vector then the joint density function of Xl XJ is 729 fur Xl 00 and Xz e The joint probability pea s Xl b lIz X b is defined as Se 1 fx x dx1tiz 730 and is represented by the volume under the surface and over the region xl xz at S Xl bj lIz sx s b as shown in Fig 712 Owen 1962 has provided a table of probabilities The bivariate normal density has five parameters These are P 0 0jt 02 and p the cor fXl Xvt x X f Figure 712 The bivariate normal density relation coefficient between XI and X2 such thar oc fJ OCt oc fl2 OX 01 0 0 0 andl p 1 The marginal densities and are given respectively as Y r Ix x lax 1e2 c 1 1 2 2 r 01 2 for OC XI oc and for oX We note that these marginal densities are normal that is and so that Xl Nu a X NJi1 a EX Jl Ex iLz VeX a veX2 O 731 732 733 734 The correlation coefficient p is the ratio of the covariance to at 01 The covariance is Thus 735 162 Chaper 7 The Normal Distribution The conditional distributions Ixx andxx are also important These condi tional densities are nonnal as shown here foraoX2 ao and 137 for 00 Xl 00 Figure 713 illustrates some of these conditional densities We first consider the distributionXZUl The mean and variance are EX i11 pajax 1 738 x EXzlx x x b Figure j13 Some typical conditional distributions a Some example conditional distributions of X2 for a few values of x b SOrtle eample conditional distributions of Xl for a few values ofX I 77 The Bivariate Normal Distribl1ton 163 and VeXIx aO p 739 Furthermore AiL is normal that is Xrlx Nl pIYJOx JiJ aJ iT 7W The locus of expected values of Xr for xl as shown in equation 738 is called the regression ojX1 on Xl and it is linear Also the variance in the conditional distributions is constant for all Xl and In the case of the distributionx the results are similar That is EX1x Ji pOIYx 11 VXIx al iT 71 72 743 b the bivariate nonnal distribution we observe that if p 0 the joint density may be fac tored into the product of the marginal densities and so X j and Xz are independent Thus for a bivariate normal density zero correlation and independence are equivalent If planes par allel to the X ll2 plane are passed through the surface shown in Fig 712 the contours cut from the bivariate normal surface are ellipses The sttdent may wish o show ms property In an atterrpt to substitute a nondestructive testing procedure for a destructve test an extensive stldy was made of shear strength Xz and wed diameter Xl of spot welds with the foUoNing findings 1 XI Xl bivariate normaL 2 Jll 020 inch Jlz 1100 pounds a 002 inch pounds and pO9 The regression of X2 on Xl is thfis ix pY5x p llOOO9 xl02 1525 002 and the variancc is 1458 107080 vxlx 01 p 5250199975 In studying these results the manager of manufacturing notes that since p 09 that is close to 1 weld diameter is highly correlated with shear strength The specification on shear strength calls for a value greater han 1080 If a weld has a diameter of 018 he asks vhat is the probability that the strength specification ill be met The process engineer notes teat EXdo lS 109705 therefore px 080 Z 108009705 9975 11171 09564 ad he recommends a policy such that if the weld diameter is not less tea 018 the weld will he clas sified as satisfactory 164 Chapter 7 The Normal Distribution Fxni7JS In developing an admissions policy for a large university the office of student testing and evaluadon has noted rlat Xl the combined score on the college board examinadons and X the student grade point average at the end of the freshman year have a bivariate normal distribution A grade point of 40 correspond to A A study indicates that 1 1300 23 06400 O25 pO6 AJJy student with a grade point average less than 15 is auomatically dropped at the end of the fresh man year however an average of20 is considered to be satisfactory Ax applicant takes the college board exams receives a combined score of 900 and is not accepted An irate parent argues that the student will do satisfactory work and specifically will have better than a 20 grade point average at the end of the freshman year Considering only the proba bilistic aspects of t1e problem the director of admissions wants to determine P t 20rc 900 Noting that and the director calculates EX900 2306 j900 1300 08 VX1900 0 16 1 12008 00013 04 which predicts only a very slim chance of the parents claim being valid 78 GENERATION OF NORMAL REALIZATIONS We will consider both direct and approximate methods for generating realizations of a standaunonnal variable Z where Z NCO 1 Recall that X 11 1Z so realizations of X N1 1 are easily obtained as x j1 17 The direc method calls for generating uniform 0 1J random number realizations in pairs u and u Then using the methods of Chapter 4 it tums out that 2 In ui2cos2nu Zz 2m usin2nuj 744 at realizations of independent NO 1 variables The values x I 1 follow directly and the process is repeated until the desired number of realizations of X are obtained An approximate method that makes use of the Central Limit Tneorem is as follows 12 zLu6 745 WIth this procedure we would begin by generating 12 uniform 0 1 random number real izations adding them and subtracting 6 This entire process is repeated until the desired number of realizations is obtained Although the direot method is exact and usually preferable approximate values are sometimes acceptable 79 SUMMARY This chapter has presented the normlll distribution with a number of example applications The normal distribution the related standard nannal and the lognormal distributions are univariate while the bivariate nonnal gives the joint density of tvlo related normal random variables The normal distribution forms the basis on which a great deal of the work in statistical inference rests The wide application of the normal distribution makes it particularly important 710 EXERCISES 7i Let Z be a standard nonal random variable and calculate the follo ing probabilities llSing sketches where appropriate a p0 Z 2 b pIZI e PZ 165 d PZ196 e pIZI 15 I P19 Z 2 g PZ 137 b pclZl 257 X 7e personnel manager of a large company requires job applicants to take a certain test and achieve a score of 500 If the test scores are aorntally distributed Vith a mean of 485 and a standard devia tion of 30 what percentage of the applicants pass the test 78 Experience indicates that thf rhwelonment time for a photographic printing pal N 30 seconds 121 seeonds2 The probability that X is at 1eas probability that X is at most 31 seconds c The prob ability that X differs from its expected value by more than 2 seconds 72 Let X NlD 9 Find PX S PX I2PCz C X S 10 t 19jA ccrtain type of light bulb has an output known roOe norroaily distributed with mea1 of 2500 end foot Ji In eapart below find the value of c that makes candles and standard devin of 75 ed footean tile probability statement true dles Determne a lower speCtlcation limit such that a c 094062 only 5 of the namfactured bulbs will be defetive b pjZl e 095 710 Show that the momentgeneratig function for c P1Zi5C099 the nO distribution is as given by equation 75 d pZ c 005 Use it to generate the mean and variance 0 0225 lfXNp 0 show that YaXb where a 14 1 PZ Zg a detemrlne Zc for a r and b are real comtants is also normally distributed Qll5 005 and 00014 Use the methods outlined in Chapter 3 10 X N80 10 compu the following a n 112 The inside diameter opiston ring is normally Ce PX 100 distributed with amem of 12 em and a standard levi b pX SO arion of 002 Glli e p75X 100 a Wbat freedon of the piston rings will buve dimn d Pl5 X etern exceeding 1205 em e p1X 801 196 b Wbat inside diameter value e has a probability of 7 The life of a particular type of dryell battery is normally distributed with a mean of 600 days and a standard deviation of 60 dayS What fraction of these Q90 being exceeded e t is the probability that tile iLside diameter fur between 1195 and 12051 batteries would be expected to survive beyond 680 7 3 plant manager orders a process shutdown and days Wat fraction would be expected to fail before etting readjustment whenever the pH of the fba1 560 days product falls above 120 or below 680 Te sample 166 Chapter 7 The Sonnal Distribution pH is normally distributed with unknown j1 and a standard deviation of 0 010 Detennine the follow ing probabilities r 7whcthe acceptable range is as in a and the hardness of each of cine randomly selected spec imens is independently detenrined what is the expected llumber of acceptable specimens among the nine specimens a Of readjusting Vhen the process is operating as intendedwithj170 1 L b Of readjusting when the process is slightly off 7 20 Prove that EZ 0 and VZt 1 where z is target with the mean pH 705 as cLfilled in Theorem 72 c ffailing to readjust hen e process is too aIka ltXii 1 2 n be independent and iden line and the mean pH IS J1 725 iruCiily distributed random variables with mean j1 and d Of falling to readjust when the process is too variance cr Consider the sample mean acidic and the mean pH is f1 675 714 The price being asked for II certain security is distributed normally I1tt a mean of 5000 and a stan dan deviation of 500 Buy are willing 0 pay an Show thatEX Ii and VX din amount iliac is also nonnaTIy ntributed Wth a m 722 A sbaft with an outside diameter OD NL20 of 4500 a standard deatlo of 250 vat 1S 00016 is inserted into a sleeve bearing having an the probability that a Ctlon ill take place in de diameter ID that is NL25 00009 Detet Wac specifications for a capacitor are that its lif e the probability of interference must be betveen 1000 d 5000 ours The life i 2 An assembly consists of three components known to be normally dtnbutcd WIth a mean 0 3000 placed side by side The length of each component is hours The revenue alize frmn e3ch capacItor IS nonnally distributed vith a mean of 2 inches and a 900 however a failed urut must be replaced t a standard deviation of 02 inch Speeifications reiuire cost of 300 to the compay Two manWactunng that all rsemblies be between 57 and 63 inches long processes can produce capacltors havng sallsfactory How JPany assemblies vill pass these requirements mean lives The standard deviation for process A is 1000 hom and for process B it is 500 hours How 241 Find the mean and variance of the linear ever process A manufacturing costs arc only half conlbinatior those for B What value of process manufacturing cost Y Xl 2Xz X3 X4 is critical so far as dictating the use of process A or B where Xl N4 3 X2 N4 4 X Nfl 4 and 716 The diameter ofa ball beaing is normally dis X 132 Vbat is the probability tbat 15 S YS 207 tnbuted random vanable With mean j1 and a standard oundoff error has a uniform distribution on deviation of 1 secifiCatio fI the ffiaete are O5J and roundoff erron are independent A 6 X 5 and a ball beanng 1thin these limits lelds um of 50 numbers is calculated where each is a proSt of CdoUars However if X 6 then the profit rounded before adding Vhat is the probability that f is Rj dollars or if X S L1e profit is Rz dollars the total rO1doff error exceeds 57 Fied the value of jl that maximizes the expected profit ZdVIn tte preceding exercise find the Optimum vatue of j1 if R R2 R 718 Usc thc results of Exercise 7 16 with C 5800 Rl 200 and R1 400 What is the value of Ii that nizes the expected profit 7e Rockwell hardess of a particuJar alloy is rmally distributed with a mean of 70 ard a standard tion of 4 a If a specimen is acceptable only jf its hardness is between 62 and 72 wha is me probability that a tandomly chosen specimen has an acceptable hardness b If the acceptable range of hardness was 70 c 70 c for what value of c would 95 of all specimens have acceptable hardness 726 One hundred small bolts are packed in a box Each bolt weights 1 ounce with a standard deviation of 001 OL1ce Find the probability that a box weighs more than 102 ounces S27An automatic machine is used to fill boxes With sp powder SpecificatiIit require that the boxes weigh betNeen land I22 ounces The only data available about inachine peiformance concern the average content of grotp nine boxes It is lnoyD that the average eontent i 119lllUces with a standard deviation ofl OOS ounce fraction of the boxes produced is ddcctive Where shOUld the me be located in order to minimize this fractioa defective Assume the weight is normally distributed 728 A busJreve1s between VIa cities but visits six Intermediate cities on theroate The means and st3L dard deviations of te traveL times are as follows 710 Exercises 167 DDrne dom variable Y In X has a lSO 2S Mean StandMd City Tune PaL hours 12 3 23 4 34 3 45 5 Deviation hours 04 06 03 12 distibution Find the mean vaance mode and median of X 7 3 Suppose independent random variables YJ Y Y are such that Y N4 I 156 7 09 In X N3 I In X3 N2 05 Find the mean and variance of W eZXXSXll Determine a set of specifications L and R such that 167 S 04 0 78 3 04 r PL W R 090 yts the probability that the bus cOC1petcs it jour nbitbin 32 hottts 2Yhowba the densi function for a lognormally distributed ra1dom variable X is given by equation 724 c A production process produces ite of which are defective A random sample of fwo items is 7 bted every y and the nunber of defve items say is cunted tsing theno appropmation to thTOTIllal find the follOWlllg I ttJX16 iY MIX 15 i5i eo f IJf fCILX 5 20 I d PX 14 74ft Consider the bivariate DOnna density x1 rnX1 where Ll is chosen so thatfis a probabiliy distribu tion Are the random variables Xl and X2 indepedent Define wo new random variahles 730 In a worksampling study it is ofte1 desired to 1 Xi pXz 1 1S i find the necessary number of observations Given that 1 pll1f2 01 1 p 01 find the necessary n such that PO05 5 s O15SO95 Yo X r se randoLl numbers generated from your 02 avorite computer package or from scalilg the random Show that the two new random variables are J integers in Table XV of the Appendix by multiplying 0 erdent by 105 to generate six realizations of a N100 4 The life of a tube X and the filament cliameter variable using the follovdng X1 are distributed as a bivwte normal random vari a The direct method ahle with the parameters fil 2000 hours fJz 010 b The approximate methOt inch 0 2500 houn f 001 inch and p 087 732 Consider a linear SOmbiDation Y 3Xt 2X The qualitycontrol manager wishes to deteaniie the where XI is NlO 3 d X1 is unifolly distributed life of eaeh tube by illcasuring the filament dimeter on 011 Generat six reations of the random If a filarrent diameter is 0098 what is the probability vaal1fe Y whetc XI andXr are mdependent that the tube will last 1950 hours Z NO 1 generate five ealizations of Zi 742 A college professor has noticed that grades on y In X aJd Y NjJy v develop a proce each of tIO quizzes have a bivariate norma clistribu mr gereralrwztions of X 2 tion with the parameters fi 75 fJz 83 25 t1jIf y XX wnere XI NJ1p v t and 0 16 and p 08 Ifa student receivC a 2 grade of 80 on the first quiz what is the probability x NJ1 a and where X and Xo are independent th h will d be th d H th i ats e 0 tterOn csecon one OWlS e deVelop a generator for producmg realizations of Y aft ed 8 wer ect bv mQlill1g p v 736 The brightness bulbs is normally dis tributed with a mean of 2500 foorcaodles and a stan Consider the surface y X whercfis the dard deviation of 50 footcandles The bulbs are tested bivariate normal density function and all those brighter than 2600 footcandles are placed in a special highqUality lot What is the prob ability distribution of the remaining bulbs Vbat is their expcced hrightness a Prove that y constant cuts the surface in an ellipse h Prove that y constant with p and 0 a cuts the surface as a circlc 168 Chapter 7 The Normal Distribution 7 44 Let Xl and Xz be independent random variables each following a normal density with a mean of zero and variance fl Find the distribution of r RXllXi The resulting distribution is known as the Rayleigh distribution and is frequently used to model the Cistri bution of radial error in a plane Hint Let Xl Rcos e and X2 Rsin 8 Obtain the joint probability disoibu clon of R and e then integrate out a sing a method similar to that in Exercise 744 obtain the distribution of cX X We say that C follows the Cauciry distribution Try to compute E C W2e X NO 1 Fmd the probability distribution of Y X 2 Y is said to follow the chisquare distribu tion with one degree of freedom It is a1 important distribution in tatistical methodology 7 48 Let we independent random variables XI NO 1 for i 1 2 n Sbow hat the probability distribution of Y t xl follows a cbisquare distli i L X 0 d ddt bution with n degrees of freedom WLere i lVI cr an IS In epen Cll 746 Let the independent random variables XI GLetX NCOI Define anew random variable Y NO 0 for i 1 2 Flld the probabiJity distribution XI Tneu firul the probability distribution of Y This of is often called the haliItrmLll distribution Chapter 8 Introduction to Statistics and Data Description 81 THE FIELD OF STATISTICS Statistics deals with the collection presentation analysis and use of data to solve probems make decisions develop estimates and design and develop both prOduCIS and pocedures An undeSlllnding of basic statistics and statistical methods would be useful to anyone in this information age however since engineers scientists and those working in mmage ment science are routinely engaged with data a knowledge of statistics and basic statistical methods is particularly vilal In this intensely competitive hightech world economy of the first decade of the tventyfirst century the expectations of consumers regarding product quality perrorIllllllce and reliability have increased significantly over expectations of the recent past Furthermore we have come to expect high levels of performance from logisti cal systems at a111eve1s and the operation as well as the refinement of these systems is largely dependent on the collection and use of data While hose who are employed in var ious service industries deal with someVbat different problems at a basic level they tOO are collecting data to be used in solving problems and improving the service so as to become more competitive in attracting market share Statistical methods are used to present describe and understand variability In observ ing a Yariable value or several variable values repeatedly where these values are assigned to units by a process e note that these repeated observations tend to yield different results Vhile this chapter wHl deal with data presentation and description issues the fonowing chaplers will utilize the probability concepts developed in prior chapters to model and develop an understanding of variability and to utilize this understanding in presenting infer ential statistics topics and methods Virtually all realworld processes exhibit variability For example consider situatiolS where we select several castings from a manufacturing process and measure a critical dimension such as a vane opening on each part If the measuring instrument has sufficient resolution the vane openings will be different there will be variability in the dimension Alternatively if we count the number of defects on printed circuit boards we will find vari ability in the counts as some boards will have few defects and others will have many defects This variability extends to all environments There is variability in the thickness of oxide coatings On silicon wafers the hourly yield of a chemical process the number of errors on purchase orders the flow time required to assemble an aircraft engine and the therms of natural gas billed to the residential custonters of a distributing utility in a given month 169 110 Chapter 8 Introduction to Statistics and Data Description Almost all experimental activity reflects similar variability in the data observed and in Chapter 12 we will employ statistical methods not only to analyze experimental data but also to construct effective experimental designs for the study of processes Why does variability occur Generally variability is the result uf changes ill the con ditions under which observations are made In a manufacturing context these changes may be differences in specimens of material differences in the way people do the work differ enceS in process variables such as temperature pressure or holding tice and differences in environmentalJactors such as relative humidity Variability also occurs because of the measurement system For example the measurement obtained from a scale may depend on where the rest item is placed on the pan The process of selecting units for observation may also cause variability For example suppose that a lot uf 1000 integrated circuit chips has exactly 100 defective chips If we inspected all 1000 chips and if our inspection process was perfect no inspection or measurement errort we would find aU 100 defective chips However suppose that we select 50 chips Now some of the chips will likely be defective and we would expect the sample to be about 10 defective but it could be 0 or 2 or 12 defective depending on the specific chips selected The field of statistics consists of methods for describing and modeling variability and for making decisions when variability is present In irerential statistics we usually want to make a decision about some population The term population refers to the colleCtion of measurements on aU elements of a universe about Which we wish to draw conclusions or make decisions In this text we make a distinction between the universe and the popula tions in that the universe is composed of the set of elementary units or simply units while a population is the set of numerical or categorical variable values for one variable associ ated with each of the universe units Obviously there may be several populations associated with a given universe An example is the universe consisting of the residential class cus tomers of an electric power company whose accounts are active during part of the month of August 2003 Example populations might be the set of energy consumption kilowatthour values billed to these customers in the August 2003 bill the set of customer demands kilo watts at the instant the company experiences the August peak demand and the set made up of dwelling catgory such as singlefamily unattached apartments mobile home etc Another example is a universe which consists of all the power supplies for a personal computer manufactured by an electronics company during a givenpctiod Suppose that the manufacturer is interested specifically in the output voltage of each power supply We may think of the output voltage levels in the power supplies as such a population In this case each population value is a numerical measurement such as 510 or 524 The data in this case would be referred to as measuremem data On the other hand the manufacturer may be interested in whether or not each power supply produced an output voltage that conforms to the requirements Vle may then visualize the population as consisting of attribute data in which each power supply is assigned a value of one if the unit is nonconforming and a value of zero if it conforms to requirements Both measurement data and attribute data are called numerical data Furthermore it is convenient to consider measurement data as being either continuous data or discrete data depending on the nature of the process assigning values to unit variables Yet another type of data is called categorical data Examples are gender day of the week when the observation is taken make of automobile etc Finally we have unit identifying data which are alphanumeric and used to identify the universe and sample unitS These data might neither exist nor have statistical interpretation however in some situations they are essential identifiers for universe and sample units Examples would be social security numbers account numbers in a bank YIN numbers for automobiles and serial numbers of cardiac pacemakers In this book we will present teclmiques far dealing with both measurement and attribute data however categorical data are also considered 81 The Field of Statistics 171 In most applications of statistics the available data result from a sample of units seected from a universe of interest and these data reflect measurement or classification of one or more variables associated with the sampled units The sample is thus a subset of the units and the measurement or classification values of these units are subsets of the respec tive universe populations Figure 81 presents an overview of the data acquisition activity It is convenient to think of a process that produces units and assigns values to the variables associated with the units An example would be the manufacturiIg process for the power supplies The power sup plies in this case are the units and the output voltage values perbaps along with other vari able values may be thought of as being assigned by the process Oftentimes a probabilistic model or a model with SOme probabilistic components is assumed to represent the value assignment process As indicated earlier the set of units is referred to as the universe a1d this set may have a firrite or an infinite membership of units Furthermore in some cases this set exists only in concept however we can describe the elements of the set without enu merating them as was the case with the sample space if associated vith a random exper iment presented in Chapter I The set of values assigned or that may be assigned to a specific variable becomes the population for that variable Now we illustrate with a few additional examples that reflect several universe struc tures and different aspects of observing processes and population data First continuing with the power supplies consider the production from one specific shift that consists of 300 units each having some voltage output To begin consider a sample of 10 units selected from these 300 units and tested with voltage measurements as previously described for sample units The universe here is finite and the set of voltage values for these universe UIJits the population is thus finite If our interest is simply to describe the sample results we have done that by enumeration of the resnlts and both graphical and quantitative meth ods for further description are given in the following sections of this chapter On the other Process Unit generation Sample units I Q Figure 81 From process to data Observation Df Data Or o 172 Chapter 8 inJOduction to Statistics and Data Dscription hand if we wish 0 employ the sample results to draw statistical inference about the popu lation consisting of 300 voltage values this is called an enumerative studyl and careful attention must be given to the method of sample selection if valid inference is to be made about the population The key to much of what may be accomplished in such a study involves either simple or more complex forms of rardom sampling or at least probability saT1pling VhUe these concepts will be developed further in the next chapter it is noted at this point that the application of simple random sampling from a finite universe results in an equal inclusion probability for each unit of the universe In the case of a finite universe where the sampling is done without replacemem and the sample size usually denoted n is the same as the universe Si7e N then this sample is called a census Suppose Our interest lies not in the voltage talues for units produced in this specific shift but rather in the process or process variable assignment model Not only must great care be given to sampling methods we must also make assumptions about the stability of the process and the structure of the process model during the period of sampling In essence the universe unit variable values may be thought of as a realization of me process and a random sample from such a universe measuring voltage values is equivalent to a rau dom sample on the process or process model voltage variable or population With our example we might assume that unit voltage E is distributed as Nu aZ during sample selection This is often called an analytic sruciy and our objective might be ior example to estimate J1 andor cr Once again random sampling is to be rigorously defined in the fol lowing chapter Even though a universe sometimes exists only in concept defined by a verbal descrip tion of the membership it is generally useful to think carefully about the entire process of observation activity and the conceptual universe Consider an example where the ingredients of concrete specirlens have been specified to achieve high strength with early cure time A batch is formulated five specimens are poured into cylindrical molds and these are cured according to test specifications Following the cure these test cylinders are subjected to lon gitudinalload until rupture at which point the rupture strength is recorded These test units with the resulting data are considered sample data from the strength measurements associ ated with the universe units each with an assigned population value that might bave been but were never actually produced Again as in the prior example inference often relates to the process or process variable assignment model and model assumptions may become cru cial to attaining meaningful inference in this analytic study Finally consider measurements taken on a fluid in order to measure some variable or characteristic Specimens are draw from wellmixed we bope fluid with eacb specimen placed in a specimen container Some difficulty arises in universe identification A conven tion here is to again view the universe as made up of all possible specimens that might have been selected Similarly an alternate view may be taken that me sample values represent a realization of the process value assignment model Getting meaningful results from such an analytic study once again requires close anention to sampling methods and to process or process model assumptions Descriptive statistics is the branch of statistics that deals with organization summa rizatioo and presentation of data Many of the techniques of deseriptive statistics bave been in use for over 200 years yith origins in surveys and census activity Modem computer technology particularly computer graphics has greatly expanded the field of descriptive statistics in recent years The techniques of descriptive statistics can be applied either to entire finite populations or to samplesj and these methods and techniques are illustrated in the following stions of this chapter A wide selection of software is available rdllging in focus sophisticatio and generality from simple spreadsbeet functions Uch as those found in Microsoft Excel to the morecomprehenshe but user friendly Minitab to large comprehensive flexible systems such as SAS Many other options are also available 83 Graphical Presertation of Dam 173 In enumerative and analj1ic studies the objective is to make a conclusion or draw inference about a finite population or about a process or variable assignment modeL This activity is called inferential statistics and most of the techniques and methods employed have been developed within the past 90 years Subsequent chapters will foeus on these topics 82 DATA Data are collected and stored electronically as well as by hUJrulD observation using tradi tional records or files Formats differ depending on the observer or observational process and reflect individual preference and ease of recording In largescale studies where unit identification exists data must often be obtained by stripping the required information from several files and mergillg it into a file format suitable for statistical analysis A table or spreadsbeettype format is often convenient and it is also compatible with most software analysis systems Rows are typically assigned to units observed and columns present numerical or categorical data on one or mare variables Furthermore a column may be assigned for a sequence or order index as well as for other unit identifying data In the case of te power supply voltage measurements no order or sequence was intended so that any permutation of the 10 data elements is the same In ather contexts for example where the index relates to the time of observation the position of the elementin the sequence may be quite important A common practice in both situations described is to employ an index say i 12 n to serve as a unit identifier where n units are observed Also an alpha bedcal character is usually employed to represent a given variable value thus if i equals te value of the ith voltage measurement in the example e 510 e2 524 e3 514 e lC 511 In a table format for these data there would be 10 rows II if a headings row is employed and one orVo columns depending on whether the index is included and pre sented in a column Where several variables andor categories are to be associated with each unit each of these may be assigned a colUIDIl as saoVffi in Table 81 from Montgomery and Runger 2003 which presents measurements of puJl strength wire length and die height made on each of25 sampled units in a semiconductor manufacturing facility In situations where cat egorical classification is involved a columo is provided for each categorical variable and a category code or identifier is recorded for the unit An example is shift of manufact1e with identifiers D N G for day night and graveyard shifts respectively 83 GRAPIDCAL PRESENTATION OF DATA In this section we will present a few of the many graphical and tabular methods for sum marizing and displayicg data In recent years the availability of computer graphics has resulted in a rapid expansion of visual displays for observational data 831 Numerical Data Dnt Plots and Scatter Pints When we are concerned with One of the variables associated with the observed units the data are often called univariate data and dot plots provide a simple attractive display that reflects spread extremes centering and voids or gaps in the data A horizontal line is scaled so that the range of data values is accommodated Each observation is then plotted as a dot directly above this scaled line and where multiple observations have the same value the dots are simply stacked vertically at that scale point bere the number of units is relatively smail say n 30 or where there are relatively few distinct values represented 174 Chp1er 8 Inrroduction to Statistics and Data Description Table 81 Wrre Bond Pull Strength Data Observation Pull Strength WueLength Die Height Number y 995 2 50 2 2445 8 110 3 3175 II 120 4 3500 10 550 5 2502 8 295 6 1686 4 200 7 1438 2 375 8 960 2 52 9 2435 9 100 10 2750 8 300 11 1708 4 412 12 3700 11 400 13 4195 12 500 14 1166 2 360 15 2165 4 205 16 1789 4 400 17 6900 20 600 18 1030 585 19 3493 10 540 20 4559 15 250 21 448 15 290 22 5412 16 510 23 5663 17 590 24 2213 6 100 25 2115 5 400 in the data set dot plots are effective displays Figure 82 shows the univariate dot plots or marginal dot plo for each of the three variables with dall presented in Table 8L These plots were produced using Minitab Where we wish to jointly display results for two variables the bivariate equivalent of the dot plot is called a scatter plDt We construct a simple rectangular coordinate graph assigning the horizontal axis to one of the variables and the vertical is to the other Each observation is then plotted as a point in this plane Figure 83 presents scatter plots for pull strength vs wire length and for pull strength vs die height for the datl in Table 8L In order to accommodate data pairs that are identical and thus that fall at the same point on the plane one convention is to employ alphabet characters as plot symbols so A is the dis played plot point where one data point falls at a specific point on the plane B is the dis played plot point if two fall at a specific point of the plane etc Another approach for this which is useful where values are close but not identicaL is to assign a randomly generated small positive or negative quantity sometimes called jil1er to one or both variables in order to make the plo better displays Wbile scatter plots show the region of the plane wbere the data points fall as well as the data density associated with this region they also suggest possible association between the variables Finally we note that the usefulness of these plots is not limited to small data sets 70 60 50 40 c 30 5 20 10 0 83 Graphical Presentation of Data 175 10 20 30 40 50 60 70 Pull strength 5 10 15 20 Wire length T t t T 100 200 300 400 500 600 Die height Figure 82 Dot plots for pull strength wire length and die height 0 10 Wire length 20 70 60 50 c 40 5 30 20 10 o 100 200 300 400 500 600 Die height Figure 83 Seatter plots for pull strength vs wire length and for pull strength vs die height from Minitabl To extend the dimensionality and graphically display the joint data pattern for three variables a threedimensional scatter plot Dlay be employed as illustrated in Figure 84 for the data in Table 81 Another option for a display not illustrated here is the bubble plot whicb is presented in two dimensions with the third variable reflected in the dot now called bubble diameter that is assigned to be proportional to the magnitude of the third variable As was the case with scatter plots these plots also suggest possible associations between the variables involved 832 Numerical Data The Frequency Distribution and Histogram Consider the data in Table 82 These data are the strengths in pounds per square incb psi of 100 glass nonrerurnable 1liter soft drink bottles These observations were obtained by testing eacb bottle until failure occurred The data were recorded in the order in whicb the bottles were tested and in this fonnat they do not convey very Dlucb infonnation about bursting strength of the bottles Questions sucb as wbat is the average bursting strength 176 Chapter 8 Introduction to Statistics and Data Desciption ttnl1o 4060 O48L12 300 8 1 00200 e 12 16 20 0 0 Wire length FigureS4 Threeilimensiona1 plots for pull strength wire length and die height Tableg2 Bursting Strength in Pounds per Square Inch for 100 Glass ILiter Nonreturnable Soft Drink Bottles 265 197 346 280 265 200 221 265 261 278 205 286 317 242 254 235 176 262 248 25G 263 274 242 260 281 246 248 271 260 265 3rJ7 243 258 321 294 328 263 245 274 270 220 231 276 228 223 296 231 301 337 298 268 267 300 250 260 276 334 280 250 257 260 281 208 299 300 264 230 274 278 210 234 265 187 258 235 269 265 253 254 280 299 214 264 267 283 235 272 287 274 269 215 318 271 293 277 290 283 258 275 251 Or what percentage of the bottles burst below 230 psi are not easy to answer when the data are presented in this form A frequeney distribution is a more useful summary of data than the simple enumera tion given in 1able 82 To construct a frequency distribution we must divide the range of the data into intervals which are usually called class intervals If possible the class inter vals should be of equal width to eohance the visual infonnation in the frequency distribu tion Some judgment must be used in selecting the number of class intervals in order ro give a reasonable display The number of class intervals used depends on the number of obser vations and the amount of scatter or dispersion in the data A frequency distribution that uses either too few or too many class intervals will not be very informative Ve generally find that between 5 and 20 intervals is satisfactory in most cases llIld that the number of class intervals should increase with n Choosing a number of class intervals approximately equal to the square root of the nwnber of observations often works well in practice A frequency distribution for the bursting slength data in Table 82 is abown in Table 83 Since the data set contains 100 observations we suspect that about fWo 10 class intervals will give a satisfactory frequency distribution The largest and smallest data val ues are 346 and 176 respectively so the class intervals must cover at least 346 176 170 psi units on the scale If we want the lower limit for the first interval to begin slightly below tile smallest data value and the upper limit for the last cell to be slightly abeve the largest data value then we might start the frequency distribution at 170 and end it at 350 This is an interval of 180 psi units Xine class intervals each of width 20 psi gives a reasonable frequency distribution ani the frequency distribution in Table 83 is thus based on nine class intervals 83 Graplrical Presentation of Data 177 Table 83 Frequency Distribution for the Bursting Strength Data in Table 82 Class Interval Relative Cumulative psi Tally Frequeney Frequeney Relative Frequency 170 x 190 2 002 002 190 x 210 1111 4 004 006 210 x 230 111 II 7 007 013 230 x 250 111 111 III 13 013 026 250sx 270 111 111 111 111 111 111 II 32 032 058 270 x 290 111 111 111 111 1111 24 024 082 290 x 310 IIIIIII 11 011 093 310x330 1111 4 004 097 330x350 III 3 003 100 100 100 The fourth column in Table 83 contains the relative frequency distribution The rela tive frequencies arc found by dividing the observed frequency in each class interval by the total number of observations The last column in Table 83 expresses the relativ frequen cies on a cumulative basis Frequency distributions are often easier to interpret than tables of data For example from Table 83 it is very easy to see that most of the bottles burst between 230 and 290 psi and that 13 of the bottles burst below 230 psi It is also helpful to present the frequency distribution in graphical form as shown in Fig 85 Such a display is called a histogram To draw a histogram use the horizontal axis to represent the measurement scale and draw the boundaries of the class intervals The ver tical axis represents the frequency or relative frequency scale If the class intervals are of equal width then the heights of the rectangles drawn on the histogram are proportional to the frequencies If the class intervals are of unequal width then it is customary to draw rec tangles whose areas are proportional to the frequencies In this case the result is called a i c o 40 30 20 u 10 r r I n 170190210230250270290310330350 Bursting strength psi Figure 85 Histogram of bursting strength for 100 glass Iliter nonreturnable soft drink bottles 178 Chapter 8 Introductioc to Statistics and Data Description density histogram For a histogram displaying relative frequency nn the vertical axis the rectangle height are calculated as I h gh rectang e el t class relative frequency class width When we find there are multiple empty class intervals after grouping data into equalwidth intervals one option is to merge the empty intervals with contiguous interVals thus creat ing some wider intervals The density histogram resulting from this may produce a more attractive display However histograms are easier to interpret when the class intervals are of equal width The histogram provides a visual impression of the shape of the distribution of the measurements as wen as information about centering and the scatter or dispersion of tho data In passing from the original dam to either a frequency distribution or a histogram a cer tain amount of information has been lost in that we no longer have the individual observa tions On the other hand this information loss is small compared to the ease of interpretation gamed in using the frequency distnoution and histogram In cases where the data assume only a few distinct values a dot plot is perhaps a better graphical display Vlhere observed data are of a discrete nature sucb as is found in counting processes then two choices are available for constrUcting a histogram One option is to center the rec tangles on the integers reflected in the count data and the other is to collapse the rectangle into a vertical line flaced directly over these integers In both cases the height of the rec tangle or the length of the line is either the frequency or relative frequency of the occurrence of the value in question In summary the histogram is a very useful gr1ljOhic display A histogram can give the decision maker a good understanding of the data and is very useful in disflaying the shape location and variability of the data However the histOgram does not allow individual data points to be identified because all observations falling in a cell are indistinguishable 833 The StemandLeafPlot Suppose that the data are represented by Xj X xlf and that each number Xi consists of at least two digits To construct a stemandleaf plot we divide each number xl into two parts a steml consisting of one or more of the 1eading digits and a lear consisting of the remaining digits For example if the data consist of the percentage of defective information between 0 and 100 on lots of semiconductor wafers then we could divide the value 76 into the stem 7 and the leaf 6 In general we should choose relatively few stemsin comparison with the number of observations It is usually best to choose between 5 and 20 stems Once a set of stems has been chosen they are listed along the lefrhand margin of the display and beside each stem all leaves corresponding to the observed data values are listed in the order in which they are encountered in the data set To illustrate the construction of a stemandleaf plot consider the bottleburstingstrength data in Table 82 To construct a stemardleafplot we select as stem values the numbers 17187 19 34 The resulting stemanaleaf plot is presented in Fig 86 Inspection of this display immediately reveals that most of tie bursting strengths lie betNeen 220 and 330 psi and that the central value is sorocNhere betllccn 260 and 270 psi Furtheunore the butSting strengths are distributed approxi mately symmetrically about the central value Therefore the stemandleaf plot like the histogram ci1OV1S US to determine quickly some important fuatures of the data that were not immediately obvi os in the original display Table 82 Note that here the original cumbers arc not lost as occurs in a 33 Graphical Presentation of Data 179 Stem 17 18 19 20 21 22 23 2A 25 26 27 28 29 30 31 32 33 34 Leaf 6 7 7 058 045 1083 511455 2826835 40800783481 5551230O5387OO595479 84 140664824 175 061 0100373 4689930 7108 78 18 74 6 Figure 86 Stemandleaf plot for the borJeburstingstrength data in Table 82 3 3 4 6 7 11 21 14 10 7 4 2 2 2 1 100 histogram Sometimes in order to assist in finding percentiles we order the leaves by magnitude pro ducing a ordered stemandleaf plot as in Fig 87 For instance since n 100 is an even number the median or midd1e observation see Section 841 is the average of the tviO observations with ranls 50 and 51 or x 265 26512 265 The lenlhpercentileis the observation withnmkOIl0005 105 halfway between the IOthaJd 11th observatiOllS or 220 221122205 Theftrst quartile is the observation withnmkO25loo 05 255 halfway between the 25th and 26th observations or 248 24812 248 and the third qu2rtie is the observation with nmk 075100 05 755 halfway between the 75th and 76th OscvatioDs or 280 2802280 The first and rhird quartiles are QCCjSionilly denoted by the syxn bos Ql and Q3 rectively and the 1nterquartile rf1tSe IQR Q3 Ql maybe used as amcasureof variability For the borJcbotsti1gstrength data the iterquartile range is IQR Q3 Q 280 248 32 The stemand1eaf displays in Figs 86 and 87 are equivalent to a histograx ith IS class iter vals In some situations it may be desirable to provide more classes or stems One way to do this would be to modify the original stems as follows divide stem 5 say into two new stems 5 and 5 Sm 5 has leaves 0 1 2 3 and 4 and stem 5 has leaves 5 6 7 8 and 9 This will double the number of orig inal stems We could increase the number of original stems by five by defining five new stems 5jO vrith leaves 0 and 1 St for twos and threes with leaves 2 and 3 Sf for fours and fives with leaves 4 and 5 5s or sixes and sevens with leaes and 7 and 5 vith leaves 8 and 9 834 The Box Plot A box plot displays the three quartiles the lllinimum and the maximum of the data on a rectangular box aligned either horizontally or vertically The box encloses the interquatile range with the left or lower line at the first quartile Q1 and the right or upper line at the third quartile Q3 A line is drawn through the box althe second quartile which is the 50th 180 Chapter 8 Introduction to Statistics and Data Description Stem Leaf 17 6 1 18 7 1 19 1 20 058 3 21 045 3 22 0138 23 114555 6 24 2235688 7 25 00013447888 11 26 000012334455555577899 21 27 01124444566188 14 28 0000113367 10 29 0346899 7 30 Ql7 4 31 78 2 32 18 2 33 47 2 34 6 1 100 Figure g 1 Ordered stcmandleaf plot for the bott1ebUX5tingstreugth data percentile or the median Q2 i A Une at either end extends to the extreme values These lines sometimes called wbiskers may extend only to the 10th and 90th percentiles or the 5th and 95th percentiles in large data sets Some authors refer to the box plot as the box andwhisker plot Figure 88 presents the box plot forthe bottleburstingstrength data This box plot indicates that the distribution of bursting strengths is fairly symmetric around the central value because the left and right whiskers and the lengths of the left and right boxes around the median are about the sarne The box plot is useful in comparing two or more samples To illustrate consider the data in Table 84 The data taken from Messina 1987 represent viscosity readings on three different mixtures of raw material used on a manufacturing line One of the objectives of the study that Messina discusses is to compare the three mixtures Figure 89 presents the box plots for the viscosity data This display permits easy interpretation of the data Mix ture I has higher viscosity than mixture 2 and mixture 2 has higher viscosity than mixture 3 The distribution of viSCOsity is not symmetric and the maximum viscosity reading from mixture 3 seems unusually large in comparison to the other readings This observation may be an outlier and it possibly warrants further exantination and analysis 176 248 25 280 346 I I I I I 175 200 225 250 275 300 325 350 Figure 8 Box plot for the botticburstingstrengJ data 83 Graphical Presentation of Data 181 Table 84 Viscosity Measurements for Three Mixtures Mixture 1 2202 2383 2667 2538 2549 2350 2590 2498 270 258 m 247 g 0 rn 235 8 5 223 212 200 2667 2570 2519 2368 2202 Mixture 2 Mixture 3 2149 2033 2267 2167 2462 2467 2418 2245 2278 2228 2256 2195 2446 2049 2379 2181 2462 2467 2432 2329 2262 2149 2 Mixture 2237 2188 2108 2033 3 Figure 89 Box plots for the mixtureviscosity data in Table 84 835 The Pareto Chart A Pareto diagram is a bar graph for count data It displays the frequency of each count on the vertical axis and the category of classification on the horizontal axis We always arrange the categories in descending order offrequency of occurrence that is the most frequently occurring is on the left followed by the next most frequently occurring type and so on Figure 810 presents a Pareto diagram for the production of transport aircraft by the Boeing Commercial Airplane Company in the year 2000 Notice that the 737 was the most popular model followed by the 777 the 757 the 767 the 717 the 747 the MD 11 and the 11D90 The line on the Pareto chart connects the cumulative percentages of the k most fre quently produced models k 1 2 3 4 5 In this example the two most frequently pro duced models account for approximately 69 of the total aUplanes manufactured in 2000 One feature of these charts is that the horizontal scale is not necessarily numeric Usually categorical classifications are employed as in the aUplane production example The Pareto chart is named for an Italian economist who theorized that in certain economies the majority of the wealth is held by a minority of the people In count data the Pareto principle frequently occurs hence the name for the chart Pareto charts are very useful in the analysis of defect data in manufacturing systems Fig ure 811 presents a Pareto chart showing the frequency with which various types of defects 182 Chapter 8 Introduction to Statistics ard Dtta Dscription 500 400 sao 1 8 200 10e 0 COLot Percent Cumo 737 281 575 575 m 757 767 717 55 45 44 92 112 92 90 65 687 779 869 935 100 80 60 1 e 8 0 40 a 20 0 747 MD11 25 4 3 51 08 06 986 994 1000 Figure S10 Airplane production in 2000 Source Boeing CornmercialAirplane Company oC1 on metal parts used in a structural component of an automobile doorfrarne Notice bOV the Pareto chart highlights the relatively few types of defects thaI are responsible for most of the observed defects in the part The Pareto chart is an imponanl parr of a qualityimprove ment program because it allows management and engineering to focus attention on the most critical defects in a product or process Once these critical defects are identified corrective actions to reduce or eliminate these defects must be deVeloped and implemented This is eas ier to do however wben we are sure that we are attacking iii legitimate problem it is much easier to reduce or eliminate frequently occng defect than rare ones 81 I iOQ if 75 e 3 ll l Q m cr 0 50 0 31 Q Q j 0 so E L25 g z 21 0 2l l Q 1ll 10 6 a 5 5 4 4 O J Out of Missing Parts net contour hoiJSslcts i greased Pans Outet Parts rot Oth urderrlmmcd sequonce drorred Defect typo Figuro S11 Pareto chart of defects in door structural elements fi l 84 umerical Description of Data 183 836 Tune Plots VIrtually everfone should be familiar with time plots since we view them daily in media presentations Examples are historical temperature profiles for a given city the closing Dow Jones Industrials Index for each trading day each month each quarter etc and the plot of the yearly Consumer Price Index for all urban consumers published by the Bureau of Labor Statistics Many other timeoriented data are routinely gathered to support infer ential statistical activity Consider the electric power demand measured in kilowatts for a given office building and presented as hourly data for each of the 24 hours of the day in which the supplying utility experiences a summer peak demaud from all cllstomers Demand data such as these are gathered using a time of use or load research mete In concept a kilowatt is a continuous variable The meter sampling interval is veri short how ever and the hourly data that are obtained are truly averages over the meter sampling inter vals contained within each hour Usually with time plots time is represented on the horizontal axis and the vertical scale is calibrated to accommodate the range of values represented in the observational results The kilowatt hourly demd data are displayed in Fig 812 Ordinarily when series like those display averages OVer a time interval the variation displayed in the data is a fune tion of the lcngth of the averaging interval with shorter intervals producing more variabil ity For example in the case of the kilowatt data if we used a 15minute interval there would be 96 points to plot and the variation in data would appear much greater 84 NUMERICAL DESCRIPrION OF DATA Just as graphs can improve the display of data numerical descriptions are also of value In this section we present several impOrtant numerical measures for describing the character istics of data 841 Measures of Central Tendency The most common measure of central tendency or location of the data is the ordinary arith metie mean Because we usually think of the data as beillg obtained from a sample of units J 50 c 2 4 6 8 10 12 14 16 18 20 22 Hour Figure 812 Summer peak doy hourly wowtt KW demand data for an office building 184 Chapter 8 Introduction to Statistics and Data Description we will refer to the arithmetic mean as the sample mean If the observationsin a sample of size n are x x2 XI then the sample mean is f X1 XZ xn n For the bottleburstingstrength data in Table 82 the sample mean is 00 X ii 26 406 64 06 lOa JOil 81 From examination of Fig 85 it seems that the sample mean 26406 psi is a typical l1 value of bursting strength since it occurs near the middle of the data wbere the observations are concentrated However this impression can be lIlisleading Snppose that the histogram looked like Fig 813 The mean of these data is still a measure of central tendencYI but it does not necessarily imply that most of the observations are concentrated around it In gen eral if we think ofllie observations as having unit mass the sample mean is just the center of mass of the data This implies that the histogram will just exactly balance if it is sup ported at the sample mean The sample mean x represents the average value of all the observations in the sample Ve can also think of calculating the average value of all the observations in a finite popu larion This average is called the population mean and as we here seen in previous chap ters it is denoted by the Greek letter J1 nen there are a finite number of possible observations say Iv in the population then the population mean is 1 1 N I N where l x Lil Xi is the finite population total for the population 82 In the followmg chapters dealing with statistical inference we will present methods for making inferences about the population mean that are based on the sample mean For exam ple we will nse the sample mean as a point estimate of fl Another measure of central tendency is the median orhe point at which the sample is divided into two equal halves Let Xl XCll It denote a sample arranged in increasing order of magnitude that iSt xli denotes the smallest observation X2 denotes the second figure 813 A histogram 84 1UIllerica1 Description of Data 185 smallest observation and xl denotes the largest observation Then the median is defined matIematically as trI2 x 221 n odd 11 even 83 The median bas the advantage that it is not influenced very much by extreme values For example suppose that the sample observations are 134276 and 8 The sample mean is 443 and the sample median is 4 Both quantities give a reasonable measure of the central tendency of the data Now suppose that the nexttoIast observation is changed so that the data are 134272519 and 8 For these dara the sample mean is 36343 Clearly in this case the sample mean does not tell us very much about the central tendency of most of the data The median however is still 4 and this is probably a much more meaningful measure of central tendency for the majority of the observations Just as is the middle value in a sample there is a middle value in the population We define fl as the median of the population that is fl is a value of the associated random vari able sucb that half the population lies below i1 and half lies above The mode is the observation that occurs most frequently in the sample For example the mode of the sample data 246 2 5 6 2 9 4 5 2 and 1 is 2 since it occurs four times and no other value occurs as often There may be more than one mode If the data are symmetric then the mean and median coincide If in addition the data bave only one mode we say the data are unimoda1 then the mean median and mode may all coincide If the data are skewed asymmetric with along tail to one side then the mean median and mode will not coincide Usually we find t13t mode median mean if the dis tribution is skewed to tIe righ while mode median mean if the distribution is skewed to the left see to Fig 814 The pistribution of the sample mean is wellknown and relatively easy to work with Furthermore the sample mean is usually more stable than the sample median in the sense that it does not vary as much from samplp to sample Consequently many analytical tatis tical techniques use the sample mean However the median and mode may also be helpful descriptve measures Negative or left skew a Symmetric b Positive or right skew e Figure 8 14 The mean and zuedian for symmetric and skewed distributions 186 Chapter 8 Introduction to Statisics and Data Description 842 Measures of Dispersion Central tendency does not necessarily provide enough information to describe data ade quately For example consider the bursting strengths obtained from two samples of six bot tles each Sample 1 Sample 2 230 250 245 258 265 240 190 228 305 240 265 260 The mean of both samples is 248 psi However note that the scatter or dispersion of Sam ple 2 is mueh greater than that of Sample 1 see Fig 815 In this section we define sev eral widely used measures of dispersion The most important measure of dispersion is the sample variance If XII 2 xI is a sample of n observations then the sample variance is n 2xxj si j S 84 nl nl Note that computation of requires calculation ofi n subtractions 3Id n squaring and adding operations The deviations Xt x may be rather tedious to work with and several decimals may have to be carried to ensure nurtencal accuracy A more efficient yet equiv alent COlOlputational formula for calculating Sa is 85 The formula for Srx presented in equation 85 requires only One computational pass but care must again be taken to keep enough decimals to prevent roundoff error To see how the sample variance measures dispersion or variability refer to l1g 816 which shows the deviations Xi x for the second sample of six bottlebursting strengths The greater the amount of variability in the burstingstrength data the larger in absolute mag nitude some of the deviationsx x Since the deviations Xi iwill always sum to zero we must use a measure of variability that changes the negative deviations to nonnegative quan tities Squaring the deviations is the approach used in the sample variance Consequently if is smaR then there is relatively little variability in the data but if SZ is large the vari ability is relatively large The units of measurements for the sample variance are the square of the original units of the variable Thus ifX is measured in pounds per square inch psi the units for the sam pIe variance are pSi2 pi1 We wi11 calculate tte sample variance of the bottlebursting streJgths for the second sample in F1g 815 The deviations Xl i for this sample are shown in Fig 810 o o o o 0 o 180 200 220 1 o 240 r2W2BO 300 320 Sample mean 248 Sample 1 0 Sample 2 Figure 815 BurstingStrength dara 84 Numerical Description of Data 187 180 x o x o x o Xo Xs o 0 x o 320 Figure 816 How the sample variance measures variability throlgh he deviations Xi t Observations x 190 58 334 228 20 400 1 305 57 3249 x4 240 i 4 x5265 17 289 xs 260 12 lUI x248 From equation 84 s2 S Ixi xl 7510 502 1 nl n1 51 pSI We may also calculate SkY from the fornwlation given in equation 85 so that II 1 r 01 2 lx llx 1 s2 Sc 11 n i1 j 367534J488 6 502psi2 nI nI 5 If we calculate the sample variance of the buISting strength for the Sample I values we find that 158 psi This is considerably smaller than the sample variance nf Sample 2 confirming OUI initial impression that Sample I has less variability than Sample 2 Because is expressed in the square of the original units it is not easy to interpret Fmthermore variability is a more difficult and unfamiliar concept than location or central tendency However we can solve the curse of diroensiooality by working with the posi tive square root of the variance called tlle sample standard deviation This gives a meas ure of dispersion expressed in the same units as the original variable The sample standard deviation of the bott1ebursting strengths for the Sample 2 bottles in Rumple 82 and Fig 815 is s l 11502 3876 psi For the Sample 1 bottles the standard deviation of burstiog strength is s isg 1257 psi 188 Cbapter 8 Introduction to Statistics and Data Description Compute the szmple variance and sample standard deviation of the bottleburstingstrength data in Table 82 Note that 00 lC 2 707425800 LX 26406 Consequently S 707425800 26406100 10148985 and 2 10148985199 102515 psi so that the sample suudard deviation is 102515 3202 psi hen the population is finite and consists of N values we may define the population Yari ance as v Lx ul 02 11 N 86 which is simply the mean of the average squared departures of the data values from the pop ulation mean A closely related quantity if is also sometimes called the population vari ance and is defined as N 0 NI 87 Obviously as N getS large a 1 f1 and oftentimes the use of iT simplifies some of the algebraic formulation presented in Chapters 9 and 10 bere severa populations are to be observed a SUbscript may be employed to identify the population characteristics and descriptive measures eg fLx O s etc if the x variable is being described We noted that the sample mean may be used to make inferences about the population mean Similarly the sample variance may be used to rnake inferences about the population variance We observe that the divisor for the sample variancet is the sample size minus I n I If we actually knew the true value of the population meanl then we could define the sample variance a the average squared deviation of the sample observations about Jl In practice the value of Jl is almost never knoWn and so the sum of the squared deviations about the sample average x must be used instead However the observations Xi tend to be closer to their average i than to the population mean jL so to compensate for this we use as a divisor n 1 rather than n Another way to think about this is to consider the sample variance s2 a being based On II I degrees affreedom The term degrees of freedom results from the fact thac the n devi ations XI X X x XII x always sum to zero so specifying the values of any 11 1 of these quantities automatically determines the remaining one Thus only n 1 of the n de1 ations Xi X are independent Another useful measure of dispersion is the sample range 88 The sample range is very simple to compute but it ignores all the information in the sam ple between the smallest and largest observations For small sample sizes say II 10 this nformation loss is not too serious in some situations The range traditionally has had wide 84 wrerical Desription of Data 189 spread application in statistical quality control where sample sizes of 4 or 5 are common and computational simplicity is a major consideration however that advantage has been largely diminished by the widespread use of electronic measurement and data storage and analysis systems and as we will later see the sample variance or standard deviation pro vides a better measure of variability We will briefly discuss the use of the range in sta tistical qualitycontrol problems in Chapter 17 Cakate the ranges of the tvo samples 0 bottleburstingstrength data from Section 84 shown in Fig 815 For the first sample we find that 26523035 whereas for the second sample 305 190 115 Note that the range of the second sample is much larger than the range of me first implying that the second sample has greater variability than the first Occasionally it is desirable to express variation as a fraction of the mean A measure of relative variation called the sample coefficient of variation is defined as 89 The coefficient of variation is useful when comparing the variability of two or more data sets that differ considerably in the magnitude of the observations For example the coeffi cient of variation might be useful i1 compaing the variability of daiiy electricity usage within samples of singlefamily residences in Atlanta Georgia and Butte Montana dnring July 843 Other Measures for One Variable Two other measnres both dimensionless are provided by spreadsheet or statistical software systems and are called skewness and kurtosis estimates The notion of skewness was graph Cally illustrated in Fig 814 These characteristics are population or population model characteristics and they are defined in terms of moments 1 as described in Chapter 2 Section 25 skewness 3 J13 1 and kurtosis 34 J1 3 1 where in the case of finite populations the kth central moment is defined as N 2xi4 N 810 811 As discussed earlier skewness reflect the degree of symmetry about the mean and nega tive skew results from an asymmetric tail toward smaller values of the variable while posi tive skew results from an asymmetric tail extending toward the larger values of the variable 190 Chapter 8 Introduction to Statistics and Data Description Symmetric variables such as those described by the normal and uniform distributions hav skewness equal zero The exponential distribution for example has skewness 33 2 Kurtosis describes the relative peakedness of a distribution as compared to a normal distribution where a negative value is associated with a relatively flat distribution and a positive value is associated with relatively peaked distributions For example the kurtosis measure for a unifonn distribution is 12 while for a nonnal variable the kurtosis is zero If the data being analyzed represent measurement of a variable made On sample units the sample estimates of f3 and 3 are as shown in equations 812 and 81 3 These values may he calculated from Excel worksheet functions using the SKEW and KURT functions n skewness 3 nlnZ n2 kurtosis 3 nn 1 4 nln2n3 i1 3 n3 If the data represent measurementS on all the units of a finite population or a census then equations 810 and 811 hould be utilized directly to detennine these measures For the pull strength data shown in Table 81 the worksheet funetions return values of 0865 and 0161 for the skewness and kurtosis measures respeetively 844 Measuring Association One measure of association between twO numerical variables in sample data is ealled the Pearson or simple correlation coefficient and it is usually denoted r Where data sets COn tain a number of variables and the correlation coefficient for only one pair of variables des ignatedx andy is to be presented a subscript notation such as rry may be used to designate the simple correlation berneen variable x and variable y The correlation coefficient is S 814 where Sa is as shown in equation 85 and Syy is similarly defined for the y variable replac ing x with yin equation 85 while 11 11 JlnJt Sx Ix y y I IXy 1 1 Ix Iy i 815 11 11 n 1 i11 Now we will return to the wire bond pull strength data as presented in Table 81 with the scatter plots shown in Fig 83 for both pull strength variable y vs wire length vari able XI and for pull strength vs die height variable x For the sake of distinguishing between the two correlation coefficients we let rl be used for y vs x and T2 for y vs Xz The eorrelation eoeffieient is a dimensionless measure which lies on the interval 1 1 It is a measure of linear association between the two variables of the pair As the strength of linear association increases r71 A positive association means that larger Xl values have larger y values associated with them and the same is true for Xz and y In other sets of data where the larger x values have smaller y values associated with them the correlation coefficient is negative When the data reflect no linear association the correlation is zero In the case of the pull strength data 0982 and r 0493 The calculations were madc 84 Numerical Description of Data 191 using lvlinitab selecting StatBasic StatisticsCorrelation Both are obviously positive It is important to note however that the above result does not imply causality We cannot claim that increasing Xl or Xz causes an increase in y This jmportant point is often missed with the potential for major rrrisinteIpretationas it may well be that a fourth 1loobserved variable is the causative variable influencing all the obserled Vdriables In the case where the data represent measures on variables associated with an entire finite universe equations 814 and 815 may be employed after repJacing x by jl the finite population mean for the x population and y by Jy the finite population mean for the y pop ulation while replacing the sample size nin the formulation by lV the universe size In tlris case it is customary to use the symbol Pcy to represent this finite population correlation coejficient belVeen variables x and y 845 Grouped Data If the data are in a frequency distribution it is necessary to modify the computing formu las for the measures of central tendency and dispersion given in Sections 841 and 842 Suppose that for each of p distinct values of x say Xl Xz XP the observed frequency is Jj Then the sample mean and sample variance may be computed as respectively p lfjXj n 816 817 A similar situation arises where original data have been either lost or destroyed but the grouped data have been preserved In such cases we can approximate the important data moments by using a convention which assigns each data value the value representing the midpoint of the class interval into which the observations were classified so that all sam ple values falling in a particular interval are asqigned the same value TIris may be done eas ily and the resulting approximate mean and variance values are as follows in equations 818 and 819lfmj denotes the midpoint ofthejth class interval and there are c class intervals then the sample mean and sample variance are approximately and ifj mj ifj mj 1 n J C I C im I fm LiJJ nJ si Jl Jl nI 818 819 192 Chapter 8 Introduction to Statistics and Data Description iiii1f To illustrate the use of equatiols 818 and 819 we compute tbe mean and variance of bursting strength for the data in the frequency distribution of Table 83 Note that there are c 9 class inter vals and ilia m ISOf 2 m 200i 4 m 2201 7 m 2401 13 m 2f1Jf 32 m 2lI0f 24 m 3001 11 m 320fs 4 m 340 andj 3 Thus and 9 LJjmJ j 26460 2 60 xm pst n 100 709190026460 100 91499 99 pSI Notice that these are very close to the vuues obtained from the ungrouped data When the data are grouped in class intervals it is also possible to approximate the median and mode The median is approximately 820 where LM is the lower limit of the class interval containing the median called the median c1assfM is the frequency in the median class T is the total of all frequencies in the class intervals preceding the median class and A is the width of the median class The model say MO is approximately MO Lw ac ab 821 where LMO is the lower limit of the modal class the class interVal with the greatest fre quency a is the absolute value of the difference in frequency between the modal class and the preceding class b is the absolute value of the difference in frequency between the modal class and the following class and C is the width of the modal class 85 SUMMARY This chapter has provided an introduction to the field of statistics including the notions of process universe population sampling and sample results called data Furthermore a variety of commonly used displays have been described and illustrated These were the dot plot frequeruy distribution histogram stemandIeaf plot Pareto cbart box plot scatter plot and time plot We have also introduced quantitative measures for summarizing data The mean median and mode describe central tendency or location while the variance standard devi ation range and interquartile range describe dispersion Or spread in the data Furtbermor measures of skew and kurtosis were presented to describe asymmetry and peakedness respectiely We also presented the correlation coefficient to describe the strength of linear assoctztion between two variables Subsequent chapters will focus on utilizing sample results to draw inferences about the process or process model or about the universe l 86 EXERCISES 81 The shelf life of a highspeed photographic film is being invesated by the manufacturer The fo1owing datl are available Life days 126 131 116 125 13 120 125 150 130 149 Life days 129 132 128 126 127 122 11 148 120 117 Life days 134 136 130 134 120 129 147 126 117 143 Life days 141 145 162 129 127 133 129 14 131 133 866 912 861 904 891 941 978 931 864 876 931 946 963 947 9U 86 ExcCises 193 894 886 841 826 831 973 968 944 961 980 837 829 873 864 845 84 An electronics company manufacures power snpples for l persOli computer They roduce sev eral hundred power supplies each shift and each unit s subjected to a 12hour burnin test The nut1ber of units failing during this 12hour test each shift is shown below a Construct a frequency distribution and histogram b Find the sample mean sample variance and sam ple standard deviation 3 Construct a histogrnm and comment on the properties 4 of tile data 2 6 7 9 10 4 8 4 10 14 8 12 4 5 14 2 8 6 10 8 6 4 6 15 4 7 5 7 2 6 9 10 7 6 4 7 4 8 7 82 The percentage of cotto in a material used to manufacture mens shirts is givc below Construct a histogram for tlC data Comment on the properties of the data 342 336 338 347 378 326 358 346 331 347 342 336 366 331 376 336 345 350 334 325 354 346 373 341 356 354 347 341 346 359 346 347 343 362 346 351 338 347 355 357 351 368 352 368 371 336 328 368 347 351 350 379 340 329 321 343 336 353 349 364 341 335 345 327 83 The following data represent the yield On 90 con secutive batches of ceramic substrate to which l metal coating has been applied by a vapordeposition proeess Constract a histogran for hese data ane comment on the properties of the data 941 932 906 914 882 861 951 90Q 924 873 873 841 901 952 8ciJ 943 932 867 830 953 941 921 964 882 864 850 849 873 896 903 924 906 891 888 864 851 840 937 877 906 846 36 854 897 876 851 896 900 901 943 854 866 917 875 842 851 905 956 883 841 5 6 10 5 4 3 II 9 2 7 8 4 2 6 5 4 3 2 8 10 9 II 13 10 9 10 3 2 4 6 4 10 8 7 14 13 12 5 4 6 5 3 2 6 9 16 11 13 3 13 3 7 3 2 9 4 3 3 6 5 10 6 7 13 12 10 2 5 7 10 4 2 2 6 10 S 14 6 4 4 8 7 9 2 3 6 7 8 4 12 6 17 5 10 8 9 J 1 7 2 8 10 7 4 3 6 8 5 Consider the shelf life data in Exercise 8 1 Com pute the sample mean sample vaiance sad sample standard deviation 86 Consider he cotton percentage daa in Exercise 32 Find the sample ean saple variarc sa1ple srandard deviation sampe median ard sample mode 194 Chapter 8 Introduction to Statistics and Data Description 87 Consider tle yiClddata in Exercise 83 Calculae the sample mean sampe variance and sample stan dard deviation SS An article in Computers rma Industrial Engineer ing 2001 p 51 describes the timetofailure data in hours for jet engines Sone of the data are repro duced below Failure Failure Engine Time Engine Tlille I 150 14 171 2 291 15 197 3 93 16 200 4 53 17 262 5 2 3 255 6 65 19 286 7 183 20 206 8 144 21 179 9 223 22 232 10 97 23 165 1 187 24 155 12 197 25 203 13 213 a Construct a frequcrcy distnbution and histogra1l for these data b Calculate the sample mean sample median sam pIe variance and sarlple standard deviation 89 For the timetofailure data in Exercise 88 sup pose the ffth oscrvation 2 hours is discarded Con struct a frequency distributioa and a histogram for the remaining data and calculate the sample mean san pIe median sample variance and sample standard deviation Compare he results with hose otained LTl Exercise 88 Wbat impact has removal of this obser vation had on he summary statistics 810 An article in Tecnnol7Urrics Vol 19 1977 p 425 presents the followilg data on motor fuel octane ratings of 5eYeral blends of gasoline 885877834867875915886 1003 956933947911910942 878 899 883876843867882 908883988 942927932910903934885901 892833853379886909890961 933918923904901 930 887 899 898896874884839912893944 927918916904911926898906 911904893897903916905937 27922922912910922900907 a CoIilltruct a stenandleaf pIal b Construct a frequency distribution and bitogram c Calculate the sample mean sample vaiance and sample standard deviation d Find the sample median and sample mode e DeteJinehe ske1lIless and kurtosis measures 811 Consider the shelflife data in Exercise 81 Corstruct a stemandleaf plot for these data Con struct an ordered stemandleaf piOL Use this plot to find the 65th and 95th percentiles 12 Consider the cotton percentage data in Exercise 82 a ConsruC a stemandleaf plot b Calculate the sample mean sample vaance and sample standard deviatiou c Consrct an ordered stemandleaf plot d Fmd the median and the fint and third quartiles e Fmd the interquartile range f Detcmirle the skeness ad kurtosis measures 813 Consider he yield data in Exercise 83 a Construct an ordered stcmandleaf plot b Find the median and the first ald third quartiles c Calculate the inlerquartile range 814 Construct a box plot for the shelflife data in Exercise 8L Interpret the data using this plot 815 Construe a box plot for the cotton percettage data in Exercise 82 Interpret the data using this plot 816 Construct a box plot for the yield data in Exer cise 83 Compare it to the histogram Exercise 83 and the stemandle3f plot Exercise 813 btcrpret the data 817 An article in the Electrical Manufacturing coa Winding Conference Proceeding 1995 p 829 presents the results for the number of returned ship ments for arecordofhemonth club The company is interested in the reason for a returned shipment The result are shown below Construct a Pareto chart and interpret the data Reason Refused Wrong selection Wrong answer Canceled Other Number of CustOtlefS 195000 50000 68000 5000 15000 r 818 The following table contains the frequency of occurrence of final letters in an adcle in the Atlcmta Journal Construct a histogram from these data Do any of the numerical descriptos in this hapter have any meatiliig for these dam a 12 n 19 b II 0 13 c Jl p 1 d 20 q 0 e 25 r 15 f 13 s 18 g 12 20 h 12 u 0 8 v 0 j 0 w 41 k 2 x 0 11 Y 15 m 12 z 819 Show the following Cal That xO 1 b That 2 22 1 x xl x 1 820 The weight of bearings produced by a forging process is being investigated A sanple of six bearings provided the weights US J21 lJ9 lJ7 120 and 12 pounds rmd the sample mean sample variane sanple standard deviation and sarrple mediat 821 The diameter of eight automotive piston rings is shown below Calculate the sample mean sample variance and sample standard deiation 74001 mm 7398 mm 74005 74000 74003 74006 74001 74002 822 The thickDess of printed circuit boards is a very imporant characteristic A sample of eight boards had the folJowig thicknesses in thousands of 3t inch 6361 65 62 61 660 ard 66 Calculate the sple mean sample variance and sample standard deviation Vllhat are the units of measuremet for each Statistic 6 ExerciseS 195 824 COding the Data Let Yi a bx i 12 II where a and b are nonzero constants Find the rela tionship between x and y and between Sx and Sy 82S Consider the quantity Xi a2 For what Ll value of a is his quantity minimized 826 The Trimmed llean Suppose hat the data are arranged in ircreasing order L of the observations removed from each end and the sample mean of L1e remaining numbers calculated The resulting qantitY is called a trimmed mean The trimmed mean gener ally lies between the sample mean i and tie sample median i why Ca Calculate the 10 trimmed mean for the yield data in Exercise S3 b Calculate the 20 trimmed mean for the yield data in Exercise 83 and compare it with the quan tit found in part 2 821 The Trimmed 1ean Suppose th2t LN is rot an integer DeveJop a procedure fur Obtaining a trimmed mean 828 Consider the shelflife data in Exercise Sl Construct a frequency distribution and histogram using a class interval width of 2 Compute the approx imate mean and standard deviation from the frequency distribution and compare it with the exact values found in Exercise 85 8 29 Consider the folloving frequency distribution a Calcclate Le sample mean variance and stan dard deviation b Calcclate the nedian and mode 11161D IUIN UOl21 UZU31M h 4 6 9 13 15 19 20 IS 15 JO S3l Consider the fQllowing frequency distribution a Calculate the sample mean variance and stan dard deviation b Calculate the median and mode x I 4 3 2 1 o 2 3 4 823 Coding the Data Consider the printed circuit board thickness data in Exercise 822 J 60 120 ISO 200 240 190 160 90 30 a Suppose that we subtract a constant 63 from eaeh number How are the sample mean sample vari ance and sample standard deviation affected b Suppose that we multiply each number by lOa How are the sample mean sample variance and sample standard deviation affected 831 For the two sets of data in Exercises 829 and S 30 compute the sample coefficients of variation 832 Compute the approximate sample mean sample variance sample median and sample mode from the data in the follolfling frequency distribution 196 Chapter 8 Introduction to Statistics and Data Description Class lnteprru Frequency 1Ox20 121 20x30 165 30 x 4 184 4Ox50 173 50 x60 142 605x70 120 70x80 118 30x90 110 90Sx 100 90 833 Compute the approximate sample meao sample variance median and mode from the data in the fol lowing frequency distribution Class Intental Frequency 1OxO 3 Ox 10 8 105x20 12 20x30 16 30x40 9 45x50 4 50Sx60 2 834 Compute the approximate sample mean sampie standard deviation sample variance median and mode for the data in the following frequency distributiOll Class Iutexval 600 x650 41 6505x 700 46 7005x750 50 750x800 52 800x850 60 850x900 64 900x950 65 950 x 1000 70 1000 x 1050 72 835 An article in the International Journal aIndus rrial ErgoroJc3 l999 p 483 describes a study con ducted to determine the relationship between exhaustion time and distance covered until exhaustion for several wheelchair exercises perfonned on a 400m outdoor track The time and distances for ten participalts are as fonows Distance Covered Exhaustion TIme until Exhaustion in seconds in meters 610 1373 310 698 720 1440 990 2228 1820 4550 475 713 890 2003 390 488 745 lll8 885 1991 Deteonine tte simple pearson correlation between time and distance Interpret your results 836 An electric utility which serves 850332 residen tial customers on May 1 2002 seleets a sample of 120 customers randomly and installs a timeofuse meter or load research meter at each se1ected residence At the time of installation the technician also records the residence size sq ft During the SUmnler peak demand period the company experienced peak demand at 531 Pt on July 30 2002 July bills for usage kwh were sent to all customers including those in the sample group Due to cyclical billing the July bills do no reflect the same timeofuse period for all customers however each customer is assigned a usage for July billing The timeofuse meters hay memory and they record time specific demand kw by time interval so for the sampled customers average 15minute demand is available for the time interval 500545 PM on July 30 2002 The data file Load data contains the kwh kw and sq ft data for each of the 120 sampled residences This file is available at wwwwileycomooliegelhinesUsing Minitzb or other software dQ the fOllowing a Construct scatter plots of 1 kw vs sq ft and 2 kw V5 kwh and COlllOlent On the observed displays specifi cilly in regard 0 the nalUre of the associatiou in parts 1 and 2 above ald for each part the general pattern of observed variation in kw across the range of sq ft and the range of kwh b Construet the threedinensional plot of lew vs kwh and sq ft c Construct histograms for kwh kw and sq ft data and comment on the patterns observed d Construct a stemandleaf plot for the kwh data and compare to the histogram for the kwh data l e Determine the sample mean median and mode for kwh kw and sq ft f Determine the sample standard deviation for kwh kw and sq ft g Determine the first and third quartilcs for kwh kw and sq ft 86 Exercises 197 h Determine the skewness and kurtosis measures for kwh kw and sq ft and compare to the measures for a normal distribution i Determine the simple pearson correlation between kw and sqft and between kw and kwh Interpret Chapter 9 Random Samples and Sampling Distributions In this chapter ve begin our study of statistical inference Recall that statistics is the sci ence of drawing conclusions about a population based on an analysis of sample data from that population There are many different ways to take a sample from a population Fur thermorej te conclusions that we can draw about the population often depend on how the sample is selected Generally we want the sample to be representative of the population One important method of selecting a sample is random sampling Most of the statistical techniques that we present in the book assume that the sample is a random sample In this chapter we wlll define a random sample and introduce several probability distributions use ful in analyzing the information in sample data 91 R4lDOM SAIlPLES To define a random sample let X be a random variable with probability distributionfx Then the set of n observations Xl X1 Xlll taken on the random variable X and having numerical outcomes Xl X is called a random satrple lithe observations are obtained by observing X independently under unchanging conditions for n times Note that the obser vations XI Xz X in a random sample are independent random variables with the same probability distributionfx That is the marginal distributions of Xl Xv X arefxl fx jx respectively and by independence the joint probability distribution of the random sample is gx Xz x fx fxj fx 91 Definition 198 X X X is a random sample of size n if a theXs are independent random variables and b eveq observation Xi bas the same probability distribution To illustrate this defmition suppose that we are investigating the bursting strength of glass Iliter soft drink bottles and that bursting strength in the population of bottles is nor mally distributed Then we would expect each of the observations on b1lSting strength X XII in a random sample ofn bottles to be independent random variables v1th exactly the same normal distribution It is not always easy to obtain a random sample Sometimes we may use tables of uni form random numbers At otber times the engineer or scientist cannot easily use formal procedures to help ensure randomness and must rely on other selection methods Ajudg ment sample is one chosen from the population by the objective judgment of an indiviual 91 Random Samples 199 Since the accuracy and statistical behavior of judgment samples cannot be described they should be avoided Suppose we ish to take a random sample of five batches of raw material out of 25 available hatches We may n1lllber the batches With the integers 1 to 25 Now using Table XV of the Appendix arbi traily choose a row and columl as a starting point Read doOD the chosen column obtaining rNO dig its at a time until five acceptable numbers are found an acceptable number lies beween 1 and 25 To illustrate suppose the above process gives ts a sequence ofnumben5 that reads 37 8 550217 617043218213136025 The bold numbes specIy which batches of raw material arc to be chosen as the random sLllpe We first present some specifics on sampling from finite universes Subsequent sections will describe various sampling distributions Sections marked by an asterisk may be omitted without loss of continuity 911 Simple Random Sampling from a Finite Universe When sampling n items without replacement from a universe of size N there are Cd possi ble samples and if the selection probabilities are 1t Iii for k I 2 then this is simple random sampling Note that eacb universe unit appears in exactly of the possible samples so each unit bas inclusion probability of N As will be seen later sampling without replacement is more efficient than samFling with replacement for estimating the finite population mean or total however we brefly dis cuss sirnple random sampling with replacement for a basis of comparison In this case there are N possible sampJes and we select each vith probability 11 liNn for k 12 N tt In this situation a universe unit may appear in no samples or in as many as n so the notion of inclusion probability is less meaningful however if we consider the probability rhat a specific omt will be selected at least once that is obviously 1 1 11 since for each unit the probability of selection on a given observation is liNt a constant and the n selections are independent so that these observations may be considered Bernoulli trials Consider a univcne consisting offive units numbered 1234sln sampling without replacement we will employ a sampleof sire two and enumerate the possible samples as 12 13 14 15 23 24 25 34 35 45 Note that there are to possible samples IT we select one from these where each has an equil selection probability of 01 assigned that is siuplcrandom sampling Consider these possible sples to be numbered 12 0 where 0 represents num 10 Now go to Table XV iJ the Appendix shol1g n1ldom integers and vAth eyes closed place a finger down Read the first digit from the five digit integer presented Suppose we pick the integer which is in row 7 of columl14 The fust digt is 6 so the sample corsists of units 2 and 4 An alternate to usg the table is to cas a singe icosohe dron die and pick the sample corresponding to the outcome Kotice also that each unit appears in exactly foUI of the possible samples and thus the inclusion probability for each unit is 0 which is simply rJN 215 May be orcitted on first reading 20t Chapet 9 Random Samples and Sa11pling Distributions It is usually not feasible to enumerate the set of possible samples For example if N 100 andn 25 there would be more than 243 X 1013 possible samples so other selec tion procedures which maintain the properties described in the definition must be employed The most commonly used procedure is to first number the universe units from 1 to N thcn use realizations from a random number process to sequentially pick numbers from 1 to N discarcfulg duplicates until n units have been picked if we are sampling with out replacement kceping duplicates if sampling is with replacement Recall from Chapter 6 that the term random numbers is used to describe a sequence of mutually independent variables U1 Uz which are identically distributed as uniform on 01 We employ a realization u1 which in sequentially selecting units as members of the sample is roughly outlined as Unit NuroberLN u I 12 1 i I 2 n ij where J is the trial number on which the nth or finaJ unit is selected In sampling without replacement J n and L J is the greatest integer contained function When sampling with replacement J n 912 Stratified Random Sampling of a Finite Universe In finiteuniverse sampling sometimes an explanatory or auxiliary variables is available that has bown values fofeach unit of the universe These may be either numerical or cat egorical variables or both If We are able to use the auxiliary vaiables to establish criteria for assigning each universe unit to exactly one of the resulting strata before sampling begins then simple random samples may be selected within each stratum and the sampling is independent from stratum to stratum which allows us to later combine with appropriate weights stratum statistics such as the means and variances obtained from various strata In general this is an efficient scheme in the sense of estimation of popUlation means and totals if following the classification the variance in the vaiable we seek to measure is small within strata while the differences in the stratum mean values are large If L strata are formed then thestrarum sizes areNt N2 ND andNI N2 NL N while the sam pIe sizes are nl Tl nL and 1t Tl nL n It is noted that the inclusion probabil ities are constant within strata as nNh for stratum h but they may differ greatly across strata Two commonly used methods for allocating the overill sample to strata are propor tional allocation and Neyman optimal allocation where proportional allocation is and Neyman allocation is n nNlI h 12 L r Nh ah 1 nh n L LN ak Lhl h 12L 92 93 The values G h are standard deviation values within strata and when designing the sampling study they are usually unknown for the variables to be observediu the study however they may often be calculated for an explanatory or auxiliary vaiable where at least one of these is numeric and if there is a reasonable correlation Ipi 06 between the variable to be measured and such an auxiliary variable then these surrogate standard deviation values denoted a will produce an allocation wbich will he reasonably close to optimal 92 Statistics and Sampling Distributions 201 In seeking to address growing COnSUmer concerns regarding the quality of claim processing a national tnaruiged iealth carebealtb insurance company has identified several characteristics to be monitored on a monthly basis The most important of these to the company are first te size of the m financial error which is the absolute value of the error overpay or underpay or a claim and second the fraction of eorrectly rued claims paid to providers within 14 days Three processing cen tern are eastern midAmerica and western Together these eenters process about 450000 claims per month and it has been observed that they differ in accuracy and tbe11ness Furthermore the corre lation between the total dollar amount of the claim end the financial error overall is historically abou 065080 If a sample of size 1000 claims is to be drawn monthly strata might be formed using cen ters E MA and Wand total claim sizes as 0200 20h900 911 claim Therefore there would be nine strata And ufor a given year there were 41357 claims these may be easily assigned to strata sinee they are natually grouped by center and each center uses the same processing system to record data by claim amount thus allowing ease of classification by claim size At this point the WOOunit pla1Jled sa1e is allocated to the tine strata as shown in Table 91 The allocation in ital ics shoWtl first employs proportional allocation while the second one uses optimal allocation The values represent the standard deviation in the claimamount metic wihin the stratum since the stan dard deviation in financial error is unknown After deciding on the allocation to use nine inde pendent sirLple random samples without replacement would be selected yielding 1000 claim forms to be inspected Stratum identifying subscripts are not shown in Table 91 It is nored that proportional allocation results in an equal inclusion probability for all units not just witbin strata while an optimal allocation draws larger samples from strata in which the product of the internal variability as measured by standard deviation often in an auxiliary variable and the number of units in the stratum is latge Thus inclusion proba bilities are the same for all units assigned to a stratum but may differ greatly across strata 92 STATISTICS AND SAMPLING DISTRIBUTIONS A statistic is any function of the observations in a random sample that does not depend on unknoYtn parameters The process of drawing conclusions about populations based on Table 91 Data fur HeaI Insurance RQDlple 93 Claim Amount Center SO S200 201 900 5901 SMax CIaitJ All E N 132365 N41321 N 10635 184321 1 42 1 255 1 6781 94154 n241371 4181454 MA N96422 N31869 N6163 134454 1 31 210 1 5128 n218115 n7235 n14163 3041213 W N82332 N 33793 N 6457 122582 1 57 1310 7 7674 n187124 n7654 n151255 2781333 All N3111l9 N 106983 N23255 441357 70516 242143 531789 100011000 202 Chapter 9 Random Samples and Sarulirtg Distributions sample data makes considerable use of statistics The procedures require that we understand the probabilistic behavior of certain statistics In general we call the probability distribu tion of a statistic a sampling distribution There are several important sampling distributions that wili be used extensively in subsequent chapters In this section we define and briefly iliustrate these sampling distributions Firs we give some relevant definitions and addi tional motivation A statistic is now defined as a value detennioed by a function of the values observed in a sample For example if XI X2 Xn represent values to be observed in a probability sample of size n on a single tdndom variable X thenK and S as described in Chapter g in equations 81 and 8 are statistics Furthermore the same is true for the median the mode the sample range the sample skeIless measure and the sample kurtosis ote that capital letters are used here as this reference is to random variables not specific numerical results as was the case in Chapter 8 921 Sampling Distributions Definition The sampling distribution of a statistic is the density function or probability function that describes the probabilistic behavior of the statistic in repeated sampling from the same uni verse or on the same process variable assignment model Examples have been presented earlier in Chapters 57 Recall that random sampling with sample size n on a process variable X provides results XI X2 XII which are mutu ally independent random variables all vtith a common distribution function Thus the sam ple mean X is a linear combination of n independent variables If EX I and VeX cr then recall that EX Jl and VX crln And if X is a measurement variable the density function of X is the sampling distribution of this statistic There are several important sampling distributions that will be used extensively in subsequent chapters In this section we wili describe and briefly illustrate these sampling distributions The form of a sampling distribution depends on the stability assumption as wen as on me form of the process variable model In Chapter 7 we observed that if X N eu cr then X NC crln and this is the sampling distribution of X for n L Now if the process vari able assignment model takes some form other than the normal model illustrated here eg the exponential model and if all stationarity assumptions hold then mathematical analysis may yield a closed form for the sampling distribution of X In this example ease if we employ an exponential process model for X me resulling sampling distribution for X is a form of the gamma distribution In Chapter 7 the important Central Limit Theorem was pre sented Recall that if the momentgenerating function M t exists for all t and if Z XI th lim Fz l n I r en crn IllZ 94 where Fiz is lhe cumulative distribution function of Z and Iz is the CDF for the stan dard normal variable Z Simply stated as n t Z t NO I random variable and this result has enormous utility in applied statistics However1 in applied work a question arises regarding how large the sample size n must be to employ the NOI model as the sampling distribution of Z or equivalently stated to describe the sampling distribution of X as NC crln This is an important question as the exact form of the process variable assign mentmodel is usually unknOIl Furthermore any response must be conditioned even when iris based on simulation evidence experience or accepted practice Assuming process sta bility a general suggestion is that if the skewess measure is close to zero impJying that the variable assignment model is symmetric or very nearly so then X approaches normality 92 Statistics and Sampling Distributions 203 quickly say for n 10 but this depends also on the standaulized knrtosis of X For exam ple if fJ 0 and Ill 075 then a sample size of n 5 may be quite adequate for many applications However It should be noted that the tail behavior of X may deviate somewhat from that predicred by a nolllW model Wnere there is considerable skew present application of the Central Limit Theorem describing the sampling distribution must be interpreted with care A Hrule of thumb that has been successfully used in survey sampling where such variable behavior is common fJ 0 is that n 25fJ 95 For instance returning to an exponential process variable assignment model this rule sug gests that a sample size of n 100 is required for employing a nolllW distribution to describe the behavior of X since fJ 2 Definition The standard error of a statistic is the standard deviation of its sampling distribution If the standard error involves unknown parnrneters whose values caD be estimated substitution of these estimates intO the standard error results in an estimated standard error To illustrate this definition suppose we are sampling from a normal distribution with mean jJ and variance fi1 Now the distribution of X is normal with mean jJ and variance dln and so the standard error of X is If we did not know J but substituted the sample standard deviation s into the above then the estimated standard error of Xi s tpgj Suppose we colicet data on the tension bond strength of a modified portland cement mortar The ten observations are 16851640 7211635165217041696171516591657 whee tension bond strength is measured in units of kgfcml We assume that the tension bond strength is welldescribed by a notnl31 distribution The sample average is x 1676 kgflcm First suppose we know or are willing to assume that the standard deiation of tension baed strength is 0 025 kgfIcm2 Then the standard error of the sample average is Jfn 025 0079 kgfcm2 If we are unwilling to assume that 0 025 kgfJcm we could use the sample standard deviation s 0316 kgfJcm2 to obtain the estimared standard error as follows sn O316jl0 00999 kgfcm2 204 Chapte 9 Random Samples and Sampling Distributions 922 Finite Populations and Enumerative Studies In the cae where sampling may be conceptually repeated on the same finite universe of units the sampling distribution of X is interpreted in a manner similar to that of sampling a process except the issue of process stability is not a COncern Any inference to be drawn is to be about the specific collection of unit population values Ordinarily in such situations sampling is without replacement as this is more efficient The general notion of expecta tion is different in such studies in that the expected value of a statisticBis defined as G EO8 L16k 96 kl where fI is the value of the Statistic if possible sample k is selected In simple random sam pling recall that nk 1J Now in the case of the sample mean statistic X we have ECX fl where I1r is the finite population mean of the random variable X Also under simple random sampling without replacement 97 where a is as defined in uation 87 The ratio nhV is called the sampling fraction and it represents the fraction of the population measures to be included in the sample Concise proofs of the results shovll in equations 96 and 97 are given by Cochran 1977 p 22 Oftentimes in studies of this sort the objective is to estimate the population total see equa tion 82 as well as the mean The mean per unitl1 Or mpu estimate of the total is simply TxN X and the variance of chis statistic is obviously Wlv VX Wlrile necessary and sufficient conditions for the distribution of X to approach normality have been developed these are of little practical utility and the rule given by equation 95 has been widely employed In sampling with replacement the quanitities EX p and vX an trhere stratification has been employed in a finite population enumeration study there are two statistics of common interest These are the aggregate sample mean and the estimate of population total The mean is given by 98 99 In these formulations Xh is the sample mean for stratum h and WII given by Nhb is called the stratum weight for stratum h Note that both of these statistics are expressed as simple linear combinations of the independent stratum statistics Xhl and both are mpu esti mators The variance of these statistics is given by 910 I I 93 The ChiSquare Distribution 205 The withlnstratum variance terms may be estimated by the sample variance terms S for the respective strata The sampling distributions of the 3oregate mean and total estimate statistics for such stratified enumeration studies are stated only for situations where sample sizes are large enough to employ the limiting normality indicated by the Cen tral Limit Theorem Note that these statistics are linear combinations across observations within strata and across strata The result is that in such cases we take 911 and j Suppose the sampltng plal in Example 93 utilizeshe optimai aliocatio as shovln Table 9L The stratum ple means and sampe staadard deviations on the fi1a1cial error variable are cakJ lated with he results shown in Table 92 where the units of measurement are error claim Then utilizing the results presented in equations 98 and 99 the aggregate statistics whel eval uated are 1836 and f 810335 Utilizing equations 910 and eoploying the withinstratum sample variance Blues s to estimate the wlt1instratum variances oh the estimates for the variance in the sampling distriblrtior of the sample mean and the estimate of total are Vfl 0574 and Hi 1118 x 101 and the estimates of the respective standard errors are thus 0785 and 334408 for the mean and tOtal esrimator distributions 93 THE elUSQUARE DISTRIBlJTlON Many other useful sampling distributions can be defined in terms of norma random vari ables The chisquare distribution is defined below Table 92 Sampe Statitics foc Health Insurance Example Claim Amount 0200 201900 901 Max Claim E N 132365 N41321 N 10635 n29 n54 n371 625 x3410 x9165 MA N96422 N31869 N6163 n IS n35 nl63 x 530 x2200 x72oo 1639 5667 VI N82332 N33793 N6457 n24 n54 n255 x 1052 4628 x 12491 986 3123 10942 All N 311119 N 106983 N31898 n67 n 143 n789 All 184321 454 134434 213 131225 333 44l357 n 1000 206 Chapter 9 Random Samples illld Sampling Distributioru Theorem 91 Let Z z Z be normally and independently distributed random variables with mean I Q and variance d I Then the random variable Z Z Z X i 21 k has the probability density funetion 1 21 42 0 J U ku e U 21211 2 otherwise 912 and is said to follow the chisquUe distribution with k degrees of freedom abbreviated X and For thc proof of Theorem 91 see Exercises 747 and 748 The mean and variance of e X distribution are Jk 913 914 Several chisquare distribltions are shown in Fig 91 Note that the chisquare random vari able is nonnegative and that the probability disuibution is skewed to the right However as k increases the distribution becomes more symmetric As k the limiting form otthe chisquare dh1ribution is the normal distribution The percentage points of the X distribution are given in Table ill of the Appendix Define Xr as the percentage point or value of the chisquare random variable with k degrees of freedom sucb that the probability that X exceeds this value is IX That is pX xd J fudu a z o 5 10 15 20 25 u Figure 91 Severll X distributions 93 The ChiSquare Distribution 207 Ims probability is shown as the shaded area in Fig 92 To illustrate the use of Table m note that p X XiIQ P X l83lJ 005 That is the 5 point of the chisquare distribution with 10 degrees of freedom is XlO 1831 Like the nonnal distribution the chisquare distribution has an important reproductive property Theorem 92 Additivity Theorem of ChiSquare Let X X X be independent chisquare nmdom variables with k k k degrees of freedom respeotiveJy Then the quantity YXXX I z 3 follows the chisquare distribution with degrees of freedom equal to p kIA pl PrOfl Note that each chisquare random variable X can be written as the sum of the squares of kj standard nonnal random variables say il 2 p Therefore u P ki YLX LLZ it 11 j and since all me random variables ZfJ are independent because the X are independent Y is just the S1n1 of the squares of k kJ independent standard normal random variables l From Theorem 91 it follows that Y is a chisquare random variable with k degrees of freedom As an example of a statisti tiat follows the chisquare distribution suppose that Xl x XI is a random sample from a normal population with mean J1 and variance 01 The functiQU of t1C sample variance fu o u Figure 92 Percentage point Xt of the chisquare distribution 208 Chapter 9 Random Sples and Sampling Distributions is distributed as X We will use this random variable extensively in Chapters 10 and 11 We will see in those chapters that becauSe the distribution of tlris random variable is chi square we can construct confidence interval estimates and test statistical hypotheses about the variance of a norwa1 population To illustrate heuristically why the distribution of the random variable in Example 96 n lSlJoi is clJisquare note that 915 If X in equation 915 were replaced byu then the distribution of l Xi 1 is i because each term x 11115 anindependeut standard normal random variable Now consider the following 1 YXXXJl 1 i II f YXX IXJl 2XJlIXX jl 11 IrxX nXI 10 Therefore or IXI 1 Z nIS lXI 2 2 j r in 916 Since X is normally distributed vith meanl and variance erIn the quantiy 5 fJif fTln is dis tributed as X FurtiJerrnore it can be shown thaJ the random Variables X anaSireaependent Therefore since 21 Xi jt1 112 is distributed as X it seems logical to use the additivity proP erty of the chiuare distrihucion Theorem 92 and conclude that t1e distribution of n 1S2til is XI 94 THE t DISTRIBUTION Another important sampling distribution is the t distribution sometimes called the student t distribution Theorem 93 Let Z NO I and V be a chisquare random variable vith k degrees of freedom If Z and V are independent then the random variable Z T r V Vk L 94 The r DistrilYtion 209 has the probability density function t fklZl 1 1 Jiikrk2 t2klfl2 oot 917 and is said to fallow the t distribution with k degrees of freedom abbreviated t Proof Since Z and V are independent their joint density function is 1kf211 fzv v e JJI2 oozOvoo J27r Z kf2 r j Using the method of Section 410 we define a new random variable U V Thus the imrerse solutions of and are and The Jacobian is Thus t Z u v Ztf IU 1 ilk I 0 vu 1 ill 1 k and so the joint probability density fwletion of T and lJ is glu j tk21 e 1Ikt 2uV2 2Jrk2kf2 918 Now since v 0 we must require that u 0 and since 00 z 00 then 00 t On rearranging equ1ition 918 we have and since 115 gtudu we obtain 210 Chapter 9 Random Samples and Sampling DistribuiODS Jt oot Primarily because of historical usage many authors make no disinction between the random variable T and the symbol t The mean and variance of the t distribution are p 0 and d kk 2 for k 2 respectively Several t distributions are shown in Fig 93 The general appearance of the r distribution is sllnilar to the standard normal distribution in that both distributions are symmetric and unimodal and the maximum ordIDate value is reached at the mean p 0 However the t distribution has beavier tails than the normal that is it has more probability further out As the number of degrees of freedom k the limiting form of the I distribution is the standard nonnal distribution In visualizing the t distribution it is sometimes useful to know that the ordinate of the density at the mean fJ 0 is approx imately four to five times larger than the ordinate at the 5th and 95th percentiles rOr exam ple th I 0 degrees of freedom fort this ratio is 48 with 20 degrees of freedom this factor is 43 and with 30 degrees of freedom this factor is 41 By comparison for the normal dis tribution this factor is 39 The pcentage points of the t distribution are given in Table IV of the Appendix Let 1 be the percentage point or value of the t random variable with k degrees of freedom such that pTZaf Jtdta a This percentage point is illustrated in Fig 94 Note that since the t distribution is symmetric about zero we find t 111 tak Tbis relationship is useful since Table N gives only upperrail percentage points that is values of t for a 050 To illustrate the use of the table note that PT tQlSIO PIT L812 005 Thus the upper 5 point of the distribution with 10 degrees of freedom is os 1812 Similarly the lowertail point too t10 I 812 Ji As an exrunple of a ra1dom variable that follows the t distribution suppose thatX Xil is a xan com sampic from a normal distribution withmcanfJ and variance rr and letX and 51 denote thesam pIe mean and variance Corsica the statistic o Figure 93 Several distributions 98 The F DIstribution 211 Figure 94 Percentage points of the t distribution Divlding both the nurneretor and denominator of equation 919 by c We obtain X1 3y Sln S72 99 Sbee X 11cI NO1 and S2dl Xn 1 and since X and S2 are independent we see from Theorem 93 mat 920 follows a t distribution with v n 1 degrees of freedom In Chapter 10 and 11 we 111 use the rdD ciom variable in equaion 920 to construct confidence intervals and test hypotheses about the cear of a normal distribution 95 TIlE F DISTRIBlTTION A very useful sampling distribution is the F distribution Theorem 94 Let Wand Ybe independent chisquare random variables with u and v degrees of freedom respectively Then the ratio F Wu Ylv has the probability density function ht u v ultl2 u21 r p 2 v 0 921 and is said to follow the F distribution with u degrees of freedom in the numerator and v degrees of freedom in the denominator It is usually abbreviated FIJi Proof Since Wand Yare independent their joint probability density distribution is 212 Chapter 9 RJmdom Samples and Sampling Distributions Iwy 121 11 W Y wy2 22rzf2rJ e OwYQQ Proceeding as in Section 410t define the new random variable M 1 The inverse solutions of wluIylu and m y are and Therefore the Jacobian w umf v y m J Lm I I v Thus the joint probability density function is given by 21 UUfm ij2 gjm l m m2ilfl 2f2r 12J1r x e 2 2 0 jm and since hfJ r gf m dIn we obtain equation 921 compleling the proof The mean and variance of the F distribution are Il vv 2 for v 2 and 2vuv2 uv2v4 v4 Several F distributions are shown in Fig 95 The Frandom variable is nonnegative and the distribution is skewed to the right The F distribution looks very similar to the chisquare distribution ill Fig 91 bowever the parameters u and v proide extra flexibility regarding Shape u5v5 f Figure 95 The F illstribution 95 The F Disribution 213 The percentage points of the F disrribution are given in Table V of the Appendix Let F be the percentage point of the F distnlJution with u and v degrees of freedom such that the probability that the random variable F exceeds this value is This is illustrated in Fig 96 For example if u 5 and v 10 we find from Table V of the Appendix that PiFe Fmlo PiF 333 005 That is the upper 5 point of FlC is FO05JO 333 Table V contains only uppertail per centage points values of F aUlo for as 050 The lowertail percentage points F Iv can be found as follows 1 flaltv av1I 922 For example to find the lowertail percentage point FO955lOl note that 474 0211 As an example of a statistic that follows the F disbution suppose we have two notmal popula1ons with variances and 0 Let independent nmdom samples of sizes n l and r1 be taken from popu1a ions 1 and 2 respectively and let S and S be the sample variances Then the rano 923 has an F distributitin vith n 1 numeratOr degrees of freedom and liz 1 denominator degrees of freedom This follows diretly from the facts that nt 110 X 1 and nzlfO Xl and from Theorem 9 The nmdom variable in equation 923 plays a key role in Chapters 10 and 11 where we address the preblems of confidence interval estimation and hypothesis testing about the variances of two independent normal populatiOI15 h0 f Figure 96 Upper and lower percentage points of the F distribution 214 Chapter 9 Random Samples and Sampling Distributions 96 SLrvL1ARY This chapter has presented the eoncept of random sampling and introduced sampling dis tributionsIn repeated samlling from a population sample statisties of the sort discussed in Chaptcr 2 vary from sample to sample and the probability distribution of suell statisties or funetions of the Statistics is called the sampling distribution The normal chisquare Stu dent t and F distributions have been presented in this chapter and will be employed exten sively in later chapters to describe sampling variation 97 EXERCISES Suppose that a random variabl is normally dis Juted with mean JJ and variance if Draw a random sample of five observations What is the joiut density tioo of the sample 92TransistorS have a life that is exponentially dis tnbuted 1th pararrerer A random sampl of It ran slstors is taken hat is the joint density function of the sample 93 Suppose thar X is uniformly distributed 00 the interval from 0 to 1 Consider a ranqom sample of size 4 from X at is the joint density function of the sample 94 A lot consists of N transistors and of these M M N are dfective We randomly select twa tran sistors without replacement from this 10t and deter M rrine whether they are defective or ooodefectivc The ra1dom variable X II if the iOO transistor is nondeftive L2 lO if the ith transistor is defective Detennine the joint probability function for Xl and Xl Woat arc the marginal probability functions for XI and i Ate Xl and X2 independent random variables 9A population of power supplies for a personal puter has an outpUt voltage that is normally dis tributed with a mean of 500 V and a standard devia tion of 010 V A random sample of eight power supplies is selected Specify the sampling distribution of X 96 Consider je power supply problem described in Exercise 95 What is the standard error off 97 Consider the power supply problem described in Exercise 95 Suppose that the papulation standard deviation is nnknown How would you obtain the esti mated standard error QtA procurement specialist has purchased 25 rcsis from vendor 1 and 30 resistors from vendor 2 Let XIP X2 Xj 2j represent the vendor 1 observed resistmces assumed normally and independently dis tributed with a mean of 100 Q and a standard devia tion of 15 n Similarly letAlXu XzJirepresent the vendor 2 observed resistances asscmed normaUy and independently distributed with a mean of 105 Q an a standard deviation of 20 0 hat is the saro pling distribution oCr X2 99 Consider the resistor problem in Exercise 98 Fmd the standard elTO of X Xl 9 to Consider the resistor problem in Exercie 98 If we could not assume that resistance is normally dis tribut what could be said about the samplicg distri bution ofX 9 11 SuppoSe that independent random samples of sizes n l and 11 are taken from wo normal populations with mean JJI and fJ ard variances j and a respectively If X and X2 are the sample means find the sampling distribution of the statistic X 12 Ill 12 Mlnl j I 9 12 A manufacturer of semicorductor devices takes a random sample of 100 chips and tests them classi fying each chip as defective or nondefetive Let X 01 if the lth chip is Ilondefective defective The sample fraction defective is Xl tXZXIOO p 100 What is the sampling distribution of p 9 13 For the semiconductor problem in Exercise 912find the standard error of p Also find the esti mated standard error of p 914 Develop the momentgenerating function of the chisquare distribution 9 15 Derive the mean and variance of the chisquare random variable wij u degrees of feedom 916 Derive the mean and variance of the I distribution 917 Derive the mean and variance of the 1 distribution 918 OrderStatistics LetXj X2 Xj be arandorr sample of size n from X a random vdriable baving distribution function F Rank the elements iu arde of increasing DIlffietCaI magnitude resciting in XI XO X where Xll is the smallest sample element Xmmin X1Xl X andX is the largest sam ple element X max XI Xl X A is called 11e ith order statistic Often de distributio of some of the order Statistics is of interest particularly the ninimum and maximurI sample valnes XiI and l respectively Prove that t1je distribution functions of Xn and XC denoted respectively by FXjt and Fxt are FXt 11 FtW Fr Fr Prove that if Xis concinuous yith probability distribu tionfx then the probability distributions of Xli and 1 are fttl nl Ft1 fit ixr nFtl fi 9R19 Continuation of Exercise 918 Let Xl Xl Xq be a random sample of a Bernoulli random vaiiable with parameter p Shov that PX 1 1 lp PXl 0 1 p tlse the results ofE1tercise 918 920 Continuation of Exercise 918 Let X X Xn be a random sample of a nonnal random variable ith mean J1 and vaiance a2 Using the resclts of Exercise 918 derive the density functions of Xl and X 921 Continuation of Exercise 918 Let Xl X Xi be a tandom sample of a1 exponential rmdom g 7 Exercises 215 variable with parameter A Derive the distribution functions and probability distributions for Xl and X Use the results of ECcrcise918 922 Let Xl Xl X be a tandom sample of a con tinuous random variable Find EFXl and 923 Using Table ill of the Appendix lind the fol lowmg values X Xll c Xcr5m d XJ such that PXJ Xll 0975 924 Using Table IV of the Appendix find the fol lowing values a iOZlO b tw c ta10 such that Pi tiC falo 095 925 Using Table V of the Appendix find the follOV ing alues a FO2S49 b F01js c e FO956S d FO902424 926 Let FI4i denote a lowerwtail point I s 050 of the F v distribution Prove that FttHy lIF avJJ Chapter 10 Parameter Estimation Statistical inference is the proeess by which information from samp1e data is used to draw oonelusions about the population from whieh the sample was selected The teehniques of statistical inferenee ean be divided into two major areas parameter estimation and hypoth esis testing This ehapter treats parameter estimation and hypothesis testing is presented in Chapter 11 As an example of a parameter estimation problem suppose that civil engineers are ana lyzing the compressive strength of eoncrete There is a natural variability in th strength of each individual eonerete speeimenonsequently the engineers are interested in estimating the average strength for the population consisting of this type of concrete They may also be interestedin estimating the variability of eompressive strength in this population We present methods for obtaining point estimates of parameters such as the population mean and variance and we alSQ discuss methods for obtaining certain kinds of interval eSPmales of parameters called confidence intervals lOl POINT ESTIMATION 216 A point estimate of a population parameter is a single numerical value of a statistic that cor responds to that parameter That is the point estimate is a unique selection for the value of an unknown parameter More precisely if X is a random variable with probability distribu tionjx characterized by the unknow parameter e and if X X X is a random sam pie of size n from X then the statistic e heX X XJ corresponding to e is called the estimator of Note that the estimate eis a random variable because it is a function of sarn pie data After the sample has been selected e takes on a particular numerical value called the point estimate of e As an examplel suppose that the random variable X is normally distributed with unknown mean J1 and known variance 02 The sample mean X is a point estimator of the unknown population mean 11 Thar is fl X After the sample has been selected the numer ical value x is the point estimate of1 Thus iX1 25 X 31 x 28 and x 30 then the point estimate of1 is X 25312830 285 4 Similarly if the population variance Ii is also unknown a point estimator for Ii is the sam ple variance S and the numerical value s 007 calculated from the sample data is the point estimate of Ii Estimation problems occur frequently in engineering We often need to estimate the following parameters The mean J1 of a single population L toI Point Estimation 217 The variance 12 or standard deviation a of a siugle population The proportion p of items in a population iliat belong to a class of interest The difference between means of two populations i1 1 The difference between two population proportions Pl P2 Reasonable point estimates of these parameters are as follaws For JL the estimate is l the sample mean For i the estimate is S the sample variance For p the estimate is p XJn the sample proportion where X is the number of items in a random sample of size n that belong to the class of interest For u I 1 the estimate is fl p X Xz the difference between the sample means of two iudependett random samples For PI P2 the estimate 15Pl P2f the difference betveen two sample proportions computed from two independent random samples There may be several different potential point estimators for a parameter For example if We wish to estillare the mean of a random variable we might cODsider the sample mean the sample mediaa or perhaps the average of the smallest and largest observations in the sample as point estimators In order to decide which point estimator of a particular param eter is the best one to llse we need to examine their statistical properties and develop some criteria for comparing estimators 1011 Properties ofEsfunators A desirable property of an estimator is that it should be close in sOme sense to the true value of the unknown parameter Formally we say that e is an unbiased estimator of the parameter 8 if 101 Thatls iHs an unbiased estimator of e ifon the averaget its values are equal to e Note that this is equivalent to requiring that the mean of the sampling distribution of e be equal to e ifl Suppose that X is a random variable with mean fland variance cr IetX1XZ X be a randomsamw pIc of sizen from X Show that the sample llleanX and samplevanance S1 are unbiased estimators of jJ and respectively ConsiT and sfuce EX 14 for all i 1 2 n 218 Chapter 10 Parameter Estimation 1 EX Ii n 1 Therefore the sample mean X is an unbiased estimator of the population mean p Now consider 1 I 1 XX2XX 1 I I Ji x1 n1 iEXlnEX It 1 1 However since IX 1 andEX 11n we have s n Il iIi 0 11 In iI 1njt2 n02 np2 all nl 0 Therefore the smnple variance S2 is an unbiased estimator of the population variance cr However the sample standard deviation S is a biased estimator of the population standard deviation 0 For latge sampes tbis bias is negligible The mean square eUor of an estimator is defined as MSE0 Ee f The mean square error can be rewritten as follows MSE0 0 fI 8 Eef Vrf bias 102 103 That is the mean square eITOr of e is equal to the variance of the estimator plus the squared bias If e is an unbiased estimator of 0 the mean square error of j is equal to the variance of e The mean square error is an important criterion for comyaring two etimators Let 81 and 0 be two timat0I of the parameter e and let MECfl0 and MSEe be the mean square errors of 0 andB Then the relative efficiency of It to 8 is defined as MSEe MSEe If this relative efficicy is less than one we would conclude that a is a more efficient esti mater of than is Oz in the sense that it has smaller mean square error For example suppose that we wish to estimate the mean il of a population We have a random sample of i I 101 Point Estimation 219 l observations XI Xz XI and we wish to compare hvo possible estimators for J1 the sample mean X and a single observation from the sample say Xi Note that both X and j are unbiased estimators of 1 consequently the mean square enor of both estimators is sim ply the variance For the sample mean we have MSEX VeX 5fn whare 5 is the population variance for an individual observation we have MSE VeX 5 There fore the relative efficiency of Xi to X is MSEX MSEX Since lfn I for sample sizes n 2 we would conclude that the sample mean is a better estimator of 1 than a single observation X r Within the class of unbiased estimaton we would like to find te estimator that has the smallest variance Such an estimator is called arninimum variance unbiased estimator Fig ure 101 shows the probability distribution of two unbiased estimaton 8 and 9 with 81 hav ing smaller variance than The estimator 81 is more likely than t to produce an estimate that is close to the true value of the unknown parameter 8 It is possible to obtain a lower bound on the variance of all unbiased estimators of Let 8 be an unbiased estimaror of the parameter 9 based on a random sample of n observa tions and let fix IJ denote the probability distribution of the random variable X Then a lower bound on the variance of fJ is 104 This inequality is called the CramerRao lower bound If an unbiased estimator e satisfies equation lO4 as an equality it is the minimum variance unbiased estimator of 8 pI81 We Will show that the sample mean X is the minimum variance unbiased estimator of the mean of a nonna distrlmtion vith known variance Figure 101 The probability distribution octVo unbiased estimators 81 and J2 ICertain conditions on the functionfX B ae required for obtaining the Cmn6Rao inequality for example see Tucker 962 These conditiOn are satisfied by most oCme standard probability distribution 220 Chapter 10 Parameter Estimation From Example lOl we observe that X is an Ulbiased estirnator of p Note tha Substituting into equation 104 we obtain VX l 2 nEJInaJ2ii X I yli ldflL 2 j 1 nEXlr 04 n Since we know hat in general the variance of the saaple mean is VX erln we see that VeX sat isfies the CamerRao lower bound as an Iqmility Therefore X is the minimum variance unbiased estimator of Jl for the normal distribution where 0 is knOVll Sometimes we find that biased estimators are preferable to unbiased estimators because they have smaller mean square error That is we can reduce the variance of the esti mator considerably by introducing a relatively small amount of bias So long as the reduc tioa in variance is greater than the squared bias an improved estimator in the mean square error sense will result For example Fig 102 sbows the probability distribution of a biased estimator 81 with smaller variance than the unbiased estimator 82 An estimate based on 81 would more likely be close to the true value of than would an estimate based on We will see an application of biased estimation in Chapter 15 An estimator lr that has a mean square error that is less than or equal to the mean square error of any other estimator for all values of the parameter e is called an optimtJl estimator of O Another way to define the closeness of an estimator to the parameter e is in terms of consistency If en is an estimator of e based on a random sample of size n we say that8r is consistent for if for 0 105 Consistency is lMgesample property since it describes the limiting behTInr of the esti mator j as the sample size tends to infinity It is usillilly difficult to prove that an estimator r 101 Point Estimation 221 Figure 102 A biased estimator 81 that has smaller variance than the unbiased estimamr e is consistent using the definition of equation 105 However estimators whose mean square error or variance if the estimator is unbiased tends to zero as the sample size approaches infinity are consistent For example X is a consistent estimator of the mean of a normal dis tribution since X is unbiased and limn VeX limll a2n O 1012 The Method of Maximum Likelihood One of the best methods for obtaining a point estimator is the method of maximum likeli hood Suppose that X is a random variable with probability distributionfx IJ where 8 is a single unknown parameter Let XI X2 0 XII be the observed values in a random sample of size n Then the likelihood function of the sample is L 8 fx 1 8 fx 8 fx IJ 106 Note that the likelihood function is now a function of only the unknown parameter e The maximum likelihood estimator vllE of e is the value of e that maximizes the likelihood function LfJ Essentially the maximum likelihood estimator is the value of e that maxi mizes the probability of occurrence of the sample results Let X be a Bernoulli random variable The probability mass function is px r1 p x 0 I 0 othenvise where p is the parameter to be estimated The likelihood function of a sample of size n would be We observe that if p maximizes Lp then p also maximizes lnLp sinee the logarithm is a monoto nieally increasing function Therefore 222 Chapter 1 Q Parameter Estimation Now dlnLp tXI lrtI dp p 1p Equating this zero and solving for p yields the MlB p as an intuitively pleasing answer Of COurse one should also perfOCl a second derivative test but we have foregone hat here Iii Let X be normally distributed win unknown mean f1 and knOWtl variance cr The likelihood function of a sample of size n is Now lnLC n2ln2l 2ar LX1 Il 1 1 dlnall arLxIl J Equating tbls last reswt to zero and solving for f1 yields It may not always be possible to use calculus methods to determine the maximum of 48 This is illustrated n the follawing example Pji LetXbe uniformly distributed on the interval 0 to a The likelihood function of a random sample X x X of size n is Notc that the slope of this function is not zero anVhcre so we cannot use calculus methods to find the maximum likelihood estimator a Hovever notice that the likelihood function increases as a decreases Therefore we would maximize La by settingd to the sIIULllest value that it could reason ably assume OearIy a can be no smaller than the largest sample value so we woJd use the largest observation as fi Thus d maxi Xi is the 1LE for a 1 101 Point Estbation 223 The method of maximum likelihood can be used in situations where there are several unknmvn parameters say 81 82 8k to estimate In such cases the likelihood function is a function of the k unknown parametern 8 a 13k and the IDamnum likelihood esti mators Ili would be found by equaring the k first partial derivatives iJLB 8 BliJ8 12 k to zero and solug the resulting system of equations iF2 LetXbenormally distibuted vithmean fI and variance cr where both uand rr are unknown Find the maximwn likelihood estimators of U and 02 Tue likelihood function for a nmdom sacple of size n is and Now LJi2 tIOetf1llC12 dlnLlla all dlnL a 1 e IV2k4 zt I Lhuo J il The solutions to the above equations yield the roaxlmurllikelihood estimators and 2 I J X X n 11 which is closely related to the unbiased sample variance Sl Namely 61 nlnS2 Maximum likelihood estimators are not necessarily unbiased see the maximum like lihood estimator of d in Example 106 but they usually may be easily modified to make them unbiased Further the bias approaches zero for large samp1es In general maximum likelihood estimators have good largesample or asymptotic properties Specifically they are asymptotically normally distributed unbiased and have a variance that approachcs the CramerRan lower bound for large More precisely if 11 is thc maximum likelihood esti mator for 8 then yJ8 8 is normally distributed with mean zero and variance IniIelvniJ I 2 Ee 1nfX1I J for large n Maximum likelihood estimators are also consistent 1n addition they possess the invariance property that is if J is the maximum likelihood estimator of and u If is a func 224 Chaprer 10 Parameter Estimation tion of II that has a inglevalued inverse then the maximum likelihood estimator of u IJ is ufJ It can be shown graphically that the maximum of the likelihood will occur at the value of the maximum likelihood estimator Consider a sample of size n 10 from a normal i distribution 1415320732302501218623702592251922592647 Assume that the population variance is known to be 4 The 11LE for the mean J1 of a nor mal distribution has been shown to beX For this set of data x 25 Figure 103 displays the loglikelihood for various values of the mean Notice that the maximum value of the loglikelihood function OCC1JlS at approximately x 25 Sometimes the likelihood function is relatively flat in the region around the maximum This may be due to the size of the sam pie taken from the population A small sample size can lead to a fairly flt log likelihood implying less precision in the estimate of the parameter of interest 1013 The Method ofMomenfs Suppose that X is either a continuous random variable with probability density fx Ii e Ii or a discrete random variable with distributionpx 8 12 Ii characterized by k unknown parameters Let X l XZl Xn be a random sample of size n from Xt and define the first k sample moments about the origin as t 12 k The first k population moments about the origin are 1 EX xfxeeZektb 2pxe18ze xGR Sample mean Figur 103 Log lilelihood for various means 27 t12 k 107 X continuous X discrete 18 lO1 Point Estimation 225 The population moments J1 will ill general be functions of the k unknown parameters 8i Equating sample moments and population moments will yield k simultaneous equa tions in k unknowns the Ii that is t I 2 k 109 The solution to equation 109 denoted 8e tI yields the moment estimators of 8 8 9 wIi917 LetX Np cr where 11 and cr are unknol1l To derive estimators for u and cr by the method of moments recaE that for the normal distributiol 11 tt 14 cfl t The sample moments are m ln Xl and rn2C ln IX From equation 109 we obtain Lt Lt which have the solution tR10 Let X be uniformly distributed on the interval 0 a To find an estimator of a by the method of moments we note that the first population moment about zero is 1 a Il xdt 0 a 2 The first sample moment is just X Therefore or the moment estimator of a is just twice the sample mean The method of moments often yields estimarors that are reasonably good In Example 107 for instance the moment estimators are identical to the maximum likelihood estima tors In general moment estimators are asymptotically normally distributed approxi mately and consistent However their variance may be larger than the variance of estimators derived by other methods such as the mcthod of maximum likelihood Occa sionally the method of moments yields estimators that are very poor as in Example 108 The estimator in that example does not always generate an estimate that is compatible with our knowledge of the situation For example if our sample observations were Xl 60 Zz 10 and x 5 then a 50 whlch is unreasonable since we know that a 60 226 Chapter 10 Parameter Estimation 1014 Bayesian Inference In the preceding chapters we made an extensive study of the use of probability Until now we have interpreted these probabilities in the frequency sense that is they refer to an experiment that can be repeated an indefinite number of times and if the probability of occurrence of an event A is 06 then we would expect A to occur in about 60 of the exper imental trials This frequency interpretation of probability is often called the objectivist Or classical viewpoint Bayesian inference requires a different interpretation of probability called the subjec tive viewpoint We often encounter subjective probabilistic statements such as There is a 30 chance of rain today Subjective statements measure a persons degree of belief concerning some event rather than a frequency interpretation Bayesian inference requires us to make use of subjective probability to measure our degree of belief about a state of nature That is we must specify a probability distribution to describe our degree of belief about an unknown parameter This procedure is totally unlike anything we have discussed previously wntil now parameters have been treated as unknoVD constants Bayesian infer ence requires us to think of parameters as random variables Suppose we letfS be the probability distribution of the parameter or state of nature fI The distributionf1t summmizes our objective information about e prior to obtaining sample information Obviously if we are reasonably certain about the value of 9 we will choosefS with a small variance while if we are less certain about 88 will be chosen with a larger variance We callf8 the prior distnbution of 6 Now consider the distribution of the random variable X The distribution of X we denote fxl8 to indicate that the clistribution depends on the unknown parameter 9 Suppose we take a random sample from X say X X X The joint density of likelihood of the sample is fxx xl8 fx8fxI8 fxI8 We define the posterior distribution of f as the conditional distribution of 8 given the sample results This is just 1010 The joint distribution of the sample and 6 in the numerator of equation 1010 is the prod uct of the prior distribution of 6 and the likelihood or ft x x 8 f8 fx x xn8 The denominator of equation 1010 which is the maIginal distribution of the sample is just a normalizing constant obtained by IOIl 0 Consequently we may write the posterior distribution of e as I f6fx x x18 f SrlXx f xllx2Xn 1012 We nOle that Bayes theorem has been used to trmsforrn or update the prior distribution to the posterior distribution The posterior distribution reflectS our degree of belief about fJ given the sample information Furthermore the posterior clistribution is proportional to the product of the prior distribution and the likelihood the constant of proportionality being the normalizing constantfx x x HH Point Estimation 227 Thus the posterior density for e expresses our degree of belief about the value of e given the result of the sample The time to failure of a transisto is known to be exponentially distributed with pttameter A For a ran dom sample of n transistors the joint densty of the sample elements giver A is ltx1 fxx xIle Suppose we feel that the prior distibution for A is also exponential jA kM A O 0 otherwise where k wou1d be chosen depending on the exact knowledge or degree of belief we have about tJe value of A The jOint density of the sample and is 2 f xXZxn t kA e and the marginal density of the sample is l f XlXZ xl1 kl e dA krn 1 Ixkri Therefore l1e posterior density for by equation lO12 is ard we see that the posterior density ot is a gzuuUli distribution With paamcters n 1 and IX k 1015 Applications to Estimation In this section we discuss the application of Bayesian inference to the problem of estimat ing an unknown parameter of a probability distribution Let Xl X X be a random sam ple of the random variable X having density fxll We want to obtain point estimare of e Letf1f be the prior distribution for e and let eccl I be the lossunction The loss function is a penalty function reflecting the payment we must make for misidentifyin e by real ization of its point estimator i Common choices for eeel I are 6 1 and e el Gen erally the less accurate a realization ofiJ is the more we must pay In conjunction vith a particular loss function the risk is defined as the expeeted value of the loss function with respect to the random variables XI X2 Xfj comprising e In other words the risk is RdO Ei Ii e r idxjxi xBfxjx xledxjdx dx where the function d x x an alternate notation for the eslimator e is simply a function of the observations Since e is considered to be a random variable the risk is itself a random variable We would like to find the function d that minimizes the expected risk We WTite the expected risk as 228 Chapler 10 Paraleter Estin1tion Bd ERd Rdfede 1013 r ldX1X2 Xn fxX2 Xnu1u u fd We define the Bayes estimator of the parameter e to be the function d ofllie sample Xl X2 Xn that minimizes the expected risk On interchanging the order of integration in equation 1013 we obtain Bd r tdx X2 xnefXIX2 xnlfd uIuCu 1014 The function B will be minimized if we can find a function d that minimizes the quantity within the large braces in equation 1014 for every set of the x values That is the Bayes estimator of is a function d of the x that minimizes J idhx x fx1x xlefejd leefxx xde fX1X2X leefelx1x xn de Thus the Bayes estimator of 8 is the value e that minimizes leeYjx1x xn de 1015 1016 If the loss function e1 1 is the squarederror loss 811 then we may show that the Bayes estimator of e say e is the mean of the posterior density for refer to Exercise 1078 iilf Consider the situation in Example 109 where it wa shown that if the random variable X is expo nentially distributed with parameter and if the prior distribution for A is exponential with parame ter k then the posterior distribution for is a gamma distribution with parameters n k 1 and Xi k Therefore if a squarederror loss function is assumed the Bayes estimator for A is the Ll mean of this gamma distribution i rJ1 iXik I Suppose that in the timetofailure problem in EXac1ple 109 a reasonable exponenial prior distri bution fox A has parameter k 140 This is equiValent to saying that the prior estimate for is 10 007142 A rmdom sample of size n 10 yields AJ4x 1500 The Bayes estimate of is rJ1 101 loo607 to 1500 140 Ixk iI We may compare this with the results that would have been obtained by classical methods The max imurn lilce1ihood estimator of me parameter A in an exponential distribution is 11 Point Estimation 229 Corsequently the maximum likelihood estimate of t based or the foregoL1g sanple data is I 10 006667 tx 1500 i Note that the results produced by the two methods differ somewhat The Bayes estimate is slightly closer to the prior estimate than is the maximum likelihood estimate Let X J XII be a random samplc from the normal density MID meal JL and variance 1 where Il is unknown Assume that the prior density for J1 is nonnal with mean 0 and variance 1 that is fll 1eIp J2r The joint conditional density of the sample given p is J1 I 1 llrXp f xlx2Xr fl 2tr2 e 1 2Jfr2Jtr nd1 e 2Z Thus the joint dClsity of the sample and ps The marginal density of the sample is f 1 Lz u 2 l xlx2x 2nif2 exp 2 xf cXPt 2lln1 JL jPx j Jl By completing the square in the exponent under te integra we obtain 1 ZX n I nlIi22fl exp 2 nlj usinghe fact hat the iategral is 2ntln llfl siace a normal density has to integrate to 1 Now the poserior density for lJ is 230 Chapte 10 Parameter Estimation There is a relationship betvleen the Bayes estimator for a parameter and the maxi mum likelihood estimator of the same parameter For large sample sizes the two are near equivalent In general the difference between the two estimators is small compared to 1 In In practical problems a moderate sample size will produce approximately the same estimate by either the Bayes Or the maximum likelihood method if the sample results are consistent with the assumed prior information If the sample results are inconsistent with the prior assumptions then the Bayes estimate may differ considerably from the maximum likelihood estimate In these clrcumstances if the sample results are accepted as being correct the prior information must be incorrect The maximum likelihood estimate would then be the better estimate to use lf the sample results do not agree with the prior information the Bayes estimator will tend to produce an estimate that is between the maxbnUnl likelihood estimate and the prior assumptions If there is more inconsistency between the prior ioformation and the sample there will be a greater difference between the two estimates For an illustration of this refer to Example 1010 1016 Precision of Estimation The Standard Error When we report the value of a point estimate it is usually necessary to give some idea of its precision The standard error is the usual measure of precision employed If tiis an esti mator of e then the standard error of is just the standard deviation of i or 1l7 If Gt involves any unknown parameters thee if we substitute estimates of these param eters into equation 117 we obtain the estimated standard error ofe say6s A small stan dard error implies that a relatively precise estimate has been reported Exlll An article in the Journal aHeat Transfer Trans ASME Scs C 96 1974 p 59 describes a method of measuring the thenlal conductivity of Anneo iron Using a temperature of 100F and a power input of 550 W the followi1g 10 measurement of thermal conductivity in BtulnrftlIF were obtained 4160 4148 4234 4195 4186 4218 4172 4226 4181 4204 A poiot estimate of mean thermal conductivity allOOF and 550 W is the sanple mea1 or 1 41924 BtuIbrftF The stardard error of the samplemean is U i GJ ald since Gis untmown we may replace it with the sample standard deviation to obtain the estimated standard error of 1 O4 00898 Jr 10 Notice that the standard error is about 02 of the sample mean implyiug that we have obtained a rel arively precise pobt estimate of thermal conductivity r 101 Point Estimation 231 When the distribution of is unknown or complicated the standard errOr of 0 may be difficult to estimate using standard statistical theory In this case a computerintensive technique called the bootstrap can be used Efron and Tibshirani 1993 provide an excel lent introduction to the bootstrap technique Suppose that the standard error of ii is denoted To Fuiher assume the population prob ability density function is by fxlJ A bootstrap estimate of T can be easily CODsnucted 1 Given a random sample fromfx Xl X estimate 8 denoted bye 2 lsing the estimate 9 generate a sample of size n from the distributionlx 8 This is the bootstrap sample 3 lsing the bootstrap sample estimate 9 This estimate we denote Ii 4 GenerateB bootstrap samples to obtain bootstrap estimates 9 for i 12 B B 100 Or 200 is often used 5 Let e t e Is represent the sample mean of the bootstrap estimates 6 The bootstrap standard error of i1 is found with the usual standard deviation formula 2 8 S In the literature B 1 is often replaced by B for large values of B bowever there is little practical difference in the estimate obtained Rmi The failure times X of an electronic component arc known to follow an exponential distribution with unknown paramete Il A random sample of ten cotlponents resulted ill the following faile times in hours 1952 2014 1830 1751 2051 1917 1886 173520082100 The meaI of the exponeritial distribution is given by EX 11 It is also known that EX 1I1l A reasonable estimate for A n is i VX From the sanple data we find X 19244 resulting in l 119244 000520 B 100 bootstrap samples of size 110 were generated using Militabll with fir 000520 000520eolrn2fu Some of the bootstrap estirrates are shoJlll in Table 101 The average ofllie bootstrap estimares is found to be X Ii 00000551 The standard ercoc of the estimate is 11 1 000169 Table 101 Bootstrap Estimates for Example 1013 Sample Smrp1e ean X 243407 000411 2 153821 000650 3 126554 000790 100 204390 000489 232 Chapter 10 Parameter Estiwltion 12 SINGLESAMPLE CONFIDENCE IiTERVAL ESTIMATION In many situations a point estimate does not provide enough information about the param eter of interest For example if we are interested in estimating the mean compression strength of concrete a single number may not be very meaningfuL An interval estimate of the form L J1 U might be more useful The end points of this interval will be random variables since they are functions of sample data In general to construct an interval estimator of the unknoll parameter e we must find two statistics L and U such that PL 1 UJ 1 c 1018 The resulting interval 1019 is called a lOO 1 a confidence irteValfor the unknovtn parameter e Land U are called the lower and upperconfidence limits respectively and I a is called the confidence coef ficiem The interpretation ill a confidence interval is that if many random samples are col lected and a 1001 a cenfidence interval on e is compured from each sample then 1001 a oithese intervals will contain the true value of e The siruationis illustrated in Fig 104 which shows several 1001 a confidence intervals for the mean J1 of a distri bution The dol at the center of each interval indicate the point estimate of J1 in this case Xi Notice that one ill the 15 intervals fuils to contain the true value of JL If this were a 95 con fidence level in the long run only 5 of the intervals would fail to contain JL Now in practice we obtain only one random sample and calculate one confidence intervaL Since this interval either will or ill not contain the true value of 8t it is not rea sonable to attach a probability level to this specific event The appropriate statement would be that e lies in the observed interval L 1 with confidence 1001 a This statement has 1 Ir l I Figure 104 Repeated construction of a confi dence interval for JL J 102 SingleSample Confidence Interval Estimation 233 a frequency interpretation that is we do not know if the statement is true for this specific sample but the method used to obtain ille interval L U yields correct statements 1001 a of the time The confidence interval in equation 1019 might be more properly called a twosided conJidence interval as it specifies both a lower and an upper limit on J Occasionally a one sided confidence interval might be more appropriate A onesided 1001 a lowercon fidence interval on is given by the interval L e 1020 where the lowerconfidence limit L is chosen so that PL 8 1 a 10211 Similaly a onesided 1001 a uppercorlidence interval on e is given by ille interval esu where the upperconfidence limit U is chosen so that P e UJ 1 a 1023 The length of the observed twosided confidence interval is an important measure of the quality of the information obtained from the sample The halfinterval length e L Or U e is called the accuracy of the estimator The longer the confidence inter1lal the more confident we are that the interval acrually contains the true value of 8 On the other hand the longer the interval the less information we have about the true value of e In an ideal situation we obtain a relatively short interval with high confidence 1021 Confidence Interval on the Mean of a Normal Distribution Variance Known LetXbe a normal random variable with unknown mean Iand known variance 0 and sup pose that a random sample of size n X x Xn is taken A 1001 a confidence interval on jl can be obtained by considering the sampling distribution of the sample mean X In Section 93 we noted that the sampling distribution of X is normal if X is normal and approximately normalif the conditions of the Central Limit Theorem are met The mean of X isu and the variance is rrln Therefore the distribution of the statistic ZXI aJn is taken to be a standad normal distribution The distribution of Z X llGjJn is sbown in Fig 105 From examination of this figure we see that Figure 105 The distrimtion of Z 234 Chapter 10 Parameter Estimaticn or This can be rearranged as pX ZI CfJn 0 J1 X Z12 OJn Ia 1024 Comparing equations 1024 and 1018 we see that the 1001 a twosided confidence interval on J1 is 1025 tpiijH Consider the thermal conductivity data in Example 1012 Suppose that We want to find a 95 con fidence interval cn The mean thecnal conductivity of Arnco iron Suppose we know that the stardrud deviation of Thermal conductivity at 100F ard 550 W is 0 010 BtuIhrftR If we assume that thermal conductivty is normally distributed or that thc conditions of the Central Limit Theorem are meo then we can use equation 1025 to construct the confidence interva1 A 95 interval implies tliat 1 a 095 so ex 005 Ind from Table IT in the Appendix Zan ZJ0512 ZO02S 196 The lower confidence limit is LiZa12 jJn and the upper confidence limit is 4 924196 01 0 50 419240062 41862 UxZat1vjJi 1924 196O10JiO 4192O062 41986 Thus the 95 twosided confidence intcrval is 41862 1 41986 This is our interval of reasonable wlues for mean thermal conductivity at 95 coniidence Confidence Level and Precision of Estimation Notice that in the previous example our choice of the 95 level of confidence was eSsen tially arbitrary What would have happened if we had chosen a higher level of confidence say 991 In fact doesnt it seern reasonable that we would want the higher level of confi dence At a 001 we find Z 20 112 Zoo 258 while for a 005 200 196 Thus the length of the 95 confidence interval is Ie 2 196CfIln 392OIJn whereas the length of tne 99 confidence interval is 225OJn 515aJn 102 SinglcSample CDnfidence Interval Estimation 235 The 99 confidence interval is longer tllan the 95 confidence interval Tbis is why we have a higher level of confidence in the 99 confidence interval Generally for a fixed sam ple size n and standard deviation a the higher the confidence level the longer the resulting confidence interval Since the length of the confidence interval measures the precision of estimation we see that precision is inversely related to the confidence level As noted earlier it is highly desir able to obtain a confidence interval that is short enough for decisionmaking purposes and that also has adequate confidence One way to achieve this is by choosing the sample size n to be large enough to give a confidence interval of specified length with prescribed confidence Choice of Sample Size The accuracy of the confidence interval in equation 1025 is ZalZa Jti This means that in using x to estimate 1 the error E IX JJi is less than Za12rr i In with confidence 100 1 0 This is shown graphically in Fig 106 In situations where the sample size Can be con trolled we can choose n to be 1001 a confident that the error in estimating il is less than a specified error E The appropriate sample size is Z 02 a Z n E 1026 If the righthand side of equation 1026 is not an integer it must be rounded up Notice that 2 E is the length of the resulting confidence interval To illustrate the use of this procedure suppose that we wanted the error in estimating the mean thermal conductivity of Armco iron in Example 1014 to be less than 005 BtuIhr ftF with 95 confidence Since rr 010 and ZUD25 196 we may find the required sample size from equation 1026 to be nZazrrJZ 1960lOT 153716 E 005 J Notice how in general the sample size behaves as a function of the length of the COn fidence interval 2 E the confidence level 1001 a and the standard deviation rr as follows As the desired length of the interval 2 E decreases the required sample size n increases for a fixed value of rr and specified confidence As crincreasest the required sample size n increases for a fixed length 2 E and spec ified cQnfidence As the level of confidence increases the required sample size n increases for fixed length 2 E and standard deviation rr Figure 106 Error in estimating I 1th X 236 Chapter 10 Parameter Estimation OneSided Confidence Intervals It is also possible to obtain ooesided confidence intervals for Il by setting either L br U and replacing Z by Z The 1001 X upperconfidence inteITal for Il is r fJ5XZa G in and the 1001 X lowerconfidence interval for Il is XZaJ JnIl 1022 Confidence Interval on the Mean of a Normal Distribution Variance Unknown 1027 1028 Suppose that We wish to find a confidence interval on the mean of a distribution but the vari ance is unknown Specifically a random sample of size 11 X1X X is avajable and X and S are the sample mean and sample variance respectively One possibility would be to replace Jin the confidence interval formulas for Il with known variance equations 1025 1027 and 1028 with the sample standard deviation s lithe sample size n is relatively large say n 30 then this is an acceptable procedure Consequently we often call the con fidence intervals in Sections 1021 and 1 022largesample confidence intervals because they are approximately valid even if the unknown population vatiances are replaced by the corresponding sample variances When sample sizes are sI1all this approach will not work and we must use another procedure To produce a valid confidence interval we must make a stronger assumption about the underlying population The usual assumptiOl is that the underlying population is normally distributed This leads to confidence intervals based on the t distribution Specif ically let Xl XZ Xn be a random sample from a normal distribution with unknown mean u and unknown variance r In Section 94 We noted that the sampling distribution of the statistic Xu t Sfn is the t distribution with n 1 degrees of freedom We nOw show how the confidence inter val on fl is obtained The distribution of r X 1lsjJn is shown in Fig 107 L11ing r be the upper al2 percentage point of the t distribution with n 1 degrees of freedom we observe from Fig 107 that Figure 107 The t distribution 1 lO2 SingleSamplc Confidence Interval Estimation 237 or p I2n S s aj2nl 4 a Rearranging this last equation yields px rtl S s1 sX H ciln1 sf 1a 1029 Comparing equatiOIl5 1029 and 1018 we see that a 1001 a twosided coniidence interval OIl 1 is Ir r X IOn1 SfIn SW X IOnl Sjvn A 1001 a lowerconiidence interval on1 is given by r X alIl S v n Jl and a 1001 a upperconiidence interval on1 is 1030 1031 1032 Remember that these procedures assume that we are sampling from a normal popula tiOIl Tbis assumption is important for small samples Fortunately the no111ility assumption holds in many practical situations When it does not we must use distributionfree or Mt parametric confidence interv Nonparametric methOsaredlscussed in Chapter 16 How ever when the population is normal the tdistribution intervals are the shonest possible 1001 a confidence intelrals and are therefore superior to the nonparametric methods Selecting the sample size n required to give a confidence interval of required length is not as easy as in the knoVU f case because the length of the interval depends on the Value of a unknmvn before the data is collected and on n Furthermore n enters the confidence interval through both lJn and tl Consequently the required n must be determined through trial and erroc tfgl An article in the Journal if Testing and Evaluation Vol 10 No4 1982 p 133 presents the following 20 measurements on residua flame time in seconds of treated specimens of cbildrens nightwcar 985 993 975 977 967 987 967 994 985 975 983 992 974 999 988 995 995 993 992 989 We msi to find a 95 confidence interval on the mean residual flame time The sample mean and standard deviation are x98475 sO0954 From Table IV of the Appendix we find to02519 2093 The lower and upper 95 confidence limits are L Xla2H s 98475 209300954J20 98029 seconds 238 Chapter 10 Parameter Estimation and U r x tlXLJ11SVn 98475 209300954 fjjj 9921 seconds Tnerefore the 95 confidence interval is 98029 sec J 98921 sec We are 95 confident that the mean residual flane time is between 9805 and 98921 seconds 1023 Confidence Interval on the Variance of a Normal Distribution Suppose that X is nOIl1ally distributed with unknown mean1 and unknovn variance a Let XX2 X be random sample of size n and let S2 be the sample variance It was shown in Section 93 that the sampling distribution of 2 nlS2 X 0 is chisquare with n 1 degrees of freedom This distribution is shown in Fig 108 To develop the confidence interval we note from Fig 108 that PXOI X X 1 a or r 2 P 2 nlS 2 1 l Xlai2111 f2 X2nl a This last equation can be rearranged to yield 2 P nlS 2 nlS ll 1 2 Y 2 r a Aaj2nt 1a2n1 1033 Comparing equations 1033 and 1018 we see that a 1001 a twosided confidence interval for a is nlS2 2 nlS 2 12 a21I1 12n1 1034 012 o Xl12n1 Figure 108 The X distribution 102 SingleSanpe Confidence Irterval EstL71ation 239 To fmd a 1001 a lowerconfidence interval on a set U and replace Xl with XUI1I glnng nIS 2 2 j Xanl 1035 The 1001 a upperconfidenee interval is found by setting L 0 and replacing Xttr2Jll with XILn resulting in 2nlS2 0 2 Xtant 1036 A manufacturer of soft drink beverages is jntcrested in the uniformity of the m1chine used to fill cans Specifically it is desirable that the standard deviation 0 of the filling process be less than 02 fluid ounces otherwise there will be a bigher than allowable perceltage of cans that are underfilled We will assume that fill volume is approxirrately normally distributed A random sample of 20 cans result in a sarople variance of S1 00225 fluid ouncesJ2 A 95 uppercOIiidence interval is found from equation 1036 as fo1ows or G 2 1900225 00423 fluidounces ml17 TIlls last statement may be cOlverted into a confidence interval on the standard deviation Oby taking the square root of both sides resulting in 0 021 fluid ounces Therefore at the 95 level of confidence the data do not supOrt the claim that the Jtoeess staIdard deviation is less than 020 fluid ources 1024 Confidence Interval on a Proportion It is often necessary to construct a 1001 a confidence interval on a proportion For example suppose that a rundom sample of size n has been talren from a large possibly infi nite population and XC n observations in this sample belong to a class of interest Then f Xn is the point estimator of the proportion of the population that belongs to this class Note that n and p are the parameters of a binomial distribution Furthennore in Section 7 5 we saw that the sampling distribution of p is approximately normal with mean p and vari ance pl pn if p is not too close to either 0 or 1 and jf n is relatively large Thus the distribution of is lProximately standard nonnal To construct the confidence inteIal on p note that PZ ZZ 1 a 240 Chapter 10 Parameter Estimation or This may be rearranged as 1037 We recognize the quantity fir p lin as the standard error of the point estimator p lnfortunately the upper and lower lintits of the confidence inrerval obtained from equation 1037 would contain the unknown parameter p However a satisfactory solution is to replace p by P in the standard error giving an estimated standard error Therefore I illill Ipliij P PZi2V n p5PZa n 1a and the approximate 1001 a twosided confidence interval onp is ilIP lillil pZaI2 n p5PZat2 n An approximate 1001 a lowerconfidence interval is Z liij5 P a p n and an approximate 1001 a uppercOllfidence interval is ijt I 1 z IP P P p a 11 1038 1039 1040 1041 In a random sample of 75 axle shafts 12 have a surlace finish t is rougher than the specifications will allow Thetefore a point estimate of he proportion p of shafts in the population that exceed he roughness specifications is pxn 12nS 016 A 95 twosided Confidence interval for p is CQIlJ puted from equation 1039 as or which simplifies to 008p 024 102 SingleSarple Confidence Interval Estimation 241 Define the error in estimating p by jJ as E iF pI Note that we are approxnnately 1001 a confident tbat this error is less than Za2 i pl p n Tnerciare in situations where the sample size can be selected we may choose n to be 1001 a confident that the error is less than some specified value E The apPrOpriate sample size is Z J plp 1042 This function is relatively flat from p 03 to p 07 An estimate of p is required to use equation 1042 If an estimate p from a previous sample is available it could be substituted for p in equation 1042 or perhaps a subjective estimate could be made If these alternatives are unsatisfactory a preliminary sample could be taken p computed and then equation 1042 used to determine how many additional observations are required to estimate p with the desired accuracy The sample size from equation 1042 will always be a maximum for p 05 that is p1 025 and this can be used to obtain an upper bound on Il In other words we are at least I OO 1 a confident that the error b estimating p using p is less than E if the sample size is nZ2 rrO25 1n order to maintain at least a 1001 a level of confidence the value for n is always rounded up to the next integer 3pij8 Consider ihe data in Example 1017 How large a sample is required if we want to be 95 con5dent that the error in usingp to estimate p is less than 005 Usingp 016 as an initial estimate of p we fiLd from equation 1042 that the required sample size is Z02 plpl96J 016084207 E 005 We note that the procedures developed in th1s section depend on the nonnal approxi mation to the binomiaL In situations where this approximation is inappropriate particularly cases where n is small other methods mUSt be used Tables of the binomial distribution could be used to obtain a confidence interval for p If n is large but p is smaIl then the Pois SOn approximation to the binomial could be used to construct confidence intervals These procedures areillustrated hy Duncan 1986 Agresti and Coull 1998 present 1I1 alternative form of a confidence interval On the population proportion p based on a largesample hypothesis test on p see Chapter 11 of this text Agresti and Coull show that the upper and lower limits of an approxnnate 1001 a confidence interval on p are Z I A 2 plp Zj2 pz l 1 2n 0 tz 4n Z2 1 n The authors refer to this as the score confidence interval Onesided confidence intervals can be constructed simply by replacing Zafl with Z 242 Chapter 10 Parameter Estimation To illustrate tins confidence interval reconsider Example 1017 whlch discusses the surface finish ofa shaft with n 75 and p 016 The lower and upper limits of a 95 con tidence interval using the approach of Agresti and Coull are 0186 0087 1051 0177 O083 1 196 75 The resulting lower and upperconfidence limits are 0094 and 0260 respectively Agresti and Coull argue that the more complicated confidence interval has several advantages over the standard largesample interval given in equation 1039 One advan tage is that their confidence interval tends to maintain the stated level of confidence better than the standard largesample interval Another advantage is that the lowerconfidence limit will always be nonnegative The Jagesample confidence interval can result in neg ative lowerconfidence limits which the practitioner will generally then set to 0 A method which can report a negtive lower limit on a parameter that is inherently nonnegative such as a proportion p is often considered an inferior method Lastly the requirements that p not be close to 0 or 1 and l be relatively large are not requirements for the approach sug gested by Agresti and CoulL In other words their approach results in an appropriate confi dence interval for any combination of nand p 103 TWOSAMPLE C01FIDENCE INTERVAL ESTIMATION 1031 Confidence Interval on the Difference between Means of Two Normal Distributions Variances Known Consider two independent random variables XI with unknown mean fll and knmvn variance cr andX with unlnownmean IL and known variance 1 We wish to fmd a 1001 a confidence interval on the difference in means1 111 Let Xli X12 XII be a random sample of n1 observatious from Xl andXZ1 Xn X2l be a random sample of nz obser vations fromXz JiX andXz a the sample means the statistic is standard nonnal if X andX are normal or approximately standard normal if the condi tions of the Central Lintit Theorem apply respectively From Fig 105 tins implies that PZZ ZnJ 1 a or 103 TwoSample Confidence Interval Estimation 243 This can be rearranged as I 2 2 100t y Xl Xl ZalI la I nJ z 1043 Comparing equations 1043 and 1018 we note that the 1001 a confidence interval for 11 11 is I 2 2 Cr2 XJXoZ ll JX1XZI 12 1044 alI1 1 a 21 l Onesided confidence intervals on 11 11 may also be obtaiued A 1001 a upper confidence interval on 11 11 is 1045 and a 1001 a lowerconfidence interval is 2 101 0 XJ X2Za 1 1 11 fll 1Lz 1046 Tensile strelgth tests were perfoIDlcd on two different grades of alumintIl1 spars used in manufac turing the ving of a commerciai trnosport aircriL From past experience with the spar manufacturing process and the testing procedure the standard deviations of tensile strengths are assumed to be knOWll The data obtained are shovU in Table 112 If fll and p denote tile true mean tensile strengths for the two grades of spars then we may 5nd a 90 confidence ioterval on the difference in mean strength 1 pz as follows L Z 101 0 T tz r 1 a nl nz 110 57 8767451645 1L i 10 2 131088 1222 kglmm TabJe 102 Tensile Strength Test Result for Aluminum Spars Sample Mean Spar Sample Telsile Strength Grnde Size kgmm 1 111 10 xl 876 2 12 745 Standard Deviation 0 10 0 15 244 Chapter 10 Paamete Estimation I 0 1 52 876745L64511l 10 12 131088 1398 kgmm 2 Therefore te 90 corfidence interval on the difference in mean tensile strength is 1222 kgImm S Il fI 1398 kgnun We are 90 confident that the mean tensile strength of grade 1 aluminum exceeds that of grade 2 a1u minum by between 1222 and 1398 kgrnm If the standard deviations 0 and 02 are known at least approximately and if the sam pIe sizes nl and ftt are equal n nz n say then we can determine the sample size required SO that the error in estimating Iii Uz using XI X2 will be less than E at 1001 a confidence The required sample size from each population is Zl2 2 2 nE 0102 1017 Remember to round up if n is not an integer 1032 Confidence Interval on the Difference between Means of Two lIIormal DistribUtions Variances Cnknown We now extend the results of Section 1022 to the case of two populations with unknown means and variances and we wish to find confidence intervals on the difference in means 11 ill If the sample sizes n and n both exceed 30 then the normal knownvariances dis tribution intera1s in Section 1 3 1 can be used However when small samples are taken we must assume that the underlying populations are normally distributed with unknown variances and base the confidence intervals on the t distribution Case I cr cr 0 Conilder two independent normal random variables say X with mean J1 and variance a and X2 with mean f1 and variance 0 Both the means Pl and Jlz and the variances d and 0 are unknown However suppose it is reasonable to assume that both variances are equal that is cr cr 0 We wish to find a 1001 a confidence interval on the difference in means J1 Jlz Random samples of size Itl and nz are taken nuXJ andXz respectively Let the sample 1 means be denoted X and X and the sample variances be denoted S 1 and S Since both S and S are estimates of the commOn variance 0 we may obtain a combined or pooled estimator of 0 s n 1sl n lsi p nl nz2 1048 r 103 TwoSample Corfiderce Interval Estitration 245 To develop the confidence interval for Jil fLJt note that the distribution of the statistic is the t distribution with 71 ll 2 degrees of freedom Therefore P lat2nrtr2S t tlli2nltllr2 1 a or This may be rearranged as ri1 t2 2S 11 Ia I f 711 J 1049 Therefore a 1001 a twosided confidence interva for the difference in means Pi JlJ is 11 1 J11 fl2 XIX2 ta211 1I2Spi Ivnl A onesided 1001 a lowerconfidence interval on U flz is and a onesided 1001 a upperconfidence interval on flI flz is 1050 1051 1052 In a batch chemical process used for etching printed circuit boards two different catalysts are being compared to dermine whether they require differert emersiol times fo rel1oval of identical quan tities of photoresist material Twelve batches were rnl with catalyst 1 restiung in a sample mean emersion time of x 246 minutes and a sample standard de iation of s 085 minutes Fifteen batches were ron with catalyst 2 resulting in a rcea emersioD time of 221 minutes and a stanw da deviation of S2 098 minutes We will find a 95 confidence interval on the difference in means 246 Olapter 10 Parameter Rtimation PI fI2 assuming that the standard deviations or variances of the two populations are equa The pooled estimate of the COIllllJon vaiance is found using equation 1048 as follows nI 1 s p 122 11085 H4O98 12152 08557 The pooled standard deviationis sf JO8557 0925 Since lQ2n2 til1lZ52i 2060 we may calculate the 95 lowet and upperconfidence limits as and 1 I 24622120600925 112 15 176 minuteS 246 221 2060092511 1 112 15 324 minutes That is the 95 confidence inteval on the difference in mean emersion times is 176 minutes s fll fl 5 324 minutes We are 95 confident tbat catalyst 1 requires an emerslon time that is between 176 minUes and 324 minutes longer than that required by catalyst 2 Case II d d In many siruations it is not reasonable to assume that Wilen this assnmption 1s unwarranted one may still find a 1001 ci confidence intervil for 111 i12 using the fact that the statistic t X X 11 Jl 21 2 js Snz is distributed approximately as t with degrees of freedom given by 2 I 2 Si in S21nz I v 2 5 Jilnzt 1053 t1 nz 1 Consequently an approximate 1001 ci twosided confidence interval for 111 1 when d is 1054 103 TwoSample Confidence mreal Estination 247 Upper lower oDesided confidence limits may be found by replacing the lower upper confidence limit with 0000 and changing cd2 to a 1033 Confidence Lterval on PI 1z for Paired Observations In Sections 1031 and 1032 we developed confidence intervals forthe differeDceiD means where two indepeldent random samples were selected from the two populations of bter est That is Il j observations were selected at random from the first popUlation and a Com pletelY independent sample of rLz observations was selected at random from the second population There are also a number of experimental situations where there are olly 11 dif ferent experimental units and the data are collected in pairs that is two observations are made on each unit For example the journal HWIlmI Faclors 1962 p 375 reports a study in which 14 subjects were asked to park two cars having subscantially differen wheelbases and turning radii The time in seconds was recorded for each car arid subject and the resulting data are shoNn in Table 103 Notice that each subject is the experimental unit referred o earlier Ve wish to obtain a colfidence interval on the difference in nean time to park the two cars say fll LIz In general suppose that the data consist of n pairs Xli X X X X X Both XI and X are assumed to be normally distributed with mean 11 and 11 respectively The random variables within different pairs are independent However because there are two measurements on the same experimental unit the two measurements within the same pair may not be independent Consider the n differences D j X1 XlI D2 X12 Xn D X1r X2r Kow the mean of me differences D say tiD is 110 ED EX X EXI EX 11 f12 because the expected value of Xl X2 is the difference in expected values regardless of whether Xl and X2 are independent Consequently we can construct a confidence interval for til Ji2just by finding a confidence interval on fla Since the differences Df are normally and independently distributed we can use the tdistribution pr0Cedure described in Section TabllO3 TlIile in Seconds to Parallel Park Tvo Automobiles Automobile 2 Difference 370 178 192 2 258 202 56 3 162 168 J6 4 242 414 172 5 220 214 06 6 334 384 50 7 238 168 70 8 582 322 260 9 336 278 58 10 244 232 12 11 234 296 62 12 212 206 06 13 362 322 40 14 298 538 240 248 Chapter 10 Parameter Estimation JO22 to find the confidence interval on uo By analogy with equation 1030 the 1001 a confidence interval on I1D 11 i11 is 1055 where 15 and SD are the sample mean and sample standard deviaoon of the differences D respecovely This confidence interval is valid for the case where j 0 because So esti mates a veX X Also for large samples say n 2 30 pairs the assumption of nOr mality is unnecessary We now return to the data in Table 103 concerning the time for n 14 subjects to parallel park two cars From the column of observed differences d we calculate d 21 and sa 1268 The 90 Con fidence interval for ftn J1 P2 is found from equation 1055 as follows dtl1053Sd nSftD5dtMSOSd L211771126SM ID 5121 17111268 M 479ID 721 otice that he confidence L1terval on PD includes zero This implies that at the 90 level of confi dence the data do not support the ciaim hat the two cars have different mean parking times PI and J1z That is the value PD PI ILl 0 is not inconsistent lit the observed data Note that when pairing data degrees of freedom are lost in comparison to the twosample confidence intervals but typically a gain in precision of estireation is achieved because 8tl is smaller than Sp 1034 Confidence Interval on the Ratio of Variances of Two Normal Distributions Suppose that Xl and X2 are independent nonnal random variables Vlith unknown means fJ and i12 and unknown variances 0 and a respectively We wish tJ find a 1001 a con fidence intenal on the ratio Ja Let two random samples of sizes n1 and YIz be taken on Xl and X21 and let 87 and S denote the sample variances To find the confidence mtervaL we note that the Sar1pling distribution of sia F Sfa is F with 1z 1 and n 1 1 degrees of freedom This distribution is shown in Fig 109 From Fig 109 we see that PF1al2JTrJnr 1 F F anny1JTIt 1 0 or Hence i 1O3 TwoSarple Confidence Interval Estimation 249 Figure 109 The distribution of Frllll Comparing equations 1056 and 1018 we see that a 1001 a twosided confi dence interval for a is sf cf 2 si JliCZ2lllIl5 O s sf FatZilllll 1O57 where the lower 1 aJ2 tail point of the FrtJ distribution is given by see equation 922 1 FI2fIl lnt 1058 We may also construct onesided confidence intervals A 1001 a lowerconfi dence limit on 0 is 0 2 sf laflzlilll 1 O 1059 while a 1001 a upperconfidence interval on aia is 1060 Ei2 Consider the batch chemical etching process described in Example 1020 Recall that two catalysts are being compared to measure their effectiveness in reducing emersioD times for printed circuit boards n 12 batches were run with catalyst 1 and llz 15 batches were rJl with catalyst 2 yield ing 085 mLlutes and 1 09S minutes We willfind a 90 coriidence interval on the ratio of vari3xccs o Froro equation 157 we find that or 12 ar rro9J4JI S2 210051411 of2 02 2 085 03 cr 085 274 098t i 098 029S a s 206 cr2 250 Chapter 10 Paramete Estimation usmgthe factiliatFQ9514H lIFo6IIA If258 039 Since this confdeace interval includes unity we could not claim that the staIldard deviations of the emersion times for the two catalysts are differ ent at the 90 level of confidence 135 Confidence Interval on tbe Difference between Two Proportions Ii there are two proportions of interest say P and P it is possible to obtain a 1001 a confidence interval on their difference PI Pl If two independent samples of size n and n are taken from infinite populations so that X and X are independent binomial random variables mth parameters n11 PI and pz respectively where Xl represents the num ber of sample observations from the first population that belong to a class ofinterest and X represents the number of sample observations from the second population that belong to the class of interes then fit Xn and P2 XIn are independent estimators of p and p respectively Furthermore under the assumption that the normal approximation to the bino mial applies the statistic p 0 p P2 Zr jPl p plp nn is distributed approximately as standard nonnaL Using an approach analogous to that oftlle previous section it follows that an approximate 1001 a twosided confidence inter val forppzis Z 101002102 PI P2 a2i I n rc pI PI 0 2Ip2 p P2 PI p Zo2 nl ll2 1061 An approximate 1001 a lowerconfidence interval for p P2 is Z VlpIpp2IfiL PIPZ a Pl P2 n1 nz 1062 and an approximate 1001 a upperconfidence interval for p p is n Z lpIpYI p P2 Yl P2 a 1 n 1063 19 Consider the data in Example 1017 Suppose that a modification is made in the surface finishing process and subsequently a second random sample of 85 axle shafts is obtained The number of defec tive shaftsin this second sample is 10 Therefore sincet 75 Pt 016 85 and 2 1085 012 we can obtain an approximate 95 confidence interval on the difference in the proportions of defectives produced under the Va processes from equation 1061 as 104 Approximae Confidence Intervals in Maximum Likelihood Estination 251 or 016012196116084 012088 75 S5 r C 016012 196116084 012088 Pl p i 75 ll5 This simplifies to 007 s P Pz 015 This UrervaJ includes zero so based on the sample datl it seems unlikely that the changes made in the strface finish process have reduced the proportion of defective axle shafts being produced 104 APPROXIMATE CONFIDENCE INTERVALS IN ifAXl1VrulVI LIKELmOOD ESTIMATION If the method of maximum likelihood is used for parameter estimation the asymptotic properties of these estimators may be used to obtain approximate confidence intervals Let Ii be the maximum likelihood estimator of fJ For large samples j is approximately normally distributed with mean e and variance V given by the CramerRan lower bouad equation 104 Therefore an approximate 1001 a confiGence interval for ii 1064 Usually the VfiJ is a timction of the unknown parameter e In these cases replace e with i iijJ Recall Example 103 where it was shoW that the maximum likelihood estimato of the parameter p of a Beruoulli distribution is p lIn XI X Using the Crun6Rao lower bound we may Lrl verify that the lower bound for the variance of ft is V 1 P 1 10pXIPjXl 1 J IX lp lp 1 nEX2 IX 2IXl p Ip Ip For the Bernoulli distribution we observe thatEX P and EX p Therefore this last expresson simplifies to 1 1 Vp 1 n P lp This result should not be surprising since we know directly that for the Bernoulli distribution VX VeX plpn Jn any case replacingp in VCP oy p the approxite 1001 a contitlence interval for p is found from equation 1004 to be 252 Chaper 10 Parameter Estimation 105 SIMDLTAcEOUS CONFIDENCE INTERVALS 11 Occasionally it is necessary to construct several confidence intervals on more than one parameter and we wish the probability to be I a LIat all such confidence intervals simultaneously produce correct statements For example suppose that we are sampling from a normal population with unknown mean and variance and we sh to construct con fidence intervals for I and a such that the probability is 1 a that both intervals simul taneously yield correct conclusions Since X and are independent we could ensure this result by constructing 1001 a2 confidence intervals for each parameter separately and both intervals would simultaneously produce correct conclusions with probability 1 ar12I al I a Ii the sample statistics on which the confidence intervals are based are not independ ent random variables then the confidence intervals are not independent and other methods must be used In general suppose that m confidence interVals are required The Bonferroni inequality states that m p all m statements are Simultaneously correct Ia 1La 1065 1 where 1 Xi is the confidence level used in the ith confidence interval In practice we select a yalue for the simultaneous confidence level 1 a and then choose the individual ai such that at a 1Jsually we set a aim Il As an illustration suppose we wished to construct two confidence intervals on the means ofVIo normal distributions such that we are at least 90 confident that both state ments are simultaneously correct Therefore since 1 a 090 we have a 010 and since two confidence intervals are required each of these should be constructed with a a12 01012 005 12 Thatls two individual 95 confidence intervals on 1 and III will simultaneously lead to correct statements with probability at least 090 106 BAESIAN CONFIDENCE INTERVALS r Previously we presented Bayesian techniques for point estimation In this section we will present the Bayesian approach to constructing confidence intervals We may use Bayesian methods to construct interVal estimates of parameters that are similar to confidence intervals If the posterior density for 8 has been Obtained we can con struct an interval usually centered at the posterior mean that contaIns 1001 a of the posterior probability Such an interval is called the 1001 a Bayes interval for the unknown parameter e While in many cascs the Bayes interval estimate for e will be quite similar to a classi cal confidence intenral with the same confidence coefficient the interpretation of the two is very different A confidence interval is an interval that before the sample is taken will include the unknon 8 th probability I a That ls the classical confidence interval relates to the relative frequency of an inreIVlll including 8 On the other hand a Bayes inter val is an interval that contain 1001 a of the posterior probability for e Since the posterior probability density measures a degree of belief about 8 given the sample results L 107 BootStrap Confidence Intervals 253 the Bayes kterval provides a subjective degree of belief about IJ radler tila a frequency interpretation The Bayes interval estimate of e is affected by the sample results but is not completely determined by them Suppose that the random variableXis normally distributed 1m meanjJ and variance 4 T1e value of jJ is mknOVll but a reasonable prior density wocld be normal Villi mean 2 and variance L Tha is We can show that the posterior density for Jt is J2 r I 81 1 eXP1 1 ji nx I L 2 4 j n4 j the methods of Section 1014 Thus the posterior distribution for jJ is normal with mean Sr 4 and variance 4n 7 4 A 95 Bayes interval for jl which is symmetric about the posterior mear would be nX8 2 nX8 2 z2SJzc n4 Vn4 n4 1066 If a raodom sample of size 16 is taken and we5nd that X 25 equation 1O66 reduces TO 152J328 If we ignore the prior information the lassical confidence interval for jJ is We sec tbat the Bayes ilterval is slightly shorter than the classical cor6dence ilterva1 because the prior riormation is equiva1en to a slight increase in lle sample size if ne prior knOWledge was 3sumed 107 BOOTSTRAP CONFIDENCE INTERVALS In Section 1016 we inLoduced the bootstrap technique for estimating the standard error of a parameter 8 The bootstrap technique can also be used to construe a confidence inter val on e For an arbitraI parameter 8 general 1001 a lower and upper limits are respectively L 1001 aI2 percentile of eil Ii u lOOeal2 percentile of e fJ 254 O1apter 10 Parameter Estimation Bootstrap samples can be generated to estimate the values of L and U Suppose B bootstrap samples are generated and ilO O and i are calculated From these estimates we then compute the differences 8 80 j 8 8 arrange the dif ferences in increasing order and find the necessary percentiles I OO 1 aJ2 and I OO aJ2 for L and U For example if B 200 and a 90 confidence interval is desired then the 1001 01012 95th percentile and the 10001012 5th percentile would be the 190th difference and the 10th difference respectively Al electronic device consists of four components The time o failure for each component follows ar exponential distribution and the conponents are identical to and bdependen of one another The elcctronic device will fail only after all four components have failed ine times to failure for the elec tronic components have been collected for lS such devices The total times to failure are 787778 135260 68291 473746 162033 275387 282515 385826 354363 802757 503861 813155 422532 339970 574312 It is of interest to construct a 90 coJiidence mterval on the exponential paramcter L By definition Lle sum of r indepcadent and identically distrlbuted exponential radom variables follows a gamma I distribution and is defined as gammar It Therefore r4 bllt A needs to be estirlated A bootstrap estimate for A can be found using the technique given in Section 1016 Using the timetofailure data above we find the average time to faibrc to be x 42545 1he mean of a gamma distribution is EX riA 2I1d A is calculated for each bootstrap sample Ruuuing MnilablS for B 100 bootstraps we found that the bootstrap estimate X 00949 Using the bootstrap estimates for each sample the dif ferences can be calculated and some of the calculations are showr in Table 104 Then the 100 differences are arranged in increasing order the 5th percentile and the 95th per centiles turn out to be 00205 and 00232 respectively Therefore the resulting confidence limits are L 00949 00232 00717 U 00949 0205 01154 We are approximately 90 confident that the rue value of Alles bet1cen OfYl 17 ad 01154 Figure 1010 displays the histogram of the bootstrap estimates 1 while Fig lQll depicts the Cifterences A T The bootstrap esfJIlates are reasonable when the estimator is unbiased and the stmdmd eITO is approximately constant Table 104 Bootstrap Estimates for Example 1026 Sample i X 1 2 3 100 0037316 0090689 0096664 0090193 OO75392 o041660 00018094 0046623 20 o 108 Other Interval Estimation Problems 255 r r I r ilh D 0065007500850095010501150125 0135 0145 0155 lambda Figure 1010 Histogram of the bootstrap estimates i 20 o 0 r r rr r Ilh 003 002 001 000 001 002 003 004 differences Figure 1011 Histogram of the differences J 108 OTHER INTERVAL ESTIMATION PROBLEMS 1081 Prediction Intervals So far in this ehapter we have presented interval estimators on population parameters sueh as the mean 11 There are many situations where the praetitioner would like to predict a sin gle future observation for the random variable of interest instead of predicting or estimat ing the average of this random variable A prediction interval ean be eonstrueted for any single observation at some future time Consider a given random sample of size n Xl X2 Xn from a nonnal popUlation vrith mean Jl and varianee cT Let the sample average be denoted by X Suppose we wish to predict the future observation Xnl Sinee X is the point predietor for this observation the prediction error is given by XlI1 X The expected value and variance of the prediction error are 256 Chapter 10 Parameter Estimation aDd Since Xltt1 and X are independent normally distributed random variables the prediction error is also Donnally distributed and iXXO Z 11 I t 1 ol1j n and is standard nonnal If cT is unknown it can be estimated by the sample variance S and then follows the distribution with nl degrees of freedom Following the usual procedure for constructing confidence intervals the twosided 1001 a prediction interval is By rearranging the inequality we obtain the final fonn for the twosided 1001 a pre diction interval I l i I 2 I 2 XtallllJS llXIlSXtai2IIIS 1 1067 The lower onesided 1001 a prediction interval on Xn is given by f1 XtlS ll5Xrl 1068 The upper onesided 1001 a prediction interval onX is given by 1069 iEiiiIi Mamum forces experienced by a transport airqaft for an airline on a particular route for 10 flights are in units of graviry g 115123156169171183183185190191 I i 108 Other Interval Estimation Problems 257 The sample average and sample standard deviation are caiculared to be i 1666 and s 0273 respectively Rmay be of importance to predict the next maximum force experienced by the LCrnft Since CcG9 2262 the 95 prediction intervu onX 1 is 666 t2IJo27321 1 I XII 1666 t21 JOZ731 I 10 I W L018X 2314 1082 Tolerance Intervals As presented earlier in this chapter confidence intervals are the intervals in which we expect the true population parameter such as Il to lie In contrast tolerance intervals are inervals in which we expect a percentage of the population values to lie Suppose that X is a normally distributed random Yariable with mean fl and variance 02 We would expect approximately 95 of all values of X to be contained within the interval fL 1645a But what if jJ and crare unknown and must be estimated Using the point esti mates x and s for a sample of size n we can construct the intervalX 1645s UnfOrtUnate y due to the variability in estimating fJ and a the resulting interval may contain less than 95 of the values In this particular instance a value larger than 1645 vill be needed to guat antee 95 coverage when using pDint estimates for the population parameters We can con struct an interval that will contain the stated percentage of population values and be relatively confident in the result For example we may want to be 90 confident that the resulting intenra coverS at leat 95 of the population values This type of interval is referred to as a tolerance interval and can be constructed easily for various confidence levels In general for 0 q 100 the twosided tolerance interval for covering at least q of the values from a normal population with 1001 a confidence is i ks The value k is a constant tabulated for various combinations of q and 1001 a Values of k are given in Table XlV of the Appendix for q 90 95 and 99 and for 1001 a 90 95 and 99 The lower onesided tolerance interval for covering at least q of the values from a nOOl1al population with 1001 a confidence is i S The upper onesided tolerllllCe interval for covering at least q of the values from anormaI population with 1001 a confidence is x ks TariOtlS values of k for onesided tolerance intervals were calculated using the technique given in Odeh and Owens 1980 and are proided in Table XlV of the Appendix Reconsider the maximum forces for the trarspon nLcraft in ExruLple 1027 A twosided toleance interval is desired that would cover 99 of all maximum forces with 95 confidence From Tabk XIV Appendix with I a 095 q 099 and n la we ftnd that k 4433 The sample aver age and sample standard deviation were calculated asx 1666 ands 0273 espectively The result ing tolerance intervalls then 1666 44330273 or 04562876 Therefore We conclude that we are 95 confident that at leaSt 99 of all maximum forcs would lie between 0456 g and 2876 g 258 Chapter 10 Parameter Estimation It is possible to construct nonparametric tolerance intervals that are based on the extreme values in a random sample of size n from any continuous population If P is the minimum proportion of the population contained between the largest and smallest obser vation with confidence 1 a then it can be shown that n11 n 11 a Further the required n i approximately I 1 P X4 n 2 lP 4 1070 Thus in order to be 95 certain that at least 90 of the population will be included between the extreme values of the sample we require a sample of size nl 9488 46 2 01 4 Note that there is a fundamental difference between confidence limits and tolerance limits Confidence limits and thus confidence intervals are used to estimate a parameter of a popu1ation while tolerance limits and tolerance intervals are used to indicate the lim it between which we can expect to find a proportion of a population As n approaches infin ity the length of a confidence interval approaches zero while tolerance limits approach the corresponding quantiles for the population 109 SlThlMARY This chapter has introduced the point and interval estimation of unknown parnmeters A number of methods of obtairring point estimators were discussed including the method of maximum likelihood and the method of moments The method of maximum likelihood usu ally leads to estimators that have good statistical properties Confidence intervals were derived for a variety of parameter estimation problems These intervals have a frequency interpretation The Nosided confidence intervals developed in Sections 102 and 103 are summarized in Table 105 In some instancet onesided coclidence intervals may be appro priate These may be obtained by setting one confidence limit in the twosided confidence interval equal to ttie lower or upper limit of a feasible region for the parameter and using a instead of cd2 as the probability level on the remaining upper or lower contidenco limit Confidence intervals using a bootstrapping technique were introduced Tolerance intervals were also presented Approximate confidence intervals in maximum likelihood estimation and simultaneous confidence intervals were also briefly introduced Table 10S Summary of Confidence Interval Procedures Problem Typ Point Estimator TwoSlded 1001 a Confidence Interval Mean II of a normal distribution varianlc dl known X O 0 Difference in means of two nonnal distributions 11 and 11 variances O and a known Me m It of a normal distribution variance ak unknown Difference in means of two normal distributions PJ lkJ variance a f unknown ljfference in means of two normal distributions for paired samples Po III Jit Variance f1 of a normal distribution Ratio of the variances a oflwo nomulI dislribulions Proportion or parameter of a binomjal distribution p Difference in two proportions or two binomial parameters PI P2 XIX X XIX Jj sJ fJ Pi P2 XI Zajl II 5 XIX1 z111 1 11 111 X tall 1 f s J1 s X t111 s In XI X2 12 I I 1 5111JJ2 sXj Xl lj2t Sp n1 V fll n1 whereS r1Sl1l2l p nn2 DtutHSVJn OJ1 S 15 tl 1 SDj s s Sq 01 S2 LsiF cr s fJj111 2 NlPc cz FHj P on p p at n n fEliJ p p Zl 5 p Pi PI Pi LJ n II o J g 260 Chapter 10 Parameter Estimation 1010 EXERCISES 101 Suppose we have a random sample of size 2n from a population denoted X and EX P and VX cf Let be two estimators of JL Vlhich is the better estimator of j1 Eplain you choice 102 Let Xl X X1 denote a a1dom sample from a population having mean j1 and variance fl Con sider the following estintators of jJ 8 l C Is either estmator Ulbiased hich estimator is bet ter In what sense is it better 103 Suppose tha 8 and are estimators of the paramet 8 Weknow thatEe eEfI el2 Vee 10 and 8 4 Arhid estilator is berer In what sense is it better 104 Suppose tha lil and t are estimators of e We ow lb EG Ee e EB e Vee 12 Veil 10 and Ei 0 6 Compare these three estimators Whicb do you prefer by 105 Let three random samples of sizes n1 10 n2 8 and n 6 be taken from a populatioa willi mean J1 and variance a Let S and S be the sampe vari ances Show t1a 82 lOSt 8Sj 6si 24 is an unbiased estinIaor of dl 106 Best Linear Cnbiased Estimators An etima tor e is called a mear estimator if it is a linear combi nation of the observations in the sample 8is called a best linear unbiased cstLl1ator if of all linear func dons of the observations it both is urbiased and has minllnum variance Show that the sample mean X is the bes linear unbiased estimator of the population meanp 10M7 Find Le tlaXimum likelihood estimator of the parameter c of the Poisson distribution based on a random sample of size n 108 Find the estinator of c in the Poisson distribu tion by the method of moments based on a random sample of size n 109 Fmd the maximum likelihood etimator of the parameter A in the exponential distribution based on a random sample of size L 1010 Find the estimator of in the exponential dis tribudon by the method of moments based on a ran dom sample of size n 1011 Find moment estL1lators of the parameters r and of the gamma distribution based on a random sarrple of size n 1012 Let X be a geometric random variable with parameter p Find an estimator of p by the method of moments based on a random sample of size n 1013 Let X be a geometric random variable with parameter p Fmd the maximum likelihood estimator of p based Or a random sample of size n 1014 Let X be a Bernoulli random variable with parameter p Find an esimator of p by the method of moments based on a random sample of size n 1015 Let X be a binorrial random vaiable with parameters n known and p Find an estimator of p by the method of moments based on arandom sample of size N 1016 Let X be a binornia random variable with parameters n and p both unknown Fmd esimators of n and p by the method of momen based on a random sample of size N 1017 Let X be a binomial random vaable with PlLrameters n unlo1own and p Find the maximum likelihood estimator of p based on a random sample of size N 1018 Set up the likelihood function for a random sample of size n from a Weibull distribution What difficulties would be encountered in Obtaining the maximum likelihood estimators of the three parame ters of the Weibull distribution 1019 Prove thatif j is an unbiased estimator of e and iflirclfe 0 then 8is a consistent estimator of e 1020 Let X be a random mabIe with mean jJ and varianee d Given two random samples of sizes n1 and n2 with sample means XI and Xl respetive1y show hat 0 a 1 is an unbiased estimator of J1 Assuming X and Xz to be ffidependent find the value of a that minimizes the vaiance of X 1021 Suppose that the random variable X has the prohability distribution fix 1x1 ox 1 0 othise Let X Xz Xii be a random sample of size fl Find the maximum likeJihood estimator of Yo I i 1022 Let X have lJe truncated on the left at x expo nential distribution fix A expAX x x x 0 0 otherwise Let X X2 X be a random sample of size n Find the Um likelihood estimator of 123 Assrme that t in the previous exercise is nmvn but xf is unknown Obtain the maximum like lihood estimator of At 1124 Let X be a random variable whh mean f1 and l2Iialce ril and let X XZ7 X be a random sample of size n from X Show thaI the estimator G lr K i Xil Xi is unbiased for an appropriate choice for K Find the appropriate value for K 1025 Let X be a normally distributed random vari able with mean 11 and variance UZ Assume that rr is known and Jl unknown The prior density for u is asSUJUed to be normal with mean Jio ani variance Determile the posterior density for 11 given a andom sample of sizet from X 126 LetAZbe normilly distributed Vith known mean Jl a1d unknown variance al Ale that the prior dcllSity for 11 cr is a ganuna distribution with parame w ters m 1 and m Detmrine the pOsterior density for ller given a random sample of size n from X 1021 Let X be a geometric random variable Vith pater p Suppose we assume a beta distributon with parameters a and b as the prior density for p Detennine the posterior density for p given a random slII1ple of size n from X 1028 Let X be a Bernoulli random variable Vllth parameter plf the prior density for p is a beta distri bution with paruneters a and b detennine t1e poste rior density for p given a nmdom sample of size n from X 1029 Let X be a Poisson random variable with parameter A The prior density for A is a ganma dis tribution with PiIatnct5 m 1 and m 1 Deter mine the posterior density for A given a random sample of sze n from X lQ3Q Suppose that X Nu 40 and let the prior density for f1 be N4 8 For a random sample of size 25 the value x 485 is obtained Vhat is the Bayes estiotate of JL assuling a squarederror loss 1031 A process manufactures printed circult boards A locating notch is drilled a distance X from a com ponent hole on the board The distance is a ta1dom aiable X Np 001 The prior density for 11 uni form between 098 and 120 inches A random sample of size 4 produces the value X LOS Assuming a squarederor loss determine the Bayes estimate of 11 1010 Exercises 261 1032 The time behleen failures of a miliiig machine is exponentially distributed with parameter A Sup pose we assume an exponential prior 01 A with a mean of 3000 hOllrs Two machines are observed and the average tite betvleen failures is i 3135 hous Assumiag a squarederror loss detemUle the Bayes estimate of 1033 The weight of boxes of candy is noonally dis tributed will mean J1 and variance It is reasoaable to assume a prior density for il that is normal Vtith a mean of 1O pounds and a variance of B Deermine the Bayes estir1ate of jJ gjven that 11 sample of size 25 produces i 005 pomds If boXes 1at weigh less than 995 pounds are defective what is the probability that defective boxes Viill be produced 1034 The number of defects that occur on a silicon wafer used in integraed circult manufacturing is known to be a Poisson random variable with patame let A Assume that the prior density for A is eAl1onen tial Vith a parameter of 025 A total of 45 defects were observed on 10 wafers Set up an integral that defines a 95 Bayes interva for t rna difficulties would you ercounter in evaluating is inregral 103S The random variable X has a density function fxW2 0x8 e and the prior density for e is Jel oe1 a Find th poslerior density for e assUttling n 1 b Find the Bayes estimator for e assuming he loss function ee e ee et and l l36 Let X follow the Bcrnotlli distribution with parameter p Asslme a reasonable prior density for p 0 be ftp 6plp 0 Opl othenvise If the loss function is squared error find the Bayes estit2tor of p if one observation is available If 1e loss funcrion is fPp 2p p find the Bayes estimator of p for n L 1037 Consider the confidence interval for Jl with known standard deviation ct X Z1 il X ZryFn where XI az a Let a 005 aad find the interval for a a a2 0025 ow find the interval for the case Xl 001 and 004 Wnich interval is shorter Is there any advantage to a symmetric carr fidence interval 262 Chapter 10 Parameter Estimatinn 1038 When Xl X2 X ae indepedent Poisson random variables each With pa1UDeter A and when n is relatively large the sample mean X is approxi mately normal 1th mean A and variance Nn a What is the distribution of the statistic XI i b Use the results of al 10 fud a 1001 a confi dence interval for 1039 A maDJfacturer produces piston rings for an automobile engjne It is known hat ring diameter is approximately normally distributed and bas standard deviation Cf 0001 mmA random sample of 15 rings has a mean diameter ofxc 74036 mm a Construct a 99 twosided confidence iutervalon the mean piston ring diameter b Construct a 95 lowerconfidence limit on the mean piston ring diameter 1040 The life in houts oia 75Wlight bulb is known to be approximately normally distribQted With a stan dard deviation CJ 25 hours A random sample of 20 bulbs has a mean life of x 014 houS a Construct a 95 twosjded confidence interval on the mean life b ConStIJc a 95 lowerconfidence interval on the mean life 1041 A civil engineer is analyzing the compressive strength of concrete Compressive strength is approx imately normally distributed with a variance cf1 1000 psi2 A random sample of 12 specimens has a mean compressive strength of x 3250 psi a Construct a 95 twosided confidence interval on mean compressive strength b Construct a 99 tVlosided confidence interval on mean compressive strength Compare the mdth of this confidence interval with the width of the one found in pat a 1042 Suppose that in Exercise 1040 we wanted to be 95 confident that the error in estimating the mean life is less than 5 hours What sample size should be used 1043 Suppose that in Exercise 1040 we wanted the total width of the confidence interval on mean life 0 be S hol1S hat sanple size should be used 1044 Suppose tbat in Exercise 1041 it is desired to estimate the compressive strength with an CIror that is less than 15 psi Vlhat sample size is required 1045 Two machines are used to fill plastic bottles mth dishwashing deterge1t The statdard deviations of fill volume are ttown to he 01 015 fluid ounces and G 018 fluid ounces for the tvo machines respectiely Two random samples of nj 12bottles from machine 1 and 10 bottles from machine 2 are selected and tie sample mean fill volUJXes are Xl 3087 fluid ounces andJ2 3068 fluid ounCes a Construct a 90 twtrsided confidence interval on the mean difference in fill volume b Construct a 95 twosided confidence interval on tk mean difference il1 fill volume Compare the vidth of this interval to the width of the interval in part a c Construct a 95 upperconfidence interval on the mean difference in fill volume 1046 The burning rates of tviO different solidfuel rocket propellants are bcirtg studied It is known that both propellants have approximately the same standard deiation of burning rate that is 0 OJ 3 cmIs Two random samples of n1 20 and 1t 20 speens are tested and the sample mean burning rates are Xl 18 cmJs and 24 cms Construct a 99 confidecce interval on the mean differenceu burning rate 1047 Two different formulations of a leadfree gaso line are being tested to study their road octane mlJllw bers The variance of road octane number for formulation 1 is a 15 and for formulation 2 it is a 12 Two random samples of size Il l 15 and liz 20 are tested and the mean road octane Ullmbers observed are XI 896 and Xz 925 COlstruct a 95 twosided confiderlee interval on the difference in nean road octane number 1048 The compressive stIength of concrete is being tested by a civil engineer He tests 16 specimens and obtains the following dara 2216 2237 2225 2301 2318 2255 2250 2238 2249 2281 2275 2300 2204 2263 2295 2217 a Construct a 95 twosided confidence inetal oc the mcan strength b Construct a 95 lowerconfidence interv on the mean strength c Comtrucra95 twOsided confidence interval on tte mean slle1gth asstlTing that a 36 Compare this interval ith the one from part a d Construct a 95 tVQsided prediction interva1 for a single compressive strength e Construct a tVosided tolerance interval that would cover 99 of all compressive strengths with 95 confidence 1049 An article in Annual Revievs Material Research 200 1 p 291 presents bond strengths for various energetic materials explosives propellants f t I I and pyrotechnics Bond strengths for 15 such materi als ateshown below Constxuct a twosided 95 con fidence interval on the mean bond strelgth 323312300284283261207183 180179174167167157 J20 1050 The wall thlckness of 25 glass 2liter bottles was measured by a qualitycontrol engineer The salll pIe mean was x 405 mm and the sample stardard deviation was s 008 mm FInd a 9000 lowerconfi dence inerVa on the mean wall thickness 1051 An industrial ergineer is inerested in estimat ing the mean time required to assemble a printed cir cuit board How large a sample is required if the engineer wishes to be 95 confident that the erro in estiJnating the mean is less than 025 minutes The standard deviatou of assemOly time is 045 minutes lO52 A random sample of size 15 fron a norma population has mean x 550 and variance S2 49 Find the following a A 95 twosided confidence interval on jl h A 95 lowerconfidence interval on u c A 95 upperconfidece intm3l o1 d A 95 twosided prediction intm3l for a single observation e A twosided tolerance interval that would eover 90 of all observations with 99 confidence 1053 An article in Computers in Cardiology 1993 p 317 presents the results of a heart stress test in which the stress is induced by a particlliar drug The heart rates in beats per minute of nine male patients after the drug is administered are recorded The aver age heartrute was found to be x 1029 bpm with a sample standare deviation of s 139 bpm Fmd a 90 eonfidence interval on the mean heart rate after he drug is adtt1hiistered 10R54 Two independent random samples of sizes n1 18 and lIz 20 are taken from two normal popula tions The sample means are Xl ZOO and Xz 190 We know that the variances are 02 15 and O 12 Fmd the folJoWng a A 95 rnosided confidence interva on1 fL b A 95 lowereonfidence interval Ot III fLl e A 95 uppercoufidenee interval on Jil JL lOS5 The output voltage from two different types of transformers is being investigated Ten transfonners of each type are seleeted at random and the voltage measured The sample zeans 1213 voltS and 1205 volts We know that the variances of output voltage fur the two types of transformers are O 07 and 0 08 respectively ConstrUct a 95 1010 Exercises 263 twosided confidence interval on the difference in mean Voltage lO56 Random samples of SlZC 20 wcC ldwn from two independent normal populations The saxrp1e means and standlrd deviations were XI 220 1 18 215 and 1 15 Assuming that er 0 find the following a A 95 twosided confidence interla on1l 1 b A 95 upperconfidence irtcrval ou Ji fLl c A 95 lowerconfidence interval 01 PI 10 10 51 The diameter of sreel rods manufactured on two different extrusion machines is being investi gated Two random samples of sizes n l 15 alld n 18 are selected and the sampe mea1S and sample variances are i 873 s 030 X1 8680 and 034 respectively Assuming that oi ai construct a 95 twosided corlictence interval on the difference in mean rod diameter 1058 Random samples of sizes n 15 and 2 10 are drawn from two mdependent Donnal populations The sample means anc variances are x 300 if 16 325 s 49 Assuming mar o construct a 95 twosided confidence interval On 11 P lO59 Consider the data in Exercise 1048 Construct the following a A 95 twosided confidence interval on fl b A 95 loweNorJidence interval on d c A 95 uppercoDfictence interval On f 1060 Consider the data in Exercise 1049 Construct the following a A 99 twosided confidence intenal 00 f b A99 lowerconfidclCeinterval on cr c A 99 upperconfideIee interval on cr 1061 Construct a 95 twosided confidence interval on he variance of the wall thickness data in Exercise 1050 1062 In a random sample of 100 light bulbs the sampe standard deviation of bulO life was found to be 126 houtS Compute a 90 apperconfcence interval on the variance of bulb life 1063 Consider the data in Exercise 1056 Construct 3 95 tvlosided eonfidence interval on the rajo of the population variances ti 1064 Consider the data in Exetcise 1051 Construct the following a A 90 twosided confidence interval on a b A 95 twosided confidence interval on 071a Compare the width of this interval 1ith the tvidth of the interval in part a 264 Chapter 10 Parameter Estimation c A 90 loweNonfidenceinterval On o d A 90 upperconfidence interval on a 1065 Construct a 95 twosided confidence interval on the ratio of the variances Jo using L1e data in Exercise 1058 10i6 Of 400 randomly selected motorists 48 were found to be uninsured Construct a 95 twosided confidence interval on the uninsured rate fer lnOtonS 1CJ67 How large a sampc would be required in Exer else 1066 to be 95 confident that ilie error in esti mating the uninsured rate for motorists is less than 0037 10M6S A manufacturer of electronic calculators is interested in estimating the fraction of defective units produced A random sample of 8000 calculators con tains 18 defectives Compute a 99 upperconfidcnce intcrval on ilie fraction defective 1069 A study is to be conducted of the percentage of homeowners who own at least two television sets How large Ii sample is required if we wish to be 99 confident that the error in estinating this quantity is less than 001 1010 A study is conducted to determine whether there is a significant difference in union membership based on sex of the person A random sample of 5000 factoryemploycd men were polled and of this group 785 were members of a union A random sample of 3000 facoryemployed women were also polled and of this group 327 were members cf a union Con struct a 99 confidence interval on the difference in proportions PI P2 1011 The fraction of defective product produced by two production lines is being analyzed A random sample of 1000 units from line 1 has 10 defectives while a random sample of 1200 units from line 2 has 25 defeetives Fmd a 99 confidence interval on the differclce in fraction defective produced by the two lines 1072 The resultS cf a study on powered wheelchair driving performance were presented in the Proceed tngs of the IEEE 24th Annual Northeast Bfoengineer ing Conference 1998 p 130 In this study the effects of two types of joysticks force sensing FSJ and position sensing pSJ on pover wheelchair control were investigated Each of 10 subjects was asked to test both joysticks One response of intert is the time in seconds to complete 3 predeterrcined couzse Data typical of this type of experiment ax as follows Subject PSJ FSI 259 334 2 302 374 3 337 480 4 276 305 5 333 278 6 346 275 7 331 369 8 306 311 9 305 271 10 254 380 Find a 95 confidence interva on t1e difference in mean completion times Is there any indication that one joystick is preferable 1073 The manager of a fleet of automobiles is test ing two brands of radial tires He assigns one tire of each brand at dom to the tvo rear wheels of eight caS and runs the cars U1til the tires wear out The data in kilometers 3C shown below Car Brand I Brand 2 1 36925 34318 2 45300 42280 3 36240 35500 4 32100 31950 5 37210 38015 6 48360 47800 7 38200 37810 8 33500 33215 Find a 95 confidence ilrerval on he difference in mean mileage Which brand do you prefer 10 74 Consider the data in Exercise IQ50 Find ecn fidenee intervals on Jl and 72 such that we are at least 90 confident that bofuinterai sixrulitaneously lead to correct conclusions 1075 Consider the data in Exercise 1056 Suppose that a dom sanpk of size n 15 is obtained from a third norwU population Vtit1 Xj 205 and 5J 12 Find twosided confidence intervals on 1 IJ fll flJ and iJJ flJ such that the probability is at least 095 that all three intervals simultaneously lead to comet c1c1sicns 1076 A random variable X is normally distributed with mean fland variance a1 10 The prior density for J1 is uniform between 6 and 12 A random sample l of size 16 yields x 8 Construct a 90 Bayes inter va for fL Could you reasonably accept the hyPothesis that J1 9 lO 77 Let X be a noIIllilly distributed random Yari able with mean fJ 5 and unknown variance d2 The prior dcrsity for 1112 is a gamma distribution with parameters r 3and A 10 Determine he posterior 1010 Exercises 265 density for lIcrIf a random sample of size 10 yelds Zx 42 492 detenrlne me Bayes estimate of lJO asstltling a squarederror loss Set up aa integral that defines a 90 Bayes interval for 162 1078 Prove that if a squaredecror loss function is used the Bayes estimator of e is the lUcan of the pos terior distribction for 8 Chapter 11 Tests of Hypotheses Many problems requLe that we decide whether to accept or reject a statement about some parameter The statement is usually called a hypothesis and the decisionmaking procedure about the hypothesis is called hypothesis testing This is one of the most useful aspects of statistical inference since many types of decision problems can be formulated as hypothesistesting problems This chapter will develop hypotheistesting procedures for severa important situations 111 INTRODUCTION 1111 Statistical Hypotheses 266 A statistical hypothesis is a statement about the probability distribution of a random vari able Statistical hypotheses often involve one or more parameters of this distribution For example suppose that we axe interested in the mean compressive strength of a particular tyPe of concrete Specifically we are interested in deciding whether or not the mean COm pressive strength say 1 is 2500 psi We may express this fonnally as Ho 1 2500 psi HI 1 2500 psi 111 The statemcntHo 1 2500 psi in equation 111 is called the null hyporhesis and the state ment HI J1 2500 psi is called the alternative hypothesis Since the alternative hypothesis specifies values of 1 that could be either greater than 2500 psi or less thaI 2500 psi it is called a twosided alternative hypothesis In some situations we may wish to formulate a onesided alternative hypothesis as in Ho 1 2500 psi HI 1 2500 psi JJ2 It is important to remember that hypotheses are always statements about the population or distribution under study not statements about the sample The value of the population parameter specified in the null hypothesis 2500 psi in the above example is usually deter mined in one of three ways First it may result from past experience or knowledge of the process or even irom prior experimentation The objective of hypothesis testing then is usu ally to determine whether the experimental situation has changed Second this value may be determined from some theory or model regarding the process under srudy Here the objective of hypothesis testing is to verify the theory or modeL A third situation arises when the value of the population parameter results from external considerations such as design or engineering specifications or from contractual obligations In this situation the usual Objective of hypothesis testing is conformance resting 111 Introdcccion 267 We are inrerested in making a decision about the truth or falsity of a hypothesis A pro cedure leading to such a decision is called a test of a hypothesis Hypothesistesting proce dures rely on using the information in a random sample from the population of interest If this information is consistent with the hypothesis then we would conclude that the hypoth esis is true however if this information is inconsistent with the hypothesis we would con clude that the hypothesis is false To test a hypothesis we must take a random sample compute an appropriate test sta Listie from the sample d and then use the information contained in this test statistic to make a decision For example in testing the null hypothesis concerning the mean com pressive strength of concrete in equation 11 1 suppose that a random sample of 1 0 concrete specimens is tested and the sample mean i is used as a test statistic 1f x 2550 psi or ifi 2450 psi we will consider the mean compressive strength of this paticular type of con crete to be different from 2500 psi That is we would reject the null hypothesis Ho j1 2500 Rejecting Ho implies that the alternative hypothesis HI is true The set of all possi ble values of that are either greater than 2550 psi or less than 2450 psi is called the criti cal region or rejection region for the testA1ternatively if 2450 psi sx S 2550 psi then we would accept the null hypothesis Ho j1 2500 Thus the interval 2450 psi 2550 psi is called the acceptance region for the test Note that the boundaries of the critical region 2450 psi and 2550 psi often called the critical values of the test statistic have been deter mined somewhat arbitrarily In subsequent sections we will show how to construct an appropriate test statistic to determine the critical region for several hypothesistesting situations 1112 Type I and Type II Errors The decision to accept or reject the null hpomesis is based on a test statistic computed from the data in a random sample nen a decision is made using the informationin a ran dom sample this decision is subject to error Two kinds of errors may be made when test ing hypotheses If the null hypothesis is rejected when it is true then a type I ettor has been made If the null hypothesis is accepted when it is false then a type It error has been made The situation is described in Table ll The probabilities of occurrence of type I and type II errors are given special symbols t P type I error P reject HOIHo is true f3 P type II error P accept HolHo is false Sometimes it is more convenient to work with the power of the test where Power 1f3 P rejectHolHo is false 113 114 115 Note that the power of the test is the probability that a false null hypothesis is correctly rejected Because the results of a test of a hypothesis ae subject to error we cannot 1Jrove or disprove a Statistical hypothesis However it is possible to design test procedures that control the error probabilities t and f3 to sitably small values Table 111 Decisions in Hypothesis Testing leptH RejectH is True NQ error lYre I error is False Type II etro No error 268 Cpter II Tests of Hypotheses The probability of type I error ais often called the significance level or size of the teSt In the concretetesting example a type 1 error would occur if the sample meanx 2550 psi or if 2450 psi when in fact the true mean compressive strength 11 2500 psi Genernlly the type I error probability is controlled by the location of the critical region Thus it is USu ally easy in practice for the analyst to set the type I error probability at or near any desited value Since the probability of wrongly rejecting Ho is directly controlled by the decision maker rejection of Ho is always a strong conclusion Now suppose that the null hypothesis Ho J1 2500 psi is false That is the true mean compressive strength J1 is some value other than 2500 psi The probability of type IJ error is not a constant but depends on the true mean compressive strength of the concrete If jJ denotes the true mean compressive strength then 31 denotes the type IJ error probability corresponding to J1 The function 31 is evaluated by finding the probability that the test statistic in this case Xl falls in the acceptance region given a particular value of j1 We define the operating characteristic curve or OC curve of a tcst as the plot of 31 against J1 An example of an operating characterisric curve for the concretetesting problem is shown in Fig 111 From this curve we see that the type II error probability depends on the extent to which He J1 2500 psiis false For example note that 32700 32600 Thus we can think of the type II error probability as a measure of the ability of the test procedure to detect a particular deviation from the noll hypothesis He Small deviations are harder to detect than large Ones We also observe that since this is a twosided alternative hypothesis the operating characteristic curve is symmetric that is 32400 32600 Futtherrnore when 11 2500 the probability of type II error 3 I a The probability of type II error is also a function of sample size as iliustrated in Fig 112 From this figure we see that for a given value of the type 1 error probability a and a given value of mean compressive strength the type 1I error probability decrcas as the sam ple size n increases That is a specified deviation of the true mean from the value specified in the null hypothesis is easier to detect for larger sample size I than for smaller ones The effect of the type I error probability a on the type IJ error probability fl for a given sample size n is illustrated in Fig 113 Decreasing acauses f3 to increase and increasing a causes 3 to decrease Because the type IJ error probability 3 is a function of both the sample size and the extent to which the null hypothesis Ho is false it is customary to think of the decision to accept Ho as a weak conclusion unless we know that 8 is acceptably small Therefore rather than saying we accept Ho we prefer the terminolQgy fail to reject Ro Failing to reject He implies that we have not found sufficient evidence to reject HOI that is to make a strong statement Thus failing to reject Ro does not necessarily mean that there is a high 00 t L I 24CO 260C I I 23GO2700 ttL Li OGC 2300 2400 2500 7Co Figure 11 1 Operating characteristic curve for the concrete testing example 111 btroducton 269 000 L 2500 L Figure 112 The effect of sample size on the operating characteristic curve 100 000 2500 Figure llR3 The effect of type I error on the operating characteristic curve probability that Ho is true It may imply that more data are required to reach a strong cOn elusion This can have important implications for the formulation of hypotheses 1113 OneSided and TwoSided Hypotheses Because rejecting Ho is always a strong conclusion while failing to reject Ho can be a weak conclusion unless f3 is knovro to be small we usually prefer to construct hypotheses such that the statement about which a strong conclusion is desired is in the alternative hypothe siI H Problems for wbkh a wosided altemative hypothesis is appropriate do not really present the analyst with a choice of formulation That is if we wish to test the hypothesis that the mean of a distribution l equals SOme arbitrary value say u and if it is important to detect values of the true mean l that could be either greater than 110 or less than 110 then One must use the tlosided alternative in He l 110 HI l u fany hypothesistesting problems naturally involve a onesided alternative hypothesis For example suppose that we want to reject He only when the true alue of the mean exceeds 110 The hypotheses would be Ho l 110 H l u 116 270 Chapter 11 Tests of Hypotheses This would imply that the critical region is located in the upper tail of the distribution of the test statistic That is if the decision is to be based on the value of the sample mean XI then we would reject Ho in equation 116 ifi is too large The operating characteristic curve for the test for this hypothesis is shown in Fig 114 along with the operating characteristic curve for a twosided test We obsene that when the true mean jJ exceeds JifJ Le when the alternative hypothesis Hi f1 JifJ is true the onesided test is superior to the twosided test in the sense that it has a steeper operating characteristic curve When the true mean jJ JifJ both the onesided and twosided tests are equivalent However when the true mean j1Is less than 4 the two operating characteristic curves differ If j1 llw the twosided test has a higher probability of detecting this departure from f1o than the onesided test This is intu itively appealing as the onesided test is designed assuming either that fJ cannot be less than f1o or if IJ lS less than f1o that it is desirable to accept the null hypothesis In effect there are two different models that can be used for the onesided alternative hypothesis For the case where the alternative hypothesis is HI Jl J1o these two models are and H 0 jJ f1o Hoc f1o Ho IJ f1o Ht IJ f1o 117 118 In equation 117 we are assnming that jJ cannot be less than I1rr and the operating charac teristic curve is undefi1ed for values of j1 Uo In equation 118 we are assuming that fJ can be less than f1o and that in such a situation it would be desirable to accept Ho Thus for equa tion 118 the operating characteristic curve is defined for all values of jJ J1o Specifically if IJ 11u we have Jrjl I afp where afp is the significance level as a function of J1 For situations in which the model of equation 118 is appropriate we define the significance level of the test as the maximum value of the type 1 error probability a that is the value of a at J1 ikJ In situations where onesided alternative hypotheses are appropriate we will usually write the null hypothesis with an equality for example Ho jJ f1o This will be interpreted as including the cases Ho jJ f1o or Ho u 11u as anropriate In problems where onesided test procedures are indicated analysts occasionally expe rienee difficulty in choosing an appropriate formulation of the alternative hypothesis For example suppose that a soft drink beverage bottler purchases 10ounce nonreturnable bot tles from a glass company The bottler wants to be sure that the bottles exceed the specifi cation on mean internal pressure or bursting strencth which for lOounce bottles is 200 psi l 100 air OC curve for a twosided test onesiaed test oool2 Figure 114 Operatirg characteristic CUJVC5 fortvvosided and onesided tests 112 Tests of HypoLeses on a Single Sample 271 The bottler has decided to formulate the decision procedure for a specific lot of bottles as a hypothesis problem There are two possible formulations for this problem either H fl 200 psi 119 Ht fl 200 psi or Ho fl 200 psi 1110 HI fl 200 15i Consider the formulation in equation 119 If the null hypothesis is rejected the bot tles will be judged satisfactory while if Ho is not rejected the implication is that the bot tles do not conform to specifications and should not be used Because rejecting Ho is a strong conclusion this formulation forces the bottle manufacturer to demonstrate that the mean bursting strength of the bottles exceeds the specification Now consider the formula tion in equation 11to In this situation the bottles will be judged satisfactory unless Ho is rejected That is we would conclude that the bottles are satisfactory unless there is strong evidence to the contrary Which fonnulation is correct equation 119 or equation 111O The answer is it deperuls For equation 119 there is some probability that II will be accepted ie we would decide that the bottles are not satisfactory even though the true mean is slightly greater than 200 psi This formulation implies that we want the bottle manufacturer to demonstrate thar the product meets or exceeds our specifications Such a formulation could be appropriate if the manufacturer has experienced difficulty in meeting specifications in the past or if product safety considerations force us to hold tightly to the 200 psi specification On the other hand for the formulation of equation 1110 there is some probability that II 0 will he accepted and the bottles judged satisfactory even though the true mean is slighCy less than 200 psi We would conclude that the bottles are unsatisfactory only when there is strong evidence that the mean does not exceed 200 psi that is when Ho fl2 200 psi is rejected This formulation assumes that we are relatively happy wiD the bottle manufacturers past per formance and that small deviations from the specification of fl2 200 psi are not harmful In formulating onesided alternative hypotheses we should remember that rejecting Ho is always a Strong conclusion and consequently we should put the statement about which it is important to make a strong conclusion in the alternative hypothesis Often this will depend on our point of view and experience with the situation 112 TESTS OF HYPOTHESES ON A SINGLE SAIPLE 1121 Tests of Hypotheses on the Mean of a Normal Distribution Variance Known Statistical Analysis Suppose that the random variable X represents some process or population of interest We assume that the distribution of X is either normal or that if it is nonnormal the conditions uf the Central Limit Theorem hold In additinn we asslllne that the mean fl of X is unknown but that the variance c is known We are interested in testing the hypothesis where 110 1s a specified constant Ho fl i1J II It i1J 1111 272 Chapter II Tests of Hypotoeses A random sample of size n Xl X2 Xll is available Each observation in this sam pre bas unknown mean J1 and known variance d The test procedure for He J1 10 uses the test statistic 1112 If the null hypothesis He J1 10 is true then EY 10 and it follows that the distribution of Zo is NO I Consequently if H J1 10 is true the probability is 1 a that a value of the test statistic Zo falls between Zqfl and Zal2 where Za2 is the percentage point of the standard normal distribution such that PZ Zan a2 ie Z is the lOOla2 per centage point of the standard normal distribution The situation is lllustrated in Fig 115 Note lilt the probability is a that a value of the test statistic Zo would fall in She region Zo Za2 or 20 Za2 whenHo fJUo is true Clearly a sample producing a value oftbe test statistic that falls in the tails of the distribution of Zo would be unusual if Ho jl 10 is true it is also an indication that Ho is false Thus we should reject Ho if either ZoZa2 1ll3a or lll3b and fail to reject Ho if 1114 Equation 1114 defines the acceptance region for Ho and equation 1113 defines the criti cal region or rejection region The type I error probability for this test procedure is ex The burning rate of a rocket propellant is being studied Specifications req that the mean burring rate must be 40 ems Furthermore suppose that we know that the standard deviation of the burning rate is approximately 2 cmfs The experinenter decides to specifY a type I errot probability a 005 and he will base the test 01 a random sample of size n 25 The hypotheses we wish to test are He Jl 40 emfs HJl40cmls Twentyfive specimens are tested and the sample mean burning rate obtained is i 4125 emso The value of u1e test statistic in equation 1112 is Crtcal region x JJo Zo JT 412540 2 f25 aJ2 il 3125 Figure 11 5 The distribution of Z when Ho J1 J1Q is true I L 112 TestS of Hypotheses oa a Single Sample 273 Since a 005 thebcundaries of the critical region arc Zom L96andZotT5 196 and we note that Zo falls in the critical region Therefore He is reJected and we conclude mat the mea burning rate is not equal to 40 cds Now suppose that we wish to test the onesided alternative say H 0 Jl 111 H Jl Lr 1115 Note that we could also wTIte He 1 JJo In defining the critical region for this test we observe that a negative value of the test statistic 2Q would never lead us to conclude that Ho jt 111 is false Therefore we would place the critical region in the upper tail of the NCO 1 dis tribution and reject H 0 on values of ZO that are too large That is we would reject Ho if Similarly to test z Z Ho Jl 111 HI jl Lr 1116 1117 we would calculate the teststaostic Zo andrejectHo on values of ZO that are too small That is the critical region is in the lower tail of the NO 1 distribution and we reject Ho if 1118 Choice of Sample Size In testing the hypotheses of equations 1111 1115 and 1117 the type I error probability a is directly selected by the analyst However the probability of type II error f3 depends on the choice of sample size In this section we will show how to select the sample size in order to arrive at a specified value of J Consider the twosided hypothesis H 0 Jl 111 HI Jl 111 Suppose that the null hypothesis is false and that the trUe value of the mean is J1 111 0 say whee 0 O ow since HI is trJe the distribution of the test statistic Zo is 1119 The distribution of the test statistic z under both the null hypothesis Ho and the alternative hypotbesisHl is shown in Fig 116 From examirring this figure we note that if Hl is true a type II error will be made ooly if Zan S z 5 Z2 where z N 15 Ja 1 That is the probability of the type II error f3 is the probability that z falls between Za12 and Z2 given that Hi is lrue This probability is shown as the shaced portion of Fig 116 Expressed mathematically this probability is 1 151 r 0 f311 ZaPa II ZO2 a 1120 where z denotes the probabilily to the left of z on the standard normal distribution Note that equation 1120 was obtained by evaluating the probability that Zo falls in the interval 274 Cbapter 11 Tests of Hypotheses Under H1 ji1kz z2 o Figure n6 The distribution of z under Ho and H Za2 Zari 01 the distribution of Zo when Hi is true These two points were standardized i to produce equation 1120 Furthermore note that equation 1120 also holds if Ii 0 due to the symmetry of the normal distribution While equation 11 20 could be used to evaluate the type II error it is more eonvenient to use the operating characteristic curves in Charts VIa and VIb of the Appendix These curves plot p as calculated from equation 1120 against a parameter d for various example sizes n Curves are provided for both a 005 and a 001 The parameter d is defined as dJI1 1101 181 J J 1121 Ve have chosen d so that one set of operating characteristie curves can be used for all prob lems regardless of the values of f1Q and 5 From examining the operating characteristic curves or equation IlM20 and Fig 116 we note the follOwing 1 The further the true value of the mean 11 fromllo the smaller the probability of type II error pfot a given n and ex That is we see that for a specified sample size and a large differences in the mean are easier to detect than small ones 2 For a given 8 and a the probability of type n error fi decreases as n increases That is to detect a specified difference in the mean 0 we may make the test more pow erful by increasing the sample size COlSider the rocket propeBant problem in Example 111 Suppose that the analyst is concerned about the probability of a type II error if the true mean burning rate is J1 41 ems We may use the operat ing characteristic curves to find J Note that 541 40 1 n 25 0 2 and a 005 Then and from Chart VIa Appendix lith n 25 we find that fJ 030 That is if the true mean burLing rate is J141 ems theJ there is approximately a 30 chance that this will nothe detected by the test withn25 iixipipiiJj Once again eomder the rocket propellaot problem in Example 111 Suppose that the analyst would like to design the test so that if the true mean hurning rate differs from 40 rtDs by as much as 1 crJs 112 Tests of Hypotheses on a Singe Sample 275 the test will detect this ie reject Ho JJ 40 ith a high probability say 090 The operatig char acteristic curves can be Ised totmd the sample size that wi give Slch a test Since d lti 14llkf 112 a 005 and fJ 010 we find fron Chart VIa Appendix tlJat tlJe required sample size is n 40 approximately In general the operating characteristic curves involve three pruameters j3 8 and n Given any two of these parameters the value of the thlrd can be determined There are two typical applications of these curves L For a given n and 8 find j3 This was ilustratedin Example 112 This kindofproh lem is often encountered when the analyst is concerned about the sensitivity of an experiment already performed or when sample size is restricted by economic or other factors 2 For a given j3 and ii find n This was ilustrated in Example 113 This kind ofproh lem is usually encountered when the analyst has the opportunity to select the sam ple size at the outet of the experiment Operating characteristic curves are given in Chats vlc and VId Appendix for the onesided alternatives If the alternative hypothesis is H t JL 10 then the abscissa scale on these charts is When the alternative hypothesis is HI JL 10 the corresponding abscissa scale Is dJ1J1 5 1122 1123 It is also possible to derive formulas to determine the appropriate sample size to use to obtain a particular value of j3for a given 0 and a These formulas are alternatives to using the operating characteristic curves For the twosided alternative hypothesis we know from equation 1120 that or if 0 0 1124 since p Za2 8 j J 0 when 0 is positive From equation 1124 We take nornal inverses to obtain or n 1125 276 Chapter 11 Tests of Hypotheses This approximation is good when 4 Zat2 oJna is small compared to 3 For either of the onesided alternative hypotheses in equation 1115 or equation lI M 17 the sample size required to produce a specified type II error with probability f3 given j and 0 is Za 2ft r2 n Ii 1126 Returning to the rocket propellant problem of Example li3 we note that 0204140 1 X 005 a1d 3 010 Since Zan 2f12S 196 and Zfj 2tO 128 the sample size required to detect this departure from He Jl 40 is found in equation 1125 0 be z 2 1961282 n li l 42 which is hJ close agreement ith the value determined from the operating characteristic curve Note that the approximation is good since IZI20rn96IJ4212520O which is small relative to 3 The Relationship Between Tests of Hypotheses and Confidence Intervals There is a close relationship between the test of a hypothesis about a parameter e and the confidence interval for O If L 1 is a 1001 a confidence interval for the parameter e then the test of size a of the hypothesis Ho 8 eo Heeo will lead to rejection of Ho if and only if 80 is not in the interval L UJ As an illustration consider the rocket propellant problem in Example 111 The null hypothesis Ho 1 40 was rejected using a 005 The 95 twosided confidence interval on 1 for these data may be computed from equation 1025 as 4047 0154203 That is the interval L UJ is 4047 4203 and since 110 40 is not included in this interval the null hypothesis Ho 1 40 is rejected Large Sample Test with Unknown Variance Although we have developed the test procedure for the null hypothesis HD 1 flo assum ing that is known in many practical situations d2 will be unknown In general if n 2 30 then the sample variance 2 can be substituted for c in the test procedures with little harmful effect Thus while we have given a teSt for known cf2 it can be converted easily in to a largesample test procedure for unknown f Exact treatment of the case where cJ1 is unknown and II is small involves the use of the t distribution and is deferred until Section 1122 PValues Computer software packages are frequently used for statistical hypothesis testing Most of these programs calculate and report the probability that the test statistic will take on a value 1 I I 12 TesTS of Hyporbeses 01 a Single Sample 277 at least as extreme as the observed value of the statistic when Ho is true This probabllity is usually called a Pvalue It represents the smallest level of significance that would lead to rejection of Ho Thus if PO04 is reported in he computer output the null hypothesis Ho would be rejected at the level a 005 but not at the level a OO GeMrally if P is less than or equal to a we would reject H 0 whereas if P exceeds a we would fail to reject Ho It is customary to call the test statistic and the data significant when the null hypoth esis Ho is rejected so we may think of thePvalues the smallest level aat which the data are significant Once the Pvalue is known the decision maker can determine for himself or herself how significant the data are without the data analyst formally imposing a prese lected level of significance It is not always easy to compute the exact Pvalue of a test However for the forego ing normal distribution tests it is relatively easy If is the computed value of the test sta tistic then the Pvalue is D2oI PjD2o 120 for a twotailed test for an upperlail tes for a lowertail test To illustrate consider the rocket propellant problem in Example 11 The computed value of the test statistic is Zo 3125 and since the alternative hypothesis is twotailed the p value is P 21 13125 00018 Thus Ho JL 40 would be rejected at any level of significance where a P 00018 For example Ho would be rejected if a 001 but it would not be rejected if aOOOl Practical Versus Statistical Significance In Chapters 10 and 11 we present confidence intervals and tests of hypotheses for both sin glesample and twosample problems In hypothesis testing we have discussed the statisti cal significance when the oui hypothesis is rejected What has not been discussed is the practical significance of rejecting the null hypothesis In hypothesis testing the goal is to make a decision about a claim or belief The decision as to whether or not the null hypoth esis is rejected in favor of the alternative is based on a sample taken from the population of interest If the null hypothesis is rejected we say there is statistically significant evidence against the null hypothesis in favor of the alternative Results that are statistically significant by rejection of the null hypothesis do not necessarily imply practically significaJt results To illustrate suppose that the average temperature on a single day throughout a par ticular state is hypothesized to 63 degrees Suppose that n 50 locations within the state bed an average temperature of x 62 degrees and standard deviation of 05 degrees If we were to test the hypothesis J1 63 against Hl J1 63 We would get a resulting Pvalue of approximately 0 and we would reject the null hypohesis Our conclusion would be that the true average temperature is not 63 degrees In other words we have illustrated a statistically significant difference bveen the hypothesized value and the sample average obtained from the data But is this a practical difference That is is 63 degrees different from 62 degrees Very few investigators would actually conclude that this difference is practical In other words statistical significance does not imply practical significance The size of the sample under investigation has a direct influence on the power of the test and thc practical significance As the sample size increases even the smallest differ ences between the hypothesized value and the sample value may be detected by the 278 Chapter 11 Tests of Hypotheses hypothesis tes Therefore care must be taken when interpreting the results of a hypothesis test when the sample sizes are large 1122 Tests of Hypotheses on the lJean of a Normal Distribution Variance Unknown When testing hypotheses about the mean J1 of a population when j2 is unknoNU we can use the test procedures discussed in Section 1121 provided that the sample size is large n 30 say These procedures are approximately valid regardless of whether or not the under lying population is normal However when the sample size is small and j2 is unlmONU1 we must make an assumption about the form of the underlying dlstribution in order to obtaill a rest procedure A reasonable assumption in many cases is that the underlying distribution is normal Many populations encountered in practice are quite well approximated by the normaJ distribution so this assumption will lead to a test procedure of wide applicability In fact moderate departure from normality will bave little effect on the test validity When the assumption is unreasonable we can either specify another distribution exponential Weibull etc and use some general method of test construction to obtain a alid procedure or we could use one of the nonparametric tests that are valid for any underlying distribution see Chapter 16 Statistical Analysis Suppose that X is a normally distributed random variable with unknown mean f1 and vari ance cr Vole wish to test the hypothesis that f1 equals a constant 110 Note that this situation is similar to that treated in Section 1121 except that now both J1 and if are unlolOWD Assume that a random sample of size nj say Xl Xl Xfl is available and letX and S2 be the sample mean and variance respectively Suppose that we wish to test the twosided alternative Ho 11 f1v HI1Uo The test procedure is based on the statistic 1127 1128 which follows the t distribution with n 1 degrees of freedom if the null hypothesis Ho 11 J1o is true To testHo J1 110 in equation 11 27 the test statistic Ie in equation 1128 is cal culated and Ho is rejected if either 1I29a or IlZ9b where tall I I and tan n I are the upper aud lower Ct2 percentage points of the t distri bution with n I degrees of freedom For the onesided alrernative hypothesis Ho 11 10 H 11 10 1130 r i L 112 Tests of Hypotheses on a Single Sample 279 we calculate the test statistic to from equation 1128 and reject Ho if For the other onesided alternative we would reject Ho if He 1 1 II 1 10 1131 1132 1133 The breaking strength of Ii textile fiber is a rormally distributed random variable Specifcations require that the tcan breaking strergth shQuld equal 150 psi The manufact1ter would like to detect any significant departure fro this alue Thus he wishes to test no 1 150 psi HIiJlSOpsi A random sample of 15 fiber specimens is elected and their breaking stregths determined The sam ple mean and variance are computed from the sample data as i 15218 and 52 1663 Therefoe the est statistic is 11 15218150 0 r 207 sjJn 166315 The type I error is specified as a 005 Therefore tW2S14 2145 and toAls4 2 VS and we would CQucludc that there is not sufficient evidence to reject the hypothesis that JJ 150 psi Choice of Sample Size The type II error probability for tests on the mean of a normal distribution with unknown variance depends on the distribution of the test statistic in equation 1128 when the null hypothesis Ho JI j1f is false VIlien the true value of the mean is JI 10 S note that the test statistic can be written as G a So zS w 1134 The distributions of Z and W in equation 1134 are NO 1 and xn i respectively and Z and Ware independent random variables Howeve J is a nonzero constant so that the numerator of equation 11 34 is a LV 8 a 1 random variable The resulting distribution is called the noncentral t distribution with n 1 degrees of freedom and 280 Chapter 11 Tests of Hypotheses nancentrality parameter oJnICi ote that if 0 0 then the noncentral tdistribution reduces ta the usual or central t distribution In any case the type II error of the twosided alterna tive far example would be 3 P ai2 1 S to S ai2n 110 0 P where to denOtes the noncentral t random variable Finding the type II error for the Itest involves fmding the probability contained between two points on the noncenttal t distribution The operating characteristic curves in Charts VIe Vlf VIg and Vlh Appendix plot 3 against a parameter d for various sample sizes n Curves are provided for both the twosided and onesided alternatives and for 0 005 or a 001 For the twosided alternative in equation 1127 the abscissa scale factar d on Charts VIe and VIis defined as d 181 1135 For the onesided alternatives if rejection is desired ie 1 Jlrr as in equation II3D we use Charts VIg and Vlh with 1136 while if rejection is desired Le 1 4j as in equation 1132 d Po 1 1137 Ci Ci Vle note that d depends on the unknown parameter dl There are several ways to avoid this difficulty In some cases we may use the results of a previous experiment or prior infor mation to make a rough initial estimate of C If We are interested in examining the operat ing characteristic after the data has been collected we could use the sample variance to estimate 32 If analysts do not have any previous experience on which to draw in estimat ing 2 they can define the difference in the mean 5iliat they wish to detect relative to 3 For example if one wishes to detect a small difference in the mean one might use a value of d 10lCiS 1 say whereas if one is interested in detecting auly maderately large differ ences in the me one might select d 101Ci 2 say That is it is the value of the ratio 10lCi that is important in determining sample size and if it is possible to specify the rela rive size of the difference in means that we are interested in detecting then a proper value of d can usually be selected itl1jiJi1i Consider tle fibertesting problem in Example 115 If the beaking strength of this fiber differs from 150 psi by as much as 25 psi the analyst would like to reject the null hpothess Ho jt 150 psi vrith a probability of at least 090 Is the sample size n 15 adequate to ensure that the test is lhis sensitive Ifwe usc the sample standard deviation s 1663 408 to estimare cr then d 1010 25408 061 By referring to the operatiJig characteristic curves in Chart VIe with d 061 and n 15 we find 045 Thus the prabability of rejecting Ho JL 150 psi if the true mean diffurs from this value by 25 psi is 1 045 055 approxioJately and we would conclude that a sample size of 1 I i v 112 Tests of HYotheses on a Single SampJe 281 11 15 is not adequate To find the sample size required to give the desired degree of protection enter the operating characteristic curves in Chart VIe with d 061 and f3 010 and read the conespon ding sample size as 11 35 approxbately 1123 Tests of Hypotheses on the Variance of a Normal Distribution There are occasions when tests assessing the variance or standard deviation of a population are needed In chis section we present tvlo procedures one based on the assumption of nor mality and the other one a largesample test Test Procedures for a Normal Population Suppose that we wish to test the hypothesis that the variance 72 of a normal distribution equals a specified value say Let X NJi 72 where1 and d are unknovn and letXt X x be a random sample of n observations from this population To test we use the test statistic Ho d Htd 2 nlS2 Xo 0 70 1138 1039 where S is the sample variance Now if Ho d is true then the test statistic ic follows the chisquare distribution with n I degrees of freedom ThereforeHo d would be rejected if 1140a or if CoXa2nt lI40b where X n 1 and i an 1 are the upper and lower ctJ2 percentage points of the chl square distribution with n 1 degrees of freedom The same test statistic is used for the onesided alternatives For the onesided hypothesis we would reject Ho if For the other onesided hypothesis we would rejeet Ho if Ho d 7 H if O 2 Xo Xarl Ho d7 Hi 02 0 1141 1142 I 43 1144 282 Chapter 11 Tests of Hypotheses Consider the machine described in Example 1016 which is used to fill cans with a soft drink bever age If the variance of the fill volume exceeds 002 fluid ourcesf1 then an unacceptably large centage ofilie cans wil be underfilled The bottler is interesred in testing the hypothesis H d 002 Hr 002 A randoCl sample of n 20 cans yields a sample variance of il 00225 Thus the test statistic is 19Q0225 2138 0Q2 Ifwe choose 0005 we find tat iolll 3014 and we would conclude that there is no strong evi deuce tha the variance offill volume eceeds 002 fluid ounces2 Choice of Sample Size Operating characteristic curves for the r tests are provided in Charts VIi through YIn Appendix for iX 005 and Ct 001 For the twosided alternative hypothesis of equation 1138 Charts VIi and VIj plot fJ against an abscissa parameter I 0 1145 for various sample sizes n where Odenotes the true value of the standard deviation Charts VIk and 11 are for the onesided alternative Ht d 0 while Charts VIm and In ardor the other onesided alternative HI al In using these charts we think of eras the value of the standard deviation that we want to detect In Example 117 find the probability of rejecting Ho a1 002 if the true vaiance is as large as d 0Q3 Since 0 jO03O1732and To J002Ol414eabscissaparameteris From Chart Ik vith1 123 and n 20 we find that J 060 ThaI is there is only about a 40 chance that Ho d1 002 will be ejected if the variance is really as large as T2 003 To reduce p a larger sample size must be used From he operating characteristic curve we note that to reduce f3 to 020 a sample size of5 is necessary A LargeSample Test Procedure The chiwsquare test procedure prescribed above is rather sensitive to the nonnality assump tion Consequently it would be desirable to develop a procedure that does not require this assumption hen the underlying population is not necessarily normal but n is large say n 35 Qr 40 then we can use the following result if Xl X2 Xn is a random sample from a population with variance cr the sample standard deviation S is approximately nonnal with mean ES and vartance VS an if n is large 112 Tests of Hypotheses on a SIngle Sample 283 Then the distribution of is approxioately standard normal To 7est Ho cf Hlcf substitute 00 for v in equation llA6 Thus the test statistic is 1146 1147 1148 and we would reject Ho if Zo Zan or if Zo Zan The same test statistic would be used for the onesided alternatives If we are testing Ho cf l HI cf 07 we would reject Ho if Zo Za while if we are testing Ho cf d HI cf d we would reject Ho if Zo Za 1149 1150 An injectionmo1ded plastic part is used in a graphics printer Before agreeing to a lQngterm contract the printer manufacturer wants to be sure using a 001 t1at the supplier can produce parts vrith a standard deviation of length of at most 0025 maL The hypotheses to be tested are Ho d 625 X 104 H d 625 X 104 smce 00252 0000625 Arandoro sample of n 50 parts is obtained and t1C sample standard devi ation is s 0021 I1t01 The test statistic is Since Zom 233 and the observed value of Zo is not smaller than this critical value Ho is aot rejected That is the evidence from the suppliers process is not strong enough to justify a longterm contract 1124 Tests of Hypotheses on a Proportion Statistical Analysis In many engineering and management problems we are concerned with a random variable that follows the binomial distribution For example consider a production process that manufactures items that are classified as either acceptable or defective It is usually 284 Chapter 11 TestS of Hypotheses reasonable to model the occurrence of defectives with the binomial distibution where the binomial parameter p represents the proportion of defective items produced We will consider testing Hoppo Hppo II51 An approximate tesl based on the nonnal approxination to the binomial will be given 11lis approximate procedure will be valid as long as p is not extremely close to zero or 1 and if the sample size is relatively large Let Xbe the number of observations in a random sample of size n that belongs to the class associated with p Then lithe null hypothesis Ho P Po is true we have X N npo nPolpf approximately To testHo p Po calculate th test statistic II52 and reject Ho p p if 1153 Critical regions for the onesided alternative hypotheses would be located in the usual manner A semieonductor firm produces logic devices The contract with their custome calls for a fraction defective of no more ilian 005 They wish to test He P 005 Hp005 A random sample of 200 devices yields six defectives The test statistic is 6200005 200 005 095 130 Using a 005 we f1d hat ZVC5 1645 and so we cannot reject the null hpothesis that 005 Choice of Sample Size It is possible ll obtain closedform equations for the 3 error for the tests in this section The 3 error for the twosided alternative H p Pos approximately 1 112 Tests of Hypotheses on a Single SaDpe 285 If the alternative is H p p then 1155 whereas if the alternative is H p p then 1156 These equations can be solved to find the sample 3ize n that gives a test ofleve1 a that has a specified f3 risk The sample size equations are 1157 for the twosided altemative and n ZaPol Po Zppl plY PPo j I 1l5B for the onesided alternatives For the situation described in Example 1110 suppose that we wish to find the 8 error of the test if p om Using equation 1 i56 the f3 error is 005007 164510050951200 f3 l 007O93OO 1030 06179 This type II error probability is not as small as one might like but ft 200 is notparticularlY large and 007 is not very fat from the null value Po 005 Suppose that we wanl the f3 error to be no larger than 010 ifhe true value of the fraction defective is as large as p 007 The required sample size would be found from equation 1158 as n L645O05O95 L2llO07093J2 007 005 1174 which is a very large sample size However notice that we are trying to detect a very small deviation fron the null value p 005 286 Chapter 11 Tests of Hypotheses 113 TESTS OF HYPOTHESES 01 TWO SAMPLES 1131 Tests of Hypotheses on the Means of Two Normal Distributions Variances Known Statistical Analysis Suppose that there are two populations of interest say X and X We assume that X has unknown mean il and known variance ri and that X2 has unknown mean iLl and known variance a We will be concerned with testing the hypothesis that the means 11 and Jlz are equal It is assumed either that the random variables Xl and X2 are normally distributed or if they are nonnormal that the conditions of the Central Limit Theorem apply Consider first the twosided alternative hypothesis Ho il iLl H il iLl II59 Suppose that a random sample of size nl is drawn from Xl say Xll Xz XIIj and that a second random sample of size n2 is drawn fromX2 say XZ1X22 Xlnt It is assumed that the XI are lndepencently distributed with mean il and variance ri that the X are independently distributed with mean iLl and variance ai and that the two samples Xlj and X are independt e test procedure is based on the distribution of the difference in sample means say X Xl In general we know that It aJ a XlXN iliLl nl llz Thus if the null hypothesis Ho il iLl is true the test statistic ZO X J7f 1 V n 2 1160 follows the NO I distribution Therefore the procedure for testing Ho ill iLl is to cal culate the test statistic z in equation 1160 and reject the null hypothesis if or Zo Z The onesided alternative hypotheses are analyzed similarly To test Bo ill iLl H ill iLl the test Statistic Zo in equation 1160 is calculated and Ho p iLl is rejected if ZoZ To test the other onesided alternative hypothesis Ho il u HI J11 fl2 use the test statistic Zo ill equation 1160 and reject Ho il iLl if ZoZa 1I6Ia 1l6Ib 1162 1163 1164 1165 113 Tests of Hypotheses on Two Samples 287 ExiinpjiiS12 The plant marager of an orange juiee canni g facility is interested in corcparing the performance of twa different production lines in her plant AI line number 1 is relatively new she suspects that its ont put in number of cases per day is greater than the n1Ullber of cases produced by the older line 2 Ten days of data are selected at random for each line for which it is found that xl 8249 cases per day and X 8186 cases per day From experience with operaticg this type of equipmectlt is laovm that d 4 and 0 50 We wish 0 test The value of the test statistic is 20 Ho J1t p HjJ1jp Xji2 j 0 ai i 8249886 210 40 50 i 10 10 Gsing a 005 we find that 21os 1645 and since 4 415 we would reject HJ and condude that tle mean number of cases per day produced by the new produetion line is greate than the mean num ber of cases per day produced by the old line Choice of Sample Size The operating characteristic curves in Charts VIa Vlb VIc and lId Appendix may be used to evaluate the type IT errorprobabJity for the hypotheses in equations 1159 1162 and 1164 These curves are also useful in sample size detertllination Curves are provided for 0 005 and 0 001 For the twosided alternative hypothesis in equation 1159 the abscissa scale of the operating characteristic curves in Charts VIa and lIb is d where 11111121 81 d rcrai vlai 1166 and one must choose equal sample sizes say n fll The onesided alternative hypothe ses requie the use ofCharts VIc and V1i For the onesided alternative HI 111 P in equa tian 1162 the abscissa scale is d 1167 I 2 Ji Ci2 where n rl nz The other onesided alternative hypothesis HI Pl f11 requires that d be defined as 1168 and n n It is not unusual to encounter problell1S where the costs of collecting data differ sub stantially between the NO populations or where one population variance is much greater than the other In those cases One often uses unequal sample sizes If11 nz the operating characteristic curves may be entered with an equivalent value of n computed from aT 02 n 2 ufju 1169 288 Chapter 11 Tests of Hypotbeses Ii and lbeir values are fixed in advance lben equation 1169 is used directly to cal culate n and the operating characteristic curves are entered With a specified d to obtain 3 If we are given d and it is necessary to determine n and to obtain a specified f3 say f3 then one guesses at trial values of n1 and i1 calculates n in equation 11 69 enters the curves with the specified value of d and finds Ii Ii f3 f3 then lbe trial values of n l and are sat isfactoryIf f3 f3 lben adjustments to and are made and the process is repeated Consider the orange juice production line problem in Exampk 11 12 If the true Cifferenee in au production rates were 10 cases per day find the sample sites required to detect this differce with a probability of 090 The appropriate value of the abscissa parameter is d 112 jf Oi 10 105 4050 and since aO05 we find trom Chart1c hat n n g It is also possible to derive formulas for the sample size required to obtain a specified f3 for a given 0 and IX These formulas occasionally are useful supplements to lbe operating characteristic curves For the twosided alternative hypothesis the sample size n i1 n is n ZX2 zl Y Yn 02 1170 This approximation is valid when r Za2 oI CJf T is small compared to i For a onesided alternative we have nl n where 1171 The derivations of equations 1170 and 1171 closely follow the singlesample case in Sec tion 112 To illustrate the use of these equations consider the situation in Example 1113 We have a onesided alternative with a 005 o 10 a 400 50 and f3 010 Thus Za 2005 1645 Zp 2010 128 and the required sample size is found from equation 1171 to be which agrees with lbe results obtained in Example 1113 1132 Tests of Hypotheses on the leans of Two Normal Distributions Variances Unknown We now considertests of hypotheses on lbe equality of the means u and Il2 of two normal distributions where the variances CJ and a are unknown A t statistic will be used to test lbes hypolbeses As noted in Section 1122 lbe normality assumption is required to r I 13 Tests of Hypotheses 01 Two Samples 289 develop the test procedure but moderate departures from normality do not adversely affect the procedure There are two different situations that must be treated In the first case we assume that the variances of the two normal distributions are unknown but equal that is i a cr In the second we assume that and a are unknown and not necessarily equal Case 1 01 01 cr Let X and X be two independent normal populations with unknown means 11 and fL2 and unknown but equal variances 17 a rr We wish to test Ho f11 P2 HI 11 12 1172 Suppose thatXw X12 XliI is a random sample ofn obsexatins ffOmXlandX1 X22 X is a random sample of observations from X Let Xl X S and S be the sam ple mes and sample variances respectively Since both S and S estimate the common aance i we may combine them to yield a single estimate say s h 1Sf Isi p n1nz2 1173 This combined or upooledn estimator was introduced in Section lO32 To test Ho 111 f1 in equation 1172 compute the test statistic x t 0 II 1 SpI nl 2 If Ho fll f1 is true 0 is distributed as tn2 Therefore if or if we reject H fLl The onesided alternatives are treated sinilarly To test Ho 11 12 HI 11 12 compute the test statistic to in equation 1174 and reject Ha 11 12 if For the other onesided alternative Ha 1 12 H 11 12 calcuinte the test statistic to and reject Ho 11 12 if 1174 l175a lH5b 1176 1177 1178 1179 The twosample Hest given in his section is often called the pooled ttest because the sample variances are combined or pooled to estimate the common variance It is also known as the independent ttest because the two normal populations are assumed to be independent 290 Chapter 11 Tests of Hypothese Example 114 Two catalysS are being analyzed to deteline how tey affect the mean yield of a chemical process Specifically catayst 1 is curently in use but catalyst 2 is acceptable Since catalyst 2 is eheaper if iI it does not change tie process yield it should be adopted Sppose we vish to test the hypotheses HOIi Hi PLot plant data yields nl 8 Xl 9173 s 389 nl 8 9375 and IH3 we find The test statistic is 7389 7402 h82 91739375 1 1 199 1 H S 203 396 402 From equation Using tX 005 we findhat tM2i4 2145 andtCCZ5 4 2145 and conseqrently Ho 111 cannot be rejected That is We do no have srrorg cidence to conclude that catalyst 2 results in a mean yield that differs fron the rnean yield when catalyst 1 is used Case 2 a u In some siruations we cannot reasonably assume that the unknown vari ances and 0 are equal There is not an exact t statistic available for testing Ho Pi 2 in this case However the statistic XlX tG 1St si P 0180 is distributed approximately as t with degrees of freedom given by St SfJ v lnl nz 2 SUn si InJ lc 1181 n 1 liz 1 if the nUL hypothesis Ho 11 i is true Therefore if a the hypotheses of equations 11721176 and 1178 are tested as before except that to is used as the test statistic and n1 1lz 2 is replaced by v in determining the degrees of freedom for the test This general problem is often called the BehrenFisher problem A manufacrurer ofYideo display units is testing two microcircrit designs to determine whether they produce equivalent current floW DevelOpment engineering has obtained the following data Desjgn 1 1ft 15 Xl 242 Design 2 12 10 Xl 239 L 113 Tests of Hypohcses on 110 Samples 291 We wish to test He 11 fJo HIplLfJ where bot populations are assumed to be normal but we are unwillilg to asSlJne that the un1caown variances ci and a are equal The test statistie is x 242239 0 184 to 1sf i 10 2ij V n 115 10 The degrees of freedom on t are found from equation 1181 to be 2 2 i E I Qt 202 v n n 2 15 10 26 sUnJ sifn lOi15 2010 16 11 ltl1 ltltl Using a 010 we find that td7v to 05J6 1746 Since It1 t0516 we cannot rejeet Hfi III k Choice of Sample Size The operating characteristic curves in Charts VIe Vlf vlg and VIh Appendix are used to evaluate the type lIerrorfor the case where 01 c if Unfortunatey when 1 the dismbution of t is unknown if the null hypothesis is false and no operating characteristic curves are available for this case For the twosided alternative in equation 1172 when a c if and n 2 n Charts VIe and VI are used with d 1 2a 2a 1182 To use these curves they must be entered with the sample size n 2n L For the one sided alternative hypothesis of equation 1176 we use Charts VIg and VIh and define d f11JLzJ U83 2a 2a whereas for the other onesided alternative hypothesis of equation 1178 we use df12f1 J 2a 2a 1184 It is noted that the parameter d is a function of a which is unknown As in the singlesample ttest Section 11 22 we may have to rely On a prior estimate of a or use a subjective estimate Alternatively we could define the differences in the mean that we wish to detect relative to 1 Iill Consider the catalyst expcment in Example 1114 Suppose that if catalyst 2 produces a yield that differs from the yield of catalyst 1 by 30 we would lile to reject the null hypothesis ith a proba bility of at leas 085 What sample size is required Usingip 199 as a rO1gh escrnlte of the 292 Chapter 11 Tests of Hypotheses common standard deviation a we have d 1000za 1300i2199 075 From Chart VIe Appen dix with d 075 and fJ 015 we find n 20 approximately Therefore since n 2n 1 n l 201 n221O5 ll say and we would use sample sizes ofnl n2 n 11 1133 ThePairedtTest A special case of the twosample ttests occurs when the observations on the two popula lions of interest are collected in pairs Each pair of observations say XJ X1j is taken under homogeneous conditions but these conditions may change from one pair to another For example suppose that we are interested in comparing two different types of tips for a hardnesstesting machine Ibis machine presses the tip into a metal specimen with a known force By measuring the deplh oflhe depression caused by lhe tip the bardness of the spec imen can be determined If several specimens were selected at random half tested with tip I half tested wilh tip 2 and lhe pooled or independent Hestin Section 1132 applied the results of the test could be invalid That is the metal specimens could have been cut from bar stock that was produced in different heats or they may not be homogeneous which is another way hardness might be affected then the obseNed differences betvleen mean hard ness readings for the two tip types also include hardness differences between specimens The correct experimental procedure is to collect the dara in pairs that is to take two hardness readings of each specimen one with each tip The test procedure would then con sist of anallzmg the differences between hardness readings of each specimen If there is no difference between tips then lhe mean of lhe differences should be zero This test procedure is called the paired ttest Let Xu Xl XI2 Xnl X XJ be a set of n paired observations where we assume lhat XI Npl cr and Xl NCp 0 Define lhe differences between each pair of obsenrations as Dj Xu X2i j 1 2 n The D are normally distributed vith mean so testing hypolheses about the equality of III and can be aecomplished by perfonDing a onesarnple Hest on JD Specifically testing Ho J1 fJ against HI Jil t f12 is equivalent to testing Eo IlD 0 HI IlD 0 The appropriate test statistic for equation ll8S is 15 where 1l8S ll86 1187 J l 113 Tests of Hypotbeses on Two Samples 293 and sD 1188 are the sample mean and variance of the differences Ve would reject Ho lv 0 implying that Il 10 if to Ian lor if to 12 1 Onesided alternatives would be treated similarly lEilllz An article in the Joul7Ul1 of Strain Analysis Vol IS No2 1983 compares several methods fur pre dicting the shear strength for steel plate girders Data fur two of these methods the Karlsruhe and Lehigh procedures when applied to nine specific girders are shown in Table 112 We wish to deter mine if there is any difference on the average between the tvlo methods The sample aYerae and standard deviation of the differences d are 1 02739 and Sd 01351 so the test statistic is 02739 01351J9 608 For the twQsided alternative HI 1lfJt 0 and a 01 we would fail to reject only if Itol tl 3 186 Since to to05 5 we conclude that the two strength prediction methods yield dfferent results Specf ically the KarhTUhe method produces on average higher stfCngtl predictiotS than does te Lehigh method Paired VersrLj Unpaired Comparisons Sometimes in performing a comparative experi ment the investigator can choose between the paired analysis and the twosample or unpaired Itest If n measurements are to be made on each population the twosample t sta tistic is Table 112 Strength Pedictions fur Nine Steel Plate Girders Pre dicted LoarliObserved Load Girder Karlsruhe Method Lehigh Melhod Difference Slil la6 1061 0125 S21 l51 0992 0159 S31l 1322 1063 0259 Soil 1339 1062 0277 S51 1200 1065 0135 S21 1402 un 0224 S22 1365 1037 0328 S23 1537 1086 0451 S24 1559 1052 0507 294 Chapter 11 Tests of HypotOeses which is compared to tOIl 211 2 and of course the paired t statistic is 15 toS I Di In which is compared to tfYi2nI Kotie that since the numerators of both statistics are identical However the denominator of the twosam pIe ttest is based on the asscmption that XI andX2 are independent In many paired exper iments there is a strong positive correlation betleen Xl and X2 That is v15 VXj X2 VX VX2CovX X 2021 p n assuming that both populations Xl and Xz hae identical variances Furthennoret S n esti llates the variance ofD Now whenever there is positive correlation within the pairs the denominator for the paired ttest will be smaller than the denominator of the tNosample ttest This can cause the tVosampJe ttest to considerably understate the significance of the data if it is incorrectly applied to paired samples Although pairing will often lead to a smaller value of the variance of Xl X it does have a disadvantage Namely the paired ttest leads 10 a loss of n 1 degrees of freedom in comparison to the twosample ttest Generally we know that increasing the degrees of freedom of a test increases the power against any fixed alternative values of the parameter So how do we decide to conduct the experimentshould we pair the observations or not Although there is no general answer to this question we can give some guidelines based on the above discussion They ate as follows 1 If the experimental units are relatively homogenous small a and the correlation between pairs is small the gain in precision due to pairing will be offset by the loss of degrees of freedom so an independentsamples experiment should be used 2 If the experimental uIUts are relatively heterogeneous large a and there is large positive correlation betveen pairs the paired experiment should be used The rules still require judgment in their implementation because j and p are usually not nmll precisely Furthermore if the number of degrees of freedom is large say 40 or 50 then the loss of n 1 of them for pairing may not be serious However if the number of degrees offreedom is small say O or 20 then losing half of them is potentially serious ifnot compensated for by an increased precision from pairing 1134 Tests for the Equality of Two Variances We now present tests for comparing two variances Following the approach in Section 1123 we present tests for normal populations and latgesample tests that may be applied to nonnormal populations 1 L 113 Tests of Hypotheses on Two Samples 295 Test Procedure for Normal Populations Suppose that two independent populations are of interes say Xl NlJ1l a and X2 NIJ1z 1 where J a pz and 1 are unknown We wish to test hypotheses about the equality of the two variances say Hol a 0 Assume that two random samples of size OJ from pop ulation 1 and of size n from populatioo 2 are available and let S and S be the sample vari ances To test the twosided alternative we use the fact that the statistic Ho cr 0 Hi o S2 Fa Si 1189 1190 is distributed as F with 1 and n I degrees of freedom if the null hypothesis llc 0 a is i1e Therefore we would reject Ho if 1191 or if Po F 1 aJ2IlI1 11 I191b where FCd2 11 11121 and F1 aI2flllII2 1 are the upper and lower J2 percentage points of the F distribution with I and n I degrees of freedom Table V Appendix gives only the upper tail points of P so to fmd F 1 a12IlI 11111 we must use Pi aj21 1n21 1 1192 The same test statistic can be used to test onesided alternative hypotheses Since the notation Xl and X2 is arbitrary let Xl denote the population that may have the largest vari ance Therefore the onesided alternative hypothesis is If we would reject Ho 0 Ho O t1 11 cr cr 1193 1194 Chemical etching is used 0 remove copper from printed circuit boards X and Xt represen pocess yields when twO different concentrations are used Suppse 6at we wis to test Ho a H j vi 0 Two samples of sizes nt liz 8 yield 37389 296 Chapter 11 Tests of Hypotheses If a 005 we find tht Foqt5 7 7 499 and FQS75 7 7 FruJ1j 7 7r 499r I 020 Therefore Ne ca1llt reject Ho 01 0 and we can conclude mat there is no strong evidence that the variance of me yield is affected by the concentration Choice of Sample Size Charts v1o VIp v1q and VIr Appendix provide operating characteristic curves for the Ftest for a 005 and a om assuming that n Cbts VIo and VIp are used with the twosided alternative of equation 1189 They plot j3 gabst the abscissa parameter A 1 1195 2 for various 111 n2 n Charts VIq and VIr are used for the onesided alternative of equa tion 1193 For the chemical process yield aralyses problem in Example 11 18 suppose that one of the concen trations affected the variance of the yield so that one of the variances was four times the other and We wished to detect this with probability at least 080 What sample size should be used Note that if one variance is four times the oilier men 1 0 By referring to ChartVIo with p 020 and 1 2 we find that a sample size of nl fl1 20 approx imately is necessary A LargeSample Test Procedure Vlhen both sample sizes lZl and 1tl are large a test procedure that does not require the nor mality assumption can be developed The test is based on the result that the sample standard deviations 51 and S1 have approximare normal disuibutions with means 0i and 02 respec tively and variances u2nl and cr2 respectively To test 2 Ho jl az HI crt 1196 we would use the test statistic r 1 1 S 1 pOj 2n 2n where Sf is the pooled estimator of the common standard deviation j This statistic has an approxiinate standard normal distribution when 0 We would reject Ho if z Zan or if ZcZ Rejection regions for the onesided alternatives have the same form as in other twosample normal teSts 1l35 Tests of Hypotheses on Two Proportions Toe teSts of Section 1124 can be extended to the case where there are two binomial parameters of interest say PI and P2 and we wish to test that they are equal That is we wish to test Hop p Hp p 1198 I 113 Tests of Hypotheses on Two Samples 297 We will present a largesample procedure based on the normal approximation to the bino mial and then outline one possible approach for small sample sizes LargeSampleTes for Hop pz Suppose that the two random samples of sizes nl and nz are taken from two populations and let Xl and X2 represent the number of observations that belong to the class of interest in samples 1 and 2 respectively Furthennore suppose that the normal approximation to the binomial applies to each population so that the estimators of the population proportions PI Xlinl and pz Xzln have approximate normal distributions Now if the null hypothe sis Ho PI P2 is true then using the fact thatpl pz p the random variable z PIP2 is distributed approximately NCO 1 An estimate of the common parameter pis Xl Xz p n1 1lz The test statistic for Ho PI P2 is then If the null hypothesis is rejected 1199 11100 Two different types offire control computers are being considered for use by the US Army in six gun IDSrom barteries The two computer systems are subjected to an operational test in which the total number of hits of the target are counted Computer system 1 gave 250 hits out of 300 rounds while computer system 2 gave 178 hits out of260 rounds Is there reason to believe that the two com puter systems differ To answer this question we test HoPIPZ HlPl pz Note thatp 250300 08333p 178260 06846 and 250178 300260 07643 The value of the test statistic is 08333 06846 413 076430235711J 300 260 If we use a 005 then 20025 196 and ZO025 196 and we would reject Ho concluding that there is a significant difference in the two computer systems 298 Ch3jter 11 Tests of Hypotheses Choice of Sample Size The computation of the f3 error for the foregoing test is somewhat more involved than in the singlesample case The problem js that the denominator of z js an estimate of the standard deviation ofpl pz under the assumption that PI P2 p hen Ho PI P2 is false the stan dard deviation ofPl pz is 11101 If the alternative hypothesis is twogided the fl risk tums out to be approximately fl Za2pqlin PI p 11102 where and lj h is given by equation 11101 If the alternative hypothesis is H P P2 then lH03 and if the alternative hypothesis is Ht P P2 then 11104 For a specified pair of values Pl and P2 we can find the sample sizes nl 2 n required to give the test of size a that has specified type II errOr fl For the twosided alter native the common sample size is approximately 11105 where ql 1 PI and q2 1 Pz For the onesided alternatives replace Zd2 in equation 11105 with2w 1 I l 113 Tests of Hypotllese on 1vo Samples 299 SrnaIlSample Test for Ho p p Most problems involving the comparison of proportions Pl and p have relatively large sam pIe sizes so the procedure based on the nonnal approximation to the binomial is widely used in practice Howeer occasionally a smallsamplesize problem is encountered In such cases the Ztests are inappropriate and an alternative procedure is required In this sec tion we describe a procedure based on the hypergeometric distribution Suppose that Xl and X2 are the number of successes in 1vto random samples of sizes nl and fi1 respectYely The test procedure requires that we view the total number of successes as fixed at the value X X Y Now consider the hypotheses HOPI p Bpp Given that Xl X Ylarge values of Xl supportH1 whereas small or moderate values of Xl support Ho Therefore we will reject Ho whenever Xl is sufficiently large Since the combined sample of n1 lLz obseryations contains Xi Xz Ytotal successes if Bo p p the successes are no more likely to be concentrated in the first sample than in the second That is all the ways in which the n responses can be divided into one sample of nl responses and a second sample of fl2 responses are equally likely The number of ways of selecting Xl successes for the first sample leaving Y X successes for the second is Because outcomes are equally likcly the probability of there being exactly Xl successes in sample I is detennined by the ratio of the number of sample I outcomes having Xl suc Cesses to the total number of outcomes or xY success in n1 fl2 responses 11106 given that Ho Pi P2 is true We recognize equation 11106 as a hypergeometric distribution To use equation 11106 for hypothesis testing we would compute the probability of finding a value of Xl at least as extreme as the observed value of X Note that this proba bility is a Pvalue If this Pvalue is sufficiently small then the null hypothesis is rejected This approach coudd also be applied to lowertailed and twotailed alternatives pi InsulatirJg cloth used in printed circuit boards is manufactured in large rolls The manufacturer is try ing to irrprove the process yield that is the number of defectfree rolls produced A sample of 10 rolls contains exactly four defectfree rolls From analysis of the defect types manufacturing cOineering Slggests S2VCa1 changes in the pocess Folowing implernettation of these chlUlges another sample of 10 rolls yields 8 defectfree rolls Do the data support the claim thzt the new process is beter tha1 the old one using aQlO 300 Chapter II Tell of Hypotheses To aIlswer this question we compute the Pvahle mour examp1e nl ns 10 Y 8 t 4 12 and the observed value of xl 8 The values of Xi that are more extreme than 8 are 9 and 10 Therefore 128 8 121 pXJ 8112 successes oy 00750 10 J 12Y81 I 19I P XI 912successes O0095 llO J I GaJ PXJ 10 12 successes 20 00003 10 The Pvalue is p 00750 00095 00003 00848 Thus at the level a 010 he nulllypothe sis is rejected and We conclude that the egineering changes have improved the process yield This test procedure i sometimes called the Fisherlrviin test Because the test depends on the asstmption that Xl X2 is fixed at some value some statisticians have argued against use of the test when X X is not actually fixed Clearly X Xis notfixed by the sampling procedure in our example However because there are no other better competing procedures the Fisher lrviin test is often used whether or not Xl X2 is acrually fixed in advance 114 TESTING FOR GOODNESS OF FIT The hypothesistesting procedures that we have discussed in previous sections are for prob lems in which the fann of the density function of the random variable is known and the hypotheses involve the parameters of the distribution Another kind of hypothesis is often encountered we do not know the probability distribution of the random variable under study say X and we wish to test the hypothesis that X follows a particular probability distribution For example we might wish to test the hypothesis that X follows the normal distribution m this section we describe a fonnal goodnessoffit test procedure based on the chisquare distribution We also describe a very useful graphical technique called probabil ity plotting Finally we give some guidelines useful in selecting the form of the population distribution The ChiSquare GoodnessofFit Test The test procedure requires a random sample of size n of the random variable X whose probability density function is unknown These n observations are arrayed in a frequency histogram having k class intervals Let 0 i be the observed frequency in the ifu class inter val From the hypothesized probability distribution we compute the expected frequeney in the ith class intervaL denoted Ej The test statistic is 2 0 Ed Xo L 11107 1 Ej It can be shown that X approximately follows the chisquare distribution with k p 1 degrees of freedom where p represents the number of parameters of the hypothesized l 114 TestigforGooCness of Fit 301 distnbution estimated by sample statistics This approximation improves as n increases We would reject the hypothesis that X eanfonns to the hypothesized distribution if xi XlXkp One point to be noted in the application of this test procedure concerns the magnimce of the expected frequencies If these expected frequencies e too slIa11 then X will not reflect the dep2ItUre of observed from expected but only the smallest of the expected fre quencies There is no general agreement regarding the minimum value of expected rre quencies but values of 3 4 and 5 are widely used as minimal Should an expected frequency be too small it can be combined with the expected frequency in an adjacent class interval The corresponding observed frequencies would then be combined also and k would be reduced by L Class intervals are not required to be of equal width We now give three examples of the test procedure Eiimiii2Z Co p A Completely Specified Distribution A computer scientist has developed an algorithm for gener ating pseudorandom itegerS over he interal 09 He codes the algorithm and generaes 1000 pseudoandom digits The data are shown ir Table 113 Is here evidence tht the rzndom number generaor is working correctly If the randoJJ number generator is working correctly then the values 09 should follow the dis crete uniform distribution which implies that each of the integers should occur about 100 tiDes That is the expecredfrequencies Ei 100 for i 0 1 9 Since these expected feGuencies ean be deter mined vrifuout estimating any parameters from the sample cata the resuling chisituate goodnessof fit test will have k p 1 10 0 1 9 degrees of freedom The observed value of the test statistie is i J xl I o J Et 94100 93100 94IOc0 100 100 100 372 Since ioM 1692 we are unable to rejeec the hypothesis tlJat the data eome from a discrete uniform distribution Therefore the random numbet generator seems to be working satisfactorily Eti83 A Discrete Distribution The number of defects in printed cirenit boards ls hypothesized to follow a Poisson distribution A random sample of n 60 printed boards have been eollected and the mID ber of defects observed The following data result Table 113 Data for Example 1122 Total 0 1 2 3 4 5 6 7 8 9 n Observed frequencies 0 94 93 112 101 104 95 100 99 108 94 1000 Expected frequencies Ej 100 100 lOa 100 100 100 100 00 100 100 1000 302 Chapter 11 Tests of Hypotheses l Number of Defects Observed Frequency o 1 2 3 32 15 9 4 Pe mean of the assumed Poisson distribution in this example is unknown and must be estimated from the sample data The estimate of the mean Dumber of defects per board is the sample IVerage that is 320 15 1 9 2 4 3160 075 From the cumulative Poisson distnoution with pararr eter 075 we may compute the expected frequencies as Ei np where Pi is the theoretic hypofue sized probability associated with the 1m class interval and n is the total number of observations The appropriate hlotheses are 075 H c oPtx x 012 HI px is Dot Poisson 1tith t 075 We may compute the erpected frequencies as follows Nu11ber of Failures Probability Expected Fequency 0 0472 2832 1 0354 224 2 0133 798 3 0Q41 2A5 The expected frequencies are obtained by multiplying the sample size times the respective pmbabU lties Since the expected frequency in the las cell is less than 3 we combine the last two cells Number of Failures o ObserVed Frequency 32 15 13 Expected Frequency 2832 2124 1044 The test statIstic which will have k P t 3 1 t 1 degree of freedom becomes 322832 152124 Q31044 294 2832 2124 1044 and since Xl 34 we cannot reject the hypothesis that the occurrence of defects follows a Poisson disJlution vitlt mean 075 defects per board A Continuous Distribution A manufacturing engineer is testing a power supply used in a word processing work station He 1shes to determine whether output voltage is adequately described by a normal distribution From arandom sample of n 100 llIlits he obtains sample estimates of the mean and standard deviation x 1204 V and s 008 V A common practice in constructing the class iutervals for the frequency distribution used in the chlsquare goodnessoffit test is to choose the cell boundaries so that the expected frequencies Ei Pi are equal for all cells To use this method we want to choose the cell boundaries 101 al at for the k cells so that all he probabilities J j I i 114 Testing for Goodness of Fit 303 are equal Suppose we decide to use k 8 cells For the standard normal distribution the intervals hat divide the scale into egh eqalIy1ely segments are 0 032 032 0675 0675 115 Ll5 l and thea fow miror image intervals on the other side at zero Denoting hese standard normal end points by Go at as it is a simple matter 0 calculate the endpoints that are necessary for the gen ernl normal problem at hand narnely we define the new class interval endpoints by the transformation ai sa i O 1 8 Por example the sixth interVals right endpoint is 1204 00 0675 12094 For each interVal Pi t 0125 so the expected cell frequencies are El The complete table of observed and expected frequences is given IT Table The computed value at the chisquare statistic is 100 0125 125 Si1ce rNO parameters in tle normal distribt1ion have been estimated we would compare to L12 to a crusjuare disttibutiol vrith k P 1 8 2 1 5 degrees of freedom Using a 010 we see that XIi 1336 and so we conclude that there is no reason to believe that output voltage is not nor mally distributed Probability Plotting Graphical methods are also useful when selecting a probability distribution to describe data Probability plotting is a graphical method for determining whether the data conform to a hypothesized distribution based on a subjective visual examination of the data The general procedure is very simple and can be performed quickly Probability plotting requires special graph paper known as probability paper that has been designed for me hypothesized distribution Probability paper is widely available for the normal lognormal Weibull and various chisquare and gamma distributions To construct a probability plot the obsenrations in the sample are first ranked from smallest to largest That is the sample Table 114 Observed iUld Etpected Frequencies Oass Observed Expected Interval Frequency 0 1 x 11948 10 125 11948 x 11986 14 125 1986x 12014 12 125 12014x 12040 13 125 12040 12066 11 125 12066 s 12094 12 125 12094sx 12132 14 125 12132 sx 14 125 100 100 304 Chapter II Tests of Hypotheses Xl X2 X is arranged as Xli Xz XIl where Xf XU 1 The ordered abserva tions XC are then plotted against their observed cumulative frequency U O5ln on the appropriate probability paper If the hypothesized distribution adequately describes the data the plotted points will fall approximately along a straight line if the plotted points deviate significantly from a straight line then the hypothesized model is not appropriate Usually the determination of whether or not the data plot as a straight line is SUbjective EieZ To illustrate probability plotting consider the followlllg data 314 10800863 179 1390 563 1436 LlS3 0504 801 We hypotheize that theC data are adequately modeled by a normal distribltion The observations are arranged in llscending order and their cumulative frequencies U 0511 calculated as follows j 05 I n 1390 005 2 801 015 3 0563 025 4 314 035 5 179 045 6 0504 055 7 0863 065 8 L080 075 9 Ll53 085 10 1436 095 The palls of values and U 051 n are now plotted on normal probability paper This plot is sboVtl in Fig 117 Most normal probability aper plots lOOj 05n on the right vertical scale ald 1001 U 05ln on the left vertical scale with the variable value plotted on tle horizontal scale We have chosen to plot Xj versus IOOU 051n On the right vertical in Fig 117 A straight line 1 I 99 2r 98 5 95 10 L 90 20 L SO l t 70 SO 0 I I 50 50 1 1 60 70 SO 20 90 10 95 5 J2 98 LL I 20 15 10 05 0 05 10 15 20 XU Figure 117 Normal probability plot J l 114 Testing for Goodness of Fit 305 chosen subjectively has been drawn through the plotted points In drawing the straight line one should be influenced more by the points near the middle than the extreme points Since the points fall generally near the line we conclude that a normal distribution describes the data We can obtain an estimate of the mean and standard deviation directly from the normal probability plot We see from the straight line in Fig 117 that the mean is estimated as the 50th percentile of the sample orjl 01 0 approximately and the standard deviation is esti mated as the difference between the 84th and 50th percentiles or 0 095 010 085 approximately A normal probability plot can also be constructed on ordinary graph paper by plotting the standardized normal scores Zj against XU where the standardized normal scores satisfy j 05 pZ Zj JZj n For example if j 051n 005 then JZ 005 implies thatZj 164 To illustrate con sider the data from Example 1125 In the table below we have shown the standardized nor mal scores in the last column j XCJ jO5n Zj 1 1390 005 164 2 0801 015 104 3 0563 025 067 4 0314 035 039 5 0179 045 013 6 0504 055 013 7 0863 065 039 8 1080 075 067 9 1153 085 104 10 1436 095 164 Figure 118 presents the plot of Zj versus XU This normal probability plot is equivalent to the one in Fig 117 Many software packages will construct probability plots for various distributions For a 1vfnitab example see Section 116 20 10 10 20 cc 20 10 0 10 20 XU Figure 118 Normal probability plot 306 Chapter 11 Tests of Hypotheses Selecting the Fonn of a Distribution The choice of the distribution hypothesized to lit the data is importan Sometimes analysts can use their knowledge of the physical phenomena to choose a distribution to model the data For example in studying the circuit board defect data in Example 1123 a Poisson distribution was hypothesized to describe the data because failures are an event per unit phenomena and such phenomena are often well modeled by a Poisson distribution Some times previous experience can suggest the choice of distribution In situations where there is no previous experience or theory to suggest a distribution that describes the data analysts must rely on other methods Inspection of a frequency his togram can often suggest an appropriate distribution One may also use the display in Fig 119 to assist in selecting a distribution that describes the data wben using Fig 119 note that the f3 axis increases downward This figure shows the regions in the 3 f3 plane for severa standard probability distributions where 14 3 5 c 6 0 c i 7 J 9 EXfl 7f yJl 232 1 p Figure 119 Regions in the P 3 plane for vmom standard dittibutions Adapted from G I Hahn and S S Shapiro Statisrical Models in Engineering John Wlley Sons New York 1961 used with permission of th publisher and Professor E S Pearson UrrieTI1ty of London 1l5 Coningency Table Tests 307 is a standardized measure of skewness and f3 X IJ t 0 is a standardized measure of kurtosis or peakedness To use Fig 119 calculate the sam pie estimates of 3 and f3 say and where 3 M 2 M 2 II M X X J n h1 j 1234 and plot the point fJ If this plotted point falls reasonably close to a point line or area that corresponds 10 one of the distributions given in the figure then this distribution is a log ical candidate to model the data From inspecting Fig 119 we note that all normal distributions are represented by the point 31 0 and 3 3 This is reasonable since all normal distributions have the same shape Similarly the exponential and uniform distributions are represented by a single point in the 3 3 plane The gamma and lognormal distributions are represented by lines because their shapes depend on their parameter values NOte that these lines are close together whieh may explain why some data sets are modeled equally well by either distri bution We also observe that there are regions of the 3 3 plane for which none of the dis tributions in Fig 119 is appropriate Other more general distributions such a the Johnson Or Pearson families of distributions may be required in these cases Procedures for fitting thesefamilies of distributions lmd fignres similar to Fig 119 are given in HaIm and Shapiro 1967 115 CONTINGENCY TABLE TESTS 11any times the n eleuents of a sample from a population may be classified according to two different criteria It is then of interest to know whether the two methods of classifica tion are statistically independent for example we may consider the population of graduat ing engineers and we rna y wish to determine whether starting salary is independent of academic disciplines Assume that the first method of classification has r levelS and that the second method of classification has c levels We willie Olj be the observed frequency for level i of the first classification method and for level j of the second classification method The data WOUld in general appear as in Table llS Sueh a table is commonly called an r X c contingency table We are interested in testing the hypothesis that the row and eolumn methods of classi fication are independent If we reject this hypothesis we conclude there is some interaction between the two eriteria of classification The exact test procedues are difficult to obtain but an approximate test statistic is valid fot large n Assume the Of to be multinomial ran dom variables and Pij to be the probability that a randomly selected element falls in the ijth 308 Chapler 11 Tests of Hypothese Table 115 An r x c Contingency Table Column Row 2 c 1 i 0 0 12 2 0 Ot 0 0 O cell given that the two classifieations are independent Then Pij Uij where u is the probability that a randomly selected element falls in row class i and Vj is the probability that a randocly selected element falls in column class j Now assuming independence the maximum likelihood estimators of Uj and Vj are Ie ujLOi 12 Il 1 r Vj 20ij n il Therefore assuming independence the expected number of each cell is Then for large n the statistic 11108 11109 2 OijES 2 XoL E XlC 111lO 111 ij approlcimately and we would reject the hypothesis of independence if X X r1I A company has to choose among three pension plans Management wishes to know whether the pref crence forplans independent of job classification The opinions of arandom sanple of 500 employ ees are shown in Table 116 We may compute ul 340500 068 1601500 032 Vi 2001500 0040 il 2001500 0040 and y 100500 020 The expecTed frequencies may be computed from equation lll09 For exanple the expected number of salaried workers favoring pet sinn plsn 1 is L Table 1l7 Expected Frequencies foc Example 1126 Pension Plan 2 3 Total Saillried workers 136 136 68 340 Hourly workers 64 64 32 160 Totals 200 200 100 500 116 Salnpi COT1puter Output 309 The expected frequencies ae shovro in Table 1l7 The test statistic is computed from equation 11110 as follom Since XO 599 we reject the typotbesis of independence and conclude that the preference for pension plans is not independent of job classification Using the tvoway contingency table to test independence between two variables of classification in a sample from a single population of interest is only one application of con tingency table methods Another common situation occurs when there are r populations of interest and each population is divided into the same c categories A sample is then taken from the ith population and the counts entered in the appropriate columns of the illl row In this situation we want to investigate whether or not the proportions in the c categories are the same for all populatioos The null hypothesis in this problem states that the populations are homogeneous with respect to the categories For example when there are only two cat egoriest such as success and failure defective and nondefective and so on then the test for bomogeneity is really a test of the equality of r binomial parameters Calculation of expected frequencies determination of degrees of freedom and computation of the chisquare statistic for the test for homogeneity are identical to the test for independence 116 SAMPLE COMPtJTEROUTPUT There are many statistical packages available that can be used to construct confidence inter vals carry out tests of hypotheses and determine sample size In this section we present results for several problems using Minitab A study was conducted on the tensile strength of a paticular fiber under various temperrtures The results of the study given in MFa are 226237272245428298345201327301317395332238367 310 Chapter 11 Tests of Hypotheses SUpPOSe it is ofiuteres to determine if the mean tensile strength is greater than 250 t1Pa That is test HoJl250 HI Jl 25Q A nonnal probability plot was constructed for the tensile strength and is given in Fig 11 10 The no mality assumption appears to be satisfied The population variance for tensile strength is asswned to be unknown ald as a result a singlesample Hest will be used for this problem The resulrs from Minitah for hypothesis testing and confidence interval OZl the mean are Tst of m 250 vs rou 250 S Meat I Variable N Mean StDev rs 5 3019 659 170 I I Vaxiable 950 LOwer Bound T DoDO S 2720 30S The Pvalue is reported as 0004 leading us to eject the null hypothesis and conclude that the mean tensile strtngth is greater tha1 250 MFa The lower onesided 95 confdence mterval is given as 272u t1qlDipJli28 Reconsider Examp1e 1111 comparing two methods for predicting the shear strength for steel plate girders The Mlnitab output for he paired ttest using Ct 010 is Faild T Karlsluhe N Mean ScDev SE Mean Kar lsruhe 9 1 3401 01460 00487 Lehigh 9 1 0662 00494 00165 Difference 9 02739 01351 00450 190 CI for mean difference I TTest of mean difference ooo 01901 03576 0999 099 095 0 080 a 050 020 005 001 0001 I I I I 200 o v5o not 0 TValce i 300 400 Tenslle strength Figure 1110 Nomal prohbilitypot fur Example 1127 608 PValcc j I i L 116 Sample Conputer Ouqmt 311 The results of tie MinitablPl output are iT agreement with the JeSJlts round in Example 11 17 Mnitabl also provides the appropriate confidence interval forhe problem Using a 010 the level of confidence is 090 the 90 confideote interval on the differelce between the two methods is 0190103576 Since the confidence interval does not coutain Zero we also conclude Liar there is a significant difference between l1e two mefuods The ncrnber of airline flights canceled is recorded for all airlines for each day of service The Dum ber of flights recorded and the Dumber of llese flights that were canceled on a single day ir Match 2001 are provided below for tNO major airlines Airline of Flights American Airlines 2128 America West Airlines 635 41 of Canceled Fligits 115 49 Proporrion Is there a significant difference in the proportion of canceled flights for the two airlines The hypothe ses of interest are He Pl h VefS1S H Pi P2 A twosample test 0 proportions and a twOsided confidece interva on proportions are Sample 1 2 X 115 49 N i128 635 Sample p 0054041 007716 Estimate for p1 p2 00231240 95 ex for 11 12 L0459949 000253139 I Test for pl p2 0 vs not 0 Z 198 FValuc The Pvalile is given as 0048 indicating that there is a significant difference between the proportions of flight canceled for American and America West airlines at a 5 level of significance The 95 confidence interval of Oi60 00003 indicates thatAmeIica West airlines had a statistically sig nificant higher proportion of canceled flights k1an American Airlines for the single day 113 The meaJ compressive strength for a particular highstrength couercte is hypothesized to be fJ 20 MPa It is knOWD that tile standard deviation of compressive strength s j 13 MPa A gOnp of engineers wants to determioe he number of concrete specimens that will be needed in the study to detect a decease ill the mean compressive strength of two stmdard deviations Ii 6c average com pressive strength is atJally less thanu 20 they want to be confident of corretly detecting this sig nificant difference In other words the test of interest would be HQ il 20 versus H J1 20 For this study the significance level is set at a 005 and the power of the test is 1 J 099 What is he win imam nrunber of conerere specimens hat should be used L1 t1is study For a difference of 2a or 26 MFa a005 and 1 f3 099 heminimllm sampie size can be found usingo1initab The result ing output is Testing tnean null versus lull Calculating power for mean Alpha 005 Sgffia 13 nul difertnce i Difference 26 Sample Target Actual SiZe Power Power 6 09900 09936 I 312 Chapter 11 Tests of Hypotheses The rninimum number of specimens to be used in the study shollid be r 6 in order to attain tte desired power and level of significance A rranuiactuet of robber belts wishes to inspect and control the number of nonconforming belts pro duced on line The proportion of noncororrning be that is acceptable is p 001 For practical pur poses if thc proportion increases top 0035 0 greatet the manufacturer wants to detect this change That is the test of interest would be Ho p 001 versus HI P 001 If the acceptable level of sig nificance is a 005 and the power is 1 3 095 how many rubber belts should be selected fat inspection For a 005 and 1 f3 095 the appropriate sample size can be determined using MinitabTheoutputis I TestiLg proportion 00 versus OCl Alpha OOS Alternative Proorior 3SOE02 Sample Size 348 Target Power 09500 Actua2 Power 09502 Therefore to adequately detect a significant change in thc proportion of nonconforming nbbcr belts random sacples of at least n 348 woUd be needed 117 SUMll4ARY This Chapter bas introduced hypothesis testing Procedures for testing hypotheses on means and variances are summarized in Table 118 The chisquare goodness of fit test was intro duced to test the hypothesis that an empirical distribution follows a particular probability law Graphical methods are also useful in goodnessoffit testing particularly when sample sizes are small 1voway contingency tables for testing the hypothesis that two method of classification of a sample are independent were also introduced Several eompurer examples were also presented 118 EXERCISES 111 The breaking strength of a fibet used in manu facuring cloth is tequited to be at least 160 psi Past experience has indicated that the standard deviation of bteaking strength is 3 psi A random sample of four specinens is tested and the average breaking strength is found to be 153 psi a Should the fiber be judged acceptable with aO05 b Vh31 is rhe probability of accepting Ho Ji 160 iftbe fber has a trne breaking strength of 165 psi 112 The yield of a chemical process is being stud ied The variance of yield is known from previous experience with this process to be 5 units of a2 per centagel The past five days of plant operation have resulted in the following yields in percentages 916 88759088995913 a Is there reason to believe chcyield is kss than 901 b What sample size would be required to detect a true mean yield of 85 with probability 095 113 The diameters of bolts are knowu to have a standad deviation of 00001 inch A tandom sample of 10 bolt yields an average diameter of 02546 inch a Test the hypotbesis that the true mean diameter of bolts equals 0255 inch using a 005 b Vihat size sample would be necessary to detect a true mean bolt diameter of 02551 inch with a probability of at least 090 114 Consider the data in Exercise 1039 a Test the hypothesis that the mean piston ring diametetis 74035 rom Use ct 001 b What sanple size is required to detect a troemean diameter of 74030 with a probability of at least 0951 llS ExeJXises 313 Table 118 Sll1llltlay of Hypothesis Testing Procedures on Means and Variances Alternative Criteia for OCCurvc Null Hypothesis Test Statistic Hypothesis Rejection Parameter Ho I i4J z x HIiJ 141 dII i4J11a a known II In Hj1 Pc 42 dIiJ H I i4J 42 d i4J Ill ItJI lalln1 d II iJjia H 1iJ X Po H i u cf unkJ1own ti HIiJ ttaII1 d I 1017 SIlI HIi4J to a d i4J Ill HI Xl X1 H 1 p Iz Za1 dIll1 Of a j flld knowL ZQ r Ja 2 a HjJl P2 Z d JlJtl 01 O Lt iJ I H JJ1jUz 4 Z dJ2 PI T a H 1 tc Hj1Uz Irol tcrn1 2 d II p112a a crunknown HIJ111 to tClI1 h712 d 1 ol2er HI p dfl2Jl2a Ho 1 0 XjX2 HI p 0 0 unknown to r 2 HjJl Jt ICta lSi H1Jll Liz to til v V fl1 1t 2 lSt s 1 v fll 111 2 1 SitI Sr It 1 ll 1 Hoif nlS Hao f z A GIGo 0 XcnI Xo ci or 2 XaXI011lI1 Had o 2 XOXVlIl A oioe H l a Of XX onl flan He 0 FoSlS Hazf cS FrFtPZnnzl A Ci1U1 or Hd Fo FIalllIl Itt FoFl7nl 7z1 Aatl1 115 Consider the data in Exercise 1040 Test the hypothesis that the mean life of the light bulbs is 1000 hours Use Ct 005 ume whether or not this yolure is 160 ounces A random sample is taken from the outpu of each machine U6 Consider the data in Exercise 1041 Test the hypothesis that mean compressive strength equals 3500 psi Gsc a 001 11 7 Two machines a used for filling plastic bottles with a net volume of 160 ounces The filling processes can be assumed normal with standard devi ations OoJ5 and oms Quality engineering s that both machinet fill to the same net vol Machine 1 1603 1601 1604 1596 1605 159S 1605 1602 1602 1599 Machine 2 1602 1603 1597 1604 1596 1602 1601 1601 1599 1600 314 Chapter 11 Tests of Hypotheses a Do you tbink that quality engileering is correct UseaM5 b Assuming equal sample sizes what sample size should be used to assure that f3 005 if the true difference in means is 00751 Asswue that aO05 c What is the power of the test in a for a true dif ference in means of 0075 118 The f1m development department of a local store s considering the replacement of its current Elmprocessing machine The time in which it takes the machine to completely process a roU of film is impoant A random sample of 12 rolls of 24expo sure color film is selected for processing by the ctr rent machine The average processirg time is 81 minutes filth a sample standard deviation of 14 mit utes A random sample of 10 rolls of the same type of film is seected for esting in the new machine The average processiltg tine is 73 minutes wit a sample standard deviation of 09 minutes The local store Vill not pUIChase the new machine unless the processing time is more than 2 minues shorter than the current machine Based on this information should they pur chase the new machine 119 Consider the data in Exercise 145 Test the hypothesis that both machines filllO the same volum Use Ct 010 11 10 Consider the data in Exercise IO4ti Test He j1 Jl against H U JIz using a 005 llll Consider the gasoline road octane number data in Exercise 1047 If fonmlation 2 produces a higher road octane number than fClfl4ulation 1 the ranufacturer would like to dctect tlis Fonnulate and test en appropriac hypohesis using X 005 11 12 The laeral deviation in yards of a certain type of mortar shell is being investigated by the propellant manufacurer The following data have been observed Round Deviation Round Deviation 1128 6 948 2 1042 7 625 3 l51 8 IOll 195 9 l65 5 647 10 068 Tes the hypothesis that the mean latetru deviation of these mortar shells is ffO Assume that lateral devia tion is normally distributed 1113 The shelf life of a photographic film is of interest to the manufacturer The manufacturer observes the following shelf life tor eight urits chosen at random fron the current production Assume that shelf life is normally distributed 108 days 128 days 134 163 124 116 159 134 a Is there any evidence that te mean shelf life is greater than or equal to 125 days b If it is important to detect a ratio of Clrr of 10 with a probability 090 is the sample size sufEcient 11 14 The titanium content of an alloy is being stud ied in the hope of ultimately increasing the tensile strength An analysis of six recent heats choSen at mc dam produces the following ritanium conents 80 77 99 116 99 146 Is thetC any evidence that the mean titanium content is greater than 951 1l15 An article in the Journal of Construction Engineering and Management 1999 p 39 presents some data on the nwuber of work hours lost per day on a construction project due to weatherrelated inci dents Over 11 workdays the following lost work hours were recorded 88 88 125 122 54 133 128 69 91 22 147 Assuming work hours are rotma11y distributed is there any evidence to conclude that the mean number of work hours lost per day is greater than 8 hours 11 16 The percentage of scrap produced in a leW finishing operation is hypothesized to be less than 75 Several days were chosen at random and the percentages of scrap were calculated 551 732 649 881 646 856 537 746 2 In your opinion is the true scrap rate less than 751 b Ifitls important to detect a ratio of JIG 15 vifu a probability of at least 090 what is the mini mum sample size hat can be used c For 5120 what is the power of the above test L 1117 Suppose that we must test the hypotheses HI1 15 H1115 where it is knoVtn that c 25 If a 005 and the trUe mean is 12 what sample Si4C is necessary to assure 11 type IT ernr of 5 1118 An engineer desires to test he hypothesis that the melting point of an alloy is lOOOQC 1f the true melting point differs from this by more thM 20CC he must change the alloys composition If we assume that the melting point is a normally distributed ran com variable 0005 thOlO and 1 lOC how many observations should be t2ken 1119 Two methods for producing gasoline from c41ce oil are being iwestigated The yields of both processes are asscrced to be normally distributed The follo1ng yied data have been obaind from the pilat plant Process 1 2 Yields 0 242 266 257 210 221 28 209 224 220 Ca Is there reason to believe that process 1 has a greater mean yield Use 0 OOL Assume that bolli variances are equal Co lssuming that in order to adopt process 1 it must produceamem yield thatls at least5 greaer than tiat of process 2 what are your reommendations e FlIlrl the power of the test in part a lithe mean yield of process 1 is 5 greater Llan that of process 2 d VIhat samp1e size is required for the test in part a to ensure that the null hypothesis will be rej with probability of 090 if tho mean yield of process J exceeds tbe mean yield of process 2 by 5 1120 An article that appeared in the Proceedings of the 1998 Winter Simulation Conference 1998 p 079 discusses the concept of validation for traffic simulation OOels The stated purpose of the study is to design aad modify the facilities roadways and con trol devices to optimize efficiency and safety of traf fle flow Pan of the study compares speed observed at various irtersections and speed simulated by a model being tested The goal is to determine whether the simulation model is representative of the actual observed speed Field data 1s collected at a particular location and then the simulation model is imple mented Fourteen speeds ftsec are measured at a particular location Fourteen observations are simu tated using the proposed model The data are 1 8 Exercises 315 Field Model 5333 5714 4740 5820 5333 5714 4980 5900 5333 6154 5190 6010 5517 654 5220 6340 5517 6154 5450 6580 5517 6957 5570 7130 5714 6957 5670 7540 Assuming the variances are equal conduct a hypoth esis test to determine whether there is a significant difference between the field data and the model simu lated data Use a 005 1121 The foUoving are the bulning times in min utes of flaes ofVo different types Type Type 2 63 82 64 56 81 68 72 63 57 59 83 74 66 75 59 82 82 73 65 82 a Test the hypothesis that the two variances are equaL Use 0005 b Using the results of a test the hypothesis that the mean buming times are equal 11 22 A new filtering device is instaled in a cberll cal unit Before its itstaEation a random sample yielded the foCoving in1ormaion about the percent age of impurity XI 125 si 10117 and n 8 Aiter installation a random sample yielded x 102 1 sl9473nz9 a Can you conclude that the two variances are equal b Has the filtering device reduced the percentage of impurity significantly 1123 Suppose that two rando samples were drawn from nornal populations with equal variances The sunple data yields x 200 nl 10 Lx Xjl 1480 X 158 n 10 and Jx xj 1425 a Test the hypotlesis that the No means are equal Use 0001 b Fwd the probability that the null hypothesis in a till be rejected if the tue difference in means is 10 Ce What sample size is required to detect a true dif ference in means of 5 with probability at least 080 if it is known at the starr of the experiment that a rough estimate of the common variance is 150 316 Chapter 11 Tests of Hypottleses 1124 Consider the data in Exercise 1056 a Test the hypothesls that the means of the two nor mal distributions are equal Use a 005 and assume that cr 0 b vhat sample size is required to detect a difference in means of20 with a probability of at least 0SS1 c Test the hypothesis that the variances of the two distributions rue equaL Use J 005 d Find the power of the tesl in c if the variance of a population is four times the other 1125 Consider the data in Exercise 1057 Asswo fig that 0 test the hypothesis that the mean rod diameters do not differ Use a 005 1126 A chemical company produces a certain drug whose weight has a standard deviation of 4 mg A new method of producing this drug has been proposed although some additioual cost is involved Manage ment wpl authorize a change in production technique only if the standard deviation of the weight in the new process is less than 4 mg If the standard deviation of weight in the new process is as small as 3 mg the company would like to switch production methods with a probability of at least 090 Assuming weight to be normally distributed and a 005 how many observations should be taken Suppose the reseaChers choose n 10 and obtain the data below Is this a good choice for n What SIlOuld be their decision 16628 grams 16622 16627 16623 16618 16630 grams 16631 16624 16622 16626 1127 A manufacturer of precision measuring instru ments clains that the standzrd deviation in the use of the isilcnt is 000002 inch A1 analyst who is u1aware of the claim uses the instrument eight times and obtain a sample standard deviation of 000005 inch a Using a 001 is the claimjustiiied b Compute a 99 confidence interval for the true variance c Vhat is the power of the test if the true standard de1ation equals 0000047 d What is the smallest sample size that can be used to detect a true standard deviation of 000004 with a probability at least ofO95 Use a 001 1128 The standard deviation of measuements made by a special thermocouple is supposed to be 0005 degree If the standard deviation is as great as 0010 we wish to detect it with a probability of at least 090 Use a 001 What sample size should be used If 1 this sanple size is used and the sample standard devi ation s 0007 what is your conclusion using a 001 Construct a 95 upperconfidence interval for he true variance 1129 The manufacturer of a power supply is inter ested in the variability of output voltage He has tested 12 units chosen at random with the following results 534 565 476 500 555 554 507 535 544 525 535 461 a Test the hypothesis t1at cr 05 Use a005 b If the true value of cr LO what is the probabil 1 ity that the hypothesis in a vill be rejected I 1130 For the data in Exercise 117 test the hypoh esis that the two variances are equal using a 001 Does the result of this test influence the manner in which a test on means would be corducted1 What sample size is necessary to detect j 25 With a probability of at least 0901 1131 Consider the fOliOVling two sampies draVtD from two normal populations Sample 1 Sa11ple 2 434 187 500 200 497 200 425 185 555 211 655 231 637 228 555 207 376 176 191 200 Is there eviderce to conclude that the variance of pop ulation 1 is greater than the variance of population 21 Use a 001 Find the probability of detecting if1c 40 1132 Two machines produce metal parts The yari ancc of the weight of these parts is of interest The fol lowing data have been collected Machine 1 1 25 x 0984 s 1346 Machine 2 1230 x 0907 965 I a Test the hypothesis that the varances of the twO macrrnes are equaL Use a005 b Test the hypothesis that the two machines prooJce parts having the same mean weight Usc a 005 1133 In a hardness test a steel ball is pressed into the material being tested at a standard load The diam eter of the indentation is measured which is related to the hardness Two types of steel balls are available and thei peiormance is compared on 10 specimens Each specinen is tested twice once with each ball The results are given below Ball x 75 46 57 43 58 32 61 56 34 65 Bally 52 41 43 47 32 49 52 44 57 60 Test the hypothesis that the two steel balls give the same expected hardness measurement Use a 005 1l34 11oypes of exercise equpnentA and B for handicapped individuals are often Sed to determine the effect of the particulu exercise aD heart rate in beats per minute Seven subjectS participated In a study to determine whether the two types of equip ment have the same effect on hem rate The results are given in the table below Subject A B 1 162 161 2 163 187 3 140 199 4 191 206 5 160 161 6 158 160 7 155 162 CQndct an appropriate tCSt of hypohesis to deter mine whether there is a significant difference in heart rate due to the type of equipmcnt used 1135 An aircraft designer has theoretical e 1dence tbat painting lhe airplane reduces its speed at a speci fied power and flap setting He tests six consecutive airplanes from the assembly line before and after painting The results are shown below Top Speed mph Airplane Painted Not Pairted 1 286 289 2 285 286 3 279 283 4 283 288 5 281 183 6 286 289 118 Exercises 317 Do the dta support the designers theory Use aOOS 1136 An article in the International Journal of Fatigue 1998 p 537 discusses the bending fatgue resistance of ge teeth when using a paticular pre stressing or presetting process Presetting of a gear tooth is obtained by applying and then removing a single overload to the machine element To detcrmine significant differences in fatigue resistance due to pre setti1g fatigue data were A preset tooth and a nonpreset tooth were paired if t1ey were present on the same gear Eleven pairs were formed ald he fatigue life measured for elCrL The final response of inrerest is Infatigue life X 10n Pair Preset Tooth Nonpreset Tooth 3813 2706 2 4025 2364 3 3042 2773 4 3831 2558 5 3320 2430 6 3080 2616 7 2498 2765 8 2417 2486 9 2462 2688 10 2236 2700 11 3932 280 Conduct a test of hypothesis to deermine whether presetfulg significantly icreascs the fatigue life of gear teeth Use aO10 1137 Consider the deta in Exercise 1066 Test the hypothesis that the uninsaw rate is 10 Use a 005 113S Consider the data in Exercise 1068 Test the hypothesis t13t the fraction of defective clLculators produced is 25 1139 Suppose that we ish to est the hypothesis Ho PI fJ against the alternative HI iJl 112 where botb variances and a are knonIl A total of fl nz N observations can be taken How should these observations be allocated to the two populations to mvimize the probability thatHo will be rejected if Hj is true and JlI P1 b O 1140 Consider the rion membership study described in Exercise 1070 Test the hypothesis that the proportion of men who belong to a union does not differ from the proportion of women who belotg to a union Use a005 1l41 Using the data in ase 1071 determine Whether it is reasonable to conclude that production line 2 proeuced a higher fraction of defective prcdlct tian ilie 1 Use a 001 318 Chapter 11 Tests of Hypothees 1142 Two different types of injectiomolding machines are used to fonn plastic parts A part is con sidered defective if it has excessive shrinkage or is discolored Two random samples each of size 500 are selected and 32 defective parr3 are found lj the sample from machine 1 while 21 defectlle part are found in the sample from machine 2 Is it reasonable to coc1ude that both machines produce tie same me don of defective parts 1143 Suppose that we wish to test Ho PI fi2 agains H flj i il2 where and 0 are known The total sample size N is fLxed bu the allocation of observations to tllC two populations such that r 2 N is to be made on ilie basis of cost If the costs of sampling for populations 1 and 2 are Ct and rcspectively find the minimum cost sample sizes that provide a specified vaiance for the difference in sam pie means 11 44 A manufacturer of a new pain relief tab1et would like to demonstrate tnat her product works twice as fast as hetcompetltors product Specifically she would like to test Ho 1 211 H 1 211 where Jl is the mean absorption time of he competi tive product and f11 is tie mean absorption time of the nevi product Assuming that he variances and 0 are known suggest a procedure for testing this hypotiesis 1145 Derive an expression siTIi1ar to cquation 11 20 for the 3 error for the test on the variance of a nor mal distribution Assume that tie twosided alternative is speCified 1146 Derive an expression similar to equation 11 20 for he 3 error for the test of the equfuity of the variances of two normal clistributions Assume that the lVtOsided alternative is specified 1147 The number of defective nits found each day by an incircci functional tester in a printed circuit board assembly process is shovU below Number of Defectives 110 1115 1620 2125 2630 3135 3640 4145 Times Observed 6 11 16 28 22 19 lL 4 a It is reasoable to conclude that these data come from a normal distibution Use a chisquare goodncssoffit test b Plot the data on nonnal probability paper Docs a1 assumption of norrnality seem justified 1148 Defects on wafer su1aces in integrated cirelli fabrication are unavoidable In a particular process the follol1ting data were collected Number of Number of Wafers Defeetsi with i Defects 0 4 13 2 34 3 56 4 70 5 70 6 58 7 42 8 25 9 15 10 9 11 3 12 1 Doos the assumption of a Poisson distribution seem appropriate as a probability model forhis process 1149 A pseudora1dom number generator is designed so that integers 0 throgh 9 have an equal probarility of occurrence Thefirst 10000 numbers are as follows 0123456789 967 1008975 122 1003 989 1001 981 1043 1011 Does this generator seem to be working properly 1150 The cycle time of an automatic machine las been observed and recorded a Does the nortXU11 distribution seem to be a reason able probability model for the cycle time Use the chisquare goodnessoffit test b Plot the data on IlQrmal probability aper DOO1l the assumption of normality seem reasonable 1151 A soft drink bottler is sudying the interrul pressure strength of lliter glass norreturnable bot tIes A random sample of 16 borGes is tested and the pressure strengLs obtained The data are shown below Plot these data on normal probabiliry paper Does it seem reasonable to conclude that presste strength is normally distributed 22616 psi 21114 psi 20220 20362 21954 18812 19373 22439 20815 22131 9545 20455 9371 20221 20081 20163 112 A company operates four machines for three shifts each day From production records the follow ing data on the number of breakdowns ate collected 2 3 31 15 II 17 9 16 14 10 Test the hypothesis that breakdowns are independent of the shift 1153 Patients in a hospital are classified as surgical or rtedicaL A record is ept of the number of tmes patients require nusing service during te night and whether these patients are or Medicare or not The data are as follows Patient Category Medicare Surgical Medical Yes 46 52 No 36 43 Test the hypothesis that caJs by surgicalmedical patients are independen of whether the patierrs are receiving Medicare 11 54 Grades L1 a statistics course and aD operations research course taken simultaneously were as follows for a group of students Statistics Operations Research Grade Grade A B C Other A 25 6 17 13 B 17 16 15 6 C 18 4 18 10 Other 10 8 11 20 fe the grades in statistics and operations research related 1155 All experiment with artillery shells yields the following data on the characteristics of laterdl deflec 118 Exercises 319 tions and ranges Would you conclude that deflection and range are independent RaDge yards 0 1999 2000 5999 6000 1l999 Latera Deflection Left Normal Rigtt 6 9 8 14 II 17 8 4 6 1156 A stcdy is being made of the failures of an electronic componert There ate four tpcs of failJIes possible arid two mounting positions for the deviee The folloVling data have been taken Fellure Type Mounting Position A B C D 22 46 18 9 2 4 17 6 12 Would you conclude that the trpe of failure is inde pendent of the moorting position 1157 An article in Research in Nursing and Health 1999 p 263 summarizes data collected from a pre vious study Research in Nursing and Healih 1998 p 285 on the relationship between physical activity and socioeconomic status of 1507 Caucasian women The data artl given in the table below Physical Activtty Sociaeconomic Stats Inactivc Active Low 216 245 Medium 226 409 Hgh 114 297 Test the hypothesis that physical activity is bdepend ent of sociaeconomic status 1158 Fabric is graded into three classifications A E and C The results below were obtained from five looms Is fabric classification independent of th loom Loom 2 3 4 5 Jber of Pieces of Fabric in Fabric Classi5catio A B C 185 16 12 190 24 21 170 35 16 58 22 7 185 22 15 320 ChaptCI 11 Tests of Hypotheses 1l59 An article in the Joumal of Marketing Research 1970 p 36 reports a study of the relation ship between facility conditions at gasoline stations and the aggressiveness of their gasoline marketing pol icy A sample of 441 gasoline stations was investigated with the results shown below obtained Is there ei dence that gasoline pricing strategy and facility CODdl tions are independent Aggressive Neutral Nonaggrcsive Condition Subst2ndard Standard Modem 24 15 17 52 73 80 58 86 36 l1O Consider the injection molding process described in liCercise 1142 3 Set up this problem as a 2 X 2 contingency table and perform the indicated statistical aalysis b State deady the hypothesis being tested Are you testing homogeneity or independence c Is tbis procedure equivalent to the test procedure used if Exercise 142 Chapter 12 Design and Analysis of SingleFactor Experiments The Analysis of Variance Experiments are a natural part of the engineering and management decisionmaking process For example suppose that a civil engineer is investigating the effect of curing methods on the mean compressive strength of concrete The experiment would consist of making up several test specimens of concrete using each of the proposed curing methods and then testing the compressive strength of each specimen The data from this experiment could be used to determine which curing method should be used to provide maximum compressive strength If thee are only two curing methods of interest the experiment could be desigled and analyzed using the methods discussed In Chapter 11 That is the experimenter has a single facwr of interest curing methodsand there are only two levels of the factor If the exper imenter is intereted in detenrining which curing method produces the maximum com pressive strength then the number of specimens to test can be determined using the operating characteristic curves in Chart VI Appendix and the ttest can be used to deter mine whether the two means differ Many singlefactor experimentS require more than two levels of the factor to be con sidered For example fPe civil engineer may have five different curing methods to investi gate In this chapter we introduce the analysis of mance for dealing with more than Wo levels of a single factor In Chapter 13 we show how to design and analyze experiments with se eral factors 121 THE COIVIPLETELY RAIDOMIZED SINGLEFACTOR EXlERIMT 1211 An Example A manufacturer of paper used for making grocery bags is interestedin improving the ten sHe strength of the product Product engineering thinks that tensile strength is a function of the hardwood concentration in the pulp and that the range of hardwood concentrations of practical interest is between 5 and 20 One of the engineers responsible for the study decides to investigate four levels of hardwood concentration 51015 and 20 She also decides to make up six test specimens at each concentration level using a pilot plant PJl 24 specimens are tested on a laboratory tensile tester in random order The data from this experinent are shown in Table 121 Thii is an example of a completely randomized singlefactor experinent with four lev els of the factor The 1evels of the factor are sometimes called treatments Each treatmem 321 322 Chapter 12 The Analysis of Vaance Table 121 TellSLe Strength of Paper psi Hardwood Observations Concentration 2 3 4 5 6 Totals Averages 5 7 8 15 11 9 10 60 1000 10 12 17 13 18 19 15 94 1567 15 14 18 19 17 16 18 102 1700 20 19 25 22 23 18 20 127 2Ll7 383 1596 has six observations or replicates The role of randomization in this experiment is extremely important By randomizing the order of the 24 runs the effect of any nuisance variable that may affect the observed tensile strength is approximately balanced Ollt For example suppose that there is a warmup effect on the tensile tester that is the longer the machine is on the greater the observed tensile strength If the 24 runs are made in order of increasing hardwood concentration ie all six 5 concentration specimens are tested first followed by all six 10 concentration specimens etc then any observed differences due to hardwood concentration could also be due to the warmup effect It is importart to gT2pwcally analyze the data from a designed experiment Figure 121 presents box plots of tensile strength at the four hardwood concentration levels This plot indicates that changing the hardwood concentration has an effect on tensile strength specifically higher hardwood concentrations produce bigher observed ensile strengill Fur thennore the distribution of tensile strength at a particular hardwood level is reasonably symmetric and the variability in tensile strength does not change dramatically as the hard wood concentration changes 30 25 20 0 S c rn c 15 i 0 0 c 10 J I 0 Hardwood conce1tration Figure 121 Box plots of hardwood concentration data 121 The Completely Randomized SingleFactor Experiment 323 Graphical interpretation of the data is always a good idea Box plots show the vari ability of the observations within a treatment factor level and the variability between treat ments We now show how the data from a singlefactor randomized experiment can be analyzed statistically 1212 The Analysis of Variance Suppose we have a different levels of a single factor treatments that we wish to compare The observed response for each of the a treatments is a random variable The data would appear as in Table 122 An entry in Table 122 say Yip represents the jth observation taken under treatment i We initially consider the case where there is an equal number of obser vations n on each treatment We may describe the observations in Table 122 by the linear statistical model i 12 a Yij J1fj Eij 12 n 121 where Yij is the Uith observation J1 is a parameter common to all treatments called the overall mean t is a parameter associated with the ith treatment called the ith treatment effect and ij is a random error component Note that Yij represents both the random vari able and its realization We would like to test certain hypotheses about the treatment effects and to estimate them For hypothesis testing the model errors are assumed to be normally and independently distributed random variables with mean zero and variance 02 abbrevi ated NIDO 02 The variance 02 is assumed constant for all levels of the factor The model of equation 121 is called the onewayclassification analysis of variance because only one factor is investigated Furthermore we will require that the observations be taken in random order so that the environment in which the treatments are used often called the experimental units is as uniform as possible This is called a completely ran domized experimental design There are two different ways that the a factor levels in the experiment could have been chosen First the a treatments could have been specifically chosen by the experimenter In this situation we wish to test hypotheses about the fl and conclusions will apply only to the factor levels considered in the analysis The conclusions cannot be extended to similar treatments that were not considered Also we may wish to estimate the 71 This is called the fixed effects model Alternatively the a treatments could be a random sample from a larger population of treatments In this situation we would like to be able to extend the conclusions whiCh are based on the sample of treatments to all treatments in the population whether they were explicitly considered in the analysis or not Here the f are random variables and knowledge about the particular ones investigated is relatively useless Instead we test hypotheses about the variability of the 7 and try to esti mate this variability This is called the random effects or components of variance model Table 122 Typical Data for OneWayClassification Analysis of Variance Treatment Observation Totals Averages y y y y y 2 y y Y2l1 y y a Yo Yo YO Yo Ya 324 Chaptcr 12 The Analysis of Variance In this section we will develop the analysis of variance for the fixedeffects model one way classification In the fixedeffects model the treatment effects are usually dnfined as deviations from the overall mean so that 122 Let y represent the total of the observations under the ith treatment and Y represent the average of the observations under the ith treatment Similarly let y represent the gnnd total of all observations and y represent the grand mean of all observations Expressed mathematically i12 a n 123 y 22ii YyN 11 ii whereN an is the total number of observations Thus the dot sUbscript notation implies summation over the subscript that it replaces We are interested lJ testing the equality of the a treatment effects tsing equation 122 the appropriate hypotheses are Ho t 1J 4 0 H 0 for at least one l 124 That is if the null hypothesis is true then each observation is made up of the overall mean fL plus a realization of the random error Ell The test procedure for the hypotheses in equation 124 is called the analysis of vari aIlee The name analysis of variance results from partitioning total variability in the data into its component pa1S The total corrected sum of squares which is a measure of total variability in the data may be written as or a a 2 22YiyYi Yi l jl Note that the crossproduct term in equation 126 is zero since iYij Yi YinYi YinYin 0 jl Thereforej we have 125 126 127 12 1 The Completely Randomized SingleFactor Experiment 325 Equation 127 shows hat he total variability in the data measured by he total corrected sum of squares can be partitioned into a sum of squares of differences betveen treatment means and the grand mean and a sum of squares of differences of observations within treat ments and the treatment mean Differences betVeen obsened treatment meanS and the grand mean measure the differences between treatments while differences of observations within a treatment from the treatment mean can be due only to random error Therefore we write equation 127 symbolically as SST SStraIme1l SSe where SST is the total sum of squaes SSreolm is the sum of squares due to treatments Le between treatments and SSE is the sum of squares due to error ie within teatments There are an Ntotal obseJlations thus SST has N 1 degrees of freedom There are a lev els of the factor so SSts has a 1 degrees of freedom Finally widrin any treatment there are n replicates providing n 1 degrees of freedom Vlith which to esthnate the exper imental error Since there are a treatmen we have an 1 an a N a degrees of freedom for error Now consider the distributional properties of these sums of squares Since we have asstmed that he errors are NIDQ a he observations Yij are NIDu i 0 Thus SSra is distributed as cbisquare with N 1 degrees of freedom since SST is a sum of squares in normal random variables Vie may also show that SSl1omi1d is chisquare with a 1 degrees of freedom if Hois true and SSfid is chisquare witt N a degrees of free dom However all three sums of squares ate not independent since SStrmnu and SSE add up to SST The followmg theorem which is a special form of one due to Cochran is useful in developing the test procedure Theorem 121 Cochran Let Z be NIDO I for i 1 2 v and let wheres v a1d Q is chisquare with Vi degrees offreedom i 1 2 3 Then QH Qtt Q are independent chisGuare random variables with Vl v2 v degrees of freedom respectively if and only if Using this theorem we note that the degrees of freedom for SSttmn and SS add up to N 1 so that SSmJa and SSea are independently distributed cbisquare random variables Therefore under the null hlothesis the statistic 128 follows the Fa IN a distribution The quantities MSrwnu and yS are rrean squares The expected values of the mean squares are used to show that Fo in equation 128 is an appropriate test statistic for Ho 1 0 and to determine the criterion for rejecting this null hypothesis Consider 326 Chapter 12 The Analysis of Variance Substituting the model equation 121 into this equation we obtain EMSE N 0 E tiC Hut tiHI elj In ow on squaring and taking the expectation of the quantities within brackets we see that terms involving e7 and Lf7j are replaced by if and nf respectively because E 1 O Furthermore all cross products involving 1 have zero expectation Therefore after squar ing taking expectation and noting that Il rl 0 we have Using a similar approach we may show that a n2rf EfS 2 1 1 IYl trearmellti J aI From the expected mean squares we see that MSE is an unbiased estimtorof cr Also uncer the null hypothesis ASueIltmeot is an urbiased estimator of cr However if the null hypothesis if false then the expected value of ofStmeJl is greater than vl Therefore under the alternative hypothesis the expected value of the numerator of the test statistic equation 128 is greater than the expected value of the denominator Consequently we should reject HJ if the test statistic is large This implies an uppertail onetail critical region Thus we would reject H if where Fe is computed from equation l28 Efficient computational formulas for the sums of squares may be obtained by expand ing and simpJifying the definitions of SSa and SST in equation 12 7 This yields SST I2 i1 Il N 129 and 1210 i r i I 12 1 The Completely Randomized SngleFactor Expriment 327 The error sum of squares 1s obtained by subtraction SS SST SSrreomenrr 12Jl The teSt procedure is summarized in Table 123 This is called an analysisofvariance table Consider the hardwood concentration experiment described in Section 1211 We can use the analy sis of wriane to test he hypothesis that different hardwood concentrations do not affect the mean tensile strength of the paper The sums of squares for analysis of variance are computed from equa dOO 129 1210 and 1211 as follows 383 7 8J 20t51296 24 60 794 102 127 6 SSE SST SStlelItnlnt 5129638279 13017 383 38279 24 The analysis of variance is sumrrarzed in Table 124 Since FocltO 494 we reject Ho and con clude that hardwood concentralio b the pulp significantly affects the strelgth of the pilper 1213 Estimation of the Model Parameters It is possible to derive estimators for the parameters in the oneway analysisofvariance model Table 123 Analysis ofVaance for the OIeWayaassification FixedEffects Model Source of Sum of Degrees of Mean Variation Squares Freedom Square Fo Betweca treatments SStmlmOOIt aI MSr1 rOT within treatments SSE Na MS Total SST NI Table 124 Analysis of Variance for the Tensile Strength Data Source of Sum of Degrees of Mean Variatio Freedom Hardwood concentration 38279 3 12760 1961 Error 13017 20 651 Total 51296 23 328 Chapter 12 The Analysis of Variance An appropriate estimation criterion is to estimate J1 and 1 such that the sum of the squares of the errors or deviations Ei I is a minimum This nethod of parameter estimation is called the method of least squares In etinating l and i by least squares the nonnality assump tion on the errors Elf is not needed To find the leastsquares estimators of J1 and 1 we form the sum of squares of the errors 1212 and find valnes of l and T say j and ii that minimize L The values p and ii are the solu tions to the a 1 simultaneous equations L al 0 jl i12 a Differentiating equation 1212 with respect to JJ and 11 and equating to zero we obtain a iLIYirjr 0 il Jl and 2IYirjiO i 12a 11 After simplification these equations become NjJnf1 rii1 nfy t m Y2 1213 Equations 1213 are called the leastsquares normal equations Notice that if we add the last a normal equations we obtain the first normal equation Therefore the normal equations are not linearly independent and there are no unique estimates for f4 tp1zt One way to overcome this difficulty is to impose a constraint on the solution to the normal equations There are many ways to c1loose this constraint Since we have defined the treatment effects as deviations from the OYerall mean it seems reasonable to apply the constraint 1214 Using this constraint we obtain as the solution to the normal equations i 1 2 a 1215 121 The Completely Randomized SingleFactor Experiment 329 This solution has considerable intuitive appeal since the overall mean is estimated by the grand average of the observations and the estimate of any treatment effect is just the differ ence bctlleen the treatment average and the grand average This solution is obviously not unique because it depends On the constraint equation 1214 that we have chosen At first this may seem unfortunate because two different experimenters could analyze the same data and obtain different results if they apply differ ent constraints However certainfimctions of the model parameter are estimated uniquely regardless of the constraint Some examples are ir which would be estimated by 1 ip and f1 ri which would be estimated by it 7 Since we are usually interested in differences in the treatment effects rather than their actual values it causes no concern that the 71 cannot be estimated uniquely In general any function of the model parameters that is a linear combination of the lefthand side of the normal equations can be estimated lI1liquely Functions that are lI1liquely esrimated regardless of which constnlint is used are called estimable functions Frequently we would like to construct a confidence interval for the ith treatment mean The mean of the ith treatment is l 1 2 a A point estimator of f1l would beil it 1i Nowt if we assume that the errors are nor mally distributed each 1 is 1D1 iln Thus if i were known we could use the nQr mal distribution to construct a confidence interval for fli V sing MS E as an estimator of cr we can base the confidence interval on the t distribution Therefore a 1001 a confi dence interval on the ith treatment mean Pi is 1216 A 1001 a confidence interval on the difference between any two treatment means say J11 lj is l12 We can use the results given previously to estimate the mean tensile strengths at different levels of hardwood concentration for the experiment in Section 12 1 L The mean tensile strength estimateS are Yl AI 1000 psi jJllYfc 1567 psi J ilm 2Ll7 psi A 95 codidence interval on the mean tensile strength at 20 hardwood is found from equation 1216 as follews 2L17 2086I65lj6 2Ll7 217 330 Chapter 12 The Analysis of Variance 1he desired confidence interval is 1900 psi Im 2334 psi bual examinarion of the data suggests that mean tensile strength at 10 and 15 hardwood is si1l1 ilar A confidence interVal on the difference in means J1lSh JllM is Yi Yj tclNQ2MS5n 17oo1j67208626516l L33307 Thus the confidence intm3l on J15 Iljl is 174 51l1YJ JiM 440 Since the confidence interval includes zero we would conclude that there is no difference in mean tensile strength at these two particula laniwooC levels 1214 Residual Analysis and Model Checking The oneway model analysis of variance assumes that the observations are normally and independently distributed with the same variance in each treatment or factor level These assumptions should be checked by examining the residuals We define a residual as ei Yl y that is the difference between an observ3tion and the corresponding treatment mean The residuals for the hardwood percentage experiment are shown in Table 125 The normality assumption can be checked by plotting the residuals on normal probabil ity paper To check the assumption of equal varia1ces at each factor level plot the residuals against the factor levels and compare the spread in the residuals It is also useful to plot the residuals agamst 1 sometines called thefitted value the variability in the residuals should not depend in any way on the value ofy Vvfien a pattern appears in these plots it usually suggests the need for transfonnation that is analyzing the data in a different metric For example if the variability in the residuals increases with Yi then a transformation such as log y or Y should be considered In some problems the dependency of residual scatter in1 is very important information It may be desirable to select the factor level that results in maximum y however this level may also cause more variation in y from run to run The independence assumption can be checked by plotting the residuals against the time or run order in which the experiment was performed A pattern in this plot such as sequences of positive and negative residuals may indicate that the observations are not independent This suggests that time or run order is important or that variables that change over tirne are important and have not been included in the experimental design A normal probability plot of the residuals from the hardwood concentration experiment is shown in Fig 122 Figures 123 and 124 present the residuals plotted against the treat Tabl125 Residuals for the Tensile Strength Experiment Hardwood Concentration Rsiduals 5 300 200 500 100 lOO 000 10 367 133 267 233 333 1J67 15 300 100 200 000 100 LOa 20 217 383 083 183 317 Ll7 121 The Completely Randomized SingleFactor Experiment 331 ment number and the fitted value YI These plots do not reveal aly model inadequacy or unusual problem with the assumptions 1215 An Cnbalanced Design In some singlefactor experiments the number of observations taken under each treatment may be different We then say that the design is unbalanced The analysis of variance described earlier is still valid but slight modifications must be made in he sums of squares formulas Let n observations be taken under treatment ii 1 2 a and let the total 2 Residual value 4 6 Figure 122 lonnal probability plot of residuals from the hardwood corcentration experiment t 4 2 1 2 3 4 2 t Figure 123 Plot of residuals YS treatment 332 Chapter 12 Tle A1alysis ofVatiance 2 f I Figure 124 Plot of residuals vs it number of observations N kln l The computational fOIDlulas for SST and SSfftltilYrJs become a SST 223 rl j N and In solving the normal equations the constraint r Intl 0 is used No other changes are required in the analysis of variance 111ere are two important advantages in choosing a balanced design First the test sta tistic is relatively insensitive to small departures from the assumption of equality of vari ances if the sample sizes are equal This is not the case for unequal sample sizes Second the power of the test is maximized if the samples are of equal size 122 TESTS ON INDIVIDUAL TREATMENT MEANS 1221 Orthogonal ConttdSts Rejecting the null bypothesis in the fixedeffectsmodeI analysis of variance implies that there are differences between the a treatment means1 but the exact nature of the differences is not specified In this siruation further comparisons between groups of treatment means may be usefuL The ith treatment mean is defined as Ili I T and 1 is estited by ii Comparisons between treatment means are usually made in tenrs of the treatment totals y Consider the hardwood concentration experiment presented in Section 1211 Since the hypothesis Eo j 0 was rejected we know that some hardwood concentrations pro duce tensile strengths different from others but which ones actually cause this difference J 122 Tests on Individual Treattnent Means 333 We might suspect at the outset of the experiment that hardwood concentrations 3 and 4 pro duce the same tensile stren implying that we would like to test the hypothesis H p H p 11 This hypothesis could be tested by using a linear combination of treatment toWs say YrYO If we had suspected that the average of hardwood concentrations 1 and 3 did not differ from the average of hardwood concentrations 2 and 4 then the hypothesis would have been Ho III P P 11 HI 1 p p 1 which implies that the linear combination of treatment totals YI y y y 0 In general the comparison of treatment means of Interest will imply a linear oornbina tion of treatment totals such as a ClCii i1 with the restriction that L tC O These linear combinations are called cOntrasts The sum of squares for any contrast is 1218 and has a single degree of freedom If the design is unbalanced then the comparison of treatment means requires that 1 1niGo 0 and equation 12 18 becomes 1219 A contrast is tested by comparing its sum of squares to the mean square error The resu1t ing statistic would be distributed as F with 1 and N a degrees of freedom A very important special case of the above procedure is that of orthogonal contrasts Two contrass with coefficients cJ and dJ are otthogonal if or for an unbalanced design if 334 Chapter 12 The Analysis of Variance For a treatments a set of aI orthogonal contrasts will partition the sum of squares due to treatments into aI independent singledegreeoffreedom components Thus tests per formed on orthogonal contrasts are independent There are many ways to choose the orthogonal contrast coefficients for a set of treat ments Usually something in the nature of the experiment should suggest which compar isons will be of interest For example if there are a3 treatments with treatment 1 a control and treatments 2 and 3 actua11evels of the facto of interest to the experimenter then appropriate orthogonal contrasts might be as follows Treatment Otthogoral Contrasts 1 control 2 0 2 levell 1 1 3 level 2 I 1 Note that contrast 1 with C i 2 1 1 compares the average effect of the factor Vith the con trol while contrast 2 with di 0 1 1 compares the two levels of the factor of interest Contrast coefficients must be chosen prior to running the experiment for if these com parisons are selected after examining the data mOSt experimenters would construct tests that compare large observed differences in means These large differences could be due to the presence of real effects or they could be due to random error If experimenters always pick the largest differences to compare they will inflate the type I error of the test since it is likely that in an unusually high percentage of the comparisons selected the observed dif ferences will be due to error Consider the hardwood concentration experiment There an four levels of hardwood concentration and the possOle sets of comparisons between these means and the associated orthogonal comparisons are Ha fi fi Pt 11 H 3fi LL Pt 3fJ4t H 1 3 3 C1 tYI Y2 Yy Y4 C2 3y 2 YJ 3Y4 C3YI3Y2 3Y3Y4 Notice that the contrast constants are orthogonaL Using the data from Table 121 we find the numer ical values of the contrasts and the sums of squares as follows C1 60941021279 9 SSe 64 38 C 36094 1023127 209 SS 209 36400 C 620 C 50 3943102 127 43 43 SSe 620 1541 These contrast sums of squares COntpletely partition the treatment sum of squares that is SSG SSe l SSC SSe 38279 These tests on the contrasts are usually incorporated into the analysis of variance such as shown in Table 126 From this analysis We conclude that there are significant dif ferences between bardwood concentrations 12 vs 3 4 but that the average of 1 and 4 does not dif fer from the average of 2 and 3 nor does the average of 1 and 3 differ from the average of 2 and 4 J 1222 122 Tests on Individual Treatment Means 335 Table 126 Analysis of Variance for the Tensile Strength Data Source of Sum of Degrees of Mean Variation Squares Freedom Square F Hardwood cOlCentrarioD 38279 3 12760 1961 C 14 s 2 3 33S 1 338 052 C 12 vs 34 36400 1 36400 5591 C 13 vs24 1541 1 1541 237 Error 13017 20 651 Total 512 23 Tukeys Test Frequently analysts do not know in advance how to construct appropriate orthogonal con trasts or they may wish to test more than aI comparisons using the same data For exam ple analysts may want to test all possible pairs of means The null bypotheses would then be Ho III Ilj for all i j If we test all possible pairs of means using Itests the probability of committing a type I error for the entire set of comparisons can be greatly increased There are several procedures available that avoid this problem Among the more popular of these procedures are the NewmanKeuls test ewman 1939 Keuls 1952 Duncans multiple range test Duncan 1955 and Tukeys test Tukey 1953 Here we describe Tukeys test Tukeys procedure makes use of another distribution called the Studentized range dis tribution The Studentized range statistic is q Y1ThlX Yn r MSEn wbere y is the largest sample mean and y is the smallest sample mean out of p sample means Let qa af repreent the upper a percentage point of q where a is the number of treatments andfis the number of degrees of freedom for error Two means y and Yj i j are considered significantly different if lYi 1 T where MSE Ta qa af In 1220 Thble XlI Appendix contains values of q afJ for a 005 and 001 and a selection of values for a and Thkeys procedure has the property that the overall significance level is exactly a for equal sample sizes and at most a for unequal sample sizes g4 We will apply Tukeys test to the hardwood concentration experiment Recill that there are a 4 rnea1S n 6 and MSc 651 The treatment means axe Y 1000 psi y 1567 psi y 1700 psi Y 2117 psi Fro Table XII Appendix with a 005 a 4 and 20 we find qOM4 20 396 336 Chapter 12 The Analysis of Variance Using Equation 1220 Therefore we would concludehat Mo means are significantly different if y 412 The differences in treatment averages are 15 11000 15671 567 If y1 100017001 700 11 31 11000 21171 ILl7 I 31 1567 17001 133 f 1 11567 21171 550 L 1 117002117 417 From this analysis we see significant differences between all pairs of means except 2 and 3 It may be of use to draw a graph of the treatment means such as Fig 125 wilo1 the means that are not dif ferent underlined Simultaneous confidence intervals can also be constructed on the differences in pairs of means using the Tukey approach It can be shown that when sample sizes are equal This expression represents a 1001 a simultaneous con fidence interval on aU pairs of means Jl jl If the sample sizes are unequal the 1001 0 simultaneous confidence interval On all pairs of means JJ is given by Interpretation of the confidence intervals is straightfonvard If zero is contained in an inter val then there is no significant difference beween the two means at the a significance level It should be noted that the significance level IX in Tukeys multiple comparison pro cedure represents an experimental error rate With respect to confidence intervals 1 rep resents the probability that one or more of the confidence intervals on the pairwise differences will not contain the true difference for equal sample sizes when sample sizes are unequal this probability becomes at most 0 91 12 fa Y4 I I I I I 100 120 140 160 1M 200 Figure 125 Results ofTukeys est L 123 The RandomEffects Model 337 123 THE RACmOMEFFECTS MODEL In many situations the facwr of interest has a large number of possible levels The analyst is interested i1 drawing conclusions about the entire population of factor levels If tlte experimenter randomly selects a of these levels from the population of factor levels then we say that the factor is a random factor Because the levels of the factor actually used in the experiment were chosen randomly the conclusions reached wil1 be valid about the entire population of factor levels We will assume that te population of factor levels is either of infinite size Or is large enough to be considered inficite The linear statistical model is i 12a Yij1ri ij J 12n 1221 where 1 and are independent random variables Note that the model is identical in struc ture to the fixedeffects case but the parameters have a different interpretation If t1e vari ance of T is then the variance of any observaion is Vy rr f The variances and tT are called variance componemsj and the model equation 1221 is called the componentsofvariance or the raru1omeffectsmodeL To test hypotheses using this model we require that the E areNIDO f tha the ti are NIDO rr and that i and j are independentThe assumption that the 1j are independent random variables implies that the usual assumption on 0 from the fixedeffects model does not apply to the randomeffects modeL The sum of squares identity 1222 still holds That is we partition the total Vmiability in the observations into a component that measures variation between treatments SSItlImI1lJ and a component that measures vari ation within treatments S5 E However instead of testing hypotheses about individual treat ment effects we test the hypotheses If 0 all treatments are identical blo1 if U 0 tlen there is variability between treat ments The quantity SSEa is distributed as chisquare with N a degrees of freedom and under the null hypothesis SSenlcf is distributed as chisquare with a 1 degrees of freedom Further the random variables are independent of each other Thus under the null hypothesis the ratio SS a I SSEeN a 1223 is distributed as F with a I and N a degrees of freedom By examining the expected mean squares we can detennme the critical region for this statistic Consider 338 Chapter 12 The Analysis of Variance If we square and take the expectation of the quantities in brackets we see that terms involv ing r are replaced by d as Er O Also terms involving I7 1 L lL I and LIL I are replaced by nO 000 and an 0 respectively Finally all crossproduct tenns involving i and ijhave zero expectation This leads to or EMSIl J1 nd 1224 A similar approach will show that EMS Jr 1225 From the expected mean squares we see that if Ho is true both the numerator and the denominator of the test statistic equation 1223 are unbiased estimators of 02 whereas if HI is true the expected value of the numerator is greater than the expected value of the denominator Therefore we should rejectHo for values of Fo that are too large This implies an uppertail onetail critical region so we reject Ho if Fo Fa a IN The computational procedure and analysisofvariance table for the randomeffects model are identical to the fixedeffects case The conclusions however are quite different because they apply to the entire population of treatments We usually need to estimate the ariance components 02 and in the model The procedure used to estimate 02 and a is called the analysisofvariance method because it uses the lines in the analysisofvariance table It does not require the normality assump tion on the observations The procedure consists of equating the expected mean squares to their observed values in the analysisofvariance table and solving for the variance compo nents When equating observed and expected mean squares in the onewayclassification randomeffects model we obtain MSuwnctJI1 02 nO and Therefore the estimators of the variance components are ldS and For unequal sample sizes replace n in equation 1227 with 1226 1227 l 123 The RandomEffects Model 339 r n21 1 a i no Lnc1 aI 1 L 1 11 L kcl Sometimes the analysisofvariance method produces a negative estimate of a variance component Sinee variance components are by defmition nonnegative a negative estimate of a variance component is unsettling One course of action is to accept the estimate and use it as evidence that the true value of the variance component is zero assuming that sampling variation led to the negative estimate rile this has intuitive appeal it will disturb the sta tistical properties of other estimates Another alternative is to reestimate the negative vari 31ce component with a method that always yields nonnegative estimates Still anodier possibility is to consider the negative eSJmate as evidence that the assumed linear model is incorrect requiring that a study of the model and its assumptions be made to find a more appropriate model Jliil In his book Design aruiAnalysis afExperiments 2001 D C Montgonery describes a singlefactor experiment involving the randomeffects model A textile manufacturing company weaves a fabric on a lagc number of loows The company is interested in loomtoloom variability in tensile stre1gth To investigate this a IruUlufacturing engineer selects four looms at FaJldom and makes fOtlf stngth determinations on fabric samples chosen at random for each loom The data are shown in Table 127 and the analysis of variance is summarized in Table 128 From the analysis of variance we conclude hat the looms n the part differ significantly in their ability to produce fabric of unifonn strength The variance components are estimated by 190 and 2973190 6 vr 4 Therefore the variace of strength in the manliactlqing process is estimated by 696 190 886 Most of this variability is attributable to dillerences between looms Table 127 Streugth Data for Example 125 Observations Loom 1 2 3 4 Totals Averages 1 98 97 99 96 390 975 2 91 90 93 92 366 915 3 96 95 97 95 383 958 4 95 96 99 98 388 970 1527 9SA 34l Chapter 12 The Analysis of Variance Table 128 Analysis of Variance for the Strength Data Source of Sum of Degrees of Mean Variation Squares Freedom Square P Looms 8919 3 2973 1568 Error 2275 12 190 Total 11194 15 This example illustrates an important application of analysis of variancethe isolation of different sources of variability in a manufacturing process Problems of excessive variability in critical functional parameters or properties frequently arise in quality improvement program For example in the previous fabricstrength example the process mean is estimated by y 9545 psi and the process standard deviation is estimated by wlJ 1886 298 psi If strength is approximately normally distributed this would imply a distribution of strength in we Outgoing product that looks like the normal distribution shown in Fig 126a If the lower specification limit LSL on strength is at 90 psi then a substantial proportion of the process defective is fallout that is scrap or defec dve material that must be sold as second quality and so aD This fallout is directly related to the excess variability resulting from differences between looms Variability in 1001P per formance could be caused by faulty setup poor maintenance inadequate supervision poorly trained operators and so forth The engineer or manager responsible for qUality improvement must identify and remove these sources of variability from the process If he can do this then strength variability will be greay reduced perhaps as low as a a 1190 138 psi as shown in 126b In this improVed process reducing the variabil ity in strength has greatly reduced the fallout This will result in lower cost higher quality a more satisfied customer and enhanced competitive position for the company Process fallout a b 110 pSI Figure 126 The distribution of fabric strengdl a Current process 0 Improved process L 124 The Randomized Block Design 341 124 THE RANDOMIZED BLOCK DESIGN 1241 Design and Statistical Analysis In many experimental problems it is necessary to design the experiment so that variability arising from nuisance variables can be controlled As an example recall the situation in Example 1117j where two different procedures were used to predict the shear strength of steel plate girders Because each girder has potentially different strength and because this variability in strength was not of direct interest we designed the experiment using the two methods on each girder and compared the difference in average strength readings to zero using the paired ttest The paired ttest is a procedure for comparing two means when aU experimental runs cannot be made under homogeneous conditions Thus the paired ttest reduces the noise in the experiment by blocking out a nuisance variable effect The ran domized block design is an extension of the paired ttest that is used in situations where the factor of interest has more than two levels As an example suppose that we wish to compare the effect of four different chemicals on the strength of a particular fabric It is knOYtll that the effect of these chemicals varies considerablY from one fabric specimen to another In this example we have only one fac tor chemical type Therefore we could select several pieces offabric and compare all four chemicals within the relatively homogeneous conditions provided by each piece of fabric This would remove any variation due to the fabric The general procedure for a randomized complete block design consists of selecting b blocks and running a complete replicate of the experiment in each block A randomized complete block design for investigating a single factor with a levels would appear as in Fig 127 There will be a observations one per factor level in each block and the order in whicb these observations arc run is randomly assigned within the block We will now describe the statistical analysis for a randomized block design Suppose that a single factor with a levels 1s of interest and the experiment is run in b blocks as shown in Fig 127 The observations may be represented by the linear statistical model fi 12 a Yiif1tifJjij J 12 b 1228 where f1 is an overall mean ti is the effect of the ith treatment PI js the effect of the jth block and e is the usual NIDO a random error term Treatments and blocks will be ClJ1 sidored initially as fixed factors Furthermore the eattnent and block effects are defined as deviations from the overall mean so thatL 1 ti 0 and L 13 O We are interested in testing the equality of the treatment effects That is Block 1 Bcck 2 BIJ 11 12 1aO H tt 0 for at least one i Brock b IY1D y he Fioure 127 The randomized complete block design 342 Chapter 12 The Analysis of Variance Let Yi be the total of all observations taken under treatment ilet Y be the total of all observations in blockj let y be the grand total of all observations and let N ab be the total number of observations Similarly y is the average of the observations taken under treatment i yO is the average of the observations in blockj and y is the grand average of all observations The total corrected sum of squares is Expanding the rigbthand side of equation 1229 and applying algebraic elbow grease yields J b 1 b 22Yij yf b 2 y a 2 j y i1 Jl i1 1 o b 2 22YjYYj i1 jl or symbolically SSr SSumrrcn SSblocul SS The degreesoffreedolI breakdown corresponding to equation 1231 is no 1 0 1 b 1 T 0 1b 1 1230 1231 1232 The null hypothesis of no treatment effects H 0 is tested by the F ratio MSIMS The analysis of variance is summarized in Table 129 Computing formu las for the sums of squares are also shown in this table The same test procedure is used in cases where treatments andor blocks are random ExnJeX26 Au experiment was performed 0 determine the effcct of four different chemicas on the strelgth of a fabric These chemicals ae used as part of the pennanentpress finishing process Five fabric samples were selected and a randomized block design was run by testing each chemica type once in random order on each fabric sunple The data are shown in Table 1210 The S of squares for the analysis of variance are corcputed as follows Table 129 Analysis of Variance for Randomized Complete Block Design Source of Variation Treatments Blocks Error Tota SUIDof Squares SSE oy subtraction Degreesof Freedom aI hI a 1h 1 abI Mean Sql13Je aI Sl 01 ss alol 124 The Randomized Block Design Table J210 Fabnc Strength Data Randomized Block Design Chemica Type 1 2 3 4 CollUlin totals YJ Colurn averages Yj Fabric Sawe1e 2 3 4 5 13 16 05 12 11 22 24 OA 20 18 18 17 06 15 13 39 44 20 41 34 92 101 35 8S 76 230 253 088 220 190 57 88 69 7178 5 SS ZL i bods a ab 1 Row Row Totas Averages y 57 88 69 178 392 y no 1804 20 Y 114 176 138 356 196 i 92 1012 35 88 76 4 392 669 20 SSE SST SSblOCiks SSlteJtmllntl 2569 669 1804 096 343 The analysis of variance is summarzed in Table 1211 We would conclude that there is a signiiicant difference in 11e chemical types as far as their effect on fabric strength is oolcernoo Table 1211 Analysis ofVarimcc for the RaIdomized Block Experiment Source of Sumo Degrees of Mean Variation Squares Freedom Squae Chemical type treatments 1804 3 601 7513 Fabic saople blocks 669 4 167 Error 096 12 OOS Total 2569 19 344 Chapter2 The A1alyss of Variance Suppose an experiment is conducted as a randomized block design and blocking was not really necessary There are ab observations and a Ib 1 degrees of freedom for error If the experiment had been run as a completely randomized singlefactor design with b replicates we would have hadab 1 degrees offreedom for error So blocking has cost alb I aI b 1 b 1 degrees of freedom for error Thus since the loss in error degrees of freedom is usually small if there is a reasonable chance that block effects may be important1 the experimenter should use the randomized block design For example consider the experiment described in Example 126 as a oneway classification analysis of variance We would have 16 degrees of freedom for error In the randomized block design there are 12 degrees of freedom for error Therefore blocking has cost only 4 degrees of freedom a tty small loss considering the possible gain in informa tion that would be achieved if block effects are really important As a genernl rule when in doubt as to the inlportaJce of block effects the experimenter should block and gamble that the block effect does exist If the experimenter is Tong the slight loss in the degrees of free dom for error will have a negligible effect unless the number of degrees of freedom is very small The reader shollid compare this discussion to the one at the end of Section 11 3 3 1242 Tests on Individual T reatnIent Means When the analysis of variance indicates that a difference exists between treannent means we usually need to perform some followup tests to isolate the specific differences Any multiple comparison method such as Tukeys test could be used to do this Tukeis test presented in Section 1222 can be used to determine differences between treatment means when blocking is involved Simply by replacing n with the number of blocks b in equation 1220 Keep in mind that the dlgrees of freedom for error have now Changed Forth rancomized block designj a Ib I To illustrate this procedure recall that the four chemical type means from Example 126 are 1 114 y 176 y 356 Therefore we would conclude that two means are significantly different if 1 51 053 The absolute values of the differences in treatment averages are IY y1 1114 1761 062 1Y yI11l4 1381024 IY y1 11143561 242 IY y1 1176 1381 038 ty Y41 1176 3561 180 ty Y4 1138 3561 218 J Too results indicate chemical types I and 3 do not differ and types 2 and 3 do not differ fignre 128 represents the results graphically where the underlined pairs do not diffeLI r 124 The Rardomized Block Design 345 y 13 2 y I I I I 100 150 2QO 250 00 350 Figure 128 Results of Tukeys esC 1243 Residual Analysis and Model Checking In any designed experiment it is always important to examine the residuals and check for violations of basic assumptions that could invalidate the results The residuals for the ran domized block design are just the differences between the observed and fitted alues where the fitted 3lues are 1233 The fitted value represents the estimate of the mean response wben the itb treatment is run in thejth block The residuals from the experiment from Example 126 are shown in Table 1212 Figures 129 1210 l2ll and 1212 present the important residual plots for the experiment There is some indication iliat fabric sample block 3 has greater variability in strength when treated with the four cbemicals than the other samples Also cheILical type Table1212 Residuals from ihe Ra1dontized Block Design Chemical Fabric Type 018 011 044 018 002 2 010 007 027 000 010 3 008 024 030 012 002 4 000 027 048 030 010 T 2 0 1il E 0 Z 2 050 025 a 025 050 Resdtal value Figure 129 Normal probability plot of residuals from the randomi2ed block design Chapter 12 The Analysis of Variance 4 which provides the greatest strength also has somewhat more mability in strength Fol lowup experiments may be necessary to confirm these findings if they are potentially important r 05 Figure 1210 Residuals by treaDent el I 05 01 O5 I 2 3 Figure 1211 Residuals by block eat l 1 I 2 O5 Figure 1212 Residuals versus Y I 4 r t t 15 6 j r l25 Determining Sample Size in SingleFactor Experiments 341 12S DETEIL1JNING SAMPLE SIZE rT SINGLEFACTOR EXPERIMENTS In any experimental design problem the choice of the sample size or number of replicates to use is important Operating characteristic curves can be used to provide guidance in mak this selection Recall that the operating characteristic curve is a plot of he type II fJ error for various sample sizes against a measure of the difference in means that it is impor tant to detect Thus if the experimenter knows how large a difference in means is of poten rial importance the operating characteristic curves can be used to determine how many replicates are required to give adequate sensitivity We first consider sample size determination in a fixedeffects model for the case of equal sample size in each treatment The pnwer 1 3 of the test is 1 3 PReject HoIH is false 1234 PFo F iH is false I To ealuate this probabili statement we need to know the distribution of the test statistic Fo if the null hypothesis is false It can be shown tllat if He is false the statistic Fo MSIMSE is distributed as a noncentral F random variable with a I and N a degrees of freedom and a noncentra1ity parameter o If Ii O then the noncentral F distri bution becomes the usual centroi F distribution The operating characteristic curves in Chart vn of the Appendix are used to calculate the pnwer of the test for the fixedeffects model These curves plot the probability of type II error 3 against 1 where nLf 1 aIr 1235 The parameter tt2 is related to the noncentraliy parameter o Curves arc available for a 005 and a I 001 and for several values of degrees of freedom for the numerator and denominator In a completely randomized design the symbol n in equation 1235 is the number of replicates In a randomized block design replace n by the number of blocks In using the operating characteristic curves we must define the difference in means that we wish to detect in tenns of 2 j 1 Also the error variance 52 is usually unknown In such cases we must choose ratios of iljf that we wish to detect Alternatively if an estimate of d is available one may replace cr with this estimate For example if we were interested in the sensitivity of an experiment that has aJready been performed we might use MS E as the estinate of cr Eij1 Suppose that five means are being compared in a completely randomized experimen with a 001 The elperimenter would like to know how IIlalJy replicates to run if it is lUpOrtall to reject HIJ withl probability of at least 090 ifL jJdl 50 The parameter 2 is in this case nt 1 1 n I 5jn QIJ 5 and the openting chalacteristic curve for a 1 5 1 4 and N a anl 511 1 euor degrees offreedomis shown in Chart VII Appendix As a fut guess try n 4 replicates This yields Ilz 4 12 and53 15 crrordegrees offrcedom Consequently from ChartVlI we find that i3 038 348 Chapter 12 TbeAnalysis of Variance Therefore the POef of the test is approxllnately 1 P 1 038 062 whicn is less than the required 090 and so we conclude that It 4 replicates are not sufficient Proceeding in a similar man ner we can construct the following display n 1 I an 1 Power 1 jJ 4 4 200 15 038 062 5 5 224 20 018 082 6 6 245 25 006 094 Thus at least 11 6 replicates must be run in order to obtain a test with the required power The power of the test for the randomeffects model is 1 J3PRejectHclHo is false PF Fo ja O 1236 Once again the distribution of the test statistic Fe under the a1ternative hypothesis is needed It can be shown that if HI is lIUe a 0 the distribution of F is centrnl F with aI and N a degrees of freedom Since the power of thc randomeffects model is based on the central F distribution we could use the tables of the F distribution in the Appendix to evaluate equation 1236 How ever it is much easier to evaluate the power of the test by llSing the operating characteris tic curves in Chart vm of the Appendix These curves plot the probability of the type II error against whcre 1237 In the randomized block design replace n with b the number ofblocks Since u is usually unknown we may either use a prior estimate or define the value of cr that we are interested in detecting in terms of the ratio d Consider a completely randomized design with five treatments selected at random with six observaw tions per treatment and aO05 We ish to determine the power of the testlf a is equal to cT Since a 511 6 and a 02 we oay compute A161 2646 From the operating characteristic curve with al 4 tV a 25 degrees of freedom and ct 005 we find that J 020 Therefore the power is approrimately 080 126 SAMPLE CO1PUTER OUTPUT Many computer packages can be implemented to carry out the analysis of variance for the situations presented in this chapter In this section computer output from 1linitab is presented j 126 SaJOple Computer Output 349 computer Output for Hardwood Concentration Example Reconsider Example 121 which investigates the effect of hardwood concentration on ten sile strength Using ANOVA in Minitab provides the following output Analysis of Source DF Concen 3 Error 20 Total 23 Level N 5 6 10 6 15 6 20 6 Pooled StDev Variance SS 38279 13017 S1296 Mean 10QOO 15667 17000 21167 or TS is 12760 651 StDev 2828 2805 1 789 2639 2551 F F 1961 0000 Indi ridual 95 Cs For Mean Based 01 Pooled StDev r 100 150 200 250 The analysis of variance results are identical to those presented in Section 1212 11initab also provides 95 confidence intervals for the means of each level of hardwood concen tration using a pOOled estimate of the standard deviation Interpretation of the confidence intervals is straightforward Factor levels with confidence lnterYals that do not overlap are said to be significantly different A hetter indicator of significant differences is provided by confidence intervals based on Tukeys test on pairwise differences an option in MinitabV The output provided is Tukeyl s pairwise comparisons Faily error rate 00500 Individual error rate 00111 Critical lue 396 IIntervals for cQla level mean row level mean 5 10 9791 1 542 15 11124 2876 20 15291 7042 10 15 5458 2791 9624 1376 8291 0042 The simultaneous confidence intervals are easily interpreted For example the 95 con fidence intenra for the difference in mean tensile strength between 5 hardwood concen tration and 10 hardwood concentration is 9791 1542 Since this confidence interval does not contain the value 0 we conclude there is a significant difference between 5 and 10 hardwood concentrations The remaining confidence intervals are interpreted simi larly The results provided by Minitab are identicil to those found in Section 1222 350 Chapter 12 The Analysis afYanancc 127 SGMMARY This chapter has introduced design and analysis methods for experiments with a single fac tor The importance of randomization in sing1efactor experiments was emphasized In a completely randomized experiment all runs are made in random order to balance out the effects of unknown nuisance variables If a known nuisance variable can be contrOlled blocking can be used as a design alternative The fixedeffects and randomeffects models of analysis of variance were presented The primary difference beteen the two models is the inference space1n the fixedeffects model inferences are valid only about thefactor lev els specifically considered in the analysis while in the randomeffects model the conclu sions may be extended to the population of factor levels Orthogonal contrasts and Tukeys test were suggested for making comparisons between factor level means in the fixedeffects experiment A procedure was also given for estimating the variance components in a ran domeffects model Residual analysis was introduced for checking the underlying assump tions of the analysis of variance 128 EXERCISES 121 A study is conducted to determine the effect of cuttirg speed on the life in hOlrs of a particular acbine tooL Four leveis of cutting speed are selected for the study with the following results Cuttirg Tool Life 41 43 33 39 36 40 2 42 36 34 45 40 39 3 34 38 34 34 36 33 4 36 37 36 38 35 35 a Does cutting speed affect tool life Draw compar ative box piots aid perform an analysis of variance b Plot average tool life agailst cutting speed and interpret the results c Use Tukeys tese to investigate Cifferences between the individual levels of cutting speed Interpet the results d Find the residuals and examine them for model inadequacy J22 Il Orthogonal Design for Process Optimiza tion and Its Application to Plasma Etching Solid Sfate Technology May 1987 G Z Ymmd D W lil lie describe 11 experiment to determine the effeet of q flow rate on the uriformity of the etch on a sili con wafer used in integrated circuit manufacturing Three flow rates are used in the experiment and the resulting unifomrity in percent for six eplicmes is as follows Observations 2 3 4 5 6 125 27 46 26 30 32 38 160 49 46 50 42 36 42 200 46 34 29 35 41 51 a Does C6 flow rate affect ech unifonrity1 Con struct box plots to compare the facor levels and perform the analysis of variance b Do the residuals indicate any problems vith the underlying assunptions tz3 The compressive strength of concrete is being studied Fom different mixing techniques are being investigated The following data have been collected Mixing TechrJque Compressive Strencth psi 3129 3000 2865 2890 2 3200 3300 2975 3150 3 2800 2900 2985 3050 4 2600 2700 2600 2765 Ca Test the hypothesis that mixing techniques affect the strength of the concrete 1 se a 005 Cb Use Thkeys test to make comparisoos bemeetl pairs of means Estimate tle treatment effects 124 A textile mill has a large nurrber of looms Each loom is supposed to provide the same output of cloth per minute To investigat this assumption fiv looms are chosen at random and the output measured at dif ferent times The following dataare obtained J r I LoOIC Output lb i miJ 40 41 42 40 41 2 39 38 39 40 40 3 41 42 41 40 39 4 36 38 40 39 37 5 38 36 39 38 40 a Is this a5xed or randoIlleffects experiment Ate the looms similar in output J Estimate the wability between looms c Estimate the experimental error variance d What is the probability of accepting Hr if is four times the experimental error variance e Analyze the residuals from this experiment and check for model itmdquacy 125 1m experiment was run to determine whether four specific 5ring tempetatures affect the density of a cerain type of brick The experiment led to the fol lowiDgda Temperatu c CF DenityL 100 218 219 217 216 217 215 218 125 217 214 215 215 150 219 218 218 216 215 217 218 217 216 218 a Does dle fixing temperature affect the density of the bricks b Estirnate the components in the model c Analyze the residuals from the expeent 126An electrotics eogineer is interested in the effect on tube conductivity of five different types of coating for cathode ray tubes used in a telecommunications system display device The folloving conductivity data are obtained 43 141 150 146 2 152 149 137 143 3 134 133 132 127 4 129 127 132 129 5 147 148 14 142 a Is there any difference in conductivity due to coat ing type Use X 005 b Estimate the overall mean and the treatment effects 128 Exercises 351 c Compute a 95 interval estimate of the mean for coating type 1 Compute a 99 interval estimate of the mean difference berneen coating types 1 and 4 d Test all pairs of means using Tukeys test with a 005 e Assuni1g tlar coating type 4 is currently in use what are your recommendations to the manufac Cf We wish to minimize conductivity 127 The response time in milliseconds was deter mined fot three different types of clrccirs tsed in an electronic calculator The results ae recorded here Circuit Type Response Time 1 2 3 19 22 20 21 16 15 20 18 25 33 27 4 18 26 17 a Test the hypothesis that the three circuit types have the same response time b Lse Tuleys test to compare pairs of treatment means c Construct a set of Ort1og0ral contrasts assuming that at the outset of the experiment you suspected the response time of circuit type 2 to be different from the other two d What is the power of this test for detecting L r ler 30 e Analyze the residuals from this experiment 12SIn The Effect of Nozzle Design on the Stability and Performance of Turbulent Water Jets Fire Safety JOUT1aI VoL 4 Angus 1981 C Theobald describes an experiment in which a shape facor was determined for several different nozzle designs at different levls of jet efflux velocity Interest in this experimer focuses primarily on nwile design and velocit is a nuisance factor The data are shown below 1 078 080 081 075 077 078 2 085 085 092 086 081 083 3 093 092 095 089 089 033 4 114 097 098 088 086 083 5 097 086 078 076 076 075 a Does nozile type affect shape factor Compare the noZ1es using box plots and the analysis of variance b Use Tukeys test to determine specific differences between the nozzles Does a graph of average or 352 Chapter 12 The Analysis of Variance standard deviation of shape factor versus nozzle type assist vith the conclusions c Analyzehe residuals from this experiment 129 In his bookDesigrt andAnalysis ajExperiments 2001 D C Montgomery describes an experiment to determine the effect of four chemical agents on the strength of a particular type of cloh Due to possible variability from cloth to cloth bolts of cloth are con sidered blocks Five bolts are scleced and all four chemicals in random order are applied to each bolt The resulting tensile strenghs are Bolt Chemical 2 3 4 5 1 73 68 74 71 67 2 73 67 75 72 70 3 75 68 78 73 68 4 73 71 15 75 69 a Is there any dIfference 11 tensile sjretgth between the chemicals b Use Tukcys test to investigate spedic differ ences between the chemicals c Analyze the residuals itom Lris experin1ent 1210 Suppose that four normal populations have COUllJOn variance f 25 ald means Il 50 60 iLl 50 and Ji4 60 How ma1Y observations should be taken on each population so that the probability of rejectiug the hypothesis of equality of means is at least 0901 Use 005 1211 Suppose hat five uonnal populations have coUllJOn variance IT 100 and means Ji 175 Ji 1901 1601 200 and1 215 How y observatiollS per population must be taken so hat the probability of rejecting the hypothesis of equallry of means is at least 0951 Use a 001 1212 Consider testing the equality of ilie means of two normal populations where t1e variances are uokuon but assumed equal The appropriate test procedure is the twosample Hest Show that he twosample ttest is equivalent to tle oneway classification analysis of variance 1213 Show that the variance of the linear combioa tion Zcy is 02Zdnic7 12J4 In a fixedeffects morul suppose that there ue n observations for each of four treatments Let and a be singledegreeof freedom components for rh ortlogO1al cootrasts Prove hat SSlttlIenu Q Q 12 1215 Consider the data shown in Exercise 127 a Write out the east squares normal equations for this problem and solve them forfl and1 making the usual coostraint Z Ji 0 Estimate C b Solve the equations in a using the constraint O Are the estimators f and fi the same as you found in a Why Now estimate 1 t and com pare your answer with a hat statement can you make about estimating contrasts in the c Estimate jl 1 21 11 1 and jl C 12 using the two solutions to the normal equations Com pae L1e results obtained n each case Chapter 13 Design of Experiments with Several Factors An experiment is just a test or a series of tests Experiments are performed in all scientific and engineering disciplines and are a major part of the discovery and learning process The conclusions that can be drawn from an experiment will depend in part on how the exper iment was conducted and so the design of the experiment plays a major role in problem solution This chapter introduces experimental design techniques useful when several fa tors are involved 131 EXAMPLES OF EXPERIMENTAL DESIGN APPLICATIONS tailiiilfr A Characterization Experiment A development engineer is working on a new process for sol dering electronic components to printed circuit hoards Specifically he is working with a rew type of flow solder machine that be hopes will reduce the number of defective solder joints A flow solder machine preheats printed circuit boards and then rroves them into contact with a wave of liquid sol der This machi1e makes all the electrical and most of me mechanical COlltections of the components to the pr 1ted circuit board Solder defects require touehup or rework which adds cost and often dam ages the boards The flow solder machine has several variables that the engineer can controI They are as follows 1 Solder temperature 2 Prebeat temperature 3 Conveyor speed 4 Flux type 5 Flu specific gravity 6 Solder wave depth 7 Conveyor angle In addition to tiese controllable factors there are several factors that cannot be easily controlled once the machine enters routine manufacturing ineluding the following 1 Thickness of the printed circuit board 2 Types of components used on the board 3 Layout of the components of the board 4 Operator 5 Enirorunental factors 6 Production rate Sometimes we call tlIe uncontrollable factors noise factors A schematic representation of the process is shOVfU in Fig lJ L 353 354 Chapter 13 Design of Experiments with Several Factors Controllable factors 1 1 r Input Process Outpl1 printed circuit boards flow solder machine defects y J ll Z1 22 Zq Uncontrollable noise factors Figure 13 1 The flow solder experiment In this sitlscion the engineer is interested in characterizing the flow solder mlClrine that is he is inter ested in detemrining which factors borb controllable and uncontrollable affect the occurrence of defects on the printed circuit boards To accomplish this he can design an experiment that vill enable him to estimate the magnitude and direction of the factor effects Sometimes we call an experiment such as this a screening experiment The information from this characterization study Or screening experiment can be used to identify the critical factors to determine the direction of adjustrJent for these factors to reduce the nnnber of defects and to assist in determining which factors should be care fully controlled dcring manufacturing to prevent high defect levels and erratic process performance Eklii3J An Optimization Experiment In a characterization experiment we are interested in determining which factors affect the response A logical next step is to determine the region in the important fac tors that leads to an optimum response For example if the response i yield we would look for a region of maxim1lll yield and if the response is cost we would look for a region of minimum cost As an illustration suppose that the yield of a chemical process is influenced by the operating temperature and the reaction time We are currently operating the process at 155F and 17 hours of reaction time and experiencing yields around 75 Figure 132 shows a view of the timetempera ture space from above In this graph we have connected points of constant yield with lines These lines are called contours and we have shown the contours at 60 70 80 90 and 95 yield To locate the optimU1l1 it is necessary to design an experiment that varies reaction time and temperature together This design is illustrated in Fig 132 The responses observed at the four pointsin the exper iment 145F 12 br 045F 22 br 16SF 12 br MId 1651 22 br indicate that we should move in the general direction of increased temperature and lower reaction time to increase yield A few additional IUIlS could be performed in this direction to locate the region of maximum yield These examples illustrate only two potential applications of experimental design meth ods In the engineering environment experimental design applications are numerous Some potential areas of use are as follows 1 Process troubleshootiIg 2 Process development and optimization 3 Evaluation of material alternatives 4 Reliability and life testing 5 Performance testing 132 Factorial Experiments 355 200 Path leading to region of higher yield 160 150 140 05 10 15 TIme hr 20 CUrrent operating conditions 25 Figure 13M2 Contour plot of yield as a function of reaction time and reaction temperature illustrat ing an optimization experiment 6 Product design configuration 7 Component tolerance detennination Experimental design methods allow these problems to be solved efficiently during the early stages of the pr6duct cycle This has the potential to dramatically lower overall prod uct cost and reduce development lead time 132 FACTORIAL EXPERIMENTS Vlben there are several factors of interest in an experiment afactorial design should be used These are designs in which factors are varied together Specifically by a factorial experiment we mean that in each complete trial or replicate of the experiment all possible combinations of the levels of the factors are investigated Thus if there are two factors A and E with a levels of factor A and b levels of factor E then each replicate contains all ab treatment combinations The effect of a factor is defined as the change in response produced by a change in the level of the factor This is called a main effect because it refers to the primary factors in the study For example consider the data in Table 131 The main effect of factor A is the dif ference between the average response at the first level of A and the average response at the second level of A or A 30 40 102020 22 356 Chapter 13 Desigl of Experirnets v1th Several Factors Table 131 A Factorial Expcrimc1t with Two Factors Factor B Factor 4 10 30 B 20 4 That is changing factor A from levell to level 2 causes an average response increase of 20 units Similarly the main effect of B is B 2040 1030 10 2 2 In some experiments the difference in response between the levels of one factor is not the same at all levels of the other factors Vhen this occurs there is an interaction betVeen the factors For example consider the data in Table 132 At the first level of factor B the A effect is A 3010 20 and at the second level of factor B the A effect is AO2020 Since the effect of A depends On the leVel chosen for factor B there is interaction between A andB Vthen an interaction is large the corresponding main effects have little meaning For example by using the data in Table B2 we find the main effect of A to be A 3001020 0 22 and we would be tempted to conclude that there is no A effect However when we exam ined the effects of A at different levels of factor B we saw that this was not the case The effect of factor A depends on the levels of factor B Thus knowledge of the AB interaction is more useful than knowledge of the main effects A significant interaction can mask the significance of main effects The concept of interaction can be illustrated grnphically Figure 133 plats De data in Table 131 against the levels of A for both levels of B Note that the B and Blines are roughly parallel indicating that factors A and B do not interact significantly Figure 134 plots the data in Table 132ln this grnpb the B and B lines are not parallel indicating the interaction between factors A and B Sucb graphical displays are often useful in presenting the results of experiments An alternative to the factorial design that is unfortunately used in practice is to change the factors one at a time rntberthan to vary them simultaneously To illustrate this onefactor atatime procedure consider the optimization experiment described in Example 132 The Table 132 A Factorial ExpCrircllt with Inteaction Factor B Faztor A A A 10 30 B 10 o i 30 D 20 i 10 O i L 50 4O 30 20 8 10 o I A Factor A 8 8 8 1 8 A A 32 Factorial Experiments 357 Figure 133 Factorial experiment no interaction Figure 13 4 Factoria experiment with interaction engineer is interested in finding the values of temperature and reaction time that maximize yield Suppose that we fix temperature at 155F the current operating level and perform five runs at different levels of time say 05 hour La hour 15 hours 20 hours and 25 hours The results of this series of runs are shown in Fig 135 This figure indicates that maximum yield is achieved at about 17 hours of reaction time To optimize temperature the engineer fixes time ar 17 hours rhe apparent optimum and perfonns five runs at different temperatures say l40F 150F 160F 170F and 180F The results of this ser of runs are plotted in Fig 136 Maximum yield occurs at about 155F Therefore we would conclude that running the process at 155F and 17 hours is the bestset of operating conditions result ingin yields around 75 BO 70 I 60 50 05 10 15 20 25 TIme hr Jrrgwe 135 YlCld versus reaction time with temperature constant at 155P 358 Chaptc 13 Design of Expeimcnrs with Several Factors t 70 Q m eo Temperature JA Figure 136 Yleld versus temperature with reaction time constant at 17 hr Figure 137 displays the contour plot of yield as a function of temperature and time with the onefactoratatime experiment shown on the contours Clearly the onefactorat atime design has failed dramatically here as the true optimum is at least 20 yield points higher and occurs at much lower reaction times andbigher temperatures The failure to dis cover the shorter reaction times is particularly important as this could have significant impact on production volume or capacity production planning manufacturing cost and total productivity The onefactoratatime method has failed here because it fails to detect the interac tion between temperature and time Factorial experiments are the only way to detect inter 18D m 0 E m t 150l 14G LL Oll 10 15 20 25 Time hr Figure 1J7 Optimization experiment using the oncfactoratatime method 133 TwoFactor Factorial Experiments 359 actions Furthermore the onefactoratatime method is inefficientj it will require more experimentation than a factorial and as we have just seen there is no assurance that it will produce the correct results The experiment shown in Fig 132 that produced the informa tion pointing to the regiOD of the optimum is a simple example of a factorial experiment 133 TWOFACTOR FACTORIAL EXPERIMENTS The simplest type of factorial experiment involves only two factorS say A and B There are levels offac1lr A and b levels off actor B The twofactor factorial is shown in Table 133 Note that there are n replicates of the experiment and that each replicate contains all ab Ireatment combinations The observation in the ijth cell in the kth replicate is denoted 11Ft In collecting the dara the abn observations would be run in random order Thus like the singlefactor experiment srudiedln Chapter 12 the twofactor factorial is a completely ran domized design The obsevations y be described by the linear statistical model 131 where J1 is the overall mean effect t is the effect of the ith level of factor A is the effect of the jth level of factor S 1llij is the effect of the interaction between A and S and il is a NIDO a norma and independently distributed random error component We are interested in testing the hypotheses of no significant factor A effect no significant factor B effect and no significant AB interaction As with the singlefactor experiments of Chapter 12 the analysis of variance will be llsed to test these hypotheses Since there are two fac tors under study the procedure used is called the twoway analysis of variance 1331 Statistical Analysis of the FixedEffects Model Suppose that factors A and S are fixed That is the levels of factor A and the b levels of factor S are specifically chosen by the experimenter and nferences are coned to these levels only In this model it is customary to define the effects fil and 13 as deviations from the mean so that T 0 ll 0 13ij 0 anlEl3i 0 Let y denote the total of the observations under the itb level of factor A let yo denote the total of the observations under the jth level of factor B let Iii denote the total of the obsevations in the ijth cell of Table 133 and let y denote the grand total of all the Table 133 Dat Arrangement for a TwoFactor Factorial Design Factor A 1 2 a Factor B 2 1 1 11 1 Yilt 121 l1ft flU Ym 11111 Ylll Y2n Ytn Jl b IWO 11 Y1Wll2 l7m 360 Chapter 13 Design of Experiments with Several Factors observations Define Yf YT YIj and y as the correspondlng row colllIIlD cell and grand averages That is Yij LYfk kl a Y LLLYij f izlkl n Y ahn The total corrected sum of squares may be written a b 2 LYilYJ fO111 a b LLLfCVIY Yj yJ rl 1k1 i 12 a i 12 a jl2 b YiJ Yio YI Y YjkYijf a 2 b 2 bnLYi Y anLY Y 11 11 ab aon 2 nLLYJ y YjYJ LLLlYk Yij 11 jl i1 11 kol 132 133 Thus the total sum of squares is partitioned into a sum of squares due to rows or factor A SSA a sum of squares due to columns or factor B SS a sum of squares due to the interaction between A and B SSM and a sum of squares due to error SSE Notice that there must be at least tvo replicates to obtain a nonzero error sum of squares The sum of squares identity in equation 133 may be written symbolically as 134 There are abn 1 total degrees of freedom The main effect A and B have a I and 1 degrees of freedom while the interaction effectAB has a 1b1 degrees cffree dom Within each of the ah cells in Table 133 there are n I degrees of freedom between the n replicates and observations in the same cell can differ only due to random error Therefore there are ahen 1 degrees of freedom for error The ratio of each sum of squares on the righthand side of equation 134 to its degrees of freedom is a mean square Assuming that factors A and B are fixed the expected values of the mean squares are 1 l 133 TwoFactor F2ctorial Experimcnt5 361 a b n L2JrJ3j EMS E SSs J 2 i j1 All albI albI and Therefore to test He t 0 no row factor effects Ho 3 0 no column factor effects and H fi 0 no interaction effects we would dividethe corresponding mean square by the mean square error Each of these ratios will follow an F distribution with numerator degrees of freedom equal to the number of degrees of freedom for the numerator mean square and abnI denominator degrees of freedom and the critical region will be located in the upper tail The test procedure is arranged in an analysisofvariance table such as is shown in Table 134 Computational fonnulas for the sums of squares in equation 134 are obtained easily The total sum of squares is computed from a b r y2 SSy LLLYi 11 jl 1 aim The sums of squares for main effects are and b 2 2 SS Yj y B L 1 an abn 135 136 137 We usually calculate the SS in two steps First we compute the sum of squares between the ab cell totals called the sum of squares due to Hsubtotals IJ b y SSSlbtOUs LLJL b 11 J1 nan Table 134 The AnalysisofVariance Table for the TwoWayaassification FixedEffects Mode Source of Variation A treatments B treatments Interaction Total Sum of Squares SS SS SSS SS SST Degrees of Freedom aI 01 a 10 I abnl abnl Mean MSA aI MS MS 01 MS MSAfj SSAS oIbI MS MS 362 Chapter 13 Design of ExperinentS with Several Factors This sum of squares also contains 5Sft and SSs Therefore the second step is to compute SSAS as SS SS SSA 5S8 The error sum of squares is found by subtraction as either or Epl133 138 139 139b Aircraft primer paints are applied to alUI1linilIl surfaces by two nethods dipping and spraying The purpose of the primer is to improve paint adhesion Some pars can be primed using either applica don method and engineering is interested i11eamiug whether thee different primers differ in their adhesio properties A factorial experiment is performed to investigate the effect of paint primer type and application method on paint adhesion Three speciners are painted with each primer using each application method a ilnish palnt applied and the adhesion force measured The data from the experiment are shown in Table 135 The circIed nUDlbers in the cells are the cell totals Yr The sums of squares required to perfow the analysis of variance are computed as follows and 4012 45 5 0 898 1072 18 Ss YL 5 or abn 287 341 270 6 3 898458 18 898 458 491 024 18 1072458 491024 099 The analysis of variance is summarized in Table 136 Sinee FJril2 3S9 and FCO11 475 we onclude that the main effects of primer type and application method affect adhesion force Futber M more since 15 FO05212 there is no indication of interaction between these factors i 133 TwoFactor FacorJ1l Experiments 363 TabJe 135 Adhesion Force Data for Example 133 Application Method Primer JYpe Dipping Spraying I 1 404543 544956 287 2 5649 54 586163 34 3 383740 555050 270 Y 402 496 898y Tb116 Analysis of Variance for Example 133 SoutCe of Sum of Degrees of Mean Variation Squares FreOOom Square Primer types 4581 2 2291 2786 Application Ilethods 4909 1 4909 5970 Interaction 0241 2 0121 147 Error 0987 12 0082 Total 10718 17 A graph of t1e cell adhesion force averages Yir versus the levels of primer type for each application method is shown in Fig 138 The absence ofintcraction is evident by the parallelism of the two lines Furthermore since a large response indicates greater adhesion force we conclude that spraying is a superior application method and that primer type 2 is most effeetive Tests 011 Individual Mean When borh factors are fixed comparisons between rhe indi vidual means of either factor may be made using Tukeys test When there is no interaction these comparisons may be made using either the rOw averages Yi or the column averages Y However when interaction is significant comparisons between the means of one factor say A may be obscured by rhe AS interaction In rhls case we may apply Tukeys test to the means of factor A With factor B set at a particular level Yit 70 f 60 r 5Or 40 aoc ng Ping 2 Prlmertypa Figure 13 Graph of average adhesion force versus primer types for Example 133 364 Chapter 13 Design of Experiments with Several Facton 1332 Model Adequacy Cbecking Just as in the singlefactor experiments discussed in Chapter 12 the residuals from a facto rial experiment play an important role in assessing model adequacy The residuals from a twofactor factorial experiment are That is the residuals are JUSt the difference between the observations and the corresponding cell averages Table 137 presents the residuals for the aircraft primer paint data in Example 133 The normal probability plot of these residuals is shown in Fig 139 This plot has tails tbat do not fall exacdy along a straight line passing through the center of the plot indicating some potential problems with the normality assumption but the deviation from normality does not appear severe Figures 1310 and 1311 plot the residuals versus the levels of primer types and application methods respectively There is some indication that primer type 3 results in slightly lower variability in adhesion force than the other two primers The graph of residuals versus fitted values Ylk YU in Fig 1312 reveals no unusual or diag nostic pattern 1333 One Observation per Cell In some cases involving a twofactor factorial experiment we may have only One replicate that is only one observation per ceIL In this situation there are exactly as many parameters Table 13M 7 Residuals for the Aircraft Primer Paint Experiment in Example 133 Application Metlod Primer Type Dippig I iJ27 023 003 2 030 04 010 3 003 013 017 20 1J ZI oJ 10 20 05 03 01 Spraying 010 OAO 030 027 003 023 033 iJ17 017 J 01 03 SIX residual Figure 139 Nonnal probability plot of the residuals from Example B3 133 TwoFactor Factorial Experiments 365 Figure 1310 Plot ofresiduals YersJS primer type 05 Or D S Application method 05 c Figure 1311 Plot of residuals versus application method 0 4 5 6 A Yijk 05 in the analysisofvariance model as there are observations and the error degrees offree dam is zero Thus it is not possible to test a hypothesis about the main effects and interac tians unless some additional assumptions are made The usual assumption is to ignore the interaction effect and use the interaction mean square as an error mean square Thus the analysis is equivalent to the analysis used in the randomized block design This 366 Chapter 13 Design of Experiments with Several Factors nointeraction assumption can be dangerous and the experimenter should carefully exam ine the data and the residuals for indications that there really is interaction present For more details see Montgomery 200 1 1334 The RandomEffects Model So far we have considered the case where A and B are fixed factors We now consider the situation in which the levels of both factors are selected at random from larger populations of factor levels and we wish to extend our conclusions to the sampled population of factor levels The observations are represented by the model i 112 a Yij 1 Hi Pj Plj H ijk j j 12 b 1310 k 12 n where the parameters 1i pj f5ii and ti are random variables Specifically we assume thati is NIOO a p is NIOO oI is NIOO 0 and ejk is NIOO 0 The vari ance of any observation is VYij a ap 0 and 0 o and 52 are called variance components The hypothcses that we are inter ested in testing are H G Ho G and Ho 0 O Notice the similarity to the one way classification randomeffects model The basic analysis of variance remains unchanged that is 8S SSe SSw SSI and SSE are all calculated as in the fixedeffects case To construct the test statistics we mUIit exam ine the expected mean squares They are and EMS 0 no bna EMS 0 na ana EMSA 0 nap EMSJ 1 1311 ote from the expected mean squares that the appropriate statistic for testing H 0 0 is 1312 since under Ho both the numerator and denominator of Fo have expectation f and only if He is false is EMSAIl greater than EMS The ratio Fo is distributed as F IJ Similarly for testing Ho 0 0 we would use 10 MS MSAB which is distributed as F4j lslb l and for testing Ho if 0 the Statistic is E MS 0 MS An 1313 1314 which is distributed as F 1011 These are all uppertail one tail tests Notice that these test Statistics are not the sarne as those used if both factors A and B are fixed The expected mean squares are always used as a guide to test statistic construction 133 TwoFactor Factorial EPeriments 367 The variance components may be estimated by equating the observed mean squares to ther expected values and solving for the variance components This yields 2 j MS 1315 Suppose that in Example 133 a large number of primers and several application methods could Ie used Three priUers say 1 2 anc 3 were seected at rancom as were he two application methods Tne analysis of vaiance assuming the randon effects node is shoWD in Table 138 Nerice that the first four colurrnsin the analysis ofvanance table are exactly as in Exa1lple 13 3 Xow however the F ratios are computed according to equations 1312 through 1314 Since F1l2 389 we conclude that interaction is not sigrlficant Also since FIIJ 190 and FMI 185 we concude that both types and application methods significantly affect adhesion force although primer type is just baely significant at j 005 The 1ariance components may be oStimated using equation 1315 as follows 008 2 012008 00133 v 3 229012 r 6 036 49 0 12 053 Clearly he twQ largest variance components are for prirrer typeS 6 036 and application meth ods 053 1335 The Mixed Model Now suppose that one of the factors A is fixed and the otherl B is random This is called the mixed model analysis of variance The linear model is iI2a f1 ti P Plil Eillj 12b k l2 n Table 1J8 Analysis of Variance for EXarlple 134 Source of Sumo Degrees of Mean Variation Squares Freedom Sqare Primer typeS 458 2 229 Applicatioo methods 491 491 Interaction 024 2 012 Error 099 12 008 Total 1072 17 F 1908 4092 15 1316 368 Chapter 13 Design of Experiments with Several Factors In this model li is a fixed effect defined such that L I t 0 Is a random effect the inter action term f3ij is a random effect and el is a NIDO a2 random error Ie is also cus tomary to assurne that Ilj is NlDO i and that the interaction elements rfJij are nonnaJ random variables ith mean zero and variance la 1ladfJ The interaction elements are not all independent The expected mean squares in this case are a bnI EMSAa2rajJ h aI 2 2 E MS f anfp EMSAB 02 na and Therefore the appropriate test statistic for testing Ho off 0 is whicb is distributed as F 1cII1 n For testing He cr1 0 the test statistic is MSB Fo MSe which is distributed as Fblabn 1 Finally for testing Hocrp 0 we would use which is distributed as FCIXb1JbI 1317 1318 1319 1320 The variance components dtjl and rr may be estimated by eliminating the first equation from eqcation 1317 t leaving three equations in three unknowns the solutions of which are and 1321 This general approach can be used to estimate the variance components in any mixed model After eliminating the mean squares containing fixed factors there will always be a set of equations remaining that can be solved for the variance components Table 139 summarizes the analysis of variance for the twofactor mixed model J I l 134 General Factorial Experbents 369 Table 139 Analysis ofVaiance for the NoFactor Mixed Model Souce of Sum of Degrees of Mean Expected Variation Squares Freedom Square Mean Square F Rows A 55 aI M5 aZncr2 bnIr2 lal MSA MSAB Columns B 55 bl MSg cr ar0 M5p M5 Interaction 55 a lb 1 MSAB r1 nO MSA MSE Error SSE abn I MS a Tota1 SSr abnl 134 GENERAL FACTORLlL EXPERIMENTS Many experiments involve more than two factors In this section we introduce the case where there are a levels of faetor A b levels of factor B c levels of factor C and so on arranged in a factorial experiment In general there will be abc n total observations if there are n replieates of the complete experiment For example consider the thteefuctor experiment with underlying model YiJ 1 Hi 3j r 131 r J3r J r j 12 a I3rlk Eiikl1j l b kl1jCj 1 12p n 1322 Assuming that A B and C are fixed the analysis of varianee is shown in Table 1310 Note that there must be at least vo replicates n 2 to compute an error sum of squares The F tests on main effects and interaetions follow directly from the expeeted mean squares Computing formulas for the sums of squares in Table 1310 are easily obtained The total sum of squares is using the obvious dot notation c b C If Ss I I I 5k i1 il kol1 aDen 1323 The sum of squares for the main effects are eomputed from the totals for factors A Yf BIy and ely as follows 1324 1325 c 2 2 SS Y k Y C 11 abn abcn 1326 310 Chapter 13 Design of Experiments with Several Factors To compute the twofactor interaction sums of squares the totals for theA x BA x C and B x C cells are needed It may be helpful to collapse the original data table into three two way tables in order to compute these totals The sums of squares are 11 b 2 2 rr Yij cLSSA SSo 1 jl en abcn 1327 SSwhrorohAlT SSA SSB 11 C 2 2 SSAC rr Yd SSA SSe il kl bn abcn 1328 SSwblotalSAC SSA SSe and b c 2 2 SSnc rr y j SSB SSe Jl Kt an abcn 1329 SSWCBC SSe SSc The threefactor interaction sum of squares is computed from the threeway cell totals Yijt as Table 1310 The Analyslsof Variance Table for the ThreeFactor FixedEffects Model SOwceof Sum of Degrees of Mean Expected Variation Squares Freedom Square Mean Squares F MS A SS aI MS 1z bcrirt at MS B SSe bI MS acnIj3 MSn CI bI MS nl MSc C SSe eI MSc 12atJ It cI MS AS SSAB a tjbi MS c rnlIPj MS albI MS AC SSAC aIeI MSc 01 brIEry MSAC alcl MSc BC SSnc b Ic I MSBC arlJlf MSec CI bII MSE 4BC SSABC a Ib 1c 1 MSJJC 0 tilITrjyjk MSAlIC alblc1 MSe Error SSE cbcn i MSr a Total Ss aCchi J L 134 Genral Factorial Experiments 371 1330b The error sum of squares may be found by subtracting the sum of squares for each main effect and interaction from the tOtal sum of squares or by 1331 A mechanical engineer is stUdyinghe swface roughness of a part produced in a metalcutting Qper arion Three factors feed rare 1 depth of cut B and tool angle C are of interest All three flc torS have been assigned two levels and two replicates of a factorial design are ron The coded data are shown in Table 1311 The trreeway celJ totals Yijk are circled in this talJle The suIlS of sqW1Cs are caculated as follows using equatio1s 1323 to 1331 a 2 y1 SSAIi I ben abcn 75 02 177 455625 8 16 SS l s 1 am cben 82 95 177 g 16 a b 2 SSAii L 2 Yu y SSA SSs j j en Men 37 38 45 57 177 4 16 75625 00625 b c 2 2 SS Yjk Y SS lJC tt e5S c j to an abcn 105625 455625 105625 138 44 4 148 J77 I I 10562530625 4 16 372 Chapter 13 Design of Experiments lith Several Factors SSE SST SSnblotlhABC 929375734375 195000 1M analysis of afiance is summarized in Table 1312 Feed rate has a significant effect on surface finish a 001 as does the depth of cut 005 a 010 There is some indication of a mild inter action betweet these factors as the Ftest for the AJ3 interaction is just less than the 10 critical value Obviously factorial experiments wit three or more factOTS are complicated and require many runs particularly if some of the factors have several more than two levels This leads us to consider a class of factorial designs with all factOrs at two levels These designs are extremely easy to set up and analyze and as we will see it is possible to greatly reduce the number of expeental runs through the tecbnique of fractional replication Table lJll Coded Surface Roughness Data for Example 135 Depth of Cut B 0Q25 Loch 0040 inch Feed Rate A Toel Angle l c Tool Angle C 15 25 W 25 Yr 9 11 9 10 20inJmln 7 10 II 8 75 10 10 12 16 30 inlmln 12 13 15 14 102 B x Ctotals 38 44 47 48 177 A X B Totals iF A X CTotals rk AlB 0025 0040 MC 15 25 20 37 38 20 36 39 30 45 57 30 49 53 82 95 y 85 92 Table 1312 AnalYStS of Variance for Example 135 Sourcof Variation Feed rate A Depth of cut B Tool angle C AB AC BC ABC Error Tow Significanr at 1 Sgni5cant at 10 Sum of Squares 455625 105625 30625 75625 00625 15625 50625 195000 929315 Degrees of Freedom 1 1 S 15 135 THE 2k FACTORIAL DESIGN 135 The 2t Factorial Design 373 Mean Square F 455625 1869 105625 433 30625 126 75625 310 00625 om 15625 064 50625 208 24375 There are certain special types of factorial designs that are very useful One of these is a fac torial design with k factors each at two levels Because each complete replicate of the design has Z runs or treatment combinations the arrangerrent is called a Z factorial design These designs have a greatly Simplified statistical analysis and they also form the basis of many other useful designs 1351 The 22 Design The simplest type of2k design is the 2z that is two factors A and B each at two levels We usually think of these levels as the low and high levels of the factor The 2 design is sho in Fig 1313 Note that the design can be represented geometrically as a square with the 21 4 runs forming the cOmers of the square A special notation is used to represent the treatment combinations In general a treatment combination is represented by a series of lowercase letters If a letter is present then the corresponding factor is run at the high level High b ab t B Low 1 Low A a High Figure 1313 The 2 lilctorial desigr 374 Chapter 13 Design of Experiments with Severa Factms in that treatment combination if it is absent the faetor is run at its low level For example treatraent combination a indicates that factor A is at the high level and faetor B is at the low leveL The treatment eombination with both factors at the low level is denoted 1 This nota tion is used throughout the 2K design series For example the treatment combination in a 24 design with A and C at the high level and B and D at the low level is denoted ac The effects of interest in the 22 design are the main effects A and B and the twofactor interaction AB Let I a b and ab also represent the totals of all n observations taken at these design points It is easy to estimate the effects of these factors To estimate the main effect of A we would average the obsenrations on the right side of the square where A is at the high level and subtract from this the average of the observations on the left side of the square where A is at the low level or A aab bl 2n 2n Ja ab b 1 2n 1332 Similarly the main effect of B is found by averaging the observations on the top of the square where B is at the high level and subtracting the average of the observations on the bottom of the square where B is at the low level B baiJ al 2n 2n I babaI 2n 1333 Finally the AB interaction is estimated by taking the difference in the diagonal averages in Fig l313i or 1334 The quantities in brackets in equations 1332 1333 and 1334 are called contrasts For example the A contrast is Contrast a ab b 1 m these equations the contrast coefficients are always either 1 Or L A table of plus and minus signs such as Table 1313 can be used to determine the sign of each treatment com bination for a particular contrast The column headings for Thble 13MB are the main effects A and B AB interaction and I which represents the total The row headirgs are the treat ment combinations Note that the signs 11 the AS column are the products of signs from Table1313 Signs for Effects in che 22 Design Teatment Factorial Effect Combinations I A B AS 1 a b ab l 135 The 2 Factorial Design 375 colullOS A and B To generate a contrast from this table multiply the signs in the apprQpri ate column of Table 1313 by the treatment combinations listed in the rows and add 1b obtain the sums of squares for A B and AB we can use equation 1218 which expresses the relationship between a sing1edegreeoffreedom contrast and its sum of squares SS ContrdSt l o L contrast coefficients t Therefore the sums of squares for A B andAB are SSA 40 SS 4n SSAB 40 The analysis of variance is completed by computing the total sum of squares SST ith 40 1 degrees offreedom as usual and obtaining the error sum of squares SSE with 40 1 degrees of freedom by subtraction gIl An article in the ATT Technical Jou MarcblApril 1986 Vol 65 p 39 describes the applica tion of twolevel experimental designs to integrated circuit manufacturing A basic processing step in t1is industry is to grow an epitaxial layer on polished silicon wafers The wafers are mounted On a sus ceptor and positioned inside a bell jrtr Chemical vapors are introduced rrrough nOl7les near the top of the jar The susceptor is rotated and heat js applied These conditions are maintained until the eplR taxiallayer is thick enough Table 1314 presen theresuJts of a 2 factorial design 1thn4replicates using the factors A deposition time and B arsenic flow rate The two leves of deposition time are short and long and the two levels of arsenic flow rate are 55 and 59 The response variable is epitaxial layer thickness urn We may find the estimates of the effects using eqJations 13 R32 1333 and 1334 as follows A n aahbll 2t4l 59299 59 1565568656081J 0836 11ble1314 The 2z Design for the Epitaxial Process Experiment Treatment Combinations A B AB Thickness um Total 1 14037141651397213907 56081 14021 a 1482114757 4843 14878 59299 14825 b 1388013860 14032 13914 55686 13922 ab 1488814921 14415 14932 59156 14789 376 Chapter 13 Design of Experiments with Several Factors BbabaI 2n 1 24 55686591565929956081 O67 ABabIab 2n 1 24 5915656081 59299556860032 The numerical estimates of the effects indicate that the effect of deposition time is large and has a pos itiYe direction increasing deposition time increases thickness since changing deposition time from low to high changes the mean epitaxial layer thickness by 0836 pm The effects of arsenic flow rate B and the AB interaction appear small The magnitude of these effects may be eonfillled with the analysis of variance The sums of squares for A E andAB are computed using equation 1335 Contrast SS n4 raabbIl SS A 16 66882 16 27956 bablllCLl SS 16 5381 16 00181 abIabj SSAB 16 O256f 16 00040 The analysis of variance is summarized in Table 1315 This confirms our conclusions obtained by examining the magnitude and direction of the effects deposition time affects epitaxial layer tbickness and from the direction of the effect estimates we know that longer deposition tines lead to thicker epi taxia11aYer5 Tablel315 Analysis ofVaciance for the EpitaXial Process Experiment Source of Sum of Degrees of Mean Variation Squares Freedom A deposition time 27956 I 27956 13450 B arsenic flow 00181 00181 087 AB 0004 00040 019 Etror 02495 12 00208 Total 30672 15 35 The 21 Factorial Design 377 ResidualAnalysis It 1s easy to obtain the residuals from a 2 design by fitting a regression mode to the data For the epitaxial process experiment the regression model is y 130 13x E since the only active variable is deposition time which is represented by Xl The low and high levels of deposition time are assigned the vaiues Xl 1 and Xl 1 respectively The fitted model is y 14389lro836ixl 2 where the intercept A is the grand average of all 16 obServatiODS Y and the slope is one half the effect estimate for deposition time The reason the regression coefficient is onehalf the effect estimate is because regression coefficients measure the effect of a unit change in Xl on the mean of y and the effect estimate is based on a twounit change from 1 to 1 This model can be used to obtain the predicted values at the four points in the design For example consider the point with low deposition time x 1 and low arsenic flow rate The predicted value is ji 14389 0836IH 13971 pm 2 J and the residuals would be 14037 13971 0066 e 14165 13971 0194 eJ 13972 J3971 0001 e4 13907 13971 Q064 It is easy to verify that for low deposition time x 1 and high arsenic flow rate ji 14389 083612 1 13971 J1rI4 the remairting predicted values and residuals are 13880 13971 0091 e 1386013971 0111 e 14032 3971 0061 e 13914 13971 0057 that for high deposition time x 1 and low arsertie flow ratejI 14389 0836121 14807 J1rI4 they are e 1482114807 0014 e 14757 14807 aoso ell 14843 14807 0036 ell 14878 14807 0071 and that for high deposition time x 1 and high arsertic flow rate9 14389 0836121 14807 J1rI4 they are e 14888 14807 0081 e 1492114807 0114 378 Chapter 13 Design of Experiments with Several Factors e 14415 14807 0392 e 14932 14807 0125 A normal proba1ility plot of these residuals is shown in Fig 1314 This plot indicates that one residUal es 0392 is an outlier Examining the four runs with high deposition time and high arsenic flow rate reveals that observation Y15 14415 is considerably smaller than the other three observations at that treatment combination This adds some additional evidence ro the tentative conclusion that observari on 15 is an outlier Another possibility is that there are some process variables that affect the variability in epitaxial layer thickness and if we could discover which variables produce this effect then it might be possible to adjust these variables to levels that would minimize the variability in epitaXial layer thick ness This would have important implications in subsequent manufacturing stages Figures 1315 and 1316 are plots ofresiduals versus deposition time and arsenic flow rate respec tively Apart from the unusually large residual associated with YU there is no strong evi dence that either deposition time or arsenic flow rate int1uences the variability in epitaxial layer thickness Figure 1317 shows the estimated Stllldard deviation of epitaxial layer thickness at all four runs in the 21 design These standard deviations were calculated using the data in Table 1314 Notice that the standard deviation of tilefour observations with A and B at the high level is considenbly larger than the standard deviations at any of the other three design T 1 1 o ZJ t 2 039200 029433 O9S67 009900 000133 009633 019400 Residual Figure 1314 Normal probability plot of residuals for the epitatial procesi experiment OLOWHi9h 05 Oeposition time A Iigure 1315 Plot of residuals versus deposition time 135 The 2k Factorial Desig 379 ol ILOWHi9h J Arsenic flow rate B Figure 13 16 Pot of residuals versus arsenic flow rate sO077 b B H 1 0110 H A a 5 0051 Figure 1317 ne estimated standard deviations of epitaxial layer thickness at the four runs in the 2design points Most of this difference is attributable to the unusually low thickness measurement associated with YIS The standard deviation of the fout observations with A and B at t1e low levels is also somewhat larger than the standard deviations at the remaining two runs This could be an indication that there are other process variables not included in this experiment that affect the variability in epitaxial layer thickness Another experiment to study this pos sibility involving other process variables could be designed and conducted indeed the original paper shows that there are two additional faetors uneonsidered in this example that affect process variability 1352 The 2 Design for k 3 Factors The methods presented in the previous section for factorial designs with k 2 factors cach at two levels can be easily extended to more than two factorS For example consider k 3 factors each at two levels This design is a 2 faetorial design and it has eight treatment combinations Geometrically the design is a cabe as sholl in Fig 1318 with the eight runs forming the comers of the cube This design allows three main effects to be estimated A B and C along with three twofactor interactions AB AC and BC and a threefactor interaction ABC The main effects can be estimated easily Remember that 1 a bab e ac be and abc represent the total of all n replicates at each of the eight treatment eombinations in the design Referring to the cube in Fig 1318 we would estimate the main effect of A by aver aging the four treatment combinations on the right side of the cube where A is at the high 380 Chapter 13 Design of Experiments with Several Factors 1 be abc Be e rT err I I I I bi ab 1 1 a 1 A Figure 1318 The 2 desigu level and sUbtracting from that quantity the average of the four treatment combinations on the left side of the cube where A is at the low level This gives I A a abac abe b e bc I 4 1336 In a similar manner the effect of B is the average difference of the four treatment combina tions in the back face of the cube and the four in the front or B rb ab be abe a c ac I 4n 1337 and the effect of C is the average difference between the four treatment combinations in the top face of the cube and the four in the bottom or 1 1 ClcacbeabcababI 4n 1338 ow consider the tvofactor interaction AB hen C is at the low level AB is just the average difference in the A effect at the tVo levels of E or ABCIOWI abbI aI 2n 2 Similarly when C is at the high leve the AB interaction is AB C high abe be ace 2n 2 The AB interaction is just the average of tbese two components or AB ab 1 abc e b a be ac 4n 1339 Using a similar approach we can show that the AC and Be interaction effect estimates are as follows 1 AC ac I abc b ac ab be 4n I BC bc 1 abca be ab ae 411 1340 1341 I 135 The 2k Factorial Design 381 The ABC interaction effect is the average difference between theAB interaction at the two levels of C Thus ABC n abcbclacclabblaI 2 abc bcac cab b a Il 4n 1342 The quantities in brackets in equations 1336 through 1342 are contrasts in the eight treatment combinations These contrasts can be obtained from a table of plus and minus signs for the 2 design shown in Table 1316 Signs for thc main effects columns A B and C are obtained by associating a plus with the high level of the faetor and a minus with the low level Once the signs for the main effeets have been established the signs for the remaining eolumns are found by multiplying the appropriate preceding columns row by row For example the signs in column AB are the produet of the signs in columns A and B Table 1316 has severnl interesting properties 1 Except for the identity column l each eolumn has an equal number of plus and minus signs 2 The SUJr of products of signs in any two columns is zero that is the columns in the table arc orthogonal 3 Multiplicating any column by column leaves the column unchanged that is I is an identity element 4 The product of any two columns yields a column in the table for example A x B AS and AB x ABC ABC C since any column multiplied by itself is the iden tity column The estimate of any main effect or interaction is determined by multiplying the treat ment combinations in the first column of the table by the signs in the corresponding main cffect or interaction colUlllll adding the result to produce a contrast and then dividing the contrast by onehalf the tolal number of runs in the expctimcnt Expressed mathematically Effect Contrast 1343 The sum of squares for any effect is SS Contrast2 1344 n2k Table 1316 Signs for Effectsin the 2 Design Treatmem Factorial Effect Combinations I A S AS C K BC IBC I a b ab C ac Ix abc 382 Chapter 13 Design of Experiments with Several Factors Fil11J Consider the surfaceroughncss experiment described originally in Example 135 This is a 23 facto rial design in the factors feed rate 4j deptl of cut B and tool angle G Yritb 10 2 replicates Table 1317 presents the observed surfacerougbness data The main effects may be estimated using equations 1336 through 1342 The effect of A is for example A lr a abacabcb cbc I 4 4t22227233020211816 1 273375 8 and the sum of squares for A is found using equation 1344 ContrastS 21 455625 28 It is easy to verify that theother effects are B 1625 C 0875 AB 1375 ACOI25 BCO625 ABC 1125 From examining the magrrimde of the effects clearly fced rate factor A is domilant followed by depth of cut 8 and the AB interaction although the interaction effect is relatively small The analy sis of Itance is summarized in Table 13181 and it confirms our interpretation of the effect estimates Table 1317 Surface RoughneiS Data for Example 137 Treatment Design Facto Surface Combinations A B C Thtals 1 1 1 1 9 7 16 a 1 1 1 1012 22 b I I 1 91 20 ah 1 1 125 27 c I 1 1110 21 ae 1 I 1013 23 be I 1 1 108 18 abc I 1 1614 30 1 35 The 2k Factorial Design 383 Tbl1318 Analysis of Variance for the SlltaeeFinish Bxperimcnt Source of Sum of Degrees of Mean Variation Squares Freedom Square F A 455625 455625 1869 B 105625 lO5625 433 C 30625 30625 126 AS 75625 75625 310 AC 00625 00625 0Q3 BC 15625 15625 064 ABC 50625 1 50625 208 Error 195000 8 24375 Total 929375 15 Other Methods for Judgfng Significance af Effects The analysis of variance is a formal way to detenni1e which effects are nonzerQ There are tO other methods that are useful In the first method we can calculate thc standard errors of the effects a compare the mag nitudes of the effects to their standard errors The second method uses normal probability plou to assess the importance of the effects The standard error of an effect is easy to find If we assume that there are ft replicates at each of the 26 runs in the design and if Yil yc Yill are the observations at the ith run design point then i 12 2 is an estimate of the variance at tle ith run where t r lyJn is the sample mean of the n obsenrations The 2 variance esGnates can be pooled to give an overall variance estimate 1 2k fi 2 S 2knlIIYijYi 11 jl 1345 Where we have obviously assumed equal variances for each design point This is also the variance estimate gNen by the mean square error from the analysis of variance procedure Each effect estimate has variance given by VEffect vcontrast n2k 1 1 T VContrast n2kr Each contrast is a linear combination of 27 treatment totals and each total consists of n ohservations Therefore VContrast n2i and the variance of an effect is VEffect 1 n2 J n2H 1346 1 2 n2k 2 J 384 Chapter 13 Design of Experiment with Several Factors The estimated standard error of an effect would be found by replacing rr with its estimate s and taking the square root of equation 1346 To illustrate for the surfaceroughness experiment we find that S 24375 and the standard error of each estimated effect is seEffect 2 32 24375 078 Therefore two standard deviation limits on the effect estimates are A 3375 156 B 1625 156 C 0875 156 AB 1375 156 AC 0125 156 BC 0625 156 ABC Ll25 156 These intervals are approximate 95 confidence intervals They indicate that the wo main effects A and 8 are important but that the other effects are not since the intervals for all effects except A and B include zero Normal probability plots can also be used to judge the significance of effects We will illustrate that method in the next section Projection o2k Designs Any zi design will collapse or project into another i design in fewer variables if One or more of the original factors are dropped Sometimes this can pro vide additional insight into the remaining factors For example consider the surfacerough ness experiment Since factor C and all its interactions are negligibl we could eliminate factor C from the design The result is to collapse the cube in Fig 13 18 into a square in the A B planehowever each of the four runs io the new design has four replicates In gen era if we delete h factors so that r k h factors remain the origioalZ design with n repli cates will project into a 2r design with n2h replicates Residual Analysis We may obtain the residuals from a 2 design by usiog the method demonstrated earlier for the 22 design As an example consider the surfaceroughness experiment The three largest effects areA B and theABinteractiou The regression model used to obtain the predicted values is y Po px Ax 3 where Xj represents factor Al1 represents factor B and xAz represents the AD interaction The regression coefficientsBlA andft are estimated by onebalfthe corresponding effect esticlates andfJo is the grand average Thus 110625 33751 16251 1375 y IX 1 0 X xxz 2 2 135 The 2 Factorial Design 385 and the predicted values would be obtained by substituting the low and high levels of A and B into this equation To illustrate at the treatment combination where A B and C are all at the low level the predicted value is j 110625 751 e25I 175IJ 925 The observed values at thls run are 9 and 7 so the residuals are 9 925 025 and 7 925 225 Residuals for the other seven runs are obtained similarly A normal probability plot of the residuals is shown in Fig 1319 Since the residuals lie approximately along a straight line we do not suspect any severe nonnormaJiry in the data There are no indications of severe outliers It would also be helpful to plot the resid uals versus the predicted values and against each of the factors A 8 and C Yates Algorithm for the 2 Instead of using the table of plus and minus signs to obtain the contrasts for the effect estimates and the sums of squares a simple tabular algorithm de1sed by Yates can be employed To use Yates algorithm construct a table with the treat ment combinations and the corresponding treatment totals recorded in standard order By standard order we mean that each factor is introduced one at a time by combining it with all factor levels above it Thus for a 22 the standard order 1s 1 a b 00 while for a 23 it is 1 a b ab c ac be abc and for a 2 it is 1 a b ab c ac be abc d ad bd abd cd acd bcd abed Then follow thls fourstep procedure L Label the adjacent column IJ Compute the entries in the top half of thls column by adding the observations in adjacent pairs Compute the entries in the bottom half of this column by changing the sign of the first entry in each pair of the original obser vation and adding the adjacent pairs 2 Label the adjacent column 2J Construct column 2 using the entries in column 1 Follow the same procedure employed to generate colurrn 1 Continue this process until k columns have been constructed Colunm k contains the contrasts designated in the rows 3 Calculate the sums of squares for the effects by squaring the entries in column k and dividing ey ni 4 Calculate the effect estimates by dividing the entries in column k by n2 2 I 1 0 J 225OO1Oa337i75OO Residual ej Figure 1319 Normal probability plot of residuals fro the surfaceroughness experimen 386 Chaper 13 Design of ECperiments vith Several Factors Table 1319 Yates Algorithm for the SurfaceRougbness Experiment SlITlof Treatment Squares Effect Estimates Corebinations lJ 2 3J Effect 31fn2J 31n2 1 6 38 85 177 Total a 22 47 92 27 A 455625 3375 b 20 44 13 13 B 105625 1625 ab 27 48 14 11 AB 75625 1375 c 21 6 9 7 C 30625 0875 ae 23 7 4 AC 00625 0125 be 18 2 1 5 BC 15625 0625 abc 30 12 10 9 ABC 50625 Ll25 Exalie138 Consider the surfaccrougrness experiment in Example 137 This is a 23 design with n 2 replicates The analysis of this data using Yaes algorithm is illustrated in Table 1319 Note that the sums of squares computed from Yates algorithm agree with the resultS obtained in Example 137 353 A Single Replicate of the zf Design As the number of factors in a factorial experirlent grows the umber of effects that can be estimated gows also For exaClple a 24 experiment has 4 main effects 6 twofactor inter actions 4 threefactor interactions and 1 fourfactor interdction whlle a 25 experiment has six main effects 15 twofactor interactions 20 threefactor interactions 15 fourfactor interactions 6 fivewfactor interactions and 1 sixfactor interaction In most situations the sparsity of effecrs principle applies that is the system is usually dominated by the main effects and loworder interactions Threefactor and higher interactions are usually negligi ble Therefore when the number of factors is moderately large say k2 4 or 5 a common practice is to ron only a single replicate of the Zl design and then pool Or combine the higherorder interactions as an estimate of error Examl 19 An article in Solid State Technology Orthogonal Design for Process Optimization and its Application in Plasma Etchirg May 1987 p 127 describes the application of factorial designs in developing a nitride etc process ou a singlewafer plasma etcher The process uses y as the reactant gas Iris pos sible to vary the gaz flow the power applied to the cathode the pressure in the reactor chamber and the spacing berweel the mode azd the cathode gap Several response variables would usually be of iter cst in this process but in this example we will concentrate on etch rate for silicon niride We will use a single replicate of a r design to ivestigate this process Since it is unikcly that the threefactor and fourfactor interactions are significant we will tentatively plan to combine diem as an estimate of error The factor levels nsed in the design are shown here Design Factor A B C D Gap Pressure cFFlow Power Level em roThrr SeeM w LowH 080 450 125 275 High 120 550 200 325 1 J 135 The 21 Factorial Design 387 Table 1320 presents the data from the 16 runs of the 24 design Table 1321 is the table of plus and minus signs for the 24 design The signs in the columns of this table can be used to estimate the factor effects To illustrate the estimate of factor A is A aabacabCad ahd acdabcdlbcd be btl cdbed S 669650642 635 749 868 860 729550 604633 6011037 0520751063J 101625 Thus the effect of increasing the gap between the anode and the catbode from 080 em to 120 em is to decrease the etch rate by 101625 Almin 1t is easy to verify that he complete set of effect estimates is A 101625 B1625 AB7875 C7375 AC 24875 BC43875 ABCIS62S D306125 ADlS3625 BDO615 ABD4125 CD212S ACD5625 BCD 25375 ABCD40125 A very helpful method in judging the significance of factors in a 2 experiment is to construct a normal probability plot of the effect estimates If none of the effects is Table 1320 The 24 Design for the Plasca Ecch Experimnt A B C D Etch Rate Gap Pressure CF Flow power 1 1 I 1 550 1 1 I 669 I 1 I 1 604 1 1 1 1 650 1 1 1 633 1 J 1 642 1 1 601 1 1 1 1 635 1 1 1 1 1037 1 1 1 749 1 1 1052 1 1 868 1 1 1075 1 1 860 1 1 1 1063 1 729 388 Chapter 13 Design ofExperiPents with Several Factors Tablel21 Contra Constants for the 2 Design A B AS C AC BC ABC D AD BD ABD CD ACD BCD ABCD 1 a b ab c ac be r abc d ad bd aha cd acd bed abed significant then the estimates will behave like a random sample drawn from a normal dis tribution with zero mean and the plotted effects will lie apProximately along a straight line Those effects that do not plot on the line are significant factors The normal probability plot of effect estimates from the plasma etch experiment is shown in Fig 1320 Clearly the main effects of A and D and the AD interaction are signif icant as they fall far from the line passing through the other points The analysis ofvmance summarized in Table 1322 confirms these findings Notice that in the analysis of variance we have pooled the three and fourfactor interactions to form the error mean square If he nonnal probability plot had indicated that any of these interactions were important they then should not be included in the error term Since A 101625 the effect of increasing the gap benveen the cathode and anode is to decrease the etch rate However D 306125 so applying higher power levels Vill increase the etch rate Figure 1321 is a plot of the AD interaction This plot indicates IIlat the effect of changing the gap width at low power settings is smaU but that increasing the 2rrDnil 1 Zj O 037 7925 Effects 15237 22980 l l 30612 Figure 1320 Normal probabilit plot of effects from the plasma etch experiment 135 The 2k Factorial Design 389 Table 1322 Analysis of Variance for the Plasma Etch Experiment Source of Sum of Degrees of Mean Variation Squares Freedom Square F A 41310563 41310563 2028 B 10563 10563 1 C 217563 217563 1 D 374850063 374850063 18399 AB 248063 248063 1 AC 2475063 2475063 121 AD 94402563 99402563 4879 BC 7700063 7700063 378 BD 1563 1563 1 CD 18063 1 18063 1 Error 10186815 5 2037363 Total 531420938 15 gap at high power settings dramatically reduces the etch rate High etch rates are obtained at high power settings and narrow gap widths The residuals from the experiment can be obtained from the regression model 776 0625 101625 306125 153625 y Xl X4 x1x4 222 For example when A and D are both at the low level the predicted value is y 7760625 O625 I e06125rI e53625I I 597 and the four residuals at this treatment combination are 1400 1200 c 1000 800 600 u W 400 200 0 e550597 47 e 604 597 7 Dhgh 325w D10w 275w Low 080 em High 120 em A Gap Figure 1321 AD interaction from the plasma etch experiment 390 Chapter 13 Design of Experlments with Several Factors co L J 300 2016 33 6650 Residua 9 Figure 1322 Normal probability plot of residuals from the plasma etch experiment e 638 597 41 e6015974 The residuals at the othr three treatment combiIations A high D low A low D high and A high D high are obtained similarly A normal probability plot of the residuals is shown in Fig 1322 The plot is satisfactory 136 CONFotJNDING IN THE 2 DESIGN It is often impossible to run a complete replicate of a factorial design under homogeneous experimental conditions Confourding is a design technique for running a factorial experi ment in blocks where the block size is smaller than the number of treatment combinations in one complete replicate The technique causes certain interaction effects to be indistin guishable from or confounded with blocb Ve will illustrate confounding in the 2 facto 1 design in 2P blocks where p k Consider a 21 design Suppose that each of the 22 4 treatment combinations requires focr hours of laboratory analysis Thus two days are required to perform the experiment If days are considered as blocks then we must assign two of the four treatment combina tions to each day Consider the design shown in Fig 1323 Notice that block 1 contains the treatment combinations 1 and ab and that block 2 contains a and b The contrasts for estimating the main effects A and B are Contrast ab a b I Contraste ab b a 1 Note that these contrasts are unaffected by blocking since in each contrast there is one plus and one minus treatment combination from each block That is my difference between block 1 and block 2 will cancel out The contrast for the AB interaction is ContrastAl ab 1 ab Since the two treatment combinations with the plus sign ao and 1 are in block I and the two with the minus sign a and b are in block 2 the block effect and theAR interaction are identical That is AS is confounded with blocks I Block 1 1 all Block 2 136 Confounding in the 2 Design 391 Block 1 Block 2 1 IJ ab 8C c be abc Figure 1323 The 2 design in two blocks Figure 1324 The 2 design in two blocksABC confounded The reason for this is apparent from the table of plus and Illlnus signs for the 2 design Table 1313 From this tablel we see that all treatment combinations that bave a plus on AB are assigned to block 1 while all treatment combinations t1at have a minus sign on AB are assigned to block 2 This scheme can be used to confound any 2k design in two blocks As a second exarI ple consider a 2 design run in two blocks Suppose we wish to confound the threefactor interaction ABC witb blocks From the table of plus and minus signs for the 2 design Table 1316 we assign the treatment combinations that are minus on ABC to block 1 and those that are plus onABCto block 2 The resulting design is shown in Fig 1324 There is a more general method of constructing the blocks The method employs a defining contrast say 1347 where Xi is the level of the ith facto appearing in a treatrlent combination and Xi is the exponent appearing on the ith factor in the effect to be confounded For the 2k system we have either lX 0 Or lj and either x 0 low level or Xi 1 high level Treatment combi natious that produce tbe sarne value af L modulus 2 will be placed in tbe sarne block Since the only possible values of L mod 2 are 0 and I this ill assign tbe 2 treatment combinations to exactly wo blocks ill an example consider a 23 design with ABC confounded with blocks Here Xl corre sponds to A XZ to E X to C and lX OJ L Thus the defining contrast for ABC is Lx1 X1X1 To assign the treatment combinations to the wo blocks we substitute the treatment com binations iJto the defining contrast as follows 1 L 10 10 10 0 0 mod 2 a L 11 10 10 I I mOd 2 b L 10 11 T 10 I 1 IDlJd 2 abo L 11 11 10 2 0 mod 2 e L 10 10 11 I 1 mod 2 ac L 11 10 11 2 0 mod 2 be L 10 11 11 2 0 mod 2 abc L 11 11 r 11 3 I mod 2 Therefore I ab ac and be are1ll1 in block I and a b c and abc arenm in block 2 This is the same design shown in Fig 1324 392 Chapier 13 Design of Experiments with Several Factors A hortcut method is useful in constructing these designs The block containing the treaunent combination I is called the principal block Any element except Il in the principal block may be generated by multiplying two other elements in the principal block modulus 2 For example consider the principal block of the 2 design with ABC con founded shown in Fig 1324 Note that ab ac abc bCj ab bcabJcac ac be abC abo Treatment combinations in the other block or blocks may be generated by multiplying one element in the Dew block by each element in the principal block modulus 2 For the 2 with ABC confounded since the principal block is I ab ac and be we know that b is in the other block Thus the elements of this second block are Jtxample13lO bIb bababa b ac abc b be bc e An experiment is performed to investigate the effects of foW factors on the terminal miss distarce of a shoulderfired groundtoair missile The four faclors are target type CAl seeker type 8 tatget altitude C and target range D Each factor may bc conveniently ron at two levels and the optical tracking system will allow termi nal miss distUlce to be measured to the nearest foot Two differett gunners are used in the flighl test and since there may be differences between individuals it was decided to conduct the 24 design in two blocks with ABeD confounded Tus the defining contrast is Lx tzxx4 The experimental design and the resulting data are Block 1 Block 2 13 7 ab 7 b 5 ae 6 c 6 be 8 d4 ad 10 abe 6 bd4 bed 7 cd lid 8 acd 9 abed It 9 abd 12 The analysis of the dcsign by rates algorithm is shown in Table 1323 A noJIDal probability plot of the effects would reveal A taget type D target nL1ge and AD to have large effects A confuming analysis of variance using threefactor interactions as error is shown in Table 132J It is possible to confound the 21 design in four blocks of 21 observations each To con struct the design twO effects are chosen to confound with blocks and their defullng con trasts obtained A third effect the generalized interaction of the tvlo initially chosen is also i l I i I i i i 36 ConfounCing in the 2k Design 393 Table 1323 Yates Algorithm for the 24 Design in Exampe 1310 Treatment Sum of Efet Combinations Response I 2 3 4 Effect Estimate 1 3 10 22 48 111 Total a 7 12 26 63 21 A 275625 2625 b 5 12 30 4 5 B 15625 0625 ab 7 14 33 17 I AB 00625 0125 e 6 14 6 4 C 30625 0875 ae 6 16 2 1 19 AC 225625 2375 be S 17 14 4 3 BC 05625 0375 abc 6 16 3 3 1 ABC 00625 0125 d 4 4 2 4 15 D 140625 1375 ad 10 2 2 3 13 AD 105625 1625 bd 4 0 2 8 3 BD 05625 0375 abd 12 2 I 11 7 ABD 30625 0875 cd 8 6 2 0 1 CD 00625 8125 cd 9 S 2 3 3 ACD 05625 0375 bed 7 2 0 3 BCD 05625 iJ375 abed 9 2 1 I ABCD 00625 0125 Table 1324 Analysis of Variance for Example 131Q Soutce of Sum of Degrees of Meml Variation SquatCS Freedom Square Fe Blocks ABCD 00625 00625 006 A 275625 1 275625 2594 B L5625 15625 147 C 30625 30625 288 D 140625 1 140625 1324 AS 00625 1 00625 006 AC 225625 1 225625 2124 AD 105625 1 105625 994 BC 05625 1 05625 053 BD 05625 05625 053 CD 00625 00625 006 ErrorSABCABD ACD BCD 42500 4 10625 Total 849375 15 confounded with blocks The generalized interaction of two effects is found by multiplying their respective columns For example consider the 24 design in four blocks If AC and BD are confounded with blocks their generalized interaction is ACBD ABCD The design is constructed by using the defining contrasts for AC and BD L X1 XJ L x 394 Chapter 13 Design of Experinents with Several Factors It is easy to verity that the four blocks are Block 1 LO 4O rmr ire i bd I i abed I Btock 2 L14O c abd bed Siock 3 tO41 l Btock 4 t 1 4 1 I be ad cd This general procedure can be extended to confounding the 2k design in 21 blocks where p k Select p effects to be confounded such that no effect chosen is a generalized interaction of the others The blocks can be constructed from the p defIning contrasts L L Lp associated with these effects 10 addition exactly 2 P 1 other effects are COD founded with blocks these being the generalized ineraction of the origina p effects cho sen Care should be taken so as not to eonfound effects of potential interest For more information on confounding refer to Montgomery 2001 Chapter 7 That book contains guidelines for selecting factors to confound with blocks so that main effects and loworder interactions are Dot confounded In particular the book contains a table of suggested confounding schemes for designs with up to seven factors and a range of block sizes some as small as two roos 137 FRACTIONAL REPLICATION OF THE 2 DESIGN As the number of factors in a 21i increases the number of rons required increases rapidly For example a 2 requires 32 runs In this design only 5 degrees of freedom correspond to main effects and 10 degrees of freedom correspond to twofactor interactions If we can assume that certain highorder interactions are negligible then a fotiona factorial design involving fewer than the complete set of 21 runs can be used to obtain information on the main effects and loworder interactions In this section we will introduce fractional repli cation of the 2 design For a more complete treatment see Montgomery 2001 Chapter 8 1371 The One Half Fraction of the 2 Design A onehalf fraction of the 2k design tontalns 2k 1 rutS and is often called a ZkI fractional factorial design As an example consider the 21 design that is a onehalf fraction of the 2 The table of plus and minus signs for the 2 design is shown in Thble 1325 Suppose we select the four treatment combinations a h C1 and abc as our onehalf fraction These Table 1325 Plus and Minus Signs for the 21 Factorial Design Treatment Factorial Effect Combinations I A B C AB AC BC ABC a b c ab ac be I 137 Fractional Replication of te 2k Design 395 treatment combinations are shown in he top half of Table 1325 We will use boh the COD ventional notation a hI c and the plus and minus notation for the treatment comblll3 tions The equivalence between the two notations is as follows Notation 1 Notation 2 a b c abc Notice that the 23 1 design is formed by selecting only those treatment combinations hat yield a plus an he ABC effect Thus ABC is called he generator of this particular frac tion Furthermore the identity element I is also plus for the four runs so we call ABC the defining relation for the design The treatment combinations in the 2 designs yield three degrees of freedom associ ated with he main effects From Table 1325 we obtain he estimates of the rna effects as 1 A zabcabc I Ba b c abc 2 I Cabcabc 2 It is also easy to verify that the estimates of the twofactor interactions are I BC zab cabe I AC 2abcabc AB 1 bcabc 2 Thus he linear combination of observations in column A say eA estimates A BC Similarly eE estimateB AC and ec estimates C AB Twa or mare effects hat have this propery are called aliases In our 21 design A and BC are aliases B andAC are aliases and C and4B are aliases iiliasing is he direct result of fractional replication In many prac tical situations it will be possible to select the fraction so that he main effectS d low order interactions of interest will be aliased with highorder interactions which are probably negligible The alias structure for this design is found by using the defining relation I ABC Mul tiplying any effect by the defining relation yields he aliases for hat effect In OUr example the alias of A AA ABC ABC BC since A I A and A 1 The aliases of B and C are BB4BCABCAC 396 Chapter 13 Design of Expeirnents with Several Factors and CCABCABCAB Now suppose that we had chosen the other onehalf fraction that is the treatment com binations in Table 1325 associated with minus on ABC The defining relation for this design is I ABC The aliases are A BC B AC and C AB Thus the estimates of 4B and C with this fraction really estimateA BC B AC and C AB In practice it usu ally does not matter which onehalf fraction we select The fraction with the plus sign in the defining relation is usually called the principal fraction and the other fraction is usually called the altemate fraction Sometimes we use sequences of fractional factorial desiglS to estimate effects For example suppose we had run the principal fraction of the 23 design From this design we have the following effect estimates eABC CBAC eccAB Suppose that we are willing to assume at this point that the tJIofactor interactions are neg ligible If tile are then tbe 23 J design has produced estimates of the three main effects A B and C However if after running the principal fraction we are uncertain about the inter actions it is possible to estimate them by running the alternate fraction The alternate frac tien produces the following effect estimates e4BC e BAC eCAC If we combine the estimates from the two fractions t we obtain the following Effect i IC I jABCABCA 1 1BACBACcB CABCABC A BCABC BC BACBAC AC 1 CAB C AB AB Thus by combining a sequence of two fractional factorial designs we can isolate both tile main effects and the twofactor interactions This property makes the fractional factorial deSign highly tLeful in experimental problems j a we can run sequences of small efficient experiments combine infonnation across several experiments and take advantage of learn ing about the process we are experimenting with as we go along A 21 design may be constructed by TIting down the treatment combinations for a full factorial with k I factOrs and then adding the kth factor by identifying irs plus and minus levels with the plus and minus signs afthe highestorder interaction ABC K I Therefore a 21 fractional factOrial is obtained by writing down the full 2 factorial and then equating factor C to the AB interaction Thust to obtain the principal fraction we would use C AB as follows r l i Full 2 A B 137 Fractional Replication of the 21 Design 397 A B CAB To obtain the alternate fraction we would equate the last column to C AB To illustrate the use of ZI onehalf fraction consider the plasma etch experiment described in Exan pIe 139 Suppose that we decide to use a 24 1 design with 1 ABeD to investigate the foU factors gap A pessure B CF6 flow rate C and power setting D This design would e conslllced by writing down a 2 in the factors A B and C and then setting D ABC Tre design ond the resulting etch rates are shovrn in Table 1326 In this design the main effects aC aliased with the threefactor interactions note that the aias of A is and sirnilaly AAABCD A2BCD BCD BACD C4BD DABC The twofactor interactiors are aliased with each otheL For exnnple the alias of AB is CD Tbe other aliases axe ABABABCD ABCD CD Table 1326 Tte 24 Design with Defining Relation 1 ABCD Treatr1ent Etch A B C DABC Combinations Rilte 1 550 ad 749 T T bd 1052 ab 650 cd 1075 ac 642 be 601 abed n9 398 Chapter 13 Design of Experiments with Several Factors ACBD ADBC The estimates of the mal effects and their aliases are found using the four colUlIlls of signs in Table 1326 For example from column A we obtain t A BCD t550 7491052 650 1075 642 601 729 12700 The other columns produce and t 8 BACD400 tcCABDlLSO tDDABC 29050 Clearly e a d e b are large and if we believe that me tlLeefactor interactions are negligible then the main effects A gap and D power setting significantly affect etch rate The interactions are estimated by formig the AE AC and AD colun1s and adding them to the table The signs in theAB column are aDd this column prodces the estimate t AB CD j5507491052 650 1075 642 601 729 1000 From theAC and AD columns we find ecACBD 2550 AoADBC 9750 The tAD estimate is large the most straightforward interptetacion of the results is that this is the AD inreractio Thus the results obtained from the 2d I design agree with the full factorial results in Example 139 Normality Probability Plots and Residuals The normal probability plot is very useful in assessing the significance of effects from a fractional factorial This is particularly true when there are many effects to be estimated Residuals can be obtained from a fractional factorial by the regression model method shown previously These residuals should be plot ted against the predicted values against the levels of the factors and on normal probability paper as we have discussed before both to assess the validity of the underlying model asSlIDptions and to gain additional insight into the experimental situation Projection of the 2k 1 Design If one or core factors from a onehalf fraction of a 2 10 can be dropped the design will project into a full factorial design For example Fig 1325 pres ents a t 1 design Notice that this design will project into a full factorial in any tNo of the three original factors Thus if we think that at most two of the three factors are important the 2 1 design is an excellent design for identifying the significant factors Sometimes we call experiments to identify a relatively few significant factors from a larger number of fac tors screening experiments This projection property is highly useful in factor screening as it allows negligible factors to be elimimted resulting in a stronger experiment in the active factors that remain j 137 fractional Replication of the 2 Design 399 B I I I I I I b l I I I I I I abc I II I A tf a I I I I I I I I I I I I I I I I V C Figure 1325 Projection of a 23 1 design iIlto three 2 designs In the 21 design used in the plasma etch experiment in Example 1311 we found that two of the four factors B and C could be dropped If we eliminate these two factors tte remaining columns in Table 1326 form a 2 design in the factors A and D with two reFli cates This design is shown in Fig 1326 Design Resolution The concept of design resolution is a useful way to catalog fractional factorial designs according to the alias patterns they produce Designs of resolution III IV and V are particularly imporraut The cefioitions of these terms and an example of each follow 1 Resolution III Designs These are designs in which no main effects are aliased with any other main effect but main effects are aliased with twofactor interactions and twofactor interactions may be aJiased with each other The 2 design with I ABC is of resolution m We usually employ a subscript Roman numeral to indicate design resolution thus this onehalf fraction is a tm 1 design 2 Resolution IV Designs These are designs in which no main effectis aliased with any other main effect or twofactor interaction but twofactor interactions are 10521075 749729 1 1550601 650642 1 1 1 A Gap Figure 1326 The 22 design obtained by dropping factors B and C from the plasma etch experiment 400 Chapter 13 Design of Experiments with Severa Factors aliased ith each other The 21 design Mth I ABCD used io Example 1311 is of resolution N 21 1 3 Resolution V Designs These are designs in which no main effect Or twofactor interaction is aliased with any other main effect or twofactor interaction but two factor interactions are aliased with threefactor interactions A 25 design with I ABCDEis ofresolution V 21 Resolution ill and N designs are particularlY useful io factor screening experiments A resolution IV design provides very good information about main effects and will provide some information about twofactor interactions 1372 Smaller Fractions The 2P Fractional Factorial Although the 2k 1 design is valuable in reducing the number of runs required for an experi ment we frequentlyfind that smaller fractions will provide almost as much useful informa tion at even greater economy In general a 2 design may be run in a 112P fraction called a 2 fractional factorial design Thus a 14 fraction is called a 2 fractional factorial design a 18 fraction is called a 2k J design and so on To illustrate a 14 fraction consider an experiment Mth six ftors and suppose that the engineer is interested primarily in main effects but would also like to get some information about the twofator interactions A 2 t design would require 32 runs and would have 31 degrees of freedom for esthnation of effects Since there are only six main effxts and 15 twofactor interactions the oneha1f fraction is inefficientit requires too many runs Suppose we consider a 114 fraction or a 26 2 design This design contains 16 runs and with 15 degrees of freedom will allow esthnation of all six main effects ith some capability for examination of the twofactor interactions To generate this design we would write dovma 24 design in the factors A B C and D and then add two columns for E and F To find the new columns we would select the two design geMrators I ABCE andl ACDF Thus column E would befouod from EABCand column Fwould be FACD and also columns ABCE and ACDF are eqnal to the identity column However we know that the product of any two columns in the table of plus and minus signs for a 2 is just another column in the table there fore the product of ABCE and ACDF or ABCE ACDF ABCDEF BDEF is also an identity column Consequently the complete defining relation for the 25 2 design is IABCEACDFBDEF To find the alias of any effect simply multiply the effect by each word io the foregoiog deflning relation The complete alias strUct1lre is A BCE CDFABDEF B ACE DEF ABCDF C ABE ADF BCDEF DACFBEFABCDE EABCBDFACDEF FACDBDEABCEF AB CE BCDFADEF ACBEDFABCDEF liD CF BCDE ABEF 1 r 13 7 Fractional Replication of the 21 Design 401 AEBC CDEFABDF AFCDBCEFABDE BDEFACDEABCF BFDE ABCD ACEF ABFCEFBCDADE CDEABDAEF CBF otice that this is a resolution IV design main effects are aliased with threefacor and higher interactions and twofactor interactions are aliased with each other TIlls design would provide very good information on the main effects and give some idea about the strength of the twofactor interactions For example if the AD interaction appears signifi can either AD andior CF are significant If A andior D are significant main effectS but C and F are not the experimenter may reasonably and tentatively attribute the significance to the AD interaction The construction of the design is shov1l in Table 1327 The same prnciples can be applied to obtain even smaller fractions Suppose we wish to investigate seven factors in 16 runs This is a 27 3 design a 118 fraction This design is constructed by writing daVID a 24 design in the factors A B C and D and then adding three new columns Reasonable choices for the three generators required ace I AliCE I BCDF and I ACDG Therefore the new columns are formed by setting E ABC F BCD and G ACD The complete defining relation is found by multiplying the generators together two at a time and then three at a time resulting in I ABCE BCDF ACDG ADEF BDEG ABFG CEFG Notice that every main effect in this design will be aliased with threefactor and higher interactions and that twofactor interactions will be aliased with each other Thus this is a resolution IV design For seven factors we can reduce the number of runs even further The 27 design is an eightrun experiment accommodating seven variables This is a 1116 fraction and is Tabl137 Construction of the 26 2 Design with Generators I ABCE and I ACDF A B C D EABC FACD 1 dej be obf eif at bcf abce dj ade bdef obd ede oedf bed obcdef 402 Chapter 13 Design of Experiments with Several actors obtained by first writing down a 2 design in the factors A B and C and then forming the four new columns from 1 ABD I ACE 1 BCF and 1 ABCG The design is shown in Table 1328 The complete defining relation is found by IDuetiplying the generators together two three and finally four at a time producing I ABD ACE BCF ABCG BCDE ACDF CDG ABEF BEGAFGDEFADEGCEFGBDFGABCDEFG The alias of any main effect is found by multiplying that effect througb each term in the defining relation For example the afuu of A is ABD CEABCFBCGABCDECDFACDG BEF ABEG FG ADEF DEG ACEFG ABDFG BCDEFG This design is of resolution rn since the main effect is aliased with twofactor interactions If we assume that all threefactor and higher interactions are negligible the aliases of the seven main effects are eABD CE FG eBADCFFG fcCAEBFDG CDDABCGEF CEACBGDF CFBCAGDE eo GCDBEAF This 27i4 design is called a saturated fractional factorial because all of the available degrees of freedom are used to estimate main effects It is possible to combine sequences of these resolution ill fractional factorials to separate the main effects from the twofactor interactions The procedure is illustrated in Montgomery 2001 Chapter 8 In constrUcting a fractional factOrial design it is important to select the best set of design generators Montgomery 2001 presents a table of optimum design generators for designs With up to 10 factors The generators in this table will produce designs of max imum resolution for any specified combination of k and p For more than 10 factors a A S C DCAS E ACI F SCI 138 Sample Computer Output 403 resolution ill design is recommended These designs may be constructed by using the same method illustrated earlier for the 2114 design For example to investigate up to 15 factors in 16 runs vrite down a 24 design in the factorS A B C andD and then generate 11 new columns by taking the productS of the original four columns two at a time three at a time and four at a time The resulting design is a 21 n fractional factoriaL These designs along witit other useful fractional factorials are discussed by Montgomery 2001 Chapter 8 138 SAMPLE COMPUTER OUllUT We provide Minitab ouput for some of the examples presented in this chapter Sample Computer Output for Example 133 Reconsider Example 133 dealing with aircraft primer paints The Minitab0 results of he 3 x 3 factorial design with tluee replicates are Ara1ysis of Variance for Force Source DF SS 4S I p Type 2 45811 22906 2786 0 000 Applicat 1 49089 49089 5970 O 000 TypeApplicat 2 02411 01206 147 0269 Error 12 09867 00822 Total 17 107178 The Minitab results are in agreement with the results given in Table 136 Sample Output for Example 137 Reconsider Example 137 dealing with surface roughness The Minitab results for the 23 design with two replicates are Term Effect Coef SE Coef T P Constant 11 0625 03903 2834 0000 A 337S0 1 6875 03903 432 0003 B 16250 08125 03903 208 0071 C 08750 04375 03903 112 0295 AB 13750 06875 03903 1 76 0116 AC 01250 00625 03903 016 0877 BC 0250 03125 03903 080 0446 ABC 11250 05625 03903 144 0188 Analysis of Variance Source DF Seq SS Adj SS Adj lS P p Mall Efect s 3 59187 59187 19729 809 0 008 2Way Ineractions 3 987 9187 3062 126 0352 3Way Interactions 1 5062 5062 5062 208 0188 Residual Error 8 19500 19500 2437 Fure Error 8 19500 19500 2438 Total 15 92937 The ouput from Minitab is slightly different from the results given in Example 137 Hests on the main effects and interactions are provided in addition to tite analysis of variance on the significance of main effects twofactor interactions and threefactor interaiions The 404 Chapter 13 Design of Experiments with Several Factors AlOVA results indicate that at least one of the main effects is significant whereas no two factor or threefactor interaction is significant 139 SUM1VlARY This chaper has introduced the design and analysis of experiments with several factors j concenrrating on factorial designs and fractional factorials Fixed randoIr and mixed mod els were considered The Ftests for main effects and interactions in these designs depend on whether the factors are fixed or random The 2k factorial designs were also introduced These are very useful designs in which all k factors appear at two levels They have a greatly simplified method of statistical analy sis In situations where the design cannot be run under homogeneous cQnditiQns the 2 design can be confounded easily in 2P blocks This requires that certain interactions be con founded with blocks The 2 design also lends itself to fractional replication in which only a parJcular subset of the 2 treatment combinations are run In fractional replication each effect is aliased with one or nore other effects The general idea is to alias main effects and loworder interactions with higherorder interactions This chapter discussed methods for construction of the 2P fractional facorial designs that is a If2P fraction of the 2 design These designs are particularly useful in industrial experimentation 1310 EXERCISES 131 An article in the Journal of Materials Process ing Technology 2000 p 113 presents results from an experiment involving tool wear estimation in milEng The objective is to minimize tool wear 1vo factos of nterest in e study were cutting speed mlmin and dpth of cut mm One response of interest is tool flank wear mm Three levels of eacll factor were selected and a factorial experiment with three repli cates is run Analyze the data and draw conclusions Depth of Cut 2 3 0170 0198 0217 12 0IS5 0210 0241 0110 0232 0223 0178 0215 0260 15 0210 0243 0289 0250 0292 0320 0212 0250 0285 1875 0238 0282 0325 0267 0321 0354 132A1 engineer suspectS that the surface finish of a meta part is lnI1uenced by the ype of paint used and the dring time He selects three ding times20 25 and 30 minutesandrandornly chooses two types of paint from several that axe available He conducts an experiment and obtains t1e data shown here Ana lyze the data and draw conclusions Estimate the vari ance components Drying Time min Paint 20 25 30 74 73 78 64 61 35 50 44 92 92 98 66 86 73 45 68 88 85 133 Suppose that in Exercise 132 paint types wce fixed effects Compute a 95 interval estimate of the mean difference between the responses for paint type 1 and painttype 2 134 The factors that influence the breaking strength of cloth are being studied Four machines and three opeators axe chosen at random and an experiment is run using cloth from the same oneyard segment The results are as follows Machine Operator 2 3 4 A l09 llO lOS 110 1I0 as 109 116 B 111 110 III 114 112 III 109 112 C 109 112 114 111 111 liS 109 112 Test for interaction and main effers at the 5 leveL Estimate the components of variance 35 Suppose that in Exercise 134 the operators were chosen at random but only four machines were available for the test Does this influence the analysis or your conclusion 136 A company employs two timestudy engine Tbei supervisor wishes to determine whether the standards set by them are bfluenced by any interac tion between engineers and operators Sbe selects three operators at random and conducts an experiment in which the engineers set standard times for the same job She ohtains the data shown here Analyze the data and draw conclusions Operator Engineer 2 3 259 238 240 278 249 272 2 215 285 266 286 272 287 137 An article in Industrial Quality Con1ltl 1956 p 5 descnbes an investigite the effect of tVo factors glass type and phosphor type on the brightness of a television tuhe The response variable measured is the current necessary in IIiicroamps to obtaizt a spedfied brightness level The daa are shOTI heJe A nalyze the data and draw conc1usonsassum irg that hoth factors are fixed Phospbor Type Glass 2 3 280 300 290 290 310 285 285 295 290 230 260 220 235 20 225 241 235 230 310 Execises 405 13M8 Consider the tool wear dara in Exercise 131 Plot the residuals from this experiment against the lev cis of cutting speed and against the depth of cut Com ment on the graphs obtained Vhat are the possible consequences of the information conveyed by the residual plots 139 The pccenrage of hardwood concentration in raw pulp ard the reeness aLd cooking time of pulp are being investigaed for their effects on the strength of paper Analyze the data shown in the following table assUltlig that all three factors are fixed Cooking Time Percenage of 15 hours Hardwood Freeness Concentration JO 500 650 10 966 977 994 960 960 99S 15 985 960984 972 969 976 20 975 956 974 966 962 981 Cooking TUM 20hou s Freeness 410 500 650 984 996 1OQ6 9S6 100 1009 975 987 996 9S1 980 990 976 970 985 984 978 998 1310 An article in Quality Engineering 1999 p 357 presents the results of an experiment conducted to determine the effects of three factors on warpage in a1 injectionmolding process Warpage is defined as the nonfiatness property in the product manufactured This parrictaar company toarufactures plastic molded components for use in television sets washing machines and automobiles The three factors of inter est each at two levels are A celt temperature B injection speed ard C injection prxess A complete 23 factorial design was carried out Nith replication fINo replicates are provided in the table below All1 lyze the data from this experiment A B C IT 1 1 1 135 140 I 1 215 220 1 1 150 150 1 110 120 1 I 1 070 070 1 1 140 135 1 120 135 LlO 100 1311 For the Warpage experimentm Exercise 1310 obtain the residuals and plot them on nonnal proba bility pape ALso plot the residuals VerSus the pre dicted values Commen on these lots 406 Chapter 13 Design of Experiments with Several Factors 13 12 Four factors are thought to possibly influence the taste of a soft drink beerage type of sweetener 4 rario of syrup to water B carbonation level C and temperature D Each factor can be run at two levels produciIlg a 24 design At each run in the design samples of the beveage are given to a test panel consisting of 20 people Each tester assigrs a point score from 1 to 10 to the beverage Total score is the response variable and the objective is to futd a formulation that maximizes tOW sCOre Two replicates of this design are run and the results shown here Analyze the data and draw conclusions Tteatment licae Combinations I II 1 190 193 a 174 178 b 181 185 ab 183 180 c 177 178 ac 181 180 be 188 182 abc 173 170 Treatment Replicate Combinations T II d 198 195 ad 172 176 bd 187 183 aba 185 186 cd 199 190 aed 179 175 bed 187 184 abed 180 180 1313 Consider the experiment in Exercisc 1312 Plot the residoals against the levels of factors A B C and D Also construct a normal probability plot of the residuals Comment on these plots 1314 Fmd the standard error of the effects for he cxperitlcnt in Exercise 13 12 Using the standard errors as a guide what factors appear significant 1315 The data shoD here represent a single repli cate of a 25 design that is used in an experiment to study the compressive strength of concrete The fac tors are nUx A time B laboratory C temperature D and drying time E Analyze the data assuming that threefactor and higher interactions are negligible Use a noma1 probability plot to assess the effects 1 700 dOOO e 800 de 1900 a 900 adll00 ae 1200 ade1500 b 3400 hd 3000 be 3500 bde4000 ab 5500 abd6100 abe 6200 abd 6500 e 600 cd SOO ce 600 cde 1500 ae 1000 acdll00 ace 1200 aede 2000 be 3000 bed 3300 oC3006 bede3400 abc 5300 abcd6000 abee 5500 abed 6300 1316 An experiment described by L G Natrella in the National Bureau of Standards Handbook of Experimental Statistics No 91 1963 involves tlametesting fabrics after applying fIreretardan treatments There are four factors type of fabric A type of fircretaIiant treatment B laundering condi ton Cthe low level is no laundering the hlgh level is aFter one laundering and the method of conducting rhe flame test D All factors are ron at two levels and the response ariable is the inches of fabric burned on a standard size test sample The data are 1 42 d 40 a 31 ad 30 b 45 hd 50 ab 29 abd 25 e 39 cd 40 DC 28 aed 25 be 46 bed 50 abc 32 abed 23 a Estimatc the effects and prepare a nonnal proba bility plot of the effects b Construct a nonnal probability plot of thc residu als and comment on the results c Construct an analysis of variance table assuming that three and fourfactor interactions are negligible 1317 Consider the data from the first replicate of Exercise 1310 Suppose rhat these oservations could not all e ron under the same conditions Set up a design to run these observations in two blocks of four observations each withABC confounded Analyze Lie data 1318 Consider the data from the firSt replicate of Elercise 13 12 Construct a desig with two blocks of eight observations each with ABCD confounded Analyze the data 13 19 Repeat Exercise 1318 assuming that four blocks are required Confound ABD and ABC and consequemJy CD wi blocks 1320 Construct a 2 design in four blocks Select the effects to be confounded so rhat we confound the highest possible interactions with blacks 1321 An article in Industrial and Engineering Chemistry Factorial Experiments in Pilot Plant Studies 1951 p 1300 repots on an experiment to investigate the effects of temperature A gas through put B and concentration C on the strength m prod uct solution in a recirculation unit TWo blocks were used with ABC confounded and the experiment was replicated twice The data are as follows r 1 2 Block 1 Block 2 Block 1 Block 2 1461 a 051 ab47 b co 10 I ac22 abc 35 bc67 a Analyze 6e data from this eperixucnt b Plot the residuals on Donna probability paper and against the predicted values Comment on the pots obtained c Comment Oli the efficiency of this design Note that we have replicated the experiment tvice yet we have no information on the ABC interaction d Suggest betterdesign specificallyono that would provide some information on aU interactions 1322 R D Snce Experimenting Vim a Large Number of Variables in Experiments in Industry Design Analysis and Interpretation of Results by R D Snee L B Hare and J B Trout Editors ASQC 1985 describes an experiment in which a 21 design with I ABCDE was used to investigatethe effects of five factors on the color of a chemical product The factors are A solventreactant B catalystreactant C temperature D reactant puity lll1d E reactant pH The results obtained are as follows e O63 d 69 a 251 ode 647 b H8 bde 3A5 abe L66 abd 568 e 06 cd 522 ace 122 aed 438 bee 209 bed 430 abc L93 abede 405 a Prepare a nnrmal probability plot of the effects Vbich factors are active P Calculate the residuals Construct a nonna ptob ahility plot of the residuals and plot the residuals versus the fitted values Comment on the plots c If any factors are negligible collapse the 25 i design into a full factorial in the active factors Coe1t on tlC resulting design and i1tepret the results 1323 An article in the Joumal of Quality Tedmol ogy VoL 17 1985 p 198 describes the use of a replicated fractional factorial to investigate the effects of five factors on the free height of leaf springs used in an automotive application The factors are A frrnace temperature B heating time C transfer time 1310 Exercises 407 D hold down time and E quench oil temperature The data are shown below A B C D E 778 778 781 815 818 788 750 756 750 759 756 775 754 800 788 769 809 806 756 752 744 756 781 769 750 725 712 788 788 744 750 756 750 763 775 756 732 744 744 756 769 762 718 718 725 18 75O 759 al Vhat is the generator for this fractio Write out the alias struCture 0 luialyze tle data Vihat factors influence mean free height c Calctlate the range of free height for each run Is there any indication that any of these factors affects variability in free height d Analyze the residuals from this experiment and comment on your findings 1324 An artic1e in Industrial and Engineering Chemistry More on Planning Experiments to increase Research Efficiency 1970 p 60 uses a 25 2 design to investigate the effects of A condensation temperature B anlount of material 1 C solvent volZlC D ondenSation time and E amount of material 2 on yield The results obtained are as follows e 232 ad 169 cd 238 bde 163 ab 15S be 162 ace 234 abed 181 a Verify that the design generators used were I ACE and 1 BDE b Write down the complete defudng relation and the aliases from this design e EsliJlate the rnain effects d Prepare a analysis of variance table Verify that the Ali and AD interactions are available to use as error e Plot the residuals versus the fitted values Also construct a normal probability plot of the residu als Corcoet ou the results 408 Chapter 13 Design of Expe6ments with Severn Fato 1325 An article in Cement and Concrete Research 2001 p1213 describes an experimenttQ investigate the effects of four metal oxides on several cement properties The four factors are aU run at two levels and one response of irterest is the mean bclk density gcm The four factors and their lelels are Low Level High Level Factor 1 1 ll FeG 0 30 8 7J10 0 15 c PbO 0 25 D 00 0 25 Typical results from this type of experiment are given in the follONing able Run ll B C D Density I 1 I I 2001 2 I 1 I 1 2062 3 1 1 1 2019 4 1 1 2059 5 1 1 1 1990 6 1 1 1 2076 7 1 1 1 2038 8 1 1 1 1 2118 a What is the generator for dlls fraction b Analyze the data What factors influence mean bulk density e Analyze the residuals from this experiment and comment on your findings 1326 Consider the 22 design in Table 1327 Sup pose that aftet analyzing the original data we find that factors C and E can be dropped Vlhat type of 2t design is left in the remaining variables 1327 Consider the 2 design in Table 1327 Sup pose that after the original data analysis we find that factors D and F can be dropped hat type of 2k design is left in the reroaining variables Compare the results with Exercise 1326 Can you explain why the answers are different 1328 Suppose that in Exercise 1312 it was possible to nm only a onehalf fracdon of the Z design Con strUCt the design and perform the statistical analysis use the data from replicate 1 1329 Suppose that in Exercise 1315 only a onehalf fraction of the 2 design could be run Construct the design and perform the analysis 1330 Consider the data in Exercise 1315 Suppose that only a onequarter fraction of the 25 design could be run Construct the design and analyze the data 1331 Construct a 2t fractional factorial design Write doWll the aiases assuming that only main effects aLd twofactor interactions are of inteCCSL J r chapterl4 Simple Linear Regression and Correlation In many problems there are two or more variables that are inherently related and it is nec essary to explore the nature of this relationship Regression analysis is a statistical technique for modeling and investigating the relationship between NO or more variables For exam ple ill a chemical process suppose that the yield of product is related to the process oper ating temperature Regression analysis can be used to build a model that expresses yield as a function of temperature This model can then be used to predict yield at a given temper ature leveL It could also be used for process optimization or process control purposes In general suppose that there is a single dependent variable or response y that is related to k independent or regressor variables say Xltz Xt The response variable y is a random variable while the regressor variables xt1 xI are measured wih neg1igi ble error The Xi are called mathematical variables and are frequently controlled by the eXperimenter Regression analysis can also be used in situations where y Xi X2 x are jointly distributed random variables such as when he data are collected as different meas urements on a common experimental unk The relationship between these variables is char acterized by a matheIratical model called a regression equation More precisely we speak of the regression of yon X1t1 Xk This regression model is fitted to aset of data In some instances the experimenter will know the exact form of the true functional relationship between y and x lt1 x say y x x2 xJ However in most cases the true func tional relationship is unknovn and the experimenter will choose an appropriate function to approximate iJ A polynomial model is usually employed as the approximating function In this chapter we discuss the case where only a single regressor variable x IS of interw est Chapter 15 will present the case involving more than one regressor variable 141 SIlvIPLE LINEAR REGRESSION We wish to detennine the relationship beteen a single regressor variable x and a response variable y The regressor variable x is assuUed to be a continuous mathematical variable controllable by he experimenter Suppose that the true relationship benveen y and x is a straight line and that the observation y at each level of is a random variable Now the expected value of y for each value of x is 141 where the intercept Pc and the slope 31 are unknown constants We assume that each obser vation y can be described by the model y 13 f3x 142 409 410 Chapter 14 Simple Linear Regresio and Correlation where is a random error with mean zero and variance cr The are also assumed to be uncorrelated random variables The regression model of equation 142 involving only a sin gle regressor variable x is often called the simple linear regression model Suppose that we have n pairs of observations say y x y x y x These data may be used to estimate the unknown parameters f30 and Pl in equation 142 Our esti mation procedure will be the method ofleast squares That is we will estimate p and Pl so that the sum of squares of the deviations between the observations and the regression line is a minimum Now using equation 142 we may write i12 n 143 and the sum of squares of the deviations of the observations from the true regression line is n n L 21 l Yi Po Plxt 144 The leastsquares estimators of Po and p say Po andfi must satisfy 145 Simplifying these two equations yields jl 11 n n Pol X PlX 146 Equations 146 are called the leastsquares normal equations The solution to the normal equations is 147 148 where y lniE y andX llnLlx Therefore equations 147 and 148 are the least squares estimators of the intercept and slope respectively The fitted simple linear regres sion model is 149 141 Simple Lbear Regression 411 Notationally it is convenient to give special symbols to the numerator and denomina tor of equation 148 That is let 1410 and n S 2x il 1411 We callSp the corrected sum of squares ofx and Sr the corrected sun of cross products of x and y The extreme righthand sides of equations 1410 and 1411 are the usual computa tional formu1as Using this new notation the leastsquares estimator of the slope is S f3 L S 1412 A chemical engineer is investigating the effect of process operating terrperature on product jield The study results irl the following data Tempernture C xl 100 110 120 130 140 150 160 170 180 190 YIeld y 45 51 54 61 66 70 74 78 85 89 These pairs of points are plotted irl Fig 141 Such a display is called a scatter diagram Examination of this scatter diagramirldicaLes that there is a sJOng relationship between yield anc temperature and the tentative assumption of the straightlile model y Pc PIX appears to be reasonable The fol lolltilg quantities IDay be computed nlO Lx 1450 C Lx 218500 il IO 2i673 I 10 Lyl 47225 fool 145 C LXy 101570 il From equations 1410 and 1411 we find S x 1l y 218 500 450 8250 n L 1OL I 10 1 j and 10 1 O i 10 I 14506731 S yI x1 y 101570 3985 ty L lOL 1 j I 10 1 1 1 Therefore the leastsquares estimates of the slope and intercept are S 3985 3 0483 S 8250 412 Chapter 14 Simple Linear Regression and Correlation 100 9C 80 70 60 5D ID 40 30 20 10 0 1 i 100 110 120 130 140 15D 160 170 160 160 Temperature x Figure 141 Scatter diagram of yield versus temperature and Po y fi ii 673 0483145 2739 The fitted simple linear regression model is y 2739 0481 Since we have only tentatively assumed the straightline model to be appropriate we will want to investigate the adequacy of the model The statistical properties of the least squares estimators Po and Pt are useful in assessing model adequacy The estimators A and PI are random variables since they are just linear combinations of the Yi and the Yl are ran dom variables We will investigate the bias and variance properties of these estimators Con sider first A The expected value of 3 is l r 141 Simple Liear Regression 413 I since 27 xlX 0 r 7X1 Sxx and by assurnptionEe OThusPI is an unbi ased estimator of the true slope PI Now consider the variance of Pt Siuce we have assumed that VeE 0 it follows that Vy 0 and 1413 The random variables y are uncorrelared because the are uncorrelated Therefore the variance of the stlmm equation 1413 is just the sum of the variances and the variance of each term in the sum say VIYXI is VZx i2 Thus I 22 I 2 1 xx S2 xx l vp 1414 02 S Using a similar approach we can show that and 1415 Note t ftc is an nbiased estimator of Po The covariance of Po and 13 is not zero iu fact Cov3e f3 rriIS It is usually necessary to obtain an estimate of fl The difference betVeen the obser vation 1 and the corresponding predicted value y say e y y is called a residual The sum of the squares of the residuals or the error sum of squares would be 2 y 1416 A more couyenint computing formula for SSe may be found by substiruting the fitted model 30 i3x into equation 1416 and simplifying The result is n SSE2Y7 11 and if we let Lftylny2 iYi y2 Syy theu we may Write SSfi as jl 1417 414 Chapter 14 Simple Linear Regression and Correlation The expected value of the error sum of squares SSe is ESSE n 21 Therefore 1418 is an unbiased estimator of oz Regression analysis is widely used and frequently misused There are several common abuses of regression that should be briefly mentioned Care should be talren in selecting vari ables with which to construct regression models and in determining the form of the approxi mating function It is quite possible to develop statistical relationships among variables that are completely unrelated in a practical sense For example one might attempt to relate the shear stren1h of spot welds with the nnUber of boxes of computer paper used by the data processing department A straight line may even appear to provide a good fit to the data but the relationship is an unreasonable one On which to rely A strong observed association benveen variables does not necessarily imply that a causal relationship exists beween those variables Designed experiments are the only way to detennine causal relationships Regression relationships are valid only for values of the independent variable vithin the range of the original data The linear relationship that we have tentatively assumed may be valid over the original range of XI but it may be unlikely to remain so as we encounter x values beyond that range In other words as we move beyond the range of values of x for which data were collected we become less certain about the validity of the assumed modeL Regression models are not necessarily valid for extrapolation purposes Finally one occasionally feels that the model y 8x is appropriate The onission of the intercept from this model implies of course that y 0 when x O This is a very strong assumption that often is unjustified Even when tVO variables such as the height and weight of men would seem to qualify for the use of this model we would usually obtain a better fit by including the intercept because of the limited range of data on the independent variable 142 HYPOTHESIS TESTING IN SlMPLE LINEAR REGRESSION An important part of assessing the adeqaacy of the simple linear regression model is test ing statistical hypotheses about the model parameters and constructing certain confidence intervals Hypothesis testing is discussed in this section and Section 143 presents methods for constructing confidence intervals To test hypotheses about the slope and intept of the regression model we must make the additional assumption that the error component t is normally distributed Thus the complete assumptions are that the errors are lJD0 d normal and independently distributed Later we will discuss how these assumptions can be checked through residual analysis Suppose we wish to test the hypothesis that the slope equals a constant say 8 o The appropriate hypotheses are B 8 8 B81 8c 1419 where we have assumed a tVosided alternative Now since the are NlDO oZ it follows directly that the observations y are XJDj3 8x 0 From equation 148 we observe that 3 is a linear combination of the observations Yr Thus 3 is a linear combination of inde pendent normal random variables and consequently fit is 3 oS using the bias and variance properties of 3 from Section 141 Furthermore 3 is independent of MS Then as a result of the normality assumption the statistic 1420 142 Hypothesis Teting in Simple Linear Regression 415 foDows the tdistribution with n 2 degrees of freedom underHo p P We would reject Ho p P 0 if Itol t 1421 where to is computed from equation 1420 A similar procedure can be used to test hypotheses about the intercept To test we wood use the statistic Ho p Po H Pn Pno and reject the null hypothesis if Itol ta1 A very important special case of the hypothesis of equation 1419 is He p 0 H 3 O 1422 1423 1424 This hypothesis relates to the significance of regression Failing to reject Ho PI 0 is equivalent to concluding that there is no linear relationship betweenx and y This situation is illusrrated in Fig 142 Note that this may imply either tbatx is of little value in explain ing the variation in y and that the best estimator of y for any x is y Y Fig 142a or that the true relationship between and y is not linear Fig 142b Alternatively if Iio 3 0 is rejected this implies that is of value in explaining the variability in y This is illustrated in Fig 143 However rejecting HOfJl 0 could mean either that the straightline model is adequate Fig 143a or that even though there is a linear effect ob better results cnuld be obtained with the addition of higherorder polynomial terms in x Fig 143b The test procedure for Iio3 0 may be developed from two approaches The frst approach starts with the following partitioning of the total corrected sum of squares for y 1425 a b Figure 142 The hypothesis Hy 3 0 is not rejected 416 Chapter 14 Simple Linear Regression and Correlation y a Figure 143 The hypothesis Ho PI 0 is rejected b x The two components of Sy measure respectively the amount of variability in the Yi accounted for by the regression line and the residual variation left unexplained by the regression line We usually call SSE ICY Yrl the error sum of squares and SSp I Yi Yl the regression sum of squares Thus equation 1425 may be vrritten 1426 Comparing equation 14f 6 with equation 1417 we note that the regression sum of squares SSES 1427 S has 1 degrees of freedom and SS and SSE have 1 and n 2 degrees of freedom respectively We may show thatESSr 2 a and ESSJ a f3S and that SSE and SS are independent Thus if Ho f3 0 is true the statistic 1 SSRl o SS 2 1428 follows the FLl distribution and we would rejectHo if Fc F ex lII1 The test procedure is usually arranged in an analysis of variance table or klOVA such as Table 141 The test for significance of regression may also be developed from equation 1420 with ilL 0 say 1429 Squaring both sides of equation 1429 we obtain 2 Jfs AS MS to MSe MSE MS 1430 Table 14 1 Analysis ofVaance for Tesing Significance of Regression Source of Sum of Degrees of Mean Variation Squares Freedom Square Regression 35 pSo MS Error or residual S5S JS n2 MS Total Syy n 1 1 143 Inteval Estlmatior in Simple Linear Regression 417 Table 142 Testilg for Significance of Regression Example 142 Source of Sum of Degrees of Mean Variation Squares Freedom Regression 192487 1 192487 23874 Ettor 723 8 090 Thal 193210 9 Note thatt in equation 1430 is identical to Fain equation 1428 It is true in general that the square of a t random variable with degrees of freedom is an F random variable with One and degrees of freedom in the numerator and denominator respectively Thus the test using is equivalent to the test based on Fo We will test the model developed in Example 141 for significance of regression The fitted model is y 2739 0483x and Syy is computed as 47225 673 0 193210 The regression sum of squares is SS PeS 04833985 192487 and the error sum of squares is 193210 192487 723 The analysis of variance for testing Ho f3 0 is summarized in Table 142 Noting that Fo t 213874 F IJO L 1126 We reject He and conclude that P L O 143 INTERVAL ESTIMATION IN SIMPLE LINEAR REGRESSION In addition to point estimates of the slope and intercept it is possible to obtain confidenoe interval estimates of these parameters The width of these confidence intervals is a mere of the overall quality of the regression line If the are normally and independently dis tribute then and 418 Chapter 14 Simple Linear Regression and Correlation are both listributed as t wilb n 2 degrees of freedom Therefore a 1001 a confidence interval on Ibe slope 3 is given by 1431 Similarly a 1001 a confidence interval on Ibe intercept 3 is I 1 1 jl 30la12 2 MS J53o S3ota22MSE J 1432 II n S n S xx xx We will find a 95 confidence interval on Llle slope of the regression line using the data in Exarnplc 141 Recall that 0483 S 8250 and MS 090 see Table 142 Then from equation 1431 we fine or 048323061 09 III 048323061 090 8250 820 TIris simplifies to 045 Il 0507 A confidence interval may be constructed for the mean response at a specified x say x This is a confidence interval about EYIxJ and is often called a confidence interval about Ibe regression line Since Eylx 3 31x we may obtain a point estimate of EYIxJ from Ibe fitted model as E yi3Pxo Now Yo is an unbiased point estimator of EYIxJ since A and p are unbiased estimators of 30 and 3 The variance of Yo is Vio O x Ln S and Yo is normally distributed as A and PI are normally distributed Therefore a 1001 a confidence interval abom the true regression line at x 0 may be computed from 1 1 xoxlJ Yo taln2 MSEl n Sa 1433 11 xx1 E YixOYoaJ22 MSEI I n Sa I r 143 btcrval Estimation in Simple Linear Regression 419 Tne width of the confidence interval for Ey1xJ is a function of xc The interval width is a minimum for Xu X and Widens as o Xl increases This widening is one reason why using regression to extrapolate is illadvised We will construct a 95 confidence interval about the regression line for the data in Example 14 1 The fited model is Yi 2739 0483x and the 95 confidence interval on Eyix is found from equation 1433 as f r 21 1 xo 14 Yc 230090 Iii 250 I 1 The fitted values Yo and the coaespomiilg 95 corfldence limits for the poits 10 ae displayed in Table 143 To illustrate the use of this table we may find the interval on the true mean process yield at Xc 140C say as 6488 071 EYix 40 648 071 or 647 EYlx 140 6549 i 1 2 confidence The fitted model and the 95 confidence interval about the regression line ac shown in Fig 144 Table 143 Confidence Interval about the Regression Line Example 144 100 110 120 130 140 150 160 70 10 190 4556 5039 5522 6005 648 6972 7455 7938 1142 8904 95 confiderce limits L30 LlO Q93 Q79 Q71 Q7t Q79 Q93 LlO 130 100 90 80 70 60 50 40 30 20 10 0 I I 100 110 120 130 140 150 160 170 180 190 Temperature x Figure 144 A 95 confidence interval about the regression line for Erampie 144 420 Chapter 14 Sircple Linear Regres10n and Correlation 144 PREDICTIO OF NEW OBSERVATIOS An important application of regression analysis is predicting new Or future observations y corresponding to a specified level of the regressor variable x If Xo is the value of the regres sor variable of interest then 1434 is the point estimate of the new or future value of the response Yo Now consider obtaining an interval estimate of this future observation Yo llis new observation is independent of the observations used to develop the regression model There fore the cOlliidence interval about the regression line equation 1433 is inappropriate since it is based only on the data used to fit the regression model The confidence interval about the regression line refers to the true mean response at x Xc that is a population parameter not to future observations Let Yo be the future observation atx xo and letyc given by equation 1434 be the esti matOr of Yo Note that the random variable Iy Yo is normally ilistributed with mean zero and variance because Yo is independent ofy Thus the 1001 a prediction interval on a future obser vations ar Xc is 1435 Notice that the prediction interval is of minimum width at Xo X and widens as J xl increases By comparing equation 1435 with equation 1433 we observe that the predic tion interval atxo is always wider than the confidence interval at x This results because the prediction interval depends on both the error from the estimated model and the error asso ciated with furore observations cr We may also find a 1001 a prediction interval on the mean of k future observa tions on the response atx XO Let Yo be the mean of k future observations at x Xo The 1001 a prediction interval on Yo is 1436 L 145 Measuring the Adequacy of the Regression Model 421 To illustrae the consruction of a prediction interval suppose we use the data in Exam ple 141 and find a 95 prediction interfal on the next observation on the process yield at Xo 160C Gsing equation 14351 we find that the prediction interval is 74552306 iO901 160145 10 8250 which simplifies to rcc 745 306 10901 1 160145 Yo L 10 8250 145 MEASLRING THE ADEQUACY OF THE REGRESSION MODEL FirJng a regression model requires several assumptions Estimation of the model parame ters requires the assumption that the errors are uncorrelated random variables with mean zero and constant variance Tests of htPotheses and inteITal estimation require that the errOrS be normally disnibuted In addition we assume that the order of the model is correct that is if we fit a firstorder polynomial then we are assuming that the phenomenon actu ally behaves in a firstorder manner The analyst should always consider the validity of these assumptions to be doubtful and conduct analyses to examine the adequacy of the model that has been tentatively enter tained In this section we discuss methods useful in this respect 1451 Residual Analysis We define the residuals as e1 Yy i 1 2 n1 where Yi Is an observation andy is the corresponding estimated value from the regression modeL Analysis of the residuals is fre quently helpful in checking the assumption that the errors are 1IDO iT and in detennin ing whether additional terms in the model would be useful As anapproximate check of normality the experimenter can construct a frequency his togram of the residuals or plot them on normal probability paper It requires judgment to assess the nonnormality of such plots One may also standardize the residuals by comput ing d eYMSE i 1 2 n If the errors are NIOO a then approximately 95 of the standardized residuals should fall in the interval 2 Residuals far outside this interval may indicatc the presence of an outlier that is an observation that is atypical of the rest of the data Various rules havc been proposed for discarding outliers However some times outliers provide important information about unusual circumstances of interest to the experimenter and should not be discarded Therefore a detected outlier should be investi gated first then discarded if warranted For further discussion of outliers sec Yontgomery Peck and Yrning 2001 It is frequently helpful to plot the residuals I in time sequence ifrnown 2 against the and 3 against the independent variable x These graphs will usually look like one of the four general patterns shoWn in Fig 145 The pattern in Fig 145a represents normal ity while those in Figs 145b Ct and d represent anomalies If the residuals appear as in Fig 14Sb then the variance of the observations may be increasing with time or with the magnitude of the i or xi If a plot of the residuals against time has the appearance of Fig 145b then the variance of the observations is increasing with time Plots agamsty and Figure145 patterns for residual plots al Satisfactory b funnel e double bow d nonlinear Adapted from Montgomery Peck and Vming 2001J Xi that look like Fig 145c also indicate inequality of variance Residual plots that look like Fig 145d indicate model inadequacy that is higherorder terms should be added to the model lxami145 The residuals for the regression model in Example 141 are computed as follows e 4500 4556 056 e 5100 5039 061 e 5400 5522 122 e 61006005 095 e 6600 6488 Ll2 e 7000 6972 028 7400 7455 055 7800 7938 138 e 8500 8421 079 e 8900 8904 004 These residuals Me plotted on normal probability paper in Fig 146 Since the residuals fall approx imately along a straight line in Fig 146 we conclude that there is no severe departure from nonnal ity The residuals are also plotted agai1st y in Fig 147a and against Xi in Fig lA7b These plots indicate no serious model inadequacies 1452 The LackofFit Test Regression models are often fit 10 data when the true funetioual relationship is unknown Naturally we would like to know whether the order of the model tentatively assumed is car rect This scction will describe a test for the validity of this assnmption 145 Measuring the Adequacy of the Regression Moce 423 2 5 10 20 30 40 j3 m 50 0 e c 60 70 BO 90 95 9BL I I 15 10 05 00 05 Residuals Figure 146 N onnal probability plot of residuals et 200 100 000 c 100 10 15 200 l 200 00 000 100 200 40 50 60 70 80 I 20 L 100 110 120 130 40 50 160 170 180 190 x 98 95 90 80 70 60 50 40 30 20 10 is 2 Figure 147 Residual plots for Example 145 0 Plot agalnsty b plot againsttr The danger of using a regression model that is a poor approximation of the true func tional relationship is illustrated in Fig 148 Obviously a polynomial of degree two or greater should have been used in this situation 424 C1apter 14 Simple Linear Regression and Correlation x Figure 148 A regression model displaying llck of fit We present a test for the goodness of fit of the regression model Specifically the hypotheses we wish to test are Ho The model adequately fits the data H The model does not fit the data The test involves partitioning the error or residual sum of squares into the NO components SSE SS SSWF where SS iE is the sum of squares attributable to pureH error and SSWF is the sum of squares attributable to the lack of fit of the model To compute SS we must have repeated obser vations OIl y for at least one level of x Suppose thar We have n total observations such that repeated observations at XI repeated observations at Yml Ynr2 YI1VI repeated observations at x Note that there are m distinct levels of x The contribution to the pureerror sum of squares at x say would be 1437 The total sum of squares for pure error would be obtained by summing equation 1437 over all levels of x as SSpE ILY y2 1438 There are n L n 1 n m degrees of freedom associated with the pureerror sum of squares The sum of squares for lack of fit is simply 1439 wid n 2 n m 2 degrees of freedom The test statistic for lack of fit would then be SSwdm2 TQ SSPEnm 1440 and we would reject it if Fo F t2IIm l J 145 Measuring the Adequacy of the Regression lodel 425 This test procedure may be easily introduced into the analysis of vafance conducted for the significance of regression If the null hypothesis of model adequacy is rejected then the model must be abandoned and attempt made to find a more appropriate model If Ho is Dot rejected then there is DO apparent reason to doubt the adequacy of the model and MSPE and MS WF are often combined to estimate rfl iXire 1ft SuppQse we have the follov1ng data x 10 LO 20 33 33 40 40 40 47 50 Y 23 L8 28 18 37 26 26 22 32 20 x 56 56 56 60 60 65 69 y 35 28 21 34 32 34 50 We may compute St 1097 Sry 1320 Su 5253 y 2847 rud x 4382 The regression model isy 1703 Q26Qx and theregressionsum of squares is SSr f3IS1 02601362 3541 The pureerror SUrl of squares is computed as follows Level of x Ey Yl Degrees of Freedom 10 01250 33 18050 40 01066 2 56 09800 2 60 00200 1 Total 30366 7 The analysis of variance is summarized in Table 144 Since Foz a 1 L 70 we cannot reject the hypothesis that the tentatve model adequatelY describes the data Ve ill poor lakoffit and pure error mean squares to fonn the denominator mean square in the test for significance of regression Also since 454 we conclude that1 O In fitting a regression model to experimental a good practice is to use the lowest degree model that adequately describes the data The lackoffit test may be useful in this Tbl144 Anilysis of Variance for Example 146 Source of Sum of Degrees of Mean Variation Squares Freedom Square Regression 3541 1 3541 715 Residual 7429 15 0495 Lack of fil 4392 8 0549 127 Pureerror 3037 7 0434 Total 10970 16 426 Chapter 14 Simple Linear Regression and Correlation respect However it is always possible to fit a polynomial of degree n I to n data points and the experimenter should not consider using a model that is saturated that is that has very nearly as many independent variables as observations on y 1453 The Coefficient of Determination The quantity I44J is called the coefficient of determination and is often used to judge the adequacy of a regression model VIe will see subsequently that in the case where x and y are jointly dis tributed random variables R2 is the square of the correlation coefficient between x and y Clearly 0 R I We often refer loosely to R as the amount of variability in the data explained or accounted for by the regression model For the data in Example 141 we have R SSFIS 192487193210 09963 that is 9963 of the variability in the data is accounted for by the model The statistic Rl should be used with caution since it is alWys possible to makeR2 unity simply by adding enougb tenns to the model For example we can obtain a perfect fit to n data points with a POlplOmiaI of degree n 1 Also Rl will always increase if we add a variable to the model but this does not necessarily mean the new model is superior to the old one Unless the error Sum of squares in the new model is reduced by an amount equal to the original error mean square the new model will have a larger error mean square than the old one because of the loss of one degree of freedom Thus the new model will actu ally be worse than the old one There are several misconceptions about R2 In general R does not measure the mag nitude of the slope of me regression line A large value of R does not imply asleep slope Furthermore Rl does not measure the appropriateness of the model since it can be artifi cially inflated by adding higberorder polynomial terms Even if y and x are related in a non linear fashion Jl will often be large For example R2 for the regression equation in Fig 143b will be relatively large even though the linear approximation is poor Finally even though R2 is large this does not necessarily imply that the regression model will pro vide accurate predictions of future observations 146 TRANSFORMATIONS TO A STRAIGHT LThE We occasionally find that the straigbtline regression model y Po PX is inappropri ate because the true regression function is nonlinear Sometimes this is visually determined from the scatter diagram and sometimes we know in advance that the mode1 is nonlinear because of prior experience or underlying theory In some situations a nonlinear function can be expressed as a straigbt line by using a suitable transformation Such nonlinear mod els are called intrinsically linear As an example of a nonlinear model that is inttinsically linear consider the exponen tial function Y 3oePe This function is intrinsically linear since it can be transformed to a straigbt line by a loga rithmic transformation lny lnP Jh In J 147 Correlation 427 This transformation requires that the transformed error tenns 1n E be normaEy and inOO pendently distributed with mean 0 and variance a Another intrinsically linear funetion is 11 y303lI x By using the reciprocal transformation z lix the model is linearized to y3o3z Sometimes several transformations can be employed jointly to linearize a function For example consider the function Letting y lly we have the linearized form lny 30 3 Several other examples of nonlinear models that are intrinsically linear are given by Daniel and Wood 1980 147 CORRELATION Our development of regression analysis thus far has assumed that x is a mathematical vari able measured with negligible error and that y is a random variable Many applications of regression analysis invulve situations where both and y are random variables In these sit uations it is usually assumed that the obserllations Yi XI i 1 2 n are jointly dis tributed random variables obtained from the distributiony x For example suppose we wish to develop a regression model relating the shear strength of spot welds to the weld diameter In this example weld diameter cannot be controlled We would randomly select n spot welds and observe a diameter Xi and a shear strength y for each Therefore y x are jointly distributed random variables We usually assume that the joint distribution ofy and Xi is the bivariate normal distri bution That is 1442 where Jil and 0 are the mean and vaiance of Y J4 and a are the mean and variance of x and p is the correlation coefficient between y and x Recall from Chapter 4 that the corre lation coefficient is defined as where 12 is the covariance between y and x The conditional distribution of y for a given value of x is see Chapter 7 428 Chapter 14 Simple Linear Regression and Correlation where and 2 01 r That is the conditional distribution of y given x is normal with mean Eylx Po PX 1443 1444a 1444b l444c 1445 and variance Oi Note that the mean of the conditional distribution of y given x is a straightline regression modeL Furthermore there is a relationship between the correlation coefficient p and the slope PI From equation 1444b we see that if p then P 0 which implies that there is no regression of y On x That is knowledge of x does not assist us in pre dicting y The method of maximum likelihood may be used to estimate the parameters 130 and JI It may be shown that the maximum likelihood estimators of these parameters are JoyJi 14400 and iI l446b We note that the estimators of the intercept and slope in equaden 1446 are identical to those given by the metlod of least squares in the case where x was assumed to be a mathe matical vaable That is the regression model with y and x jointly normally distributed is equivalent to the model with x considered as a mathematical variable This follows because the random variables y given x are independently and normally distributed with mean Po PIX and constant variance 0 These results will also hold for any joint distribution of y and x such that tle conditional distribution of y given x is noxmaL It is possible to draw inferences about the correlation coefficient p in this model The estimator of p is the sample correlation coefficient 1447 J 147 Correlation 429 Note that S 12 YY j III s r a 1448 so the slope PI is JUSt the sample correlation coefficient r multiplied by a scale factor that is the square root of the spread of the y values divided by the spread of the x values Thus il and r are closely related although they provide somewhat different information The sample correlation coefficient r mea1Ues the linear association between y and x while il measures the predicted change in the mean of y for a mtit change in x In the case of a mathematical variable x r has no meaning because the magnitude of r depends on the choice of spacing for x We may also write from equation 1448 which we recognize from equation 1441 as the coefficient of determination That is the coefficient of determination RZ is just the square of the sample correlation coefficient between y and x It is often useful to test the hypothesis HpO HpeO The appropriate test statistic for this hypothesis is 1449 1450 which follows the t distribution with n 2 degrees of freedom if Ho P 0 is true There fore we would reject the null hypothesis if Itl 11 This test is equivalentto the test of the hypothesis He Il 0 given in Section 142 This equivalence follows directly from equation 1448 The test procedure for the hypothesis H PPa Hppo 1451 where Pe 0 is somewhat more complicated For moderately large samples say n 25 the statistic 1 lr Z arctanh r ln 2 lr is approximately normally distributed with mean 1 Ip Ilz arctanh P ln 2 Ip 1452 430 Chapter 14 Simple Linear Regression and Correlation and variance r n3t Therefore to test the hypothesis Ho P Po we may compute the statistic Zo arctanh T arctanh pOln 3ln and reject H P Po if IZo Z 1453 It is also possible to construct a 1001 a confidence interval for P using the trans formation in equation 1452 The 1001 a eonfidence intelval is Zai2 J Z2 tanh arctanh r r In3 p tanh arctanh r In3 1454 where tanh u e Ie e 1Efi Montgomery Peck and VUJing 2001 descnbe an application of regression analysis in which an engineer at a softdrlnk bottler is investigating the product distribution and route service operations for vending machines She suspects that the time required to load and service a machine is related to the number of cases of produt delivered A random sample of 25 etail outlets having vending machines is selected and tOe inoutlet delivery time in minutes and volume of product delivered in cases is observed for each outlet The dataare shown in Table 45 We assume that delivery time and volume of product delivered ate jointly normally distributed Using the data in Table 145 we may calculate S 61059447 The regression model is s 6985600 y 51145 29027 S 20277132 The sample correlation coefficient between x and y is computed from equation 1447 as Sry 20277132 09818 T l t 6985600610594471 SnSYJ Table 145 Data for Example 147 Delivery iumber of Delivery Number of Observation Tune y cses x Observation TUlle y Cx 995 2 14 1166 2 2 2445 8 15 2165 4 3 3175 11 16 1789 4 4 3500 10 17 6900 20 5 2502 8 18 1030 1 6 1686 4 19 3493 O 7 1438 2 20 4659 15 8 960 2 21 4488 15 9 2435 9 22 5412 16 10 2750 8 23 5663 17 11 1708 4 24 2213 6 12 3700 11 25 2115 5 13 4195 12 i J r I 149 Summary 431 Note that Rl 098182 09640 or that approximately 964000 of the variability in delivery time is explained by the linear relationship with delivery volume To test the hypothesis Iio p 0 Ii p 0 we can compute the test statistic of equation 1450 as follows n2 0981823 2480 to jl2 jl09640 Since toOli1 2069 we reject Ho and conclude fuat the correlation coefficient pt O Fially we may construct an approximate 95 confidence interval on p from equation 1454 Since arctanh r arctmh 09818 23452 equation 1454 becot1S tmh 23452 p tmhIZ3452 196 I 22 fii which reduces to 09585 s P s 09921 148 SAMPLE COMPUTER OUTPUT Many of the procedures presented in this chapter can be implemented usiug statistical soft ware In this seetion we present the Minitab output for the data in Example 141 Recall that Example 141 provides data On the effect of process operating temperature on product yield The Minitab output is The regression equation is Yield 274 0483 Temp Predictor Coef Constant 2739 Tp 048303 SE Coe 1546 O0046 T 77 467 P 0114 0000 S 09503 RSq 996 RSqadj 996 Aalysis of Variance Source DF Regression 1 Residual Error 8 Total 9 55 9249 72 932 MS 9249 09 F F 2131 57 0000 The regression equation is protided along with the results from the Hests on the individual coefficients The Pvalues indicate that the intercept does not appear to be significant Pvalue 0114 while the regressor variable temperarure is statistically significant Pvaue 0 The analysis of variance is also testing the hypothesis that Ii 31 0 and can be rejected Pvalue 0 Note also that r 4617 for temperature and f 4617 213167 F Aside from rounding the computer results are in agreement with those found earlier in the chapter 149 SUMMARY This chapter has introduced the simple linear regression model and shown how least squares estimates of the model parameters may be obtained Hypothesis testing procedures 432 Chapter 14 Simple Linear Regression and Correlation and confidence interval estimates of the model parameters have also been developed Tests of hypotheses and confidence intervals requrre the assumption that the observations y are nonnally and independently distnouted random 3Iiables Procedures for testing model adequacy including a lackoffit test and residual analysis were presented The correlation model was also introduced to deal with the case where x and yare jointly normally distrib uted The equivalence of the regression model parameter estimation problem for the case where x and y are jointly nonnal to the case where x is a mathematical variable was also dis cussed Procedures for obtaining point and interval estimates of the correlation coefficient and for testing hypotheses about the correlation coefficient were developed 1410 EXERCISES 141 Montgomery Peck and Vwing 2001 present data concerning the performance of the 28 National Football League teams in 1976 It is suspected that the number of games won y is related to the number of yards gainedUShing by an opponert x The data are shown below Yards Rushing by Teams Games Won y Opponent x Washington 10 2205 Minnesota 11 2096 New Englaod 11 1847 Oakland 13 1903 Pittsbcrgh 10 4151 Baltimore 11 1848 Los Angeles 10 1564 Dallas 1 1321 Atlanta 4 2577 Buffalo 2 2476 Chicago 7 1984 Cinc1Ilati 10 1917 Cleveland 9 1761 Denver 9 1709 Detroit 6 1901 Green Bay 5 2288 Houston 5 2072 Kansas Cit 5 2861 Miami 6 2411 ew Orleans 4 2289 New York Giaats 3 2203 Sew York Jets 3 2592 Philadelphia 4 2053 St Louis 10 1979 San Diego 6 2048 San Francisco 8 1786 Seattc 2 2876 Tampa Bay 0 2560 a Fit a linear regression model relating games won to yards gained by an opponent b Test for significance of regression e Find a 95 confidence interval for me slope d Vha percentage of total variability is explained by the lIlodel e Findhe residuals and prepare appropriate resid ual plots 142 Suppose We would like to use the model devel oped in Exercise 141 to prediet he number of games a team will n if it can limit the opponents to 1800 yards rushing Find a poin estimate of the number of games won if the opponents gain only 1800 yards rushing Find a 95 prediction interval on the number of games WOn 143 Motor Trend magazine frequently presents per formance data for automobiles The table below pres ents data from the 1975 volume of Motor Trend concerniog the gasoline milage perfonnance and the engine displacement for 15 automobiles files I Displacement Automobile Gallon y Cubic x Apollo 1890 350 Omega 1700 350 tova 2000 250 Monarch 1825 351 Duster 2001 225 Jensen ConY 1120 40 Skybawk 2212 231 Monza 2147 262 Corolla SR5 3040 969 Carnaro 1650 350 Eldorado 1439 500 Trans Am 1659 400 Charger SE 1973 318 Cougar 1390 351 Corvette 1650 350 J l a Fit a regression model relating mileage perform ance to engine displacemenL b Test far significance of regression c What percentage of total arizbility in mileage is explained by the model d Fird a 90 confidence intena On the mean mileage if the engine displacement is 275 cubic inches 144 Suppose that we wish to predict the gasolirc mileage from l car vith a 275 cubic inch displacement engine Find a point estimate using the model devel oped in Exercise 143 ard an appropriate 90 interval estimate Compare this interval to the Onc obraned in Exercise l43d Which one is wider and why 145 Fmd the rcsidlals from the moder in Exercise 143 Prepare appropriate residua plors anC commert on model adequacy 146 An article in TechnomeIrics hy S C Narula and J E Wellington Prediction Linear Regession and a MininnntJ SUDl of Relative Errors VoL 19 1977 presents data On the selling price and annual taxes for 27 houses The data are shown below Taxes Local School Sale Price f 1000 County 11000 259 49176 295 50208 279 45429 259 45573 299 50597 299 38910 309 5898Q 289 56039 359 58282 315 53003 3LO 62712 309 59592 300 50500 369 82464 419 66969 45 77841 439 90384 375 59894 379 75422 445 87951 379 60831 389 83607 369 81400 458 91416 410 Exexoises 433 a Fit aregression model relating sales price to taxes paid b Test for significance of regression c What percentage of the variability in selling price is explained by the taxes paid Cd Find the residuals for this model Construe 11 nor mal probability plot for the residuals Plot the residuals versus y and versus x Does the model seem satisfactory 147 The strergth of paper used ill the manufacture of cardboard boxes y is related to the percentage of hardwood concentration in the original pulp x UnCer controlled corditions a pilot plant manufaetuCS 16 samples each from a different batch of pup and measures the tensie strength The data are shown here 4 7411 11211911609118 19 y 113 1230 121 1452 1343 45 J431 1469 x zlZSl28 I 2 30 30 32 I B a Fit a simple linear regression model to the data b Test for lack of fit and significa1ce of regression c Construct a 90 confidence interval on the slope fl Cd Construct a 90 confidence interval 00 the inter cept flo e Construct a 95 confidence interval on the true regression line at t 25 148 Compute the residuals for the regression model in Exercise 147 Prepare appropriate residual plots and comment on model adequacy 14 9 The number of pounds of steam used per month hy a chemical plant is thought to he related to the aver age ambient temperature for that month The past years usage a1d temperatures are shown in the following table Month Temp Usage 1 1000 Jan 21 18579 Feb 24 2147 Mar 32 28803 Apr 47 428 May 50 4558 June 59 53903 July 68 62155 Aug 74 67506 Sept 62 56203 Oct 50 45293 No 41 36995 Dec 30 27398 434 Chapter 14 Simple Linear Regression and Correlation a Pit a simple linear rcgression model to the data b Test for significance of regression e Test the hypothesi that the slope 131 10 d Construct a 99 confidence interval about ilie true regression line atX 58 e Construct a 99 prediction interval on the steam usage in tlie net month having a mean ambient temperatJlc of 58 1410 Compute the residuals for the regession model i Exercise 149 Prepare appropriate residual ploto and comment on model adequacy 1411 The percentage of impurity in oxygen gas pro eueen by a distilling process is iliought to be related to the percentage of Ilydrocabon in the main condenser of the processor One mon1s operating data are avail able as shown in the table at the bottom of this page a Pit a simple linear regresso model to the data b Test fot lack of fit and significance of regresston c Calculate R2 for this model d Calculate a 95 confidence interval for the slope 13 1412 Compute the residuals for the data in Exercise 1411 a Plot the residuas on norma probability paper and draw appropriate coucbsions b Plot the residuals againsty and x Interpret these displays 1413 NJ article in Transportation Research 1999 p 183 presents a study On world maritime employ ment The pwpose of the Stdy was to determine a relationship between average manning level and the average size of the fleet Marrung level refers to the ratio of number of posts that must be manned by a seaman per ship postsship Data collected for ships of the lIned Kingdom over a 16yeax period are Average Size Purity 9154 9277 9221 998 8705 HYGrocarbon Purity Hydrocarbon 8691 102 73 146 Level 2027 1998 2028 1965 1881 continues Average Size Level 8530 1820 8544 1805 7964 1681 7440 1556 6432 1398 6032 1451 5125 1099 4418 1283 4327 1185 4133 1133 3765 1025 a Fit a linear regression model relating average manning level to avemge ship size b Test for significance of regression c Find a 95 confidence interval 0 the slope d hat percentage of total variability is explained by ilie model e Find the residuals and construct appropriate resid ual plots 1414 The final averages for 20 randomly selected stvdents taking a course in engineering statistics and a course in operations researcb at Georgia Tech ae shown here Assume that the final averages are jointly normally distributed Statistics OR Statistics OR a Find the regression line relating the statistics final average to the OR final average b Estimate the correlation coefficient c Test the hypothesis that p O d Test the bypothesi that p 05 e COIlSQUct a 95 confidence interval estinate of the correlation coefficient 1415 The weight and systolic blood pressure of 26 randomly selected males in the age glOJp 2530 are 8733 1 8629 8986 095 I 111 102 9500 I 9685 8520 9056 101 i 099 095 098 shown in the following table Assume that weight and blood pressure are jointly normally distributed Subject 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Weight 165 167 180 155 212 175 190 210 200 149 158 169 170 172 159 168 174 183 215 195 180 143 240 235 192 187 Systolic BP 130 133 150 128 lSI 146 150 140 148 125 133 135 150 153 128 132 149 158 150 163 156 124 170 165 160 159 a Find a regression tine relatingsystolic blood pres sure to weight b Estimate the correlation coefficient e Test the hypothesis that p O d Test the hypothesis that p 06 e Construct a 95 confidence interval estitoae of the correlation coefficient 1416 Consider the siJrple linear regression model Y 130 f3x c Snow that EMS 12 flStt 1417 Suppose that we have assumed the straightline regression model but that the response is affected by a second variable tz such tt the true regression function is 1410 Exercises 435 Is the esir1ator of t1e slope in the simple linear regression model unbiased 1418 Suppose that we are fitting a straight line and we wish to make the variance of the s10pe Pl as small as possible Vtbere should the observations x i 1 2 n be taken so as to minimize Vfi l Disss the practical implications of this allocation of the X 14M19 Weighted Least Squares Suppose thar we are fitting the straight line y flo l iJjx but that the vzriance of the y values now depends on the level of x that is i12 n where the Wi are unknown constants often called weights Show that the resulting easHquares normal equations are 50 t WI PIt wx t WY l 11 loLwixlPLwixr LwxY tl i 14M20 Consider the data shOWll below Suppose that the relationship between y and x is hypothesized to be y iJ PIX Erl Fit an appropriate model to the data Does the assumed model form seem appropriate x 6 y 024 1421 Consider the weightand blood pressure data b Exercise 1415 Fit a nowbtercept model to the data and compare it to the model obtained in Exercise 1415 Which model is superior 1422 The following data adapted from Montgomery Peck and Vming 2001 present the number of certified mental defectives per 10000 of estimated population in the United Kingdom y and the number of radio receiver licenses issued x by the BBC in millions for the years 19241937 Fit a regression model relating y to x COIllment on the model Specifically does the existence of a strong cor relation imply a causeandeffect relationship 436 Chapter 14 Simple Linear Regression and Correlation Number of Certified Number of Radio Mental Defectives per Receiver Licenses 10000 of Estimated Issued Millions Year UK Population y in the UK xl 1924 8 1350 1925 8 1960 1926 9 2270 1927 10 2483 1928 1 2730 1929 11 3091 1930 12 3674 1931 16 4620 1932 18 5497 1933 19 6260 1934 20 7012 1935 21 7618 1936 22 3m 1937 23 8593 J Chapter 15 Multiple Regression Many regression problems involve more than one regressor variable Such models are called multiple regression models Multiple regression is one of the most widely used sta tistical techniques This chapter presents the basic techniques of pardII1eter estimation con fidence interval estimation and model adequacy checking for multiple regression We also introduce some of the special problems often encountered in the practical use of multiple regression j including model building and variable selection autocorrelation in the errors t and multicollinearity or nearlinear dependence among the regressors 151 MULTIPLE REGRESSION MODELS A regression model that involves more than one regressor variable is called a multiple regression model As an example suppose that the effective life of a cutting tool depends on the cutting speed and the tool angle A mnltiple regression model that might describe this relationship is 151 where y represents the tool life XI represents the cutting speed and x2 represents the tool angle This is a multiple linear regression model with two regressors The term linear is used because equation 151 is a linear function of the unknown parameters f3o f31 and Note that the model describes a plane in the twodimensionalx1 s space The parameter f30 defines the intercept of the plane We sometimes call 3 and A partial regression coeffi cients because 3J measures the expected change in y per unit change in x when A2 is held constant and A measures the expected change in y per unit change in X when Xl is held constant In general the dependent variable or response y may be related to k independent vari abies The model 152 is called a mnltiple linear regression model with k independent variables The parameters j 0 I k are called the regression coefficients This model describes a hyperplane in the kdimensional space of the regressor variables x The parameter f3 represents the expected change in response y per unit cbange in Xj when all the remaining independent variables Xi i t are held constant The parameters j 1 2 k are often called par rial regression coefficients because they describe the partial effect of oue independent tari able when the other independent variables in the model are held constant Multiple linear regression models are often used as approximating functions That is the true functional relationship between y and x X x is unlnoD but over certain ranges of the independent variables the linear regression model is an adequate approximation 437 438 Chapter 15 Multiple Regression Models that are more complex in appearance than equation 152 may often still be ana lyzed by multiple linear regression techniques For example consider the cubic polynomial I model in one independent variable i y Po Px pzX f3i e If we let x x A1 r andxJ 3 then equation 153 can be written y Po PIXI 3x 3x e 153 154 which is a multiple linear regression model wilh three regressor variables Models that include interaction effects may also be analyzed by multiple linear regressiOTIlllethods For example suppose that the model is y Po PIX f3x Pixlx s If we let x XIx and p Pll then equation 155 can be written y Po Pxl f3x 3x e 155 156 which is a linear regression model In general any regression model that is linear in the paramJJrers 111e 3s is linear regression model regardless of the shape of the surface 111t it generates 152 ESTIMATION OF THE P4RAMETERS The method of least squares may be used to estimate the regression coefficients in equation 152 Suppose that n kobservations are available and let Xlj denote the ith observation Or level of variable x The data will appear as in Table 151 We ssume that the error term 8 in the model bas EE 0 Vs 0 and 111at the if are uncorrelted random variables We may write the model equation 152 in terms of the observations Yi 10 f31xil fllxf2 flkxik ej k J30 LJ3j xJj ei rl The leastsquares function is Table 15 1 Data for Multiple Linear Regression y Xu V X x x il2 lL 157 158 J 152 Estimation of the Parameters 439 The function L is to be minimized with respect to A 3 31 The leastsquares estiroa tors of 30 3 31 must satisfy 59 and j 12 k 159b Simplifying equation 159 we obtain the leastsquares nonnal equations r n n n3o 3IXil h2i2 flLXik Ly il i ll jl 2 ffi Lx 31 L Xi ffi2L XilX 3k2 xilxi LXilYi v 1510 rl il 11 i i1 Note that there are p k 1 normal equations one for each of the unknown regression coef ficients The solution to the norrnaI equations will be the leastsquares estimators of the regression coefficients 10 Jll Jr 1ti5 simpler to solve the normal equations if they are expresseiin matrix notation We now give a matrix development of the normal equations that parallels the development of equation 1510 The model in terms of the observations equation 157 may be written in matrix notation yXIHe where 1 x11 x 1 y J xl X21 22 Xu X 2 x 01 and 1 1 In general y is an n X 1 vector of the observations X is an n xp matrix of the levels of the independent variahles II is a p x I vector of the regression coefficients and s is ail n X I vector of random errors We wish to fud the vector of leastsquares estimators that clitlimizes L LSY eeyX3yXII il 440 Chaprer 15 Mcltiple Regression Note that L may be expressed as L yy IlXy yX IlXX yy 21lXy IlXXf3 1511 since IfXy is a 1 x 1 matrix hence a scalar and its transpose IfXy yX is the same scalar The leastsquares estimators must satisfy aLI 2Xy2XXO a which simpliftes to XX Xy 1512 Equations 1512 are the leastsquares normal equations They are identical to equations 1510 To solve the normal equations multiply both sides of equation 1512 by the inverse of XX Thus the leastsquares estimator of is p XXtXy 1513 It is easy to see that the matrix form of the normal equations is identical to the scalar form Writing out equation 1512 in detail we obtain n n cpo l n n li LX2 LXi LV ie tl 1 ttil n n n Lxa LxA LXiXn LXnXik A 1 LXllY jl 11 l1 11 1 n I n LX LXuxil L Xik Xl2 LX LXikYi L i 11 il PkJ 11 If the indicated matrix multiplication is performed the scalar form of the normal equations that is equation 1510 will result In this fonn it is easy to see that XX is a p X p sym metric matrix and Xy is a p x 1 column vector Note the special structure of the XX matrix The diagonal elements of XX are the sums of squares of the elements in the columns QfX and the offdiagonal elements are the sums of cross products of the elements in the columns ofX Furthermore note that the elements ofXfy are the sums of cross prod ucts afthe columns of X and the observations yJ The fitted regression model is YXp 1514 In scalar notation the fitted model is k y Po LPj j1 The difference between the observationy and the fitted valuey is a residual say e Yt Yi The n xl vector of residuals is denoted oyy ISIS 152 Estimation of the Parameters 441 iul article in the Journal of Agricultural Engineering and Research 2001 p 275 describes the use of a regression model to relate the damage susceptibility of peaches to the height at which they are dropped drop height measured in rom Cd the density of the peach measured in glcmJ One gocl of the anaysis is to provide a predictive model for peach damage to serve as a guideline for harvest ing and postharvesting operations Data typical of this type of experiment is given in 1able 152 We will fit the multiple linear rege sion model y io pjx1 f3J to these data The X matrx and y vector for this model are The XX matrix is and the X y vector is I 3037 O90i 362 3667 U14 727 3368 101 266 1 3045 095 153 1 3468 098 491 il 6000 104 1036 1 3690 096 526 1 4180 100 609 1 2690 101 657 1 3230 094 424 x 101 y 5622 so4 2842 097 346 5586 103 850 1 4150 101 934 I 3495 104 555 1 4628 102 811 1 3331 105 732 5021 110 1258 3114 091 015 3514 096J 523 J l 3037 oro XX307 3667 104 3667 090 104 096 c 3514 096 20 77678 1993 J 77678 3201646 7791878 1993 7791878 199077 I I Xy 3037 3667 090 104 1 JCl 3 62 r 12079 c 3514 77 5112917J 096 5 12270 25 442 Chaplel15 Multiple Regression Table 152 Peach Darrage Data for Example 151 Observation Damage rom Drop Height mm Fruit Density gIcm Number 362 3037 090 2 727 3667 104 3 266 3368 101 4 153 3045 095 5 491 3468 098 6 1036 6000 UJ4 7 526 3690 096 8 609 4180 100 9 657 2690 101 10 424 3230 094 11 804 5622 101 12 346 2842 097 13 850 5586 103 14 934 4150 l01 15 555 3495 104 16 811 4628 102 17 732 3331 105 18 1258 5021 110 19 015 3114 091 20 S23 3514 096 The leatsquares estimators are found from equation 1513 to be XXtXy or Ol 20 lj 77678 1993 77678 3201646 7791878 1993 TC 12079 7791878J lS1l2917 199077 12270 r 2463666 l 0005321 2674679 33831 001314 34890 0005321 00000077 008353 2674679C 12079 008353 rl 5112917J 30096389 j 12270 Therefore the fitted regression model is y 33831 O01314x 34890x Table 153 shows the fitted values of and the residuals The fitted values andresidua1s are calculated to the same accuracy as the original data 1 J l 152 Estimation of the Parameters 443 Table 153 Observations Fitted Values and ResidJa1s for Example 151 Observation Number y 1 362 156 206 2 717 727 000 3 266 583 311 4 153 331 l7S 5 491 492 001 6 1036 1034 002 7 526 451 075 8 609 655 046 9 651 494 163 10 424 321 103 11 804 879 075 12 346 375 029 13 850 944 094 14 934 686 248 15 555 705 150 16 811 784 027 17 132 718 014 18 1258 1114 144 19 015 201 186 20 523 428 095 The statistical properties of the leastRsquares estimator may be easily demonstrated Consider first bias Eii E xXtXy E xXXxP Ell E XxrxXl3 XXYX EJ I since Ee 0 and XXYXX 1 Thus is an unbiased estimator of fl The variance property of is expressed by the covariance matrix Cov E E E The covariance matrix of is a p xp symmetric matrix whosejjth element is the variance of and hose ijth element is the covariance between i and A The covariance matrix of I is Cov crXX It is usually necessary to estimate fil To develop this estimator consider the sum of squares of the residuals say 2A 1 ee 444 Chapter 15 Multiple Regression Substituting eyy y X we bave SS y Xy X yy PXy yX XX yy2Xy XXp Since XXP Xy this last equation becomes SS yy PXy 1516 Equation 1516 is called the error or residual sum of squares and it bas n p degrees of freedom associated with it The mean square for errOr is MS np 1517 It can be shown that the expected value of MS is 02 thus an unbiased estimator of fl is given by MS 1518 tpif5 We Vill estima tle error vaciance f for the multiple regression problem in Exanp1e 151 Using the dara in Table 152 we find and 20 yy I0 90460 11 12079 1 XY33831 001314 34890J 5112917J 12270 86639 Iberefore the ector sum of squares is The estimate of IT is SSe yy XY 90460 86639 382 2 SSe 3821 2 04 i n p 203 153 CONFIDENCE INTERVALS IN M1LTIPLE LThEAR REGRESSION It is often necessary to construct confidence intervcl estimates for the regression coefficients fl The development of a procedure for obtaining these confidence intervals requires that w assume the errors o to be normally and independently distributed with mean zero and variance cr Therefore the observations Y are normally and independently distributed 153 Confidence Intervals in Multiple Linear Regresskn 445 witb mean fio r 1 f1Xi1 and variance rr Since the leastsquares estimator lis a linear com bination of the observations it follows that Ii is nonnally distributed with mean vector II and covariance matrix uXXt Then each of the qUlllltities P j Pj 12 lCf ejj j Ol k 1519 is distributed as twith n p degrees oifreedom where CJJ is tbeith element oitbe XXt matrix and if is the estimate of the error variance obtained from equation 1513 Therefore a 1001 a confidence interval for the regression coefficient Py j 0 1 k is 1520 We will construct a 95 confidence interval on the parameter f31 in Example 151 Note thathe point estinate of PL is fil 001314 and me diagonal element of XX corresponding to fJl is en 00000077 The estimate of f was obtained in Example 152 as 247 and tilo11 2110 Therefo the 95 confu1ence interval on 3 is computed from equation 1520 as O0131421l02247ooOOO077 3 i 0013142 0224700000077 which reduces to 000436 3 00219 We may also obtain a confidence interval on the mean response at a particular point say X01 Xw xoJ To estimate the mean response at this point define the vector 1 The estimated mean response at this pomt is Yox 1521 This estimatoris unbiased since EGo Ex x11 EVe and the variance afY is 1522 Therefore a 1001 a confidence interval on the mean response at the point x x tJtis Equation 1523 is a confidence interval about the regreSSion hyperplane It is tbe multiple regression generalization of equation 1433 446 Chaptr 15 Mtiple Regression The scientists conducting the experiment on damaged peactes in Example 151 would like to OOn struct a 95 confidence interval on the mean damage for a peach dropped froma height of Xl 325 if its density is Xi 098 glcm Therefore The estimated mean response at this point is found from equation 1521 to be The variance of Yo is estimated by 33831 325 098 001314 463 34890 OxXXrxc 22471 325 2463666 0981 000321 c2614679 224700718 01613 0005321 00000077 l008353 l008353 325 2674679 1 30096389lo98 Therefore a 95 confidence interval On the me3l1 damage at this point is found from equation 1523 to be which reGuces to 378 S Ey 548 154 PREDICTION OF EW OBSERVATIONS The regression model can be used to predict future observations on y corresponding to par ticular alues of the independent variables say Acl xC XQI If 1 XI XC Xct then a point estimate of the future observation Yo at the point xm Xoo Xlk is 1524 A 1001 a prediction interval for this future observation is A ClI Xl YCtctj2I1p1cJ 0 A Xc 1525 c XXtC sYo Yo t12pd Ixc 0i TIlls prediction interval is a generalization of the prediction inteIla for a future observation in simple linearregressio equation 1435 In predicting new observations and in estimating the mean response at a given point xc Xw XOk one must be careful about extrapolating beyond the region containing the original observations It is very possible that a model that fits well in the region of the orig inal data will no longer fit well outside that region In multiple regresion it is oiren easy to inadvertently extrapolate since the levels of the variables XI1 xa Xi i 1 2 n jointly define the region comaining the data As an example consider Fig 151 which illus l I 155 Hyporbesis Testing in Multiple Linear Regmision 447 XOj L Onginal rangs for x1 Figure 151 An example 0 extrapolation in multiple regression trates the region containing the observations for a twovariable regression model Note that the point tol xO lies within the ranges of both independent variables Xl and but it is out side the region of the original observations Thus either predicting the value of a new observation or estimating the mean response at this point is an extrapolation of the original regression modeL Suppose mat the scientists in Example lSffl wish to construct a 95 prediction interval on the dam age on a peach that is dropped from a heignt of XI 325 IIlIl1 and has a density ofs 098 glcmJ Note that x 1325 098 and the point estimate of the damageisYn x 463 rom Also inEample 154 we calculated xXXrlx 00718 Therefore from equation 1525 we have 463 21l022471 00718 5 Y 463 211022471 00718 and the 95 prediction interval is 155 HYPOTHESIS TESTING IN MULTIPLE LINEAR REGRESSION In multiple linear regression problems certain tests of hypotheses abeut the model param eters are useful in measuring model adequacy In this section we describe several impor tan hypothesistesting procedures We continue to require the nonnality assumption on the errors which was introduced in the prevlous section 1551 Test for Significance of Regression The test for significance of regression is a test to determine whether there is a linear rela tionship between the dependent variable y and a subset of the independent variables XI hzl f xt The appropriate hypotheses are 448 Chapter 15 Multiple Regression Hl 3 30 H f3 for at least onej 1526 Rejection of He 3 0 implies that at least one of the independent variables XI x xk contributes significantly to the model The test procedure is a generalization of the proce dure used in simple linear regression The total sum of squares SfY is partitioned into a sum of squares due to regression and a sum of squares due to error say SyySSSS and if H 3 0 is rue then S510 xi where the number of degrees of freedom for the x is equal to the number of regressor variables in the model Also we can show that S510 X and SSe and SS are independent The test procedure for H J3 a is to compute E SSlk MSR e SSEnkl MSE 1527 and to reject Ho if FI F aklIiI The procedure is usually summarized in an analysis of vari ance table such as Table 154 A computational formula for SS may be found easily We have derived a computa tion formula for SSE in equation 1516 that is SSEyy Xy Now since SyJ ly Olyy1n yy CZlylln we may revrite the foregoing equation or Therefore the regression sum of squares is 1528 the error sum of squares is 1529 and the total surn of sqWeS is 1530 155 Hypothesis Testing in Multiple Lirear Regrssion 449 Table 154 Analysis of Variance for Significance of RegreSSIon in MultipJe Regression Source of Sillll of Degrees of Mean Variation Scum Freedom Square Regression SS k MS Error or residual SSE nkl MS Total S n j We vill test for significance of regression using the damaged peaches data from Example 15 L Some of the numerical quantities required are calculated in Example J52 Note that I r fi 2 Syyyy Ly n i 90460 12079 20 175089 SSRXY r nl 86639 3688 2079 20 SS Syy 5S yYXY 3821 The analysis of variance is shown in Table 155 To testRJ f31 A 0 we calculae the statistic Po MS 6844 3046 MS 2247 Since Fo FoIJ1 359 peach damage is related to drop height froitdensity or both However we note that this does not necessarily imply that the relationship found is an apprOpriate oae for predict ing damage as a function of drop height or fruit density Further tests of model adequacy are required Table 155 Test for Significance of Regression for Example 156 Source of Sumo Degrees of Mean Variation Squares Freedom Square F Regression 13688 2 684 3046 EIror 3821 17 2247 Total 17509 19 450 Chapter 15 Multiple Regression 1552 Tests on lndividual Regression Coefficients We are frequently interested in testing hypotheses on the individual regression coefficients Such tests would be useful 10 determining the value of each of the independent variables in the regression model For example the model might be more effective with the inclusion of additional variables or perhaps with the deletion of one or lUore of the variables already in the modeL Adding a variable to a regression model always causes the sum of squares for regres sion to increase and the error sum of squares to decrease We must decide whether the increase in the regression sum of squares is sufficient to warrant using the additional vari able in the model Furthermore adding an unimPOrtant variable to the model can actually increase the mean square error thereby decreasing the usefulness of the model The hypotheses for testing the significance of any individual regression coefficient say f3i are Ho f3j 0 HlO 1531 If H f3j 0 i not rejected then this indicates that xJ can possibly be deleted from the model The test statistic for this hypothesis is p to ac 1532 where ejj is the diagonal element of Xxyt corresponding to A The null hypothesis He 0 is rejected if Itol 1 Note that this is really a partial or marginal test because the regression coefficient p depends on all the other regressor variables xli that are in the modeL To illustrate the se of this test consider the data in Example 151 and suppose that we want to test He f3 0 H 3 O The main diagonal element of XXt corresponding to jj is en 30096 so the r statistic in equation 1532 is 3489 424 224730096 Since to 22 2110 we reject Ho 3 0 and conclude that the variable x density con tributes sigiuficantly to the modeL Note that this test measures the marginal or partial con tribution of X1 given that Xl is in the model We may also examine the contribution to the regression sum of squares of a variable say x given that other variables x i J are included in the model The procedure used to do this is called the general regression significance test or the extra sum of squares method This procedure can also be used to investigate the contribution of a subset of the regressor variables to the model Consider the regression model with k regressor variables yXll whereyis nX I Xis nxp II is px I is nx I andp k 1 We would like to determine whether the subset of regressor variables Xt Xl xr r k contributes J 155 Hypoesis Testing in Multipe Lineru Regression 451 significantly to the regression model Let the vector of regression coefficients be pa1itioced as follows where P is rx I and 2 is P r X IJ We wisb to test the hypotheses Ho Pl 0 H p cO The model may be VIitten l533 1534 where Xl represents the columns of X associated with PI and represents the columns of X associated itb FortbeftII1110del including both 13 and we know that XXrXy Also the regression sum of squares for all varables including the intercept is p degrees of freedom and yy Xy MS n p SSI3 is called the regression sum of squares due to 13 To find the contribution of the terms in 13 to the regression fit the model assuming the null bypothesis Ho 13 0 to be true Tbe reduced model is found from equation 1534 to be y X2 e The leastsquares estimator of is Xxr and S5J Xy p rdegrees of freedom The egression sum of squares due to 13 given that az is already in the model is SSI3 SSR SSl 1535 1536 l537 This sum of sq1lZlIes has r degrees of freedom It is sometimes called the extra sum of squares due to Note tblIt S5 1 is the increase in the regression sum of squares due to including the variables x x x in the model 1ow S513 113 is independent of MS E and the null hypothesis 0 may be tested by the statistic sSRi3d2lr 1533 If Fo Fa f p we reject HCr concluding that at least one of the parameters in 11 is not zero and consequently at least one of the variables XI Xz x in Xj contributes significantly to the regression model Some authors call the test in equation 1533 a partial Ftest The partial Ftest is very useful We can use it to measure the contribution of xJ as if it were the last varable added to the model by computing SSiflip p PiI 5 pJ 452 Chapter 15 Multiple Regression This is the inerease in the regression sum of squares due to adding X to a model that already includes XI XjI H xk Note that the partial Ftest on a single variable Xj is equiva lent to the ttest in equation 1532 However the partial Ftes is a more general procedure in that we can measure the effect of sets of variables In Section 1511 we will show how the partial Ftest plays a major role in model building that is in searebing for the best set of regressor variables to use in the model Consider the damaged peaches data in Example 151 We will investigate the contribution of the vari able x density to tte model That is we wish to test He fl 0 H 3 0 To test this hypothesis we need the extra sum of squares due to 3 or SS3I3 flc SScf3 fl f3J SS3 I30l SSJ31 f3I3J sS3lf3 In Example 156 we calcJlated and if the model y flo f3 XI e is fit we have SSfllfJol fjs 962 Theefore we have sS3I3 flu 13688 9621 4067 2 degrees of freedom 1 degee of freedom 1 degree of freedom This is the increase in the regression sum of squares attributable to adtfng x to a model aJJeady con taining XI To testHo fl1 0 onn the test statistic SSRfllflflc1 4067 1810 MSE 2247 Note that theMS from thefull model usig both x ard x is used in the denominator of the test sta tsuc Since F CdI7 445 we reject Hi 132 0 and conclude bat density x2 contributes significantly to the model Since this partial Ftes involves a single variable it is equivalent to the Mest To see this recall that the ttest on Ho B resulted in the tes statistic tc 424 Furtbcrmore recall that he square of a trmdom variable with v degrees of freedom is an Frandom variable with one and v degrees of freedom and we note that 424 1798 FfJ 156 MEASURES OF MODEL ADEQUACY A number of techniques can be used to measure the adequacy of a multiple regression modeL TItis section will present several of these techniques Model validation is an impor 156 Measures of Model Adequacy 453 tant part of the multiple regression model building process A good paper on this sUbject is Snee 1977 see also Montgomery Peck and VIning 2001 1561 The Coefficient of Multiple Detennination The coefficient of multiple determination R2 is defined as 1539 R2 is a measure of the amount of reduction in the variability of y obtained by using the regressor variables xis xk As in the simple linear regression case we must have a s R2s 1 However as before a large value of Rl does not necessarily imply that the regres sion model is a good one Adding a variable to the model will always increase R2 regard less of whether the additional variable is statistically significant or not Thus it is possible for models that have large values of If to yield poor predictions of new observations or esti mates of the mean response The positive square root of R2 is the multiple correlation coefficient between y and the set of regressor variables xl s xk That is R is a measure of the linear association between y andx Is x k When k 1 this becomes the simple correlation between y and x The coefficient of multiple determination for the regression model estimated in Example 15 1 is R SSR 13688 0782 S 17509 That is about 782 of the variability in damage y is explained when the two regressor variables drop height XL and fruit density s are used The model relating density to Xl ouly was developed The value of R2 for this model turns out to be Rl 0549 Therefore adding the variable X to the model has increased R2 from 0549 to 0782 AdjustedR Some practitioners prefer to use the adjusted coefficient amultiple determination adjusted R2 defined as 1540 The value Syn 1 will be constant regardless of the number of variables in the model SSE1n p is the mean square for error which will change with the addition or removal of tenns new regressor variables interaction terms higherorder terms from the model Therefore R dj will increase only if the addition of a new term significantly reduces the mean square for error In other words the Rdj will penalize adding terms to the model that are not significant in modeling the response Interpretation of the adjusted coefficient of multiple detemrination is identical to that of R2 454 Chapter 15 Multiple Regression We can calculate Kj for the model fit in Example 151 Prom Example 156 we found that SSE 3821 and Srr 17509 The estimate R is then I 3821203 1 2247 1750920 I 9215 0756 The adjusted R will playa significant role in variable selection and model building later in this chapter 1562 Residual Analysis The residuals from the estimated multiple regression model defined by yY play an important role in judging mode adequacy JUSt as they do in simple linear regression 5 Doted in Section 1451 there are several residual plots that are often useful These are illus trated in Example 15 9 It is also helpful to plot the residuals against variables nIlt presently in the model that are possible candidates for mc1usion Patterns in these plots similar to those in Fig 145 indicate that the model may be improved by adding the candidate variable The residtaL for the nodel estimated in Example 151 are shownin Table 153 These residuals are plotted 001 a normal probability plot in Fig 152 No severe deviatiollS from rocrnaliry are obvious although the smallest residual e 311 does not fall near the remaining residuals The standard ized residual 3l7j2247 211 appears to be large and could indicate an unUSta1 observation The residuals are plotted against y in Fig 153 and againstxj andA1 in Figs 154 and 55 respec tivey In Fig 154 there is some i1dication that the assumption of constant variance may not be sat isfied Removal ofthe unusual observation may improve the model fit but there is no indication of error in data colleetion Therefore the point kill be retained We will see subsequently Example 15 16 iliatVo other regressor variables are required to adequately model these data 2 1 0 0 m ro 0 E 1 I 2 2 1 0 2 3 3 Figure 152 Normal probability plot of residuals for Eample 1510 r 156 easures of Model Adequacy 455 3 i 2 j i o I l 2 7 12 Fitted value Figure 153 Plot of residuals againsty for Exanple 1510 3 2 i 2 r 300 400 500 600 x Figure 154 Plot of residuals against x or Example 1510 Figure J55 Plot of residuals against for Example 15 I 0 456 Chapter 15 Multiple Regression 157 POLYNOMIAL REGRESSION The linear model y X e is a general model that ean be used to fit any relationship that is linear in the unknown parameters This includes the important elass of polynomial regression models For example the seconddegree polynomial in one variable y 3 3x 3 S and the seconddegree polynomial in two variables y 3 3x 3x 3 x f3lX 3x s are linear regression models 1541 1542 Polynomial regression models are widely used in cases where the response is curvilin ear because the general principles of multiple regression can be applied The following example illustrates some of the types of analyses that can be performed Eplsii Sidewall panels for the interior of an airplane are formed il a 1500ton press The unit manufactur ing cost varies 1th the production lot size The data shown below give he average cost per unit in hundreds of dollars for this product y and he production lot size x The scatter diagram shown in F1g 156 indicates that a secondorder polynomiul may be appropriate y LSI 170 65 55 lAS lAO 130 126 124 121 120 118 x 20 25 30 35 40 50 60 65 70 75 80 90 We will fit fue model y f30 3x 3r 90 180 170 i50 0 150 0 u 140 m 130 120 110 100 I 20 30 40 50 60 70 8 90 Lot size x Figure 156 Dta for Example 1511 f 1 I 157 Polyromial Regression 457 The y vector X matrix and vector are as follows j811 170 165 155 L48 140 y 130 126 L24 121 x 1 20 400 25 625 I 30 900 I 35 1225 1 40 1600 1 50 2500 60 3600 I 65 4225 70 4900 1 75 5625 120 80 6400 cLl8j 1 90 8100 Solving the normal equations XXtl X y gives the fitted model 21983 00225 00001251 The test fosignificance ofegrcssioIl is shoVtO in Table 56 Since Fo 217107 is significant at 1 we conclude iliat at least Oe of the parameters fJ and 311 is not zero Furthermore the sta1dad tests for model adequacy reveal no unusucl behavior In fitting polynomials we generally like to use the lowestdegree model consistent with the data In this example it would seem logical to investigate dropping the quadratic term from the modeL That is we would like to test H 13 0 H 13 O The general regression significance test can be used to test this hypothesis We need to determine the extra sum of squares due to 111 or SSf3ulf3f3J SSRfi f3lilf3 SSf3If3J The sum of squares 5Sf3 f31f3J 05254 from Table 156 To find SSRfilf30 we fit a simple linear regression model to the original data yielding y 19004 00091 It can be easU y verified that the regression sum of squares for this model is 5813113 04942 Table 156 Test for Significance of Regression for the Seconder Model in Example 1511 Source of Sum of Degrees of Mean Variation Squares Freedom Square F Regression 05254 2 02627 217L07 Error 00011 9 0000121 Total 05265 11 458 Chapter 15 Multipe Regression Table 157 Analysis ofVariazce of Example 15 11 Showing the Test for He 31 0 Source of lU of Degree of Mean Variation Squaces Freedom Square Regession SS3 Ji3 05254 2 02627 217107 Lin SS3I3 04942 1 04942 408430 Quadratic SS813 8 00312 I 00312 25785 Error 00011 9 0000121 Thtal 05265 11 Therefore the extra sum of squares due to Pll given that 31 and Pc are in the model is SSfilA 3 SSk3 3113 SSifiJ3 05254 04942 00312 The analysis of variance with the test of H 311 0 incorporated into the procedure is dis played in Table 157 Note that the quedratic term contributes significantly to the model 158 INDICATOR VARIABLES The regression models presented in previous sections have been based on quantitative vari abIes that is variables that are measured on a numerical scale For example variables such as temperature pressure distance and age are quantitative variables Occasionally we need to incorporate qualitative variables in a regression model For example suppose that one of the variables in a regression model is the operator who is associated with each obser arion y Assume that only two operators are involved We may wish to assign different lev els to the two operators to account for the possibility that each operator may have a different effect on the response The usual method of accounting for the different levels of a qualitative variable is by using indicator variables For instance to introduce the effect of two different operators into a regression model we could define an indicator variable as follows x 0 if the observation is from operator 1 x 1 if the observation is from operator 2 In general a qualitative variable with t levels is represented by t 1 indicator variables which are signed values of either 0 or 1 Thus if there were three operators the different levels would be accounted for by two indicator variables defined as follows x X 0 0 if the observation is fmm operator 1 1 0 if the observation is from operator 2 0 1 if the obsertation is from operator 3 Indicator variables are also referred to as dummy variables The following example illus trates some of the uses of indicator variables For other applications see Montgomery PeCk and Vining 200 1 1 158 Indicator Variables 459 Epli2 Adapted from Montgomery Peck and Vmiog 2001 A mechanlcal engineer is investigating the sur face finish of metal parts ptoduced on a lathe and its relationship to the speed in RPM of the lathe The data are shown in Table 158 Note that the data have been collected usiGg two different typCs of cuttilg tools Since it is likely that the type of cutting tool affects tbe surlace fiIrish we will fit the model y 3 fil Ph E where y is the surface finLh XI is the lathe speed in RPM and X is a1 indicator vaahle denoting the type of cttting tool used ilia is o for tool type 302 x 1 for tool type 416 The parameters if ttis model may be easily interpreted If x 0 then the model becomes Y Pc ftjX t e which is a straightline model with slope Pi and intercept f3y However if X 1 then the model becomes y 13 fix 13l E p 3 f3x E which is a straightJine model vith slope PI and intercept f3ij 32 Taus the model y Po f31X f3zXz e implies that surface finish is linearly related to lathe speed and that the slope f31 does Dot depend on the type of cutting tool used However the type of cutting rool does affect rbe intereept and 32 indi cates the ehange in the intercept associated with a change in too type from 302 to 416 Thble158 Swiace Furish Daafor Example 1512 Observation Su face F1nish Type of Cutting NlrIber i y RPM Tool 4544 225 302 2 4203 200 302 3 5010 250 302 4 4875 245 302 5 4192 235 302 6 4779 231 30 7 5226 265 30 8 5052 259 30 9 4558 221 30 0 4478 218 302 II 3350 224 416 12 3L23 212 416 13 3752 248 416 14 373 260 416 15 3470 243 416 16 3392 238 416 11 3213 224 416 18 3547 251 416 19 3349 232 416 20 3229 216 416 460 Chapter 15 Multiple Regression The X matrix and y vector for this problem are as follows 225 0 4544 200 0 4203 1 250 0 5010 1 245 0 4875 1 235 0 4792 1 237 0 4779 1 265 0 5226 259 0 5052 221 0 4553 1 218 0 4478 X 224 1 y 3350 1 1 212 3123 I 248 1 3752 1 260 3713 1 243 370 1 238 3392 1 224 1 3213 251 1 3547 232 1 i 3349 216 d L3229 J The fitted model is 5 142762 O141u 132802x The ataysis of variance for this model is shohIl in Table 159 Note that the hypothesis Eo PI 32 0 of regression is reje1ed This table aw contains he sum of squares SSg SS3 3iPJ ssg3I3 SS3I3 3 S a test of the hypothesis Ho 32 0 can be made This hypothesis is also rejected so we conclude that tool type has an effect on surface finish It is atsopossihle to use indicarorvariables tomvesligate whether tool type affects both slopemui intercept Let the model be Table 159 Analysis of Variance of Example 1512 SouCc of Sumo Degrees af Mean Variation Squares Freedom Square Regression 10120595 2 5060297 110369 SSflIfJ 1306091 1 1306091 28487 55 3IfJ 3o 8814504 1 8814504 192252 Error 77943 7 04508 Total 10198538 19 Significant at 1 cJ 159 The Correlation Matrix 461 wheres is the inmcam variable Now if too type 302 is useds O and the model is y Jxe lftool rype 416 is used s 1 and the model becomes y Jo J A f3x e fJ fiJ 3 3JX S Note that A is the change in the intercept and A is the change in slope produced by a change in tool type Another method of analyzing these data set is to fit separate regression models to the data for each tool type However the indicator variable approach has several advantages rmt only one regression model must be estimated Second by pooling the data on both 001 types more degrees of freedom for error are obtained Thlrd tests of both hypotheses on the parameters 13 end 33 ae just special cases of the general regression signific3IWe test 159 THE CORRELATION llATRIX Suppose we wish to estimate the parameters in the model i 12 n We may rewrite this model with a transformed intercept f3 as Y 3 3x x Ax z or since A y yy 3X x Ax cr The xx matrix for this model is where Skj IxikxxgXj kj 12 il It is possible to express this XX matrix in correlation fo Let Tk I 12 I SS Ii JJ kj12 1543 1544 1545 1546 1547 1548 and note that r r 1 Then the correlation form of the XX matrix equation 1546 is RI12 r21 1549 The quantity r t2 is the sample correlation betveenx and We may also define the sam ple correlation between Aj and y as j12 1550 462 Chapter 15 Multiple Regression where Sj ixJ XIYy jI2 1551 1 is the corrected sum of cross products between Xj and y and SIJ is the usual total corrected sum of squares of y These transfurnations result in a new regression model where Yy Yi Slf2 yy x XI z11 12 B j 12 1552 The relationship between the parameters b and b2 in the new model equation 1552 and the paramerers J JI and A in the original model equation 1543 is as follows 1553 J 1554 1555 The leastsquares normal equations for the transformed model equation 1552 are 1556 The solution to equation 1556 is or 1557a rZy lizll 11 J55ib The regression coefficients equations 1557 are usually called staruiardized regression coefficients Many multiple regression computer programs use this transfornation to reduce roundoff errors in the XXl mati These roundoff errors may be very serious if the l 159 The Correlation Matrix 463 original variables differ considerably in magnitude Some of these computer programs also display both the original regression coefficients and the standardized coefficients The standardized regression coefficients are dimensionless and this may make it easier to com pare regression coefficients in situations where the original variables Xj differ considerably in their units of measuremenL In interpreting these standardized regression coefficients hovvever we must remember that they aoe still partial regression coefficients ie b s1ows the effect of z glven iliat other Zi i are in the model Furhermore the hJ are attected by the spacing of the levels of the x Consequently we should not use the magnitude of the bj as a measure of the importance of the regressor variables While we have explicitly treated only the case of two regressor variables the results generaIize If there are kregressor variables XI xk one may vrrite the XX matrix in correlation form r 1 12 13 Jk 12 rn r Rr13 TZ3 1 1558 1 Lik r2k r wtere rj SSSlLf2 is dIe sample correlation between x and x and Sij IXi Xi XI x The correlations between Xj and y are ly g r2j 1559 lrky j where Y 1 x xy Y The vector of standardized regression coefficients b h b b s 0 R1g 1560 The relationship between the standardized regression coefficients and the original regres sion coefficients is lil131 For the data in Example 151 we find Therefore S 175089 SIV4215372 S 233 j 0 12 k S 18471016 S 0047755 Sll 512873 512873 18471O16 0047755 05460 1561 464 Chapter 15 Multiple Regression 425372 118471O166i089 07412 S 233 r2v ljl l 08060 SS JrO047755175089 f and the correlation matrix for this problem is 054601 I J From equation 1556 the normal equations in terms of the standardized regression coefficients are I 1 05460lt07412 c 05460 1 L 08060 Consequently Le standardized regression coefficients are rJ 1 05460074121 bJ L 05460 1 L 08060 1424737 77791J07412 j77791 1424737 08060 0429022 0571754 These standardized regression coefficients could also have been computed directly from elfuer equation 1557 or equation 1561 Note that alfuough b b we should be cautious about ooncluding that the fruit density X is more important fuan drop height x smce h and b2 are still partial regression coefficients 1510 PROBLEMS IN MULTIPLE REGRESSION There are a number of problems often eneountered in the use of multiple regression In this section we briefly discuss three of these problem areas the effect of multicollinearity on the regression model the effect of outlying points in the xspace on the regression coeffi cients and autocorrelation in the errors 15101 Multicollinearity In most multiple regression problems the independent or regressor variables Xi are inter correlated In situations whieh this intercorrelation is very large we say that multi collinearity exists lv1ulticollinearity ean have serious effects on the estimates of the regression coefficients and on fue genernl applicability of fue estimated model The effects of multicollinearity may be easily demonstrated Consider a regression model with two regressor variables Xl and x and suppose thatxi and have been stan dardized as in Section 159 so that the XX matrix is in correlation form as in equation 1549 The model is J r 1510 Problems in Multiple Regression 465 The XXr matrix for this model is Il CXXf 2 Ir and the estimators of the parameters are rI rall ljlfl J P iY12X2Y 2 1r12 XY xy 3 2 12 ll where T2 is the sample correlation between x andtj and xy and xy are the elements of the Xy vector Now if multicollilearity is presen x and are highly correlated and If1 1 In such a situation the variances and covariances of the regression coefficients become very large since vli ejF as If1 1 and COy A Ii Cd depending on whether r12 1 The large variances for A imply that the regresion coefficient are very poorly estimated Note that the effect of multicollinearity is to introduce a neat linear dependency in the columns ufthe X matrix As r 1 this linear dependency becomes exact Furthermore if we assume that xy x as Irnl 1 then the estimates of the regression coefficients become equal in magnitude btll opposite in sign that is Ii A regardless of the true values of p and p Similar problems occur when multicollinearity Lt present and there are more than tvo regressor variables In general the diagonal elements of the matrix C XXJ can be written j 12 k 1562 where RJ is the coefficient of multiple determination resulting from regressing Xi on the other k 1 regressor variables Clearly the stronger the linear dependency of Xj on the remaining regressor vatiables and hence the stronger the multicollinearity the larger the value of If will be We say that the variance of Ii is inflated by the quantity 1 Rr Consequently we usually call j 12 k 1563 the variance inflation factor for j ote that these factors are the main diagonal elements of the inverse of the correlation matrix They are an important measure of the extent to which multicollinearity is present Although the estimates of the regression coefficients are very imprecise when multi collinearity is present the estimated equation may still be useful For example suppose we v1sh to predict new observations If these predictioQi are required in the region of the xspace where the multicollinearity is in effect then often satisfactory results will be obtained because while individual f3jmay be poorly estimated the function Lf3 may be estimated quite well On the other band if the prediction of new observations requires extrapolation then generally we would expect to obtain poor results Exlrdpolation usually requires good estimates of the individual model parameters 466 Chapter 15 Multiple Regressio Multicollinearity arises for several reasons It will occur when the analyst collects the data such that constraint of the form L ojX 0 holds among the colnnms of the X matrix the a are constants not all zero For example if four regressor variables are the components f a mixture then such a constraint will always exist because the sum of the cOClponents is always constant Usually these constraints do not hold exactly and the analyst does not know that they exist There are several ways to detect the presence of multicollinearity Some of the more important of these are briefly discussed 1 The variance ioilation factors defined in equation 1563 are very useful measures of multicollinearity The larger the variance inflation factor the more severe the multicollinearity Some authors have suggested that if any variance inflation factors exceed 10j then multicollinearity is a problem Other authors CQusiderthis value too liberal and suggest that the variance inflation factors should not exceed 4 or S 2 The determinant of the correlation matrix may also be used as a measure of multi collinearity The value of this determinant can range between 0 and 1 When the value of the determinant is I the columns of the X matrix are orthogonal ie there is no intercorre1ation between the regression variables and when the value is O there is an exact linear dependency among the columns of X The smaller the value of the determinant the greater the degree of multicollinearity 3 The eigenvalues or characteristic roots of the correla ion matrix provide a measure of multicollinearity If XX is in correlation form then the eigenvalues of XX are the roots of the equation IXX All o One or more eigenvalues near zero implies that multiCOllinearity is present If Am and AmllC denote the largest and smallest eigenvalues of XX then the ratio AmruAmin can also be used as a measure of multicollinearity The larger the value of this ratio the greater the degree of multicollinearity Generally if the ratio VAm is less than 10 there is little problem with multicollinearity 4 SOrletimes inspection of the individual elements of the correlation matrix can be helpful in detecting multicollinearity If an element h1 is close to 1 then Xi and X may be strongly multicollinear However when more than tvlO regressor vari ables are involved in a multicollinear fashion the individual Tlj are not necessarily large Thus this method will not always enable us to detect the presence of multicollinearity 5 f the Ftest for significance of regression is significant but testt on the individual regression coefficients are not significant then multicollicearity may be present Several remedial measures have been prpJsed for resolving the problem of multi collinearity Augmenting the data with new observations specmcally designed to break up the approximate linear dependencies that currently exist is often suggested However some times this is impossible for economic reasons or because of the physical constraints that relate the X Another possibility is to delete certain variables from the model This suffers from the disadvantage that one must discard the information contained in the deleted variables Since multicollinearity primarily affects the stability of the regression coefficients it would seem that estimating these parameters by some method that is less sensitive to mul tieollinearity than ordinary least squares would be helpful Several methods bave beeD sug gested for this Hoed and Kennard 1970 b bave proposed ridge regression as an 15 0 Problems in Multiple Regression 467 alternative to ordinary least squares In ridge regression the parameter estimates are obtained by solving 111 XX IfrXy 1564 where I 0 is a constant Generally mues of 1 in the interval 0 s I I ae appropriate The ridge estimator 111 is not an unbiased estimator of I as is the ordinary leastsquares esti mator but the mean square error of 111 will be smaller than the mean square error of Thus ridge regression seeks to find a set of regression coefficients that is more stablelt in the sense of having a small mean square error Since mUlticollinearity usually results in ordinary leastsquares estimators that may have extremely large variances ridge regression is suitable for situations where the multicollinearity problem exists To obtain the ridge regression estimator from equation 1564 one must specify a value for the constant I Of course there is an optimum I for any problem but the simplest approach is to solve equation 1564 for several values of I in the inrenlll 0 s I L Then a plot of the values of Ill against 1 is constructed This display is called the ridge trace The appropriate value of 1 is chosen subjectively by inspection of the ridge trace TypicallYI a value for I is chosen such that relatively stable parameter estimates are obtained In generall the variance of 111 is a decreasing function of I while the squared bias 1lIlIJ is an increasing function of l Choosing the value of involves trading off these two properties of 131 A good discussion of the practical use of ridge regression is in Marquardt and Snee 1975 Also there are several other biased estimation techniques that have been proposed for dealing with multicollinearity Several of these are discussed in Montgomery Peck and Vrning 2001 itmp1514 Based on an example in Rald 1952 The heat geoerated in calories per gxa for a particular type of cement as a function of the qua1tities of foll additives ZI z z andz is shown in Table 1510 We wish to fit a nultiple linear regreslon model to these daa Tabl1510 Data or Example 1514 Observation Number y z Z z 1 2825 10 31 5 45 2 2480 12 35 5 52 3 1186 5 15 3 24 4 3660 17 42 9 65 5 1580 8 6 5 19 6 1623 6 17 3 25 7 2950 12 36 6 55 8 2875 10 34 5 50 9 4320 18 40 10 70 10 3847 o 50 10 80 II 1014 16 37 5 61 12 3892 20 40 II 70 13 3670 15 45 8 68 14 1531 7 22 2 30 15 840 9 12 3 24 468 Chapler 15 Multiple Regression The data Will be coded by defining a new set of regressor variables as iI2 15 i 1234 where Sjj rt Zij i is the corrected sum of squares of the levels of The coded data are shown in Table 1511 This tranfonnation makes the intercept orthogonal to the or regression coeffi cients since thefirst column of the X matrix consists of ones Therefore the intercept in this model will always be estimated by y The 4 x 4 XX matrix for the foUl coded variables is the correlation marix 100000 084894 091412 0933671 084894 100000 076899 Qg7567 XX 091412 076899 100000 086784 093367 097567 086784 100000 This matrix contains several large correlation coefficients and this may indicate significant mu1 dcolli1earity The inverse of XX is r 20769 XXrl 25813 0608 l44042 25813 74486 12597 107710 0608 440421 12597 107710 8274 18903 18903 163620 The ariance inflation factors are the main diagonal clements of this matrix Note Jlat three of te variance inflation factors exceed 10 a good indication that multicollinearity is present The eigenval ues of XX are 1 3657 i 02679 A 007127 and 4 0004014 Two of the eigenvaluest and A4 are re1atively close to zero Also the ratio of the largest to the smallest eigervalue is 365291106 Amill 0004014 which is considerably larger than 10 Therefore since examination of the variance inflation factors and the eigenvalues indicates otential problems with multicollinearity we will use ridge regression 0 estiroate the model parameters Tblelsn Coded Data for Example 1514 Observdtion Number y x X X x 2825 J12515 000405 JD9206 J05538 2 2480 J02635 008495 J09206 003692 3 1186 037217 J31957 027617 J33226 4 3660 022066 022653 027617 020832 5 i580 J22396 J50161 J09206 J39819 6 1623 032276 J27912 J276l7 J3l907 7 2950 J02635 010518 000000 007641 8 2875 J12515 006472 009206 001055 9 4320 027007 018608 036823 027425 10 3847 051709 038834 036823 0040609 11 1014 017126 012540 J09206 015558 12 3892 036887 018608 046029 027425 13 3670 012186 028721 OI84Jl 024788 14 1531 027336 017799 J36823 J25315 IS 840 J17456 J38025 J27617 J33226 1510 Problems it Multiple Regression 469 We solved Equation 1564 for various values of l and the results are summarized in Tabie 1512 The ridge trace is shown in Fig 157 The instabLity of the leastsquares estimates tJU 0 is evident from inspectio1 of the ridge tace It is often difficult to choose a value of 1 from the ridge trace e3r simultaneously stabilizes the estimates of all regression coefficients We will choose l 0064 which implies that the regression model is y 2553 180566x1 r 172202xz 30743 47242J using A y 2553 Coavening the model to the original variables Zt we haC y 29913 08920z 03483 33209 00623 Table 1512 Ridge Regression Estimates for Example 1514 0000 0001 0002 0004 0008 0016 0032 0064 0128 0256 0512 Pilf 70 60 Ie o 10 2C 30 40 50 283318 310360 326441 341071 343195 319710 263451 180566 91786 19896 24922 P4 I PI 659996 641479 572491 570244 619645 440901 509649 603899 353088 432358 580266 243241 351426 547018 133348 279534 500949 45489 220347 438309 12950 172202 360743 47242 134944 279363 65914 109160 208028 75076 92014 15397 77224 lo OOS 00 015 020 025 030 035 040 D45 050 055 Figure 15 7 Ridge trace for Example 1514 470 Chap 15 Multiple Regression 15102 Influential Observations in Regression When using multiple regression we occasionally find that some small subset of the obser varions is unusually influential Sometimes these influential observations are relatively far away from the vicinity where the rest of the data were collected A hypometical situation for two variables is depicted in Fig 158 where one observation in xSface is remote from the rest of the data The disposition of points in the xspace is important in dete1nining the propetties of the mode For example the point x xQ in Fig 158 may be very influen tial in detenrJning the estimates oime regression coefficients the value of R2 and the value of MS Ve would like to examine the data points used to build aregrvSsion model to determine if they control many model properties If these influential points are badl points or are erroneous in any way then they should be eliminated On the other hand there may be nom ing Wlong with these points but at least we would like to determine whether or not they produce results consistent with the rest of the data In any event even if an influential pomt is a valid one if it controls important model properties we would like to know this since it could have an impt on the use of the modeL Montgomery Peck and Vrning 2001 describe several methods for detecting influen tial observations An excellent diagnostic is the Cook 1977 1979 distance measure Thls is a measure of the squared distance between the least squares estimate of p based on all n observations and the estimate v based on removal of the ith point The Cook distance measure is XXil D J pMSE il j 2 n Clearly if the it point is influential its removal will result in 1 changing considerably from me value p Thus a large value of D implies that the ith point is influential The sta tiseic Di is acually computed using where f ej JMSEIh and hii is me ith diagonal element of the matrix H XxXtX Xl XI I Region containing 1 all obServattons I except the ith 1 I I I Figure 158 A point that is remote in xspace 1565 1 1510 Problems in Multiple Regression 471 The H matrix is sometimes called the haC matrix since yx XXXrXy Hy Thus H is a projection matrix that transforms the obsened values of y into a set of fitted values Y From equation 1565 we note that D is made up of a component that reflects how well the model fits the ith observatioll Yi the quantity eJ MSEI hii is called a Studentized residual and it is a method of scaling residuals so that they have unit variance and a com ponent that measures how far that point is from the rest of the data lil1 h is the dis tance of the ith point from the centroid of the remaining n 1 points A value of DI 1 woold indicate that the point is influential Either component of D or both may contribute to a large value IIte5Ii5 Table 513lists the values of D for the canaged peaches data in Example 15 1 Ib illusnte the ca1 culations CODSider the first observation Di Table 15 13 Irfb1cnce Diagnostics for the Damaged Peaches Data in Example 1515 Observation Cooks Distance Measure 0249 0277 2 0126 0000 3 0088 0156 4 0104 0061 5 0060 0000 6 0299 0000 7 0081 0008 8 0055 0002 9 0193 0116 10 0117 0024 11 0250 0037 12 0109 0002 13 0213 0045 14 0055 0056 15 0147 0067 16 0080 0001 17 0209 0001 18 0276 0160 19 0210 0171 20 0078 0012 472 Chapter 15 Multiple Regression 206N22471O249 0249 3 10249 0277 The values in Table 1513 were calculated using Minitab The Cook distance measue Di does not identify any potentially influential observations in the data as no value of Di exceeds unity 15103 Autocorrelation The regression models developed thus far have assumed that the model error components S are uncorrelated random variables Many applications of regression analysis involve data for which this assumption may be inappropriate In regression problems where the depend em and independent variables are time oriented or are timeseries data the assumption of uncorrelated errors is often untenable For example suppose we regressed the quarterly sales of a product against the quarterly pointofsa1e advertising expenditures Both vari abIes are time series and if they are positively correlated with other factors such as dispos able income and population which are not included in the model then it is likely that the error terms in the regression model are positively correlated over time Variables that exhibit correlation over time are referred to as autocorrelated variables 11any regression problems in economics business and agriculture involve autocorrelated errorS The occurrence of positively autocorrelated errors has several potentially serious con sequences The ordinary leastsquares estimators of the parameters are affected in that they are no longer minimum variance estimator51 although they are still unbiased Furthermore the mean square error lfS J may underestimate the error variance cr Also confidence inter vals and tests of hypotheses which are developed assuming uncorrelated errors are not valid if autocorrelation is present There are several statistical procedures that Can be used to determine whether the error terms in the model are uncorrelated We Yin describe one of these the Durbin WatSOn test This test assumes that the data are generated by the firstorder autoregressive model tl2 n 1566 where t is the index of time and the error terms are generated according to the process 1567 where Ipi 1 is an unknown parameter and 0 is a NlDO f random vanable Equation 1566 is a sjmple linear regression model except for the errors which are generated from equation 1567 The parameter p in equation 1567 is the autocorrelation coefficient The DurbinWatson test can be applied to the hypotheses He pO H1pO l56S Note that if Ho P 0 is not rejected we are implying that there is no autocorrelation in the errors and the ordinary linear regression model is appropriate 1510 Problems in Multiple Regression 473 To test II p 0 first fit the regression model by ordinary least squares Then ealcu late the DuibinWatson test statistic 1569 where e is the rth residual For a suitable value of a obtain the critical values D a u and D qJ from Table 1514lf D D do not reject II p 0 but if D D t reject II p 0 and conclude that the errorS are pOSitively autocorrelated If D 13 L D D 1Z lj the test is Table 1514 Critical Values of the DurbinWatsor Statistic Pobability LowetTail SampIe Significance Size Level a I k K umber of Regressors Exchding he Intercept 2 3 4 5 001 081 107 070 125 059 146 049 170 039 196 15 20 25 30 50 60 80 100 0025 005 001 0025 005 001 0025 005 001 0025 005 001 0025 005 00 0025 005 001 0025 005 001 0025 005 001 0025 005 095 123 083 lAO 071 161 108 136 095 154 082 175 095 115 086 127 077 141 108 128 099 141 089 155 120 141 110 154 100 168 105 121 098 130 090 141 113 134 110 143 102 154 120 145 121 155 112 166 113 126 107 134 101 142 125 138 118 146 Ll2 154 135 149 128 157 121 165 125 134 120 lAO 115 146 135 145 130 151 125 157 144 154 139 160 134 166 132 140 128 1045 24 149 142 150 138 154 134 159 l50 159 146 163 142 167 138 145 135 1048 132 152 147 154 144 157 140 161 55 162 l51 165 148 169 147 152 144 154 142 157 154 159 152 162 149 165 161 166 159 169 156 172 154 156 150 158 148 160 159 163 157 165 155 167 165 169 163 172 161 174 059 L8L 0048 209 069 197 056 221 063 157 060 174 079 170 070 187 090 183 079 199 083 52 075 165 094 165 086 177 104 177 095 189 094 151 088 161 105 163 098 173 114 174 107 183 110 152 105 158 120 163 115 169 129 172 123 179 120 154 116 159 130 164 126 169 138 72 134 177 128 56 125 160 137 165 133 169 144 173 141 177 139 60 136 162 147 167 144 170 153 174 151 177 145 163 144 165 153 170 151 172 159 76 157 178 Source Adapted from Econometrics by R 1 Wonnacott and T H Wonaaeon Joron Wucy Sons New York 2970 villi pennlssion of rhe publisher 474 Chapter 15 Multiple Regression inconclusive vnen the test is inconclusive the implication is that more data must be col lected In many problems this is difficult to do To test for negative autocorrelation that is if the alternative hypothesis in equation 1568 is HI P 0 then use lY 4 D as the test statistic where D is defined in equation 1569 If a twosided alternative is specified then use both of the onesided procedures not ing that the type I error for the twosided test is 2a where a is the type I error for the one sided tests The only effective remedial measure when autocorrelation is present is to build a model that accounts explicitly for the autocorrelative structure of the errors For an introductory treatment of these methods refer 10 Montgomery Peck and Vining 2001 1511 SELECTIOl OF VARIABLES IN MULTIPLE REGRESSION 15111 The ModelBuilding Problem An important problem ill many applications of regression analysis is the selection of the set of independent or regressor variables to be used in the model Sometimes previous experi ence or underlying theoretical considerations can help the analyst specify the set of inde pendent variables Usually however the problem consists of selecting an appropriate set of regressors from a set that quite likely illc1udes all the important variables but we are sure that not all these candidate variables are necessary to adequately model the response y In such a situation we are interested in screening the candidate variables to obtain a regression model that contains the best subset of regressor variables We would like the final model to contain enough regressor variables so that in the intended use of the model prediction for example it will perform satisfactorily On the other band to keep model maintenance costs to a minimum we would like the model to use as few regressor variables as possible The compromise betvleen these conflicting objectives is often called finding the best regression equation However in most problems there is nO single regression model that is best in teons of the varions evaluation criteria that have been proposed A great deal of judgment and experience with the system being modeled is usually necessary to select an appropriate set of independent variabes for a regression equation No algorithm will always produce a good solution to the variable selection problem Most currently available procedures are search techniques To perform satisfactorily they require interaction with and judgment by the analyst We now briefly discuss some of the more popular variable selection techniques 15112 Computational Procedures for Variable Selection Ve assume that there are k candidate variables XI Xl XI and a single dependent vari able y All models will include an intercept term 3 so that the model with all variables included would have k 1 terms Furthermore the functional fonn of each candidate vari able for example x lIxtx In x etc is correct An Possible Regressions This approach requires that the analyst fit all the regression equations involving one candidate vaable all regression equations involving two candi date variables and so on Then these equations are evaluated according to some suitable cri teria to select the best regression modeL If there are k candidate variables there are 2k total equations to be examined For example if k 4 there are 24 16 possible regression equations while if k 10 there are 2 1024 possible regression equations Hence the number of equations to be examined increases rapidly as the number of candidate variables increases 1511 Selection of Variables in Multiple Regression 475 There are a number of criteria that may be used for evaluating and comparing the dif ferent regression models obtained Perhaps the most commonly used crierion is based on he coefficient of multiple detennination Let Ii denote the coefficient of determination for a regression model with p terms that is p 1 candidate variables and an intercept eml note that p 5 k 1 Computationally we have R RP l P S 1570 where SScp and SSEP denote the regression sum of squares d the error sum of squares respectively for the pvariable equation Now JS increases as p increases and is a maximum when p k 1 Therefore the analyst uses this criterion by adding variables to the model up to the point where an additional variable is not useful in tha it gives only a small increase in R The general approach is illustrated in Fig l59 which gives a hypothetical plot of R against p 1iJically one examines a display such as this md chooses the number of variables in the model as the point at which the knee in the curve becomes apparent Clearly this requires judgment on the part of the malyst A second criterion is to consider the mean square error for the pvariable equation say MSEP SSpn p Generally MSEP decreases as p increases but this is not neces sarily so If the addition ofa variable to the model withp 1 terms does not reduce the error sum of squares in the new p term model by an amount equal to the error mean square in the old p 1 term model dSsP will increase because of the loss of one degree of freedom for error Therefore a logical criterion is to select p as the value that lIinimizes 11SE P or since fSp is usually relatively flat in the vicinity of the minimum we could choose p such that adding more variables to the model produces only very small reductions in MSEP The general procedure is illustrated in Fig 1510 A third criterion is the Cp statistic which is a measure of the total mean square error for the regression modeL We define the total standardized mean square error as 1 n 2 rp rEly EHl 1 2 tEyJEYj t Vy 1 b2 j2 las vanance J o Figure 159 Plot of against p 476 Chapter 15 Multiple Regression Minimum MSEo L Figure 15lO Plot of MSEP against p Ve use the mean square error from the full k 1 term model as an estimate of al that is IT MSsk I An estimator of ris 1571 If the ptel1I model has negligible bias men it can be shown that E Clzero bias p Therefore the values of Cp for each regression model under consideration should be plot ted against p The regression equations that have negligible bias will have values of Cp that fall near the line Cp P while those with significant bias will have values of Cp that pwt above this line One then chooses as the best regression equation either a model with min imum Cp or a model with a sJJghdy larger CD that contains less bias than the minimum Another criterion is based on a modification of R that accounts for the number of vari ables in the model We presented this statistic in Section 1561 the adjusted R for the model fit in Example 15L This statistic is called the adjusted R defined as nl RoO p 1 111 7 n p P 1572 Note that RjP may decrease as p increases if the decrease in n ll R is not CDW pensated for by the loss of one degree of freedom in n p The experimenter would usually select the regression model that has the caximum value of RP However note that this is equivalent to the model that minimizes MSP since j 1511 Seleetion of Variables in Multiple Rcgrassion 477 The data in Tabe 15 15 are an expanded set of data fo the damaged peach data in Example 151 Tnere are now five candidate variables drop height Xl fruit density x fruit height at impact point xJ fruit pulp thickness x4 and potential energy of me fruit before me impact x5 Tabe 1516 presents the rescl of ruruing all possible regressions except the tivial modei wim ody an intercept on these data The values of R Rlp MSp and ep are given in fre table A plot of the maximum R for each subset of size p is shown in Fig 15 1 L Based on this plot there does not appear to be much gain in zdding the fifth variable The value of R does Dot seem to bcrease sigrif icantly Jth the addition of x over the fourvariable model with the highest R value A plot of the minimum lrfSp for each subset of size p is sholAJ1 in FIg 1512 The best twovariable model is either x X or tz X the best thUvariable model is xpx xJ the besl fourwuiablc model is either xtJs xJ or xl 4 x3 X5 There are several models with relatively small values of MSip but either the threevariable model xl 4xJ or me fourvariable model x Xz xJ would be supe rior to the other models based on the MSdJ criterioll Further investigation oill be necessary A Cp pot is shown in Fig 1513 Only fue fivvariable model has a Cp 5 P specifically Cp 60 but the Cp value for the fourvariable model Xl x xJ is Cft 61732 There appears to be insufficert gain in the Cp value Q justify including x5 To illustrate the calculations for this equation for the rrwdel including X x x xJ we woud find Table1 Observation 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 C SSp n2p p 02 1829715 202561732 113132 Darraged Peach Data fur Example 1516 Delivery Drop Fnrit Fnrit TIme Height Density Heigh4 y x X X 362 3037 090 2il 727 3667 Ul4 180 26 3368 101 390 153 3045 095 485 491 3468 098 431 1036 6000 104 210 526 3690 096 127 609 4180 100 460 657 2690 101 26 424 3230 094 69 804 5622 101 273 346 2842 097 306 850 5586 103 377 934 4150 101 2il 555 3495 104 480 811 4628 102 328 732 3331 105 292 1258 5021 110 0 015 3114 091 392 523 3514 096 363 Fruit Pulp Potential Thickness Energy x x 223 1845 215 1852 229 1284 2004 1730 187 1396 170 1465 20L 1555 181 1292 215 1546 244 1528 195 1996 202 775 226 2100 171 165 215 1953 237 1710 219 1639 169 1408 260 541 235 1946 478 Clapter 15 Multiple Regression 07 5 6 P Figure 1511 110 maximwn R plot for Example 1516 oJ w 08 r J j t 2 3 4 5 6 P Figure 1512 The MSP plot for ample 1516 35 F u 25 15 i 5L j I 2 3 4 5 6 P Figure 1513 The C plot for Exanple 1516 noting thaf1 113132 is obtained from the full equation xtS x3 x4 xj Slnee all other models with the exclusion of the fivevariable model contain substantial bias we would conclude on the basis of the C criterion that the best subset of the regressor variables is x Xs xJ Since this model also results in relaively small MSsp and a relatively high R we would selecr it as the best regression equation The final model is jk199 00 123x 273x 00655 0196 Keep in mind though that fitrher analysis should be conducted on this model as well as other pos sible candidare models VV1th additional investigation it is possible to discover an even bcttcrfitting model We vill discuss this in mOre detail later in this chapter 15 11 Selection of Variables in Multiple Regression 479 The allpossIbleregressions approach requires considerable computational effort even when k is moderately small However if the yst is willing to look at something less than the estimated model and all its associated statistics ir possible to devise algorithms for all possible regressions that prOOuce less infonnarion about each model but which are more efficient computationally For example suppose that we could efficiently calculate only the MSEfor each model Since models with large MSE are not likely to be selected as the best regression equations we would then have only to examine in detail the models with small values of MS There are se ern1 approaches to developing a computationaUy efficient algo rithm for all possible regressions for example see Furnival and Wison 1974 Both Minitab and SAS computer packages provide the Fumival and Wilson 1974 algorithm as an option The SAS output is provided in Table 1516 Stepwise Regression This is probably the most widely used variable selection technique The procedure iteratively constructs a sequence of regression models by adding or Table 1516 Al1 Possible Regressions for the Data in Example 15 16 unber in Variables in ModelpI Ii ifcp Gp MSfp Model 06530 06337 377047 337540 05495 05245 537211 438205 x 03553 03194 837824 627144 x 1 01980 01535 1081144 780074 X 00021 00534 1384424 970689 x 2 07805 07547 199697 226063 xXz 2 07393 07086 263536 2685M x 2 07086 06744 310914 300076 Xzx4 2 07030 06681 319641 305883 Xl x 2 06601 06201 386031 350064 Xz x1 2 06412 05990 415311 369550 X x 2 05528 05002 552140 460607 x1 2 04940 04345 643102 521141 xJ X4 2 04020 03316 785531 615925 x4 2 02125 01199 1078731 811045 xp 3 08756 08523 72532 136135 xp XzS 3 08049 07683 18J949 213501 x1Xzx 3 07898 07503 205368 230060 XzSx 3 07807 07396 219371 239961 xl Xxj 3 07721 07294 232681 249372 x 1XX4 3 07568 07112 256336 26609B Xz Xl x 3 07337 06837 292199 291456 Xz x4 Xs 3 07032 06475 339410 324838 xlS Xj 3 06448 05782 429705 388683 xl XM Xs 3 05666 04853 550797 474304 X X 4 4 08955 08676 61732 121981 x XzxJx 4 08795 08474 86459 140630 xtz AXS 4 08316 07866 160687 196614 Xz x X4 Xl 4 08103 07597 193611 221446 x X x4 Xl 4 07854 07282 232090 250467 XX4XS 5 09095 08772 60000 113132 Xl X x xs 480 Chaptirr 15 Multiple Regression removing variables at each step The criterion for adding or removing a variable at any step is usually expressed in terms of a pattial Ftes Let Fi be the value of the F statistic for adding a variable to the model and letF oot be the value of the Fstatistic for removing a vari able from the model We must have Fin Fout and usually F F 00 Stepwise regression begirs by forming a oneMvariable model using the regressor vari able that has the highest correlation with the response vaiable y This will also be the vari able producing the largest F statistic fno F statistic exceeds Fin the procedure terrninates For example suppose that at this step XI is selected At the second step the remaining k 1 candidate variables are examined and the variable for which the statistic SSRPJlP Po ldSExJ1X1 l573 is a maximum is added 10 the equation proided that F F In equation 1573 MSxr x denotes the mean square for error for the model containing both XI andx Suppose that this procedure now indicates thats should be added to the model Now the stepwise regression algorithm de1lermines whether the variable x added at the first step should be removed This is done by calculating the F statistic 1574 If F I FuI the variable x is removed In general a each step the set of remaining candidate variables is examined and the variable with the largest partial F statistic is entered provided that the observed value of F exceeds F Then the partial F statistic for each variablein the model is calculated and the variable with the smallest observed value of F is deleted if the observed F F 00 The pro cedure continues until no other variables can be added to or removed from the model Stepwise regression is usually performed using a computer program The analyst exer cises control over the procedure by the choice of FlO and Ftu Some stepwise regression computer programs require that numerical values be specified for Fin and FQu Since the number of degrees of freedom on MS e depends on the number of variables in the model which ch3lges from step to step a fixed value of FIE and FOUl causes the type I and type II error rates to vary Some computer programs allow the analyst to specify the type I error levels for Fm and Fowt However the advertised significance level is not the true level because the variable selected is the one that maximizes the parial F statistic at that stage Sometimes it is useful to experiment With different values of Fjj and Fjt or different adver tised type I eITOr rates in several runs to see if this substantially affects the choice of the final model mi4t We wJlapply stepwise regression to the damaged peaches data in Thble 1515 Miuitab2i Otput is pnr vided in Fig 1514 From this figure we see that variables XIS andx are sigcificalt this is because the last column contains entries for only xl andtJ Figure 1515 provides the SAS computer output that vill support the computatioas ro be calculated nett Instead of specifflg uca1 values of F andF Q we use an advertised type I ITOr of aO10 The first Step consists ofbcilding a sirnpe linear regression model using the variable that gives the largest F statistic This is Xz and since F SSRPIPo 11432885 3387 F F 301 MSx 337540 151 Selection of Variables in Multiple Regressior 481 AlpratoEnter Ql AlphatoRemove Response is y on predictors with N 20 Step Constalt x2 Tlalue PValue xl TValue PValue x3 Value PValue S RSg RSq adj Cp 1 42 87 491 582 0000 184 653C 6337 377 2 3333 349 423 0001 00131 314 C C06 1 SO 7S0S 7547 200 3 2789 307 471 0000 00136 419 0001 0067 350 0003 117 8756 S523 73 01 Figure 1514 Minitabl out for stepwise regression in Example 1517 L is entered into the modeL The second step betns by finding Lie vaiable xJ that has the largest partial F statistic giver that L is in the mode This is xI and since sSlldIl2llc MSExx 4645691 226063 2055 F Fi1 303 x is added to he model ow the procedure evaluates whetier or not Xz should be retained given that x is in the modeL rus mvol yes calculating 4044627 1789 F 226063 m FOOJ17 303 TherefOiet should be retained Step 2 terminates INith boti x and X in tie model The bird step finds the next variable fer entry asS Since ss Il I Il Ilc MSExxx lq64910 L36135 1223 Fin FOIQIIS 305 l is added to the model Partial Ftests on Xz given Xl andx and Xl given and xJ indicate that these variables sholtld be retained Therefore the third step coucludes with the variables XI x2 and Xj in the model At the fourth step neither of the remainirg terms x or x is significant enough to be included in the model Therefore the stepwise proced4e tes The stepwise regression procee would conehlde that the bes model includes XI andS The usual checks of medel adequacy such as residual analysis and Cp plots should be applied to the equation These results are similar to those found by all possible regressions with the exception that Xi was also considered a possible significant variable with all possible regressions 482 Chapter 15 Multiple Regression Forward Selection This variable selection procedure is based on the principle that vari ables should be added to the model one at a time until no remairing candidate variables produce a significant increase in the regression sum of squares That is variables a added one at a time as long as F Fin Forward selection is a simplification of stepwise regression that omits the partial Ftest for deleting variables from the model that have been added at previous steps TIlls is a potential weakness of forward selection the procedure does not explore the effect that adding a variable at the current step has on variables added at ear lier steps The REG Procedure Depende Variable y Porward Selection Step 1 Vaiable x2 Entered RSquare 37707 06530 ald Cp Source Model DF 1 Sum of Squares 11432885 6075725 17508609 Mean Square 11432885 337540 P Value 3387 Pr f 0001 Eror Corrected variable Intercept x2 18 Total 19 Pareter Estimate 4287237 4908366 St andard Eror 841429 843377 Type II 5S 8762858 11432885 Bounds on conciion nu1lber 1 1 F Value 2596 3387 Pr P OC01 0001 Forward Selection Step 2 Variable xl Enterec RSquare 07805 and Cp 199697 Source Model DF 2 17 19 Sum of Squares 13665541 3843068 175086C Mean Squore 633277 226063 F Value 3023 Pr F 0001 Error Corrected Total Variabe Intercept xl 2 Parameter Estimate 3383110 001314 3488963 Standard Error 746286 000418 824844 Type II S5 4645691 2232656 4044627 Bounds on condition number 14282 57129 F Valle 2C55 988 1789 Pr F 00003 C0059 00006 Fo ard Selection Step 3 Varlable x3 Entered RSqaare 0 08756 and Cp 72532 Source Model Error Dr 3 16 19 Sum of Squaes 15330451 21 78159 17508609 Mean Square 5110150 136135 F Value 3754 Pr F 0001 COrected Tota Variable Intercept xl 2 x3 Parameter Esti tate 2789190 001360 3068486 006701 Bounds on Standard Error 603518 000325 650286 001916 condition number Type II S5 F Value P F 2907675 2136 00003 2387130 1754 00007 3021859 2240 00002 166490 1223 00030 1786 1185 No other variable met the 01000 significance level for Summary of Forward Seection entry into the model Variabe Number Partial Model Step Entered vars In RSquare RSquare 1 x2 1 06530 06530 2 xl 2 01275 07805 3 x3 3 00951 08756 Figure 1515 SAS output for stepwise regression in Example l5 17 Cp 377047 199697 72532 F Value 3387 988 1223 Pr P 0001 00059 00030 15 1 Selection of Variables in Multiple Regression 483 ExunplelS18 AppEcarion of the forward selection algoriilim to the damaged peach data ir Tabre 15 15 would begin by adding Xz to the node1 Thel th variable that induces the WbeSt partial Ftest given tha X2 is in the model is addedthis is variable XI The third step enters X which produces tIre Largest patial F statistic given 1fat xt and x are in the modeL Since the partial F Statlsrcs for XJ alld are not signif ieaIt the procedure terminates The SAS outputfo forward selection is given in Fig 1516 Note tat forward selection leads to the samefinal model as stepwise regression This is not always the case Backward Eliminotion This algorithm begins with all k candldate variables in the modeL Then the variable Vith tbe smallest partial F statistic is deleted if this F statistic is insignif kant thatis ifF Foul Next the model with k 1 3riables is estimated and the next vari able for potential elimination is found The algorithm terminates when no further variables can be deleted To apply backward elimination to ilie dara in Table 1515 we begil by estimating t1e fur model in a five variables This model is y 2089732 001 lO2x 2737046x 006929 025695x OOl66X The SAS COffiputer output is given in Fig 1517 The partial Ftests for eaeh variable are as follows 1j SS Pl3 3 p p Po 1365360 12m MSe 113132 I F SS 3 f3 3 p P 30 J 2173153 192J MSz 113132 F SS R 3 3 8 f34 f3 p 1737834 1536 MSE 113132 F SSR3P f3 3 3 f30 525602 465 MS 113132 F SSf33 8 3f34 80 245862 217 MSE 113132 Te variablex has the smallest F statistic Fs 217 F WI FoOt4 310 therefore AS is removed from the model at step L The model is now fit with only the four xmainilg variables 11 step 2 the F statistic for ott F 286 is less than Foot FO1011 307 therefore x4 is emoved from the model No rernairing variables have F statistics less than the appropriate F fJl values and the procedure is ter miDated Te thCevariable model x Xz Xv bas all variables significant according to the partial Ftest criterion Note that backward cliUnation has resulted in the same model that was found by forward selection and stepwise regression This rwy not always happen Some Comments on Final Jodel Selection livre have illustrated several different approaches to the seleetion of variables in multiple linear regession The final model obtained from any modelbuilding procedure should be subjected to the usual adequacy checks such as residual analysis and examination of the effects of outermost points The analyst may also consider augmenting the original set of candidate variables with cross products polynomial terms or other transformations of the original variables that might improve the model 484 Chap 15 Multiple Regression The REG Procedure Dependent Vaiable y Backward Eli inatior Step 0 All Vaxiables Enered RSquate Cp 60000 Sum of Mean Source DP Squares Square F Value Model 5 15924760 31 84952 2815 Error 14 1583850 113132 Coected Total 19 1708609 Paameter S1al1dard Variable Estimate Error Type II S5 F Value Intercept 2089732 716035 963604 852 xl 001102 000317 1365360 1207 x2 2737046 624496 21 73153 1921 x3 OO6929 001768 1737834 1536 x4 025695 011921 525602 465 x5 0016B8 001132 245862 217 Bounds on condition number 1 6438 35628 09095 and Pr F 0001 Pr P 00112 00037 00006 00015 00490 01626 Backward E1lmination Step 1 Variable xS Removed RSquare Q8955 and Cp 61732 Source Mode Eror Corrected Total DF 4 IS 19 Va1iable Intercept xl 2 x3 Parameter Estimate 1993175 001233 2728797 006549 Q19641 5llll of Squares 15678898 1829712 17508509 Standard Exror 740393 000316 648433 001816 011621 Mean Square 3919724 121981 Type II S3 884010 1851245 21 60246 1586299 348447 P Valle 3213 F Value 725 IS18 1771 1300 286 Bolnds on condition nUJbcr 16358 2231 Pr F 0001 Pr F 00167 00014 00008 00026 01117 Backward Elimination Step 2 Variable x4 Removec RSqlare 08756 and Cp 72532 Source Model Error Correctd Total DF 3 16 19 Variable Intercept x x2 x3 ParaTctcr Estirrcatc 2789190 001360 3068486 006701 Sum of Squares 15330451 21 78159 17508609 Standard Erra 603518 000325 65286 001916 Mean Square 5110150 136135 Type II 55 2907675 2387130 3021859 1664910 F Value 3754 F Value 2136 1754 2220 223 Bounds on condition number 14786 1185 All variables left k the model are significant at the Summary of Bac1cNard ElilIination Variable Nu11ber Partial Model Step Entered Vaxs In RSquare RSquare Cp Pr F 0001 lr F OoooS 00007 00002 00030 01000 level F Value PO F 1 x5 4 00140 08955 61732 217 01626 2 x4 3 0 199 08756 72532 286 017 Figure 15 16 SAS output for forward selection in Example 1518 1511 Selection of Variables in Multiple Regression 485 he REG PQcedre Dependent Variable y Sepwise Selectior Sep 1 Variable x2 Enteed RSquare 377047 Source DF Made 1 Error 18 Corrected Tcta 19 arameter Variable Estimate Intercept 4287237 x2 49C8366 Sum of Squares 11432885 6075725 17508609 Standard Error 841429 843377 Meal Square 11432885 337540 Type II SS 8762858 11432885 Value 3387 F Value 2596 3387 Bounds On condition ncnber 1 1 Sepse Selection Step 2 Variabe xl Entered RSquaxe 199697 Slln of Mean Souce DE Squares Square F Value Model 2 13665541 6332771 3023 Error 17 3843068 226063 Corrected Total 19 7508609 Pararteter Sta1dard Variabe Estimae Error ype 55 F Value Intercept 3383110 746286 4645691 2055 xl 003l4 000418 2232656 9Sa x2 3488963 824844 4044627 1789 06530 a1d Cep PI F 0001 Pr F 0001 0001 07805 ad Cp Pr F OCOl Pr F C0003 00059 000C6 Bounds on condi tioD number 14282 57129 Stepwise SElEction Step 3 Variable x3 Entered RSquare C 8756 and Cp 72532 SCJce D Model 3 Error 16 Ccrrected 70tal 19 Pararneter Variabe Esimate ntercept 2789190 xl 001360 x2 3068486 x3 0C6701 Bounds SUl1I of Mean Squares Sqtare F Value 15330451 5110150 3754 2178159 1 36135 17508609 S1dard Error TJ II 55 F Va1le 60358 2907675 2136 000326 2387130 1754 651286 3021859 2220 001916 1664910 1223 on coruli ion nurnbe 1 4786 1185 Pr F 0001 Pr F 00003 000C7 00002 0C030 AI variables left in the model are significt a he OlCOO leveL Ke other variable met the 01000 significance level for entry into the model Summary of Stepwise Selecton Varable Variable Number Partial Mode Step Entered x2 2 3 xl 3 Removed Vars In aSquar RSquare 1 06530 06530 2 01275 07805 3 00951 08756 Figure 1517 SAS output for bacbvard elimination ill Example 1519 Cep 377047 199697 72532 F Value 3387 988 1223 Pr F 0001 C0059 0C030 A major criticism of variable selection methods such as stepwise regression is that the analyst may conclude that there is one best H regression equation This generally is not the case because there are often several equally good regression models that can be used One 486 Chapter 15 Multiple Regression way to avoid this problem is to use several different modelbuilding techniques and see if different models result For example we have found the same model for the damaged peach data by using stepwise regression forward selection and backward elimination This is a good indication that the threewvariable model is the best regression equation Furthennore there are variable selection techniques that are designed to find the best onevariable model the best twovariable model and so forth For a discussion of these methods and the van able selection problem in general see Montgomery Peck and Vining 200l If the number of candidate regressors is not too large the allpossiberegressions method is recommended It is not distorted by multicollinearity among the regressors as stepwisetype methods are 1512 SU1 This chapter has introduced multiple linear regression including leastsquares estimation of the parameters interval estimation prediction of new observations and methods for hypothesis testing Various tests of model adequacy including residual plots have been dis cussed It was shown that polynomial regression models can be handled by the usual mul tiple linear regression methods Indicator variables were introduced for dealing with qualitative variables It also was observed that the problem of multicollinearity or inter correlation between the regressor variables can greatly complicate the regression problem and often leads to a regression model that may not predict new observations well Several causes and remedial measures of this problem inc1udilg biased estimation techniques were discussed Finally the variable selection problem in multiple regression was intro duced A number ofmodelbui1ding procedures including all possible regressions stepwise regression forward selection and backward e1imination were illustrated 1513 EXERCISES lSl Consider the damaged peach data in Table 1515 a Fit a regression model using XI drop height and x fruit pulp thickness to these data h Test for significance of regression c Compute the residuals from this modeL Analyze these residuals using the methods discussed in this chapter Cd How does this twOMable r1ode compare Veith the twovariable model using x and tz from Example 15 j 152 Consider the damaged peaen data in Table 1515 a Fit a regression nodel using XI top heighttz fruit density and fruit height at impact point to cese data b Test for sigrlficance of regression c Compute the residcals from this model Analyze these residuals using the methods discussed in Us chapter 153 Using the results of Exercise 151 find a 95 confidenee interval on A lS4 Using the results of Exercise 152 find a 95 confidence interval on j3 155 The data in the table at the top of page 487 are the 1976 team performance statistics for the teams in the National Football League Source The Sparring lyews a Fit a multiple regression nodel relating the num ber of games won to the teams passing yardage x the percentage of rushing plays x and the opponentS yards rosblng x b Construct the appropriate residual plots ad com ment on model adequacy e Test the significance of each variable to the model using either the Nest Or the ptrtial Ftest 1S6 The table at the top of page 488 presents gaso line rrileage perforrrace for 25 attomobiles Source Motor Trend 1975 a Fit a multiple regression lodel relating gasoline mileage to engine displacement Xl and number of carburetor bacels xJ b Analyze the residuals and comment on mood adequacy 1513 Exercises 487 National Football League 1976 Team Performance Team y x Washington 10 2113 1985 389 647 4 868 597 2205 1917 Mionesota II 2003 2855 388 613 3 615 550 2096 1575 New England 11 2957 1737 4DJ 600 l4 914 656 1847 2175 Oadand 13 2285 2905 16 453 4 957 614 1903 2476 Potsburgh 10 2971 1666 392 538 15 836 661 1457 1866 Baltimore II 2309 2927 397 741 8 786 610 1848 2339 Los Angeles 10 2528 2341 381 654 12 754 661 1564 2092 Pallas 11 2147 2737 370 783 l 761 580 1821 1909 Atlanta 4 1689 1414 421 476 3 714 570 2577 2001 Buffalo 2 2566 1838 423 542 1 797 589 2476 2254 Chieago 7 2363 1480 373 480 19 984 675 1984 2217 Cincinnati 0 2109 2191 395 519 6 700 572 1917 1758 Cleveland 9 2295 2229 374 536 5 1037 588 1761 2032 Denver 9 1932 2204 351 714 3 986 586 1709 2025 Detroit 6 2213 2140 388 583 6 819 592 1901 1686 Green Bay 5 1722 1730 366 526 19 791 544 2288 835 Houston 5 1498 2072 353 593 5 776 496 2072 1914 Kansas City 5 1873 2929 4Ll 553 10 789 543 2861 2496 Miami 6 2118 2268 382 696 6 582 587 2411 2670 New Orleans 4 1775 1983 393 783 7 901 517 2289 2202 New York Gimts 3 1904 1792 397 381 9 734 619 2203 1988 New York Jets 3 1929 1606 397 688 21 627 527 2592 2324 Philadelphia 4 2080 1492 355 688 8 722 578 2053 2550 St Louis 10 2301 2835 353 741 2 683 597 1979 210 San Diego 6 2040 2416 387 500 0 576 549 2048 2628 San hancisco 8 2447 1638 399 571 1l 848 653 1786 1776 SeatJe 2 1416 2649 374 563 22 684 438 2876 2524 TIllIlpa Bay 0 1503 1503 393 470 9 S7S 535 256 2241 y Gimes won pc 14 gaUc season t Rusbing yards season x Passing yards season xl Punting average yds I punt t Field goal percentage Igs made I fgs attempted xl TJTIoler Cifferenial rornovers acquirCd ilDnOVrs lost Penalty yards season x Percent rushing rushing prays I total plays OppoIk1lt5 rushing yards season Oppocents passing yards season c What is the value of addirtg 46 to a model aut amocn temperature xa the number of days in jre already containsX lIIDnth r the average product purity x and he tOllS of proouct produced xJ The past years hlston 15M7 The electric power COJ1SllD1ed each month by a cal data is available and is presented in the table at the chemieal plant is thought to be related to the average bottom of page 488 488 Chapter 15 lutipl Rogression Automobile y Apollo Nova MoarCh Duster Jenson Cony Skyhawk Scirocco Corolla SR5 Camara DatsunB210 Capri II Pacer Granada Eldorado Imperial NovaLN Starlirc Cordoba Trans Am Corolla E5 MklV Celica GT Charger SE 1890 350 165 260 801 2000 250 105 185 8251 1825 351 143 255 801 2007 225 95 170 841 1120 440 215 330 821 2212 231 110 175 801 3470 897 70 81 821 30040 969 75 83 901 1650 350 155 250 851 3650 853 80 83 851 2150 171 109 146 821 1970 258 110 195 801 1780 302 129 220 801 1439 500 190 360 851 1489 440 25 330 821 1780 350 155 250 851 2354 231 no 175 801 2147 360 180 290 841 1659 400 185 NA 761 3190 969 75 83 901 1327 460 223 366 801 2390 1336 96 120 841 1973 318 140 255 851 Cougar 1390 351 148 243 80 1 Corvette 1650 350 165 255 851 y Miles I gallon XL Displacement cubc in x Horsepower ftlb f Torque ftlb XA Compression noo x Rear atle ratio r CWJrctor barrels Xl No of trasnlssion speeds x Overall length in x Yidth in X10 Weiglt Oos XI i Type of tnlnmlssion Aauomatic MmanU3l j 240 236 290 274 301 316 25 31 45 60 65 72 24 21 24 25 25 26 91 90 88 87 91 94 100 95 110 88 94 99 2561 4 2731 3001 2 2761 1 2881 4 2561 2 3901 2 4301 2 3081 4 3891 2 3221 2 3081 I 3001 2 2731 4 2711 4 3081 4 2561 2 2451 2 3081 4 4301 2 3001 4 3911 2 2711 2 3251 2 2731 4 y 300 296 267 276 288 261 x 80 84 75 60 50 38 3 3 3 3 3 3 5 3 4 3 3 3 3 3 3 3 3 5 3 5 3 2003 1967 1999 1941 1845 1793 1557 1652 1954 1606 17004 1715 1999 2241 2310 1967 1793 2142 196 1652 228 1715 2153 3 2155 3 1852 25 25 24 25 25 23 699 3910 A 722 3510 A 740 3890 A 718 3365 M 690 4215 A 654 3020 A 640 1905 M 650 2320 M 744 3885 A 622 2009 M 669 2655 M no 3375 A 740 3890 A 798 5290 A 797 5185 A 722 3910 A 6504 3050 A 763 4250 A 730 3850 A 618 2275 1 798 5430 A 634 2535 M 763 4370 A 785 4540 A 690 3660 A 87 86 88 91 90 89 x 97 96 110 105 100 98 a Fit a multiple regression model to these data b Test for sigrllficance of regression c Use partial F statistics to test H J 0 and H J 0 d Compute the residuals from this model Analyze the resduals using the merhods discussed in this chapter 158 Hald 1952 reports data on the heat evolved in calories per gram of cement y for various amounts of four ingredients x x x xJ Observation uber y x X x 735 7 26 6 60 2 743 1 29 15 52 3 1043 11 56 8 20 4 816 11 31 3 47 5 959 1 52 6 33 6 1092 11 55 9 22 7 1027 3 71 17 6 8 725 1 31 22 44 9 931 2 54 18 22 10 1159 21 47 4 26 11 88 I 4 23 34 12 1133 11 66 9 12 13 1094 10 68 8 12 a Fit a multiple regression model to dlese data b Test for sigific2nce of regession c Test the hypothesis fl 0 using the partial Ftest Cd Compute the r statiscics for each independent vari able What conclusions C2n you draw e Test the hypothesis J fl 3 0 using the par tial Ftest f ConstrJct a 95 confidence interval estimate for 3 159 An article entitled P Method for Improving the Accuracy of PolY1omial Regression Analysis in the loumal of Q1aliry Technology 1971 p 19 reported the following data on y ultitlate shear strength of a rnbler compound psi and x cure temperature F y 770 800 840 810 735 640 590 560 c Test the hypothesis thar Pn O d Compute the residuals and test for model adequacy 1510 Consider the followJlg data which result from an experiment to derernine the effect of x test time in hours at a particular temperature on y change in oil viscosity y 442 139 155 189 243 315 405 515 643 789 x O2S 050 075 00 125 L50 175 200 225 250 a Fit a secondOlderpolynotial to the data b Test for significance of regresion 0 Test the hypothesis that Pu O d Compute the residuals and check for model adequacy lS 11 For many polyromial regression models We subtract x from each x value to produce a centered regressor Xl xx Using the data from Exercise l59 fit the model y e f3 Px PIX e Use the results to estimate the coefficients in the uncentered model y Jc Jx f3 i E 15u Suppose that we use a standa1nzed variabie xo x xlsft where Sy is the standard deviation of x in constructing a polyromial regression model Using he data in Exercise 159 and the standardized varilble approach fit the model y fJ Px j3IXZ e a What value of y do you predict when x 285F b Estimate the regression coefficients in the unstan datdized model y Po PiX PllX e c What can you say about the relationship beween SSe and RZ for the standardized and unstandard ized models d Suppose that y y Yfsv is used in the model along with x Fit tbe Lodel and comment on the relationship between SSE and If in the standard ized model and the unstandardized model 1513 The data shown at the botwm of this page were collected during an experiment to deterntine he change in thrust ef5ciency y as the divergence angle of a rocket nozzle x clanges a Fit a secondorder model to the data b Test for significance of regression and lack of fit c Test the hypothesis that 3 o x 280 284 292 295 298 305 308 315 1514 Discuss the hazards inherent in fitting polyno ll1i2l models a Fir a secondorder polynomial to this data b Test for significance of regression 40 50 50 1515 Consider the data in Example 1512 Test the hypothesis that two diffexentregression models with 65 65 675 70 73 490 Chapter 15 Multiple Regresion different slopes and intercepts are required to ade quately model the data 15 16 Piecewise Linear Regression I Suppose that y is piecewise linearly related to x Thar is differ ent linear relationships are appropriate over the inter vals ox X l 3lld x x l Show how indicator variables em be used to fit such a piecewise linear regression model assuming that point x is knoWU 1517 Piecewise Linear Regression II Consider the piecewise linear regression mode3 described in Exercise 1516 Suppose that at point x a discontinu ity occurs in the regression function Show how indi cator variables can be us to iIlcorporate the discontinuity into the modeI 1518 Piecewise Linear Regression ill Consider the piecewise linear regression model described in Exercise 1516 Suppose that point x is not known with certainty aud must be estimated Develop an approach that could be used to fit the piecewise linear regression model 1s19 Calcu1ate the standardized regression coeffi cients for the regression model developed in Exercise lSL 1520 Calculate the standardized regressioo coeffi cients for the regression model developed in Exercise 152 1521 Find the variance inflation factors for the regression model developed L1 Example 151 Do they irdicate tat multicollinearity is a prOblem in this model 1522 Use the National Football League Team Per fonnance data in Exercise 155 to build regression models using rhe following techniques a All possible regressions b Stepwise regression c Fonvard selection d Backward elimination e COIllmellt on the various models obtained 1523 Use the gasoline mileage data in Exercise 156 to build regression models using the following tech niques a All possible regressors b Stepwise regression e Forward selection d Backward atio e Comment on the various models obtained 1524 Consider the Rald cement data in Exercise 158 Build regression models for the data using me following techniques a il1 possible regressions b Stepwise regression c FoIWdld selection Cd Backward elimination 1525 Consider the Hald cement data in Exercise 158 Fit a regression model involving all four regs sors and find t variance inflation factors Is multi collinearity a problem i1 this model Use ridge regression to estimate the coefficients in this model Compare the ridge model to the models obtal1ed in ECercise 1525 using variable selection methods Chapter 16 Nonparametric Statistics 161 INTRODUCTION Most of the hypothesis testing and eonfidenee interval proeedures in previous chapters are based on the assumption that we are working with random samples from normal popula tions Fortunately most of these proeedures are relatively insensitive to slight departures from nonnality In general the t and Ftests and t eonfidenee intervals will have aetuallev ets of signifieance or eonfidenee levels that differ from the nominal or advertised levels chosen hy the experimenter although the difference between the actual and advertised lev els is usually fairly small when the underlying population is not too different from the nor mal distribution Traditionally we have called these procedures parametric methods beeause they are based on a partieular parametric family of distributionsin this case the nonna Alternatively sometimes we say that these procedures are not distnbution free because they depend on the assumption of normality In this chapter we describe procedures ealled nonparametric or distributionwfree meth ods and usually make no assumptions about the distribution of the underlying population other than that itis eontinuous These procedures have actual levels of signifieanee a or con fidence levels 1001 a for many different types of distributions These procedures also have considerable appeal One of their advantages is that the data need not be quantitative it could be categorical such a yes orno defective or nondefective etc or rank data Another advantage is that l10nparametrie procedures are usually very quick and easy to perform The procedures deseribed in this chapter are competitors of the parametric t and FprocedUIes describd earlier Consequently it is important to compare the performance ofbotb parametric and nonparametric methods under the assumptions of both normal and nonnormal populations In general nonparametrie procedUIes do not utilize all the infor mation provided by the sample and as a result a nonparametric procedure will be less effi cient than the corresponding parametric procedure when the underlying population is norma1 This loss of efficiency usually is reflected by a requirement for a larger sample size for the nonparametric procedure than would be required by the parametric procedure in order to achieve the same probability of type II error On the other band this loss of effi ciency is usually not large and often the difference in sample size is very smalL When the underlying distributions are not nonnal then nonparametric methods have much to offer They often provide considerable improvement over the nOrmaltheory parametric methods 162 THE SIGN TEST 1621 A Description of the Sign Test The sign test is used to rest hypotheses about the median j1 of a continuous distribution Recall that the median of a distribution is a value of the random variable such that the prob ability is 05 that an observed value of X is less than or equal to the median and the 491 492 Cbapter 16 Nonpa3Juetric Statistics probability is 05 that an observed value of X is greater than or equal to the median That is PX sl PXfl 05 Since the normal distribution is symmetric the mean of a normal distribution equals the median Therefore the sign test can be used to test hypotheses about the mean of a nor mal distribution This is the same problem for which we used the Itest in Chapter 11 We will discuss the relative merits of the two procedures iu Section 1624 Note that while the ttest was designed for samples from a normal distribution the sign test is appropriate for samples from any continuous distribution Thus the sign test is a nonparametric procedure Suppose that the hypotheses are Ho l 10 H l 10 161 The test procedure is as follows Suppose that Xl X2 Xtl is a random sample of n obser vations from the population of mterest Form the differences Xi pJ i 1 2 n Now if Ho l1o is true any differenceX1o is equally likely to be positive or negative There fore let R denote the number of these differences X p that are positive and let K denote the number of these differences that are negative where R min CRt R When the null bypothesis is true R has a binomial distribution with parameters n and p 05 Therefore we would find a critical value say R from the binomial distribution that ensures that Ptype I erroD Preject Ho whenHo is true a A table of these critical val uesRis given in the Appendix Table X If the test statistic Rf R then the null hypothe sis Ho P 10 should be rejeeted 1ontgomery Peck and Vining 2001 report on a study in which a rocker motor is formed by bind Llg an igniter propellant and a sustainer propellant together inside a metal housing The shear strength of the bond between je two propellant types is an important characteristic Resllts of testing 20 ran comly selected motors are shown in Table 161 We would like to test the hypothesis that me median shear strength il 2000 psi Tne formal statement of the hypotheses of interest is H fl 2000 H fl 2000 The last two columns of Table 16 1 show the differences Xi 2000 fnr i 12 20 and the cor responding sigus Note that Jr 14 and R 6 Theefore R min Jr R min 1 6 6 Fron the Appendix Table X with n 20 we find that the critical value for a 005 is 05 5 Therefore smceR 6 is not less than or equal to the critica va1ue1l 5 we cannot rejecthe lull hypothesis that the median shear strength is 2000 psi We note that since R is a b1romial random variable we could test the hypothesis of interest by directly calculating a Pvalue from the binomial distribution When Ho fl 2000 is true R has a bino mial distributior with paratl1eters n 20 and p 05 Thus the probability of observing six or fewer negative signs in a sample of20 observations is 20 PR S 6 1 1050520 1 r rO 0058 Since the Pya1ue is not less than the desired level of Significance we cannot reject thelull hypoth n esis of a 2000 psi r I 162 The Sigr Test 493 rble161 Propellant Shear Streng Daa Observation i Shear Strength X Differences X 2000 Sign 215870 15870 2 167815 32L85 3 231600 31600 4 206130 L30 5 220750 20750 6 170830 29170 7 178470 21530 8 257510 57500 9 235790 35790 10 225670 25670 11 216520 16520 12 239955 39955 13 177980 22020 14 233675 33675 15 176530 23470 16 205350 5350 17 241440 41440 18 220050 20050 19 265420 65420 20 175370 24630 Exact Significance Levels When a test statistic has a discrete distribution1 such as R does in the sign test it may be impossible to choose a critical value R that has a level of signif icance exactly equal to a The usual approach is to choose R to yield an a as close to the advertised level a as possible Ties in the Sign Test Since the underlying population is assumed to be continuous it is theoretically impossible to find a that is a value of XI exactly equal to Po However this may sometimes happen in practice because of the way the dzta are collected Vhen ties occur they should be set aside aJd the sign test applied to the remaining data OneSided Alternative Hypotheses We can also use the sign test wben a onesided alter native bypothesis is appropriate If the alternative is H p p then reject Ho p p if R R if the alternative is H p p then rejectHo P p if R R Tne level of signif icance of a onesided test is onehalf the value sbo in the Appendix Table X It is also possible to calculate Pvalue from the binomial distribution for the onesided case The Normal Approximation When p 05 the binomial distribution is well approxi mated by a norma distribution when n is at least 10 Thus since the mean of the binomial is np a1d the varice is nplp the distribution of R is approximately normal with mean O5n and variance 025n wbenever n is moderately large Thereforel in these cases the null hypothesis can be tested with the statistic RO5l 162 494 Chapter 16 Nonparaetric Slatistics The twosided alternative would be rejected if 1201 Zoo and the critical regions of the one sided alternative would be chosen to reflect the sense of the alternative if the alternative is H f1 f1c rejectH if 20 Z for example 1622 The Sign Test for Paired Samples The sign test can also be applied to paired observations drawn from continuous popu1a aons Let X Xj 1 2 n be a collection of paired observatiocs from two contin uous population and let 12 n be the paired differences We wish to test the hypothesis that the two populations have a common median that is thatil This is equivalent to testing that the median ofllie du ferences fld O This can be done by applying the sign tcst to the r differences D as illus tratedin the following example Exampk162 An autorootive engincer is investigating two different types of metering devices for an electronic bel injection system to determine if they differ in their fuelmieage performance The system is installed on 12 different cars and a tesis run with each meterslg system on each car The observed fuel mileage performance data corresponding differences ard their signs are show in Table 162 Note that K 8 andR4 ThmforeR min KR1 min 8 4 4 From the Appendix Table X with n 12 we find the criticlli va1e for a 005 isO 2 SinceR is not less than the critical value jj we cannot reject the null hypothesis that the two metering devices produce the same fuel mileage performanee 1623 Type n Error fJ for the Sign Test The sign test will control the probability of type I error at an advertised level a for testing the null hypothesis H f1 A fo any continuous distribution As with any hypothesis testing procedue it is lnportant to investigate the type IT error p The test should be able to effectively detect departures from the null hypothesis and a good measure of this effec Tablel6Z Pcrfurrance of Flow Metering Devices Metering Device Car 2 Difference 176 168 08 2 194 200 06 3 195 182 13 4 171 164 07 5 153 160 07 6 159 15 05 7 163 165 02 8 184 180 04 9 173 164 09 10 191 201 l0 1l 178 167 Jl 12 182 179 03 162 The Sign Test 495 tiveness is the value of P for departures that are important A small value of P implies an effective test procedure In determining p it is important to realize that not only must a particular value of Jl say A Ll be used also the form of the underlying distribution will affect the calculations To illustrate suppose that the underlying distribution is nonnal with 0 I and we are testing the hypothesis that 1 2 since P f1 in the normal distribution this is equivalent to testing that the mean equals 2 It is important to detect a departure from1 2 to1 3 The situa tion is illustrated graphically in Fig 1610 hen the alternative hypothesis is rue H 1 3 the probability that the random variable X exceeds the value 2 is p PX 2 PZI 1 41 08413 Suppose we have taken samples of size 12 At the a 005 level Appendix Table X indi cates that we would reject Ho 1 2 if R R 2 Therefore the P error is the probabil ity that we do not reject No 1 2 when in fact 1 3 or 2 112 J P 1 l x 01 587O8413ltx 02944 If the distribution of X has been exponential rather than nonnal then the situation would be as shown in Fig 161b and the probability that the random variable X exceeds the valuex 2 when fl 3 note when the median of an exponential distribution is 3 the mean is 433 is 1 0 1 2 pPX2 r1433X dx33 06301 h 433 Under He il 2 J2 JL289 Under H ji 2 a b 2 3 4 5 6 Under 3 Figure 11 Calcuation of 3for the sign test a Normal distributions b exponential distributions 496 Chzpter 16 Nonparametric Statistics The p error in this case is 2 121 f3 1I x J03699 0630112x 08794 x l1ms the 3 error for the sign test depends not only on the alternative value of l but on the area to the right of the value specified in the null hypo thesis under the population prob ability distribution This area is highly dependent on the sbape of that particular probabil ity distribution 1624 Comparison of the Sign Test and the tTest If the underlying population is normal then either the sign test or the ttest could be used to test Ho p flo 1be Hest is known to have the smallest value of 3 possible among all tests that have significance level ex so it is sUperior to the sign test in the normal distribution case When the population distribution is symmetric and nonnormal but with finite mean i P then the ttest will have a f3 error that is smaller than f3 for the sign test unless the distribu tion has very heavy tails compared with the nonnal Thus the SlgTl test is usually consid ered a test procedure for the median rather than a serious competitor for the ttesL The Wucoxon signed rank rest in the next section is preferable to the sign test and compares well with the test for symmetric distributions 163 THE WILCOXON SIGNED RANK TEST Suppose that we are willing to assume that the population of interest is continuous and sym metric As in the previous section our interest focuses on the medianJl or equivalently the mean fJ since f1 fL for symmetric distributions A disadvantage of the sign test in this sit uation is that it considers O11ly the signs of the deviations Xi flo and not their magnitudes The Wilcoxon signed rank test is designed to overcome that disadvantage 1631 A Description of the Test We are interested in testing He IJ Jlo against the usual alternatives LAssume that Xl X2 Xis a random sample from a continuous and symmetric distribution with mean and medi an fl Compute the differences X 11 i I 2 n Rank the absolilte differences XI L4 i L 2 n in ascending order and then give theranks the signs of their cor responding differences Let R be the sum of the positive ranks and R be the absolute value of the sum of the negative ranks and let R min R R Appendix Table XI con tains critical values of R say R If the alternative hypothesis is Hi i 11 then if R R the null hypothesis He i 11 is rejected For onesided tests if the altemative is Hj u Jiol reject Ho Ii J10 1 R R and if the alternative is HI i 11 reject Ho i 11 if R R The significance level for onesided tests is onehalf the advertised level in Appendix Table Xl 2FiiUli11 To illustrate the Wucoxon signed rank test consider the propellant shear strength data presented in Table 161 The signed ranks are 163 The WIlcoxon Signed Rank Test 497 Observation Difference XI 2000 Signed Ran 16 5350 11 4 130 2 1 15870 3 11 16520 4 18 20050 5 5 20750 i 7 21530 7 13 22020 8 15 23470 9 20 24630 10 O 25670 11 6 29170 12 3 31600 13 2 32185 14 14 133675 15 9 35790 16 12 39955 17 17 41440 18 8 57500 19 19 i5420 20 The sum of the positiveranks is R 1 2 3 4 5 611 13 15 16 7 18 19 20 150 and the sum of the negaive ranks is lC 7 8 9 10 12 14 60 Therefore R nilil P R nilil 150 60 60 Proll Appendix able Xl with n 20 and a 005 we find the critical value RJ1 52 Since R exceeds R we cannot reject the null uypothesls that the mean or median since the popUlations are assumed to be symmetric shear strength is 2000 psi Ties in the Wilcoxon Signed Rank Test Because the underlying population is continuous ties are theoretically impossible although bey will sometimes occur in practice If several observations have the same absolute mag nitude they are assigned the average of the ranks that they would receive if they differed slightly from one another 1632 A LargeSample Approximation If the sample size is moderately large say n 20 then it can be shown that R has approxi mately a normal distribution with mean and variance nn 1 4 nn 12n 1 24 498 Chapter 16 Nonparanetric Statistics Therefore a test of lIo J1 J1r can be based on the statistic R nn 14 Zo 00 jnn 12n 124 163 An appropriate critical region can be chosen from a table of the standard normal distribution 1633 Paired Observations The Wilcoxon signed rank est can be applied to paired data Let X Xj 12 n be a coLlection of paired observations from continuous distributions that differ only with respect to their means it is not necessal that the distributions of X and X be symmetric This assures that the distribution of the differences DJ X lj Xl is continuous and symmetric To use the WIlcoxon signed rank test the differences are fLSt ranked in ascending order of their absolute values and then the ranks are given the signs of the differences Ties are assigned average ranks Let K be the sum of the positive ranks and R be the absolute value of the sum of the negative ranks and let R min R R We reject the hypothesis of equal ity of means if R R where R is chosen from Appendix Table Xl For onesided tests lithe alternative is H P 110 or H Pn 0 rejectH if R R and if H P 110 or H PD OJ reject Ho if R R Note that the significance level for onesided tests is onehalf the value given in Table XI Consider th fuel metering device data examined in Example 162 The signed anks are shown below Cat Difference Signed RanI 7 02 1 12 03 2 8 04 3 6 05 4 2 06 5 4 07 65 5 07 65 1 08 8 9 09 9 10 10 10 11 l 1 3 13 12 Note that K 555 and It 225 thereforeRmin Irt10 min 555 225 225 FromAppcn dix Table Xl with It 12 and 1 005 we findhe critical value iJ 13 Since R exceeds 1 we cannot reject tbe null hypothesis that the two metering devices produce the same mileage performance 164 The Wilcoxon RankSum Test 499 1634 Comparison with the t Test Vlhen the underlying population is normal either the ttest or the Wilcoxon signed rank test can be used to test hypotheses about 1 The ttest is the best test in such situations in the sense that ir produces a minimum value of fJ for all tests with significance level a Howeverj since it is not always clear that the nonnal distribution is appropriate and since there are many situations in which we know it to be inappropriate it is of interest to com pare the rNO procedures for both normal and nonnormal populations Unfommatey such a comparison is not easy The problem is that 3 for the Vlilcoxon signed rank test L1i very difficult to obtain and 13 for the ttest is difficult to obtain for non normal distributions Because type II error comparisons are difficult other measures of comparison have been developed One widely used measure is asymptotic relative effi ciency ARE The ARE of one test relative to another is the Umting ratio of the sample sizes necessary to obtain identica error probabilities for the two procedures For example if the ARE of one test relative to a competitor is 05 then when sample sizes are large the first test will require a sample tlce as large as the second one to obtain similar error per formance While this does not tell us anything for small sample sizes we can say the following 1 For normal populations the ARE of the Wilcoxon signed rank test relative to the test is approximately 095 2 For nonnormal populations the ARE is at leab 086 and in many cases it will exceed unity VVben it exceeds unity the Wilcoxon signed rank test requires a smaller sample size than does the ttest Although these are largesample results we generally conclude that the Wilcoxon signed rank test will never be much worse than the ttest and in many cases where the pop ulation is nonnormal it may be superior Thus the Vlikoxon signed rank test is a useful alternative to the ttest 164 THE WILCOXON RANKSUM TEST Suppose that we have nvo independent continuous populations XJ andXz with means uj and p The distributions of X and X have the same shape and spread and differ only possibly in their means The Wilcoxon ranksur test can be used to test the hypothesis flo 11 fL Sometimes this procedure is called the MannWhitney test although the MallllVihitney test statistic is usually expressed in a different form 1641 A Description of the Test LetX11 X12 XII andX21Xn XoJ be two independent random samples from the con tinuous populations X andXj described earlier We assume that n l s 1t ATange all nJ n2 observations in ascending order of magnitude and assign ranks to them If two or more observations are tied identical then use the mean of the ranks that would have been assigned nthe observations differed Let R be the sum of the rmks in the smaller X sam ple and define 164 Now if the two means do not differ we would expect the surrt of the rmks to be nearly equal for both samples Consequently if the sums of the ranlcs differ geatly we would conclude that the means are not equal 500 Chapter 16 KonpararJetric Statistics Appendix Table IX contains the critical values R of the rank sums for 0 005 and a 00 I Refer to Appendix Table IX with the appropriate sample sizes n and Ilz The null hypothesis Ho Ill is rejected in favor of HI J11t JLif either R or R is less than or equal to the tabulated critical value R The procedure can also be used for onesided alcematives If the alternative is H j Jlt 1 then reject He if R R while for He 1 fJz1 reject 8 0 if Rz s R For these onesided tests the tabulated critical values R correspond to levels of significance of a 0025 and 0 0005 Exartlple16S The mean axial stress in tensile members used in an aircraft stucture is bei1g studied Two alloys are being investigated Alloy I is a traditional material and aloy 2 is a new aluminumlithium tilloy that is much lighter than the standard materiaL Ten specimens of each alloy type are tested and the axial stress measlXd The sample data are assembled in the following table Alloy 1 Alloy 2 3238 psi 3254 psi 3261 psi 3248 psi 3195 3229 3187 3215 3246 3225 3209 3226 3190 3217 3212 3240 3204 3241 3258 3234 The data fIe arranged in ascending ordet and ranked as follows Alloy Nutnber 2 2 2 2 2 2 1 2 2 2 2 Axial Stress 3187 psi 3190 3195 3204 3209 3212 3215 3217 3225 3226 3229 3234 3238 3240 3241 3246 3248 3254 3258 3261 Rank 2 3 4 5 6 7 g 9 10 11 2 13 14 15 6 17 18 19 20 r 165 Norparametric Methods in the Analysis ofVaiauce 501 The sum of the ranks for alloy 1 are R 2 3 4 S 9 11 3 15 16 IS 99 and for alloy 2 they are R2 n1nl L2 1 R j 1010 10 1 99 111 FromAppendix Table IX wit1 ftJ nl 10 and a 005 we fiad that Rc 78 Since neither RI nor is less than Rils we cannot reject the hypothesis that both alloys exhibit the Same mean axial stress 1642 A LargeSample Approximation When both nj and 1tz are mOderately large say greater than 8 the distribution of R j can be well approximated by the normal distribution with mean nn 1 1 IlR 2 and ariance n1nI OR 12 Therefore for nJ and nz 8 we could use z 165 as a test statistic and the appropriate critical region as 2 Zm z Z or Z Z dependiag on whether the test is a twotailed uppertail or lowertail test 1643 Comparison with the t Test In Section 1634 we discussed the comparison of the ttest with the Wilcoxon signed rank test The results for tle twosample problem are identical to the ooesample case that is when me normality assumption is correct me Wilcoxon ranksum testis approximately 95 as efficient as the ttest in large samples On the other band regardless of the form of the distributions the Wilcoxon ranksum test will always be at least 86 as efficientif the underlying distributions are very nonnormal The efficiency of the ViilClJxon test relative to me Itest is usually high if me underlying distribution has heavier tails thao the normal hecause the behavior of the ttest is very dependent on the sample me which is quite unstable in heavytailed distributions 165 NOPARA1ETRIC METHODS IN THE ANALYSIS OF VARIAiiCE 1651 The KruskalWallis Test The sitglefactor analysis of variance model developed in Chapter 12 for comparing a population means is t it2a Yij J1 1 ij 1 2 nl 166 502 Chapter 16 Nonpanunctric Statistics In this model the error tenns E are assumed to be normally and independently distributed with mean zero and variance 0 The assumption of normality led directly to the Ftest described in Chapter 12 The KruskalWallis test is a nonparametric alternative to the Ftest it requires only that the Eli have the same continuous distribution for all treatments j I 2 D Suppose that N Lln is the total number of observations Rank all N observations from smallest to largest and assign the smallest observation rank 1 the next smallest rank 2 and the largest observation ranklY If the null hypothesis Ho 1 l 1 is true theN obsenations come from the same distribution and all possible assignments of the 11 ranls to the a sIple are equally likely then we would expect the ranls I 2 11 to be miICed throughout the a samples If however the null hypothesis Ho is false then some samples will consist of observations having predominantly small ranks while other samples will consist of observations having predominantly large ranks Let Rj be the rank of observation Y and let R and denote the total and average of the ni ranks in the ith treatment Whenthe null hypothesis is true then and The KruskalWallis test statistic lI1eaSuteS the degree to which the actual observed average ranks RI differ from their expected value N 12lfthis difference is large then the oull hypothesis Ho is rejected The test statistic is K 12 111 n R NN1 2 167 An alternate computing formula is K 12 112 3111 NN I i n 168 We would usually prefer equation 168 to equation 167 as it involves the rank totals rather than the averages The null hypothesis He should be rejected if the sample data generate a large value fOr K The null distribution for K has been obtained by using the fact that under He each possi ble assignment of mlks to the a treatments is equaliy likely Thus we could enumerate all possible assignments and count the number of times each value of K occurs This has led to tables of the critical values of K although most tables are restricted to small sample sizes nf In practice We usually employ the following largesample approximation Vlhenever He is true and ei ther a 3 and n26 for i 1 2 3 or for i 1 2 G 165 Nonparametric Met100s IT the Analysis of Variance 503 then K has approximately a chisquare distribution with aI degrees of freedom Since large values of K imply that Ho is false we would reject He if Kil The test has approximate significance level a Tus in the Kruskal Vallis Test When observations are tied assign an average rank to each of the tied observations 1len there are ties we shQuld replace the test statistic in equation l68 with 169 where ni is the number of observations in the itb treatment N is the total number of obser vatioMand 52 Il C iIR lN 12 N 1 Id 11 4 1610 Note that f is just the variance of the ranks When the number of ties is moderate there will be little difference between equations 168 and 169 and the simpler form equation 168 may be used In Design and Analysis oj Experiments 5tl Edition John Weey SODS 201 0 C Montgomery presents data from an experiment in which five different levels of cotton content in a synetic Eber were tested to daermine if eottOn content has any effect on fiber tensile strength The sallple data and ranks from this experimert are shown in Table 13 Since here is a fairly large number of ties we use equation 169 as the test statistic From equation 16 10 we fud S2 fiiili NNl f lil Jl 4 1549779 2526 1 24 4 j 5303 Table 163 Data and Rans for he Tensile Testing Experiment Percentage of Cotton IS 20 25 30 35 7 20 12 95 14 110 19 205 7 20 7 20 17 140 18 165 25 250 10 50 IS 125 12 95 IS 165 22 230 11 70 11 70 18 165 19 205 19 205 15 125 9 40 18 165 19 205 23 240 11 70 lit 275 660 850 1130 335 504 Chapter 16 Nonparametric Statistics and the test Statistic is 1 a R2 NtNl21 K L S2liQ1 nJ 4 J lr 52450 2526 5303 4 1925 Silce K xc 1328 we would reject the nullhypoiliesis andcondude that treatnents differ This is the same conclusion given by the usual analysis of variance Fwtest 1652 The Rank Transfonnation The procedure used in the previous section of replacing the observations by their ranks is called the rank transformation It is a very powerful and widely useful technique If we were to apply the ordinary Ptest to the rarJs rather than to the orginal data we would obtain Kial NIKiNa as the test statistic Note that as the KruskalWallis statistic K increases or decreases F also increases or decreases so the KruskalWallis test is nearly equivalent to applying the usual analysis of variance to the ranks The rank transformation has wide applicability in experimental design problems for which no nonparametric alternative to the analysis of variance exists If the data are ranked and the ordinary F wtest applied an approximate procedure results but oue that has good sta tistical properties Villeo we are concerned about the normality assumption or the effect of outliers or wild values we recommend that the usual analysis of variance be perfotmed on both the original data and the ranks 11en both procedures give similar results the analysis of variance assumptions are probably satisfied reasonably well and the standard analysis is satisfactory Vihen the two procedures differ the rank tralllformation should be preferred since it is less likely to be distorted by nonnormality and unusual observations In such cases the experimenter may want to investigate the use of transformations for non normality and examine the data and the experimental procedure to detennine whether out liers are present and if so why they have occurred 166 S1ARY This chapter has introduced nonparametric or distributionfree statistical methods These procedures are alternatives to the usual parametric t and Ftests when the normality assumption for the underlying population is not satisfied The sign test can be used to test hPotheses about the median of a continuous distribution It can also be applied to paired observations The Wilcoxon signed rank test can be used to test hypotheses about the mean of a symmetric continuous distribution It can also be applied to paired observations The Wilcoxon signed rank test is a good alternative to the ttest The twosample hypothesis testing problem on means of continuous symmetric distributions is approached using the Wilcoxon ranksum test This procedure compares very favorably with the twosample Hest The KruskalWallis test is a useful alternative to the Ftestin the analysis of variance 167 EXERCISES 161 Ten samples were taken Emma plating bath used in an electronics manufacturing process and the bath pH determined The sample pH values are given below 791785682 8Ql 746 695 705 735 725742 Manufacturing engineering believes that pH has a median value of7 0 Do the sample claa indicate that this statement is correct Use the sign tes to investi gate this hypothesis 16 2 The titanium content in an aitcraftgTade ilioy is an important determinant of strength A sample of 20 test coupons reveals the following titanium contents in percent 832805893865825846852835 836841 8A2 830871875860883 850 838 829 846 The median titanium content should be 85 Use jre sign test to investigate this hypottesis 16 3 The distribution time between arivals in a telecommtDicauon system is exponential and the sys tem manager wishes to test the hypothesis that HG p 35lIlin versus HI p 35 min a hat is the value of the mean of the exponential distribution under Hij fl 35 0 Suppose that we have taken a sample of It 10 observations and we observe R 3 Would the sig1 test reject Ho at a 005 c What is the type n error of this test ifjl 457 164 Suppose that we take a sample of n 10 meas urementS from a normal distributiol with 1 1 We wish to test Ho J1 against HI J1 O The norrnal test statistic is x 0 arn and we decide to use a critical region of 196 hat is rejectRc if4 2 196 a What is ex for this test b What is j1fortlllstestifu I c If a sign tes is used specify the critical region that gives an a value consistelt with a for the nor nal test d What is the t value for the sign test if fJ 1 Compare this with the esult obed in part b 165 TWo different types of tips can be used in a Rockwell hardness tester Eight COUP0lS from test ingots of a nickelbased alloy are selected and each coupon is tested twice once with each tip The Rock well Csca1e hardness readings are shown next Use 167 Eercises 505 the sigc test to determine whether or not the CWO tips produce equivalent hardness readings Coupon Tip 1 Tip 2 1 63 60 2 52 51 3 58 56 4 60 59 5 55 58 6 57 54 7 53 52 8 59 61 166 Testing for Trends A turbocharger whee is manfactured ming an investTIent casting process The shaft fits into me wheel opening and this wheel opening is a critical dimension As wheel wax patterns are fanned the hard tool producing the wax patterns wears This may cause growth in the wheelopening dimension Ten wheelopening measurements in time order of production are shown below 400 mm 402 403 401 400 403 44 402 403 403 a Suppose that p is the probability that observation Xi5 exceeds observation X If t1ere is no upward or downward trend then X5 is no more or Iess likely to exceed XI or lie below Xt VVhat is the value ofp 0 Let V be he number of values of i for which Xi Xr If there is no upward or dovllward trend in the measurements what is the probability dis tribution of V c Use te data above and the results of parts a and b to test He there is 10 trend versus HI there is ipwatd trend Use aO05 Note that this test is a modification of the sign test It Was dc eloped by Cox and Suart 16 7 Consider the Wilcoxon signed rank test and suppose hat n 5 Assume t Ho jJ l1tJ is trJe a How many different sequences of signed rmks are pOSsible Errtrnerate these sequences b How ma1Y different values of R are there Find the probability tlSsociated with each value of R c Suppose that we define the critical region of the test to be R such that we would reject if R R and R 13 What is the approximate a level of this test d Can you see from this exercse how the critical values for the Vlilcoxon signed xank test were deveoped Explain 506 Chapter 16 Nonparametric Statistics 16 Considerhe data in Exercise 161 and asstllXte that the distribution of pH is symmetric and continu ous Use the Wilcoxon signed rank test to test the hyp1thesis Ho u 7 against H 1 7 169 Consider the data in Exercise 162 Suppose that the distribution of titanium content is symmetric and contiruous Use the Wilcoxon signed rank test to test the hypotheses Ho Jl 85 versus H Jl 5 1610 Consider t1e data in Exercise 162 Use the largesample approximation for the Wilcoxon signed rank test to test he hypotheses Ho J1 85 Versus H J1 85 Assume that he distribution of titanium con tent is continuos and symmetric Hill For the largesample approximation to the Wilcoxon signed rank test derive the mean and stan dard deviation of the test statistic used in t1e procedtlte 1612 Consider the Rockwell hardness test data in Exercise 165 Assume Lia bot1 distributions are continuous and use lle Vilcoxon signed rank test to rest that the mear difference iL hardress readings between the two tips is zero 1613 An electrical engineer must design a circuit to deliver the rlaximum amount of current to a display rube to achieve sufficient image brightness Within rJs allowable design constraints he has developed tNO candidate circuits and tests prototypes of each The resulting data in microamperes is shown below Circuit 1 25 255 258 257 250 251 254 2S0 248 Circuit 2 250 253 249 256 259 252 250 251 Use he Wilcoxon rmsum test 10 test Ho J1 u1 against the alternative HI J11 u1 1614 A consultant frequently travels from Phoenix Arizona to Los Angeles California He will use one of two airlires Unied or Southwest The number of minutes that his flight arrived late for the last six trips on each airlbe is sboVll belOW Is there evi dence that either airlile has superior ontime arrival perfonl3lce1 United 2 19 2 8 0 minutes late Southwest 20 4 8 8 3 5 minues late li15 The manufacrurer of a bot tub is interested in testing twO different heating elements fOf his product The element that produces the maximum heat gain after 15 rcinutes would be preferable He obtains 10 samples of each heating unit and tests each one The heat gain after 15 minutes in F is shown below Is there any rea son to suspect that one unit is superior to the other Unit I 25 27 29 31 30 26 24 32 33 38 Unit 2 31 33 32 35 34 29 38 35 37 30 1616 In Design and halysis 0 Experiments 5th Edition John Wiley Sons 2001 D C Mont gomery presents the results of an experiment to com pare four different niting techrUques on the tensile strength of portland cement The results are shoVn below Is tere any indication that rnixing echnique affects tlJe stength Mixing Technique Tensile Stength lb i In 1 3129 3000 2865 2890 2 3200 3000 2975 3150 3 2800 2900 2985 3050 4 2600 2700 2600 2765 1617 An article in the Quality Control Handbook Srd Edition McGrawHill 1962 presents the results of an experiment perfonned to investigate tle effect of thee differert conditioning methods on the brealirg strength of cement briquettes Tne data are shown below Is there any indication that conditioning method affects breaking strength Conditioning Method Brealdng Ib i in 1 553 550 568 541 537 2 553 599 579 545 540 3 492 530 528 510 571 161S In Statisticsor Research John Wiley Sons 1983 S Dowdy and We3rden present the results of an experiment to measure stress resulting from oper ating handheld chain saws The experimenters meas ured the kickback angle through which the saw is deflected when it begins to cut a 3inch stock syn thetic board Shown below are deflection angles for five saws chosen at random from each of four differ eut manufacturers Is there any evidence that the man ufacturers products Ciffcr with respect to kickback argle llanufucturer Kickback Angle A 42 17 24 39 43 B 28 50 44 32 61 C 57 45 48 41 54 D 29 40 22 34 30 Chapter 17 Statistical Quality Control and Reliability Engineering The quality of the products and services used by our society has become a major consumer decision factor in many if not most businesses today Regardless of whether the consumer is an individual a corporation a military defense program or a retail store the consumer is likely to consider quality of equal importance to cost and schedule Consequently qual ity improvement has become a major concern of many US corporations This chapter is about statistieal quality control and reliability engineering methods two sets of tools that are essential in qualityimprovement activities 171 QlJALITY IMPROVEMENT ArD STATISTICS Quality means fitness for use For example we may purcbase automobiles that we expect to be free of manofactering defects and that should provide reliable and economieal trans portation a retailer buys finished goods with the expectation that they are properly pack aged and arranged for easy storage and display or a manufacturer buys raw material and expeets to process it with minimal rework Or scrap In other words all consumers expect that the products and serviees they buy will meet their requirements and those requirements define fitness for use Quality or fitness for use is detemtined through the ilJternetion of quality of design and quality of conformance By quality of design we mean the different grades or levels of per formance reliability serviceability and function that are the result of deliberate engineer ing and management decisions By quality of conformance we mean the systematic reduction of variability and elimination of defects until every unit produced is identical and defect free There is some confusion in our SOCiety about qlJt1lity improvement some people still think that it means gold plating a product or spending more money to develop a product or process This thinking is Tong Quality improvement means the systematic elimination of waste Examples of waste include serap and rework in manufacturing inspection and test errors on documents such as engineering drawings checks purchase orders and plans customer complaiDt hotlines1 warranty costs and the time required to do things over again that could have been done right the first time A successful qualityimprovement effort can eliminate much of this waste and lead to lower costs higher productivity increased cus tomer satisfaction increased business reputation higher market share and ultimately bigher profits for the company 507 508 Chapter 17 Statistical Quality Control and Reliability Engineering Statistical methods playa vital role in quality improvement Some applications include the following 1 In product design and development statistical methods including designed exper iments can be used to compare difterent materials and different components or ingredients1 and to help in both system and component tolerance determination This can significantly lower development costs and reduce development time 2 Statistical methods can be used to determine the capability of a manufacturing process Statistical process control can be used to systematically improve a process by reduction of variability 3 Experiment design methods can be used to investigate improvements in the process These improvements can lead to higher yields and lower manufacturing costs 4 Life testing provides reliability and other performance data about the product This can lead to new and improved designs and products that have longer useful lives and lower operating and maintenance costs Some of these applications have been illustrated in earlier chapters of this book It is essen tial that engineers and managers have an indepth understanding of these statistical tools in any industry or business that wants to be the bighquality lowcost producer In this chap ter we give an introduction to the basic methods of statistical quality control and reliability engineering that along with experimental design form the basis of a successful quality improvement effolt 172 STATISTICAL QUALITY CONTROL The field of statistical quality control can be broadly defined as consisting of those statistical and engineering methods useful in the measurement monilorng contro and improvement of quality In this chapter a somewhat more narrow definition is employed We will define statistical quality control as the statistical and engineering methods for process controL Statistical quality control is a relatively new field dating back to the 19205 Dr Walter A Shewbart of the Bell Telephone Laboratories was One of the early pioneers of the field In 1924 he TOte a memorandum shoViing a modem oontrol chart one of the basic tools of statistical process control Harold F Dodge and Harry G Romig two other Bell System employees provided mucb of the leadership in the development of statistically based sam pling and inspection methods The work of these three men forms the basis of the modem field of statistical quality controL World War II saw the widespread introduction of these methods to US icdustry Dr W Edwards Deming and Dr Joseph M Juran bave been instrumental in spreading statistical qualitycontrol methods since World War II The Japanese have been particularly successful in deploying statistieal qualitycontrol methods and have used statistical methods to gain significant advantage relative to their competitors In the 1970 American industry suffered extensively from Japanese and other foreign competition and that has led in tum to renewed interest in statistical quality control methods in the United States Mucb of this interest focuses on statistical process control and expenmental design Many US companies have begun extensive programs to implement these methods into their manufacturing engineering and other business organizations 173 STATISTICAL PROCESS CONTROL It is impossible to inspect quality into a product the product must be built right the first time This implies that the manufacturing process must be stable or repeatable and capable 173 Statistical Process CenITol 509 of operating with little variability around the target or nominal dimension Online statisti cal process controls are powerful tools useful in achieving process stability and improving capability through the reduction of variability It is customary to think of statistical process control SPC as a set of problemsolving tools that may be applied to any process The major IDols of SPC are the following 1 Histogram 2 Pareto chart 3 Causeandeffect diagram 4 Defectconcentration diagram 5 Control chart 6 Scatter diagram 7 Check sheet While these tools are an important part of SPC they really constitute only the technical aspect of the subject SPC is an attitudea desire of all individuals in the organization for continuous improvement in quality and productivity by the systematic reduction of vari ability The control chart is the most powerful of the SPC tools We now give an introduc tion to several basic types of control charts 1731 Introduction to Control Charts The basic theory of the control chart was developed by Walter Shewhart in the 1920s To understand how a control chart works we must first understand Shewharfs theory of vari ation Shewhart theorized t1at all processes however good are characterized by a certain amount of variation if we measure with an instrument of sufficient resolution When this variabilicy is confined to random or chance variation ouly the process is said to be in a state of statistical control However another situation may exist in which the process variability is also affected by some assignable cause such as a faulty machine setting operator error unsatisfactory raw material worn machine components and sOonl These assignable causes of variation usually have an adverse effect on product quality so it is important to have some systematic technique for detecting serious departures from a state of statistical control as soon after thy occur as possible Control charts are principally used for this pur pose The power of the control chart lies in its ability to distinguish assignable causes from random variation It is the job of the individual using the control chart to identify the under lying root Cause responsible for the outofcontrol conditio develop and implement an appropriate corrective action and then follow up to ensure that the assignable cause has been eliminated from the process There are three points to remember 1 A state of statistical control is not a natural state for wost processes 2 The attentive use of control charts will result in the elimination of assignable causes yielding an incontrol process and reduced process variability 3 The control chart is ineffective without the system to develop and implement cor rective actions that attack the root causes of problems Management and enginee ing involvement is usually necessary to accomplish this ISooetimes contnum cJuse is usedinsreld of random or chAnce cause and special cause is used ifIstead of assignabe cause 510 Chapter 17 Statistical QUality Control and Reliability Engineering We distinguisb between control charts for measurements and control charts for attrib utes depending on whether the observations on the quality characteristic are measurements or enumeration data For example we may choose to measure the diameter of a shaft say with a micrometer and utilize these data in conjunction with a control chart for measure ments On the oilier hand we may judge each unit of product as either defective or nonde fetive and use the fraction of defective units found or the total number of defects in conjunction with a control chart for attributes Obviously certain products and quality char acteristics lend themselves to analysis by either method and a clearcut choice between the two methods may be difficult A control chart whetberfor measurements or attributes consists of a cemerline corre sponding to the average quallty at which the process should perform when statistical ontrol is exhibited and two control Ujts called the upper and lower control limits UCL and LCL A typical control chartls shown in Fig 171 The control limits are chosen so that val ues falling between them can he attributed to chance variation while values falling beyond them can be taken to indicate a lack of statistical conrroL The general approach consists of periodically taking a random sample from the process computing some appropriate quan tity and plotting that quantity on the control chart When a sample value falls outside the control limits we search for some assignable cause of variation However even if a sample value falls between the conrrollimits a trend Or some other systematic pattern may indicate that some action is necessary usually to avoid more serious trouble The samples should be selected in such a way that each sample is as homogeneous as possible and at the same time maximizes the opporturUty for variation due to an assignable cause to be present This is usu ally called the ratiolllll subgroup concept Order of production and source if more than one source exists are commonly used bases forobtaining rational subgroups The ability to interpret control charts accurately is usually acquired with experience It is necessary that the user he thoroughly familiar with both the statistical foundation of con trol charts and the nature of the production process itself 1732 Control Charts for Measurements When dealing with a quality characteristic that can be expressed as a measurement it is cus tomary to exercise control over both the average value of the qUality characteristic and its variability Control over the average quality is exercised by the control chart for means usu ally called the X chat Process variability can be controlled by either a range R chat or a Upper control limit UCL Sample number Figure 171 A typical control chart 173 Statistico1 Process Control 511 standard deviation chart depending on how the population standard deviation is estImated We will discuss only the R chart Suppose that the process mean and standard deviation say p and a are known and furthermore that we can assume that the quality characteristic follows the normal distribu tion LetXbe the sample mean based on a random sample of size n from this process Then the probability is 1 IX that the mean of such random samples wn fall between flZa2aJi and I1Zl2a Therefore we could use these two values as the upper and lower control limits respectively However we usually do not know f1 and a and they must be estimated In addition we may not be able to make the normality assumption For these reasons the probability limit 1 a is seldom used in practice Usually Zan is replaCed by 3 and 3sigma control limits are used Wben J1 and a are unknown we often estimate them On the basis of preliminary sam pIes taken when the process is thought to be in control We recommend the use of at least 2025 preliminary samples Suppose k preliminary samples are aIlilable each of size n Typically n will be 45 Or 6 these relatively small sampie sizes are widely used and often arise from the COnstruction of rational subgroups Let the sample mean for the ith sample be Xi Then we estimate the mean of the population j by the grand mean l X ILX 1 171 Thus we may take X as the centerline On the X control chart We may estimate afrom either the standard deviations or the ranges of the k samples Since it is more frequently used in practice we confine Our discussion to the range metbod The sample size is relalively small so there is little loss in efficiency in estimating afrorn the sample ranges The relationship between the range R of a sample from a normal pop ulation with known parameters and the standard deviation of that population is needed Since R is a random variable the quantity W RIO called the relative range is also a ran dom ariable The parameters of the distribution of W have been determined for any sam ple size n The mean of the distribution of W is called d and a table of d for various n is given in Table XllI of the Appendix Let R be the range of the ith sample and let 1 k R 211 k IL be the average range Then an estimate of j would be R 0 d Therefore we may use as our upper and lower control limits for the X chart We note that the quantity 3 UCL X F R d2n LCLX3 J 17 2 73 174 512 Chapte 17 Statistical Quality Control and Reliability Eugineering is a constant depending on the sample size so it is possible to rewrite equations 174 as UCL XAR 175 LCL X AJ The constant A is tabulated for various sample sizes in Table XIII of the Appendix The parameters of the R chart may also be easily determined The centerline will obvi ously be R To determine the control limits we need an estimate of jR the standard devia tion of R Once again assuming the process is in control the distribution of the relative range W will be useful The standard deviation of W say jWI is a function of n which has been determined Thus since RWO we may obtain the standard deiation of R as jR jlj As a is unknown we may estimate OR as and we would use as the upper and lower control limits on the R chart UCL R 30w R d2 LCLR 30w R d Setting D3 1 30w1d2 and D4 1 30w1d2 we may rewrite equation 176 as UCL D4R LCLDR where D and D4 are tabulated in Table XIII of the Appendix 176 177 When preliminary samples are used to construct limits for control charts it is cus tomary to treat these limits as trial values Therefore the k sample means and ranges should be plotted on the appropriate charts and any points that exceed the control limits should be investigated If assignable causes for these points are discovered they should be eliminated and new limits for thecontrel charts determined fn this way the process may eventually be brought into statistical control and its inherent capabilities assessed Other changes in process centering and dispersion may then be contemplated Exaxnpie171 A component part for a jet aircraft engine is manufactured by ax investment casting process The vane opening on this casting is an imporant functiOIlal Pararleer of the part We will illustrate the use of X ld R control chatS to assess the statistical stability of this process Table 171 presents 20 samples of five parts each The values given in the table have been coded by using the last three digits of the dimension that is 316 should be 050316 inch The quantities X 3333 ad R 585 are shown at the foot of Table 17L Notice that even though X X R and R are now realizations of random variables we have still Vitten them as 173 Statistica Process Control 513 TabJe 171 Vane Opening Measurenents Sample Number x x x X R 33 29 32 33 316 4 0 2 35 33 31 37 31 332 6 3 35 37 33 34 36 350 4 4 30 31 33 34 33 322 4 5 33 34 35 33 34 338 2 6 38 37 39 40 38 384 3 7 30 31 32 34 31 316 4 8 29 39 38 39 39 368 10 9 28 34 35 36 43 352 15 10 39 33 32 34 32 340 7 11 28 30 28 32 31 29S L 12 31 3S 35 35 34 340 4 13 27 32 34 35 37 330 10 14 33 33 35 37 36 348 4 15 35 37 32 35 39 356 7 16 33 33 27 31 30 308 6 17 35 34 34 30 32 330 5 18 32 33 30 30 33 316 3 19 25 27 34 27 28 282 9 20 35 35 36 33 30 338 6 X3333 R585 uppercase letters This is the usual convention in quality control and it will always be clear from the context what the notation implies The trial concrollimits are for the X cbart X AR 333 0577585 3333 337 or UCL 3670 LCL2996 For the R chart the trial cortrollimits are UCL DR 2115585 1237 LCL DR 0585 0 The X aodR cootrol charts 1tb these trial cootrollimit are shoNU in Fig 172 Notice that salples 6 S 11 and 19 are out of cOlltrol on the X chart and that sample 9 is out of control on the R chart Suppose that all of these assigrablc causes can be traced to a defective tool io the waxmolding area We should discard these lve samples and recompute the limits for the X and R charnL These new revised limits are fur the X chart and for ce R chart they are UCL X AJ 3290 05775313 3596 LCL X 3290 05775313 2984 UCL 21155067 1O7l LCL 05067 0 514 Chapter 17 Statistical Quality ContrQI and Reliability Engineering c I iii 3S E o UCL 3670 Mean 3333 J ILCL i 1O 2O Sample number 2996 m 0 C l c E M j o 1S 10 S O 10 Sample number 20 UCL 1237 LCLO Figure 17M2 The X and R control charts for vane opening The revised control charts are shOWTl in Fig 173 otice that we have treated the firs 20 preliminary samples S estimation data with which to establish control limits These limits can now be used to judge the statistical control offuturc production As each l1eW sample becomes available the values ofX and R should be cotUputed and plotted on te cOIltrol charts It may be desirable to revise the lim hs periodically even if the process rCJllJlins stable The limits should alvrays be revised when process improvements are made Estimating Process Capability It is usually necessary to obtain some information about the capability of the proc that is about the performance of the process when it is operating in control Two grapbical tools the tolerance chart or tier chart and the histogram are helpful in assessing process capa bility The tolerance chart for all 20 samples from the vane manufacturing process is shovll in Fig 174 The specifications on vane opening 05030 0001 inch are also shown on the chart In terms of the coded data the upper specification limit is USL 40 and the lower specification limit is LSL 20 The tolerance chart is useful in revealing pattems oyer time in the individual measurements or it may show that a particular value niX or R was pro duced by one or two unusual observations in the sample For example note the tWO unusual ObserY3tions in sample 9 and the single unusual observation in sample 8 Note also that it 40r S 35 w c g m I I 0 10 Subgroup 15 m 10 e c E J 0 10 Subgroup 8 Not used in computing control ilmfs 17 3 Statistica Process Control 515 i UCL3596 L 1 f VS LCL2984 20 UCL 1071 R5C67 LCLC 20 Figure 173 X ad R control charts fOT vane openi1g revised limits 40 USL40 r Lit ItT 11 35 ITI I i 0 30 Nominal ID C ID dirrension 30 25 20 LSL Sample rumber Figure 17 4 Tolerance diagram of vane openings 516 Chapter 17 Statistical Quality Contro and Raliability Engineerillg is appropriate to plot the specification limits on the tolerance chart since it is a chart of indi vidual measurements It is never appropriate to plot specijicationlimits on a control chart Or 10 use the specifications in determining the control limits Specification limits and con trollimits are unrelated FInally note from Fig 174 that the process is running off center from the nonrillal dimension of 05030 inch The histogram for the vane opening measurements is shown in Fig 175 The obser vations from samples 6 8 9 II and 19 have been deleted from his histogram The gen era impression from examining this histogram is that the process is capable of meeting the specifications but that it is running off center Another way to express process capability is in terms of the process capability ratiQ PCR defined as peR USLLSL 60 178 Notice that the 60 spread 30 on either side of the mean is sometimes called the basic capability of the process The limits 3a on either side of the process mean are sometimes called natural tolerance limits as these represent limits that an incontrol process should meet with most of tIle units produced For the vane opening we could estimate aM J R 206 d 2326 Therefore an estimator for the peR is peR USLLSL 60 4020 6206 162 20 15 5 o rHrTyrHrryrrl 18 20 22 24 26 28 30 32 34 36 38 40 42 44 LSL t USL Ncmina dimension Vane opening 173 Statistical Process Contral 517 The PCR has a natural intetpretation IPCRIOO is just the percentage of the toler ance band used by the process Thus the vane opening process uses approximately 11162100 617 of the tolerance band Figure 176a shows a process for which the peR exceeds unity Since the process nat ural tolerance limits lie inside the specifications very few defective or nonconforming units will be produced If PCR I as shown in Fig 176b more nonconforming units result In fact for a normally distributed process if peR 1 the fraction nonconforming Is 027 or 2700 parts per million Finally when the PCR is less than mity as in Fig 176 the process is very yield sensitive and a large number of nonconforming U1l3 will be produced The definition of the peR given in equation 178 implicitly assumes that the process is centered at the nominal dimension If the process is running off center itS actual capa biliry will be less than that indicated by the peR It is convenient to think of PCR as a meas ure of potential capability that is capability with a centered process If the process is not centered then a measure of actUal capability is giVe1 by C D r USLX X LSL 1 P k romi J 179 31 31 cLc PCR 1 LSL L3U I 3uJ USL NonconformIng units LSL Nonconforming units al b PCR Nonconforming unilS USL lSL 3T13u e Figure 176 Process fallout and the process capability ratio peR 518 Chapter 17 StatistiC Qa1ity Cootrol and Reliability Engineering In effect peRk is a onesided process capability ratio that is calculated relative to the spec ification limit nearest to the process mean For the vane opening process we find that PCRt minUSLX X LSLJ 30 30 min403319 110 331920 2131 3206 3206 j Note that if PCR PCR the process is centered at the nominal dimension Since PCR 11 0 for the vane opening process and PCR 162 the process is obviously running off center as was first noted in Figs 174 and 175 This offcenter operation was ultimately traced to an oversized wax tOoL Changing the tooling resulted in a substantial improvement in the process Montgomery 2001 provides guidelines on appropriate values of the peR and a table relating fallout for a normally distributed process in statistical control as a function of peR Many US companies USe PCR 133 as a minimum acceptable target and PCR 166 as a minillUIn target for strength safety or critical characteristics Also some Us compaM rues particularly the auomobile industry have adopted the Japanese terminology Cp PCR and Cpk peRk As Cp has another meaning in statistics in multiple regression see Chap ter 15 we prefer the traditional notation PCR and PCR 1133 Control Charts for Individual Measurements Many situations exist in which the sample consists of a single observation that is n L These situations occur when production is very slow or costly and itis impractical to allow the sample size to be greater than one Other cases include processes where every observa tion can be measured due to automated inspection for example The Shewhart control chartjor individual measurements is appropriate for this type of situation We will see later in this chapter that the exponentially weighted moving average control cbart and the cumu lative sum control chart may be more informative than the individual chart The Shewhart control chart uses the moving range MR of two successive observations for estimating the process variability The moving range is defined For example for m observations m I moving ranges are calculated as MR lx XII MR3 lx x1 MRm Ixm xll Simultaneous control chans can be established on the individual observations and on the moving range The control limits for the individuals control chart are calculated as UCL x3 MR d 2 Centerline X LCL MR xi where MR is the saJ1ple mean of the MRi 1710 l I 173 Statisticul Process Control 519 If a moving range of size n 2 is used then d2 1128 from Table XIII of the Appen dix The control limits for the moving range control chart are UCL D4MR Centerline MR LCLDMR 1711 Batches of a particular chemical product are seected from a proess ard the purity measured on each Daa for 15 successive bathes have been collected mld are given in Table 172 The moving ranges of size n 2are also displayed in Table 172 To Set up the control chart for individuals we t need the sample average of the 15 purity measurements This average is found to bex 0357 The average of the moving ranges of two obser vations is MR 0046 The control limits for the individuals chart with moving angcs of size 2 using me linritsin equation 17lQ are UCL 0757 3 0046 0879 1128 Centerline 0757 LCL 0757 3 0046 0635 1128 The control limits forhe movingrange chart ae found using the Emits giver in Equation 1711 UCL 32670046 0150 Centerline 0046 LCL 00046 O Table172 Purity of Olernical Product Batch Moving Range ill 077 2 076 00 3 077 001 4 072 005 5 073 om 6 073 000 7 085 012 8 070 015 9 075 005 10 074 om 11 075 am 12 084 009 13 079 005 14 072 007 15 074 002 70757 MRO06 520 Chapter 17 Statisticil Quality Control and Reliability Engineering 09rlUCL0879 t Mea 0757 LLCL 0635 06 0 5 18 15 Subgroup a 015 rl UCL Q150 010 r i 005 r R 0046 000 bL J1 LCL 0 o 5 10 15 Subgroup b Figure 177 Control charts for a the individual observations and b the moving range ap puity The control charts for individual observations and for the moving range are provided in Fig 111 Since there are no pointz beyond the controllirojts the process appears to be in statistical canuoI The individuals chart can be interpreted much like the X control chan An outofontrol situa rion would be indicated by either a poiu or points plotting beyond the controllimis or a pattern such as a run on one side of the centerline The moving range dwt cannot be itterpreted in the same way Although a point orp0lnts plot ling beyond the control limits would likely indicate al outofcontrolsituation a parem or run on one side of the centerline is not necessarily an indicaion that the process is out of connoL This is due to the fact that the moving ranges are correlated and this cone1ationlll3Y natu13lly cause patterns or trends on the chart 1734 Control Charts for Attributes The p Chart Fraction Defective or Nonconforming Often it is desirable to classify a product as either defective or nondefectlve on the basis of comparison with a standard This IS usually done to achieve economy and simplicity in the inspection operation For example the diameter of a ball bearing may be checked by deter mining whether it will pass through a gauge consisting of circular holes cut in a template This would be much simpler than measuring the diameter with a micrometet Control charts for attributes are used in these situations However attribute control charts require a con siderably larger sample size than do their measurements counterparts Vie will discuss the fractiondefective chart or p chart and two chartS for defects the c and u chartS Note that it is possible for a unit to have many defects and be either defective or nondefective In some applications a unit can have several defects yet he classified as nondefective 173 StatjstiaJ Process Control 521 SUFPose D is the number of defective units in a random sample of size n We assume that D is a binomial random variable with unknown parameter p Now the sample fraction defective is an estimator of p that is D 1712 n Furthennore the variance of the statistic p is pl p n so we may estimate a as 1 P 1713 n The centerline and control limits for the fractiondefective control chart may now be easily determined Suppose k preliminary samples are available each of size n andD is the number of defectives in the ith sample Then we may take as the centerline and i I UCLP3 P LeL p3 IPl p n 1714 1715 as the upper and lower control limits respectively These control limits are based On the normal approximation to the binomial distribution When p is sman the normal approxi mation may not always be adequate In such cases it is best to use control limlts obtained directly from a table ofbinornial probabilities or perhaps from the Poisson approximation to the binomial distribution If p is small the lower control limit may be a negative number If this should occur it is customazy to consider zero as the lower control limit I1IiZ Suppose we wish to construct a ractiondefectiye control char for a ceramic substrate pro duction line We have 20 preliminary samples each of size 100 the number of defectives in each sample are shown in Table 173 Assume that the samples are numbered in the sequence of production Note that Ii 80012000 OAO and therefore the trial parameters for the control chart are Centerline 0395 UCL 0395 3 03950605 100 05417 LCLO39530395O605 100 02483 522 Chapter 17 Statistical QUality Control and Reliability Engineering fibl173 NuUher of Defectives in S3tI1pies of 100 Cerar1ic Substrates Sample No of Defectives Sample No of Defectives 1 44 11 36 2 48 12 52 3 32 13 35 4 50 14 41 5 29 15 42 6 31 16 30 7 46 17 46 8 52 18 38 9 44 19 26 10 38 20 30 The control chart is shoVrTI in Fig 178 All samples are in control If they were not we would search for assignable causes of variation and revise the limits accordingly Although this process exhibits statistical conlrol its capability 0 0395 is very poor We should take appropriate steps to investigate the process to detenrJne why such a large number of defective units are being produced Defective units should be analyzed to determine the specific types of defects present Once the defeet types are known process changes should be investigated to determine their impact on defeet levels Designed exper iments may be useful in this regard ElIi74 Attributes Versus Measuremenl Control Charts Tae advantage of measurement control charts relarlve to the p chart With respect to size of sample may be easily illustrated Suppose that a DOnnally distributed quality characteristic has a standard devia tion of 4 ad specification limits of 52 and 68 The process is centered at 60 which results iri a frnc tiOD defective of 00454 Let the process mean shift to 56 ow the fraction defective is 01601 If the OSS C 1 1 UCL 05417 1045 035 025 o 10 Sample number Figure 178 The p chart for a ceramic substrate LCL 02483 20 173 Statistical Process Control 523 probability of dtecting the shi on he first sample following the mift is to be 050 hen the sample size mus be such tlat he lower 3sigma limit will be at 56 This implies whose sOlution is It 9 For a p chart using the no approximation to the binomial we must have 0045430045409546 0160L whose solution is n 30 Thus unless the cost of measurement inspection is more than three times ali costly as the attributes inspection the measurement control chart is cheaper to operate The c Chart Derects In some situations it may be necessary to control the number of defects in a unit of product rather than the fraction defective In these situations we may use the control chart for defects or the c chart Suppose that in the production of cloth it is necessary to control the number of defects per yard or that in asembling an aircraft wing the number of missing rivets must be controlled Many defectsperunit situations can be modeled by the Poisson dismbution Let c be the numher of defects in a unit where c is a Poisson random variable with parameter a Now the mean and vaiance of this distribution are both ex Therefore if k unit are availahIe and c is the number of defects in unit i the centerline of the control chart is and 1 k cLc K il UCLcW LCLc 3 are the upper and lower control limits respectively 1716 1717 Printed circuit boards are assembled by a combinaon of manual assembly and automation A flow soldr machine is used to make the mechanical and elecrrical connections of the leaded components to the board The boards are run thtough the flow solder process almost continuously and every bour five boards are selected ad ilSpectcd for processcontrol purposes The number of defects l each sample of five boards is noted Results for 20 samples are shown in Table 74 Now c 160120 8 and ttlerefore UCL83S 16414 LCL S 38 0 set to 0 Fom lite control chart io Fig 179 we see that the process is in controL However eight defects per group of five printed circuit boards is too many about 85 16 defectslboord and the process needs improvement An investigation needs to be made of the specific types of defects found on the printed circuit boards This will usually suggest POteJltial avemes for process improvement 524 Chapter 17 Statistical Quality Control and Reliability Engineering Table 17 4 Number of Defects in Samples of Five Printed Circuit Boards Sample No of Defects Sample lo of Defects 6 11 9 2 4 12 15 3 8 13 8 4 10 14 10 5 9 15 8 6 12 16 2 7 16 17 7 g 2 18 1 9 3 19 7 10 13 20 13 20 UCL 16484 c r1 J w 10 0 0 E z 0 Sample number Figure 179 The c chart for defeCts in samples of five prirred circuit boards The u Chart Defects per Unit In some processes it may be preferable to work with the number of defects per unit rather than the total number of defects Thus if the sample consists of n units and there are c total defects in the sample then c un is the average number of defects per unit A chart may be constructed for such data If there are k preliminary samples each with 11 Uz Uk defects per unit then the center line on the u chart is l718 and the control limits are given by 1719 173 Statistical Process Conl 525 A u chart may be constructed for the printed clrcuir board defect data in Exampe 175 Since each sample contains r 5 printed circuit boards the values of u for eacb sample may be calculated as shown in the following display Sample Sample size n Number of defects c Defects per unit 5 6 2 5 4 3 5 S 4 5 10 5 5 9 6 5 12 7 5 16 8 5 2 9 5 3 1O 5 10 11 5 9 12 5 15 13 5 8 14 5 10 15 5 8 16 5 2 17 5 7 3 5 19 5 7 20 5 13 The centerline for the u chart is 1 20 U2 20 1 32 16 and the upper and lower control limits are 117 f6 UCLu3 1635 33 LCLu 3 f 10i50 set to O 1n 12 03 16 20 18 24 32 04 06 20 18 30 16 20 16 04 14 02 L4 26 The control char is plotted in Fig 171O Notice that the u chart in this example is equivalent to the c chart in Fig 17 9 In some cases particuLtrly when the sle size is lot constant the u chart will be preferable to the c chart For a disc1ssioc of vaiable sample sizes on control charts see Mont gomery 2001 1735 CUSLM and EWMA Control Charts Up to this point in Chapter 17 we have presented the most basic of control charts the Shewbart control cham A major disadvantage of these control cham is their insensitivity to small shifts in the process shifts often less than 150 This disadvantage is due to the fact that the Shewhart charts use information only from the current observation 526 Chapter 17 Sttistical Quality Control and Reliability Engineering w 1 ill 1 r i 0 0 10 Sample number 20 Figure 1710 The u chart of defects peUDit on printed circuit boards Example 17wo Alternatives to Shewhart eontrol charts include the cumulative sum control chart and the exponentially weighted moving average control chart These control charts are more sensitive to small shifts in the process because they incorporate i1formation from current and recent past observations Tabular CUStJM Control Cilarts for lb Process Mean The cumulative sum CUSUM control chart was first introduced by Page 1954 and incor porates information from a sequenee of sample observations The chart plots the cumula tive SU of deviations of the observations from a target value To illustrate let Xj represent the jthsample mean let J1J represent the target value for the process mean and say the sam ple size is n 1 The CUSUM control ebllrt plots the quantity n Ci Ilxj J1e 1720 jl against the sample i The quantity C is the cumulative sum up to and including the ith sam pIe As long as the process is in control at the target value flo then Ci in equation 1720 rep resents a random walk with a mean of zero On the other hand if the process shifts away from the target mean then either an upward or downward drift in Ci will be evident By incorporating information from a sequence of observations the CUSUM chart is able to detect a small shifi in the process more quickly than a standard Shewhart chan The CUSUM eharts Can be easily implemented for both subgroup data and individual observa tions We will present the tabular CUSUM for indiyidual observations The tabular CUSUM involves tvlo statistics c and C which are the accumulation of deviations above and below the target mean respectively C is called the onesided upper CUSrM and c is called the onesided lower CUSUM The statistics are eomputed as follows 1721 1722 Nith initial values of C CO O The eonstant K is referred to as the reference value and is often chosen approximately halfway between the target mean 4 and the outofcontrol 173 Statistical Process Control 527 mean that we are interested in detecting denoted Jll In other words K is half oftbe mag nitude of the shift from 110 10 11 or The Sllltistics given in equations 1721 and 1722 accumulate the deviations from target that are larger than K and reset to zero when either qmutity becomes negative The CrSUM control chart plots the values of C and C for each sample If either statistic plots beyond the decision interval H the process is considered out of controL We will discuss the choice of H later in this cher but a good rule of thumb is often II 5a A study presented in Food Control 200t p 119 gives the results of measuring the drymatter cortent in buttercream from a batch process One goal of t1e study is to monitor the amount of Cry matter from batch to batch Table 175 displays some data hatrnay be tpical of this type of process Th reported values Xi are percentage of drylllater content examinee after mixing The target amount of drymat ter cortent is 45 ald assnme that a 04 Let us also assume t1at weare interested in detectirg a shift in the process of mean of at least 10 that is J11 IJ 1045 1084 4584 We will use the Table 175 CUSUM Calculations for Example 177 Batch i x xi 4542 4458 4621 079 079 163 0 2 4573 031 110 115 0 3 4437 105 005 021 021 4 4419 123 0 039 060 5 4373 169 0 085 145 6 4566 024 024 108 037 7 4424 LIS 0 034 071 8 4440 094 0 010 OSI 9 4604 062 062 146 0 10 4404 38 0 054 054 11 4296 246 0 162 216 12 4602 060 060 144 072 13 4482 060 0 D24 048 14 4502 D40 0 D44 o 15 4577 035 035 LJ9 0 16 4740 198 233 282 0 17 4755 213 446 297 0 18 4664 122 568 2 0 19 4631 089 657 173 0 20 4482 060 597 D24 0 21 4539 003 594 D81 0 22 4780 238 832 322 0 23 4569 J I 959 21 1 0 24 4699 157 1116 241 0 25 4453 089 1027 005 0 528 Chapter 17 Statistical Quality Control and Reliability Egioeering recommended decision ilterval of H 5j 5084 42 The reference valu K is found to be The vaues of C and c are given ir Table 175 To illustrate the calculations consider the first two sample batches Recall that C CO 0 and using equations 1721 and 1722 itb K 042 and Jlo 45 we have and Fcrbatth lx 4621 and For batch 2 4573 and c O45 M2CJ maxlOxj 45A2tC C maxO 45 042 xl Ci maxO4458x CQJ maxO 45214542 OJ mazO 079 079 C maxO 4458 4521 OJ maxO L63J 0 c ll3XO 4573 4542 079 maxO 110 110 C maxO 4458 4573 0 maxO 115J 0 The CUSLM calcJlations give in Table 175 indicate rhat the uppersided CUSUM for batch 17 is C77 446 which CAceeds the dedsion value of H 42 Therefore the process appears to have shifted out of concrol The CUSUM status chart created using 1initab withH 42 is given in Fig 171 L The outofcontrol control shuation is also evident On this chat at batch 17 The CliSUM control chart is a powerluJ quality tool lor detecting a process that has shifted from the target process mean The correct choices of H and K can greatly improve the sensitivity of the control chart while protecting against t1e occurrence of false alarms the process is actually in control but the control chart signals out of conol Design rec ommendations for the CliSUM will be provided later in this chapter whcn the concept of average run length is introduced 173 Stltistical Process Control 529 6 Upper CUSUM r 3 2 1 0 r o B 1 Jsv j 2 I 3 4 t Lower CUSUv o 5 10 15 Subgrouo nurrber I 20 25 142 We bave presented the upper and lower CUStiM control charts for situations in which a shift in either direction away froe the process target is of interest There are many instances w ben we may be interested in a shift IT only one direction either upward or dovnward Onesided CUSUM charts can be constructed for these situations For a thor ough development of these charts and more details see Montgomery 2001 EWMA Control Charts The exponentially weighted moving average EVMA control cht is also a good alteITl3 tive to the Shewhart control chart when detecting a small srift in the process mean is of interest We will present the EWMA for individual measurements although the procedure can also be modified for subgroups of size n 1 The EWMA control chart was firSt introduced by Roberts 1959 The EVMA is defined as 1723 where Ais a weight 0 A 1 The procedure to be presented is initialized withZo Po the process target mean If a target mean is unknown then the average of preliminary data x is used as the initial value of the EWlYfA The definitior given in equation 1723 demon strates that information from past observations is incorporated into the current value of z The value Zi is a weighted average of all previous sample means To illustrate we can replace Zll on the righthand side of equation 1723 to obtain Zi x 1 AAxi1 I Azd Axi AI Axil 1 A2Z2 B recursively replacing Zipj 12 t we find iI Z A 21 Ai XJ 1 Ai 530 Chapter 17 Statistical Quality Control ad Reliability Engineering The EWMA can be thought of as a weighted average of all past and current observations Note that the weights decrease geometrically with the age of the observation giving less weight to observations that occurred early in the process The EvMA is often used in fore casting but the EWMA control chart bas been used extensively for monitoring many types of processes fme observations are independent random variables with variance f the variance of the EVNIA Zi is Given a target mean Po and the variance of the EVlMA the upper control limit centerline and lower control limit for the Em1A control chart are UCL Centerline ilJ I A J LCLUoL1 2y1At where L is the width of the comrollimits Note that the term 1 1 2i approaches 1 as i increases Therefore as the process continues running the control limits for the EVM1 approach the steady srate values 1724 Although the control limits given in equation 17 24 provide good approximations it is rec ommended that the exact limits be used for smalJ values of i Fxlrnp i 78 We will now impcmenr the EVlMA control chart with 1 02 and L 27 for the drymatter content data provided in Table 175 Recall that the target meanis JiJ 45 and the process standard devia tion is assumed to be 5 04 Tae EWMA calculations are provided in Thble 176 To demonsmtte some of the calculations consider the first observation with XI 4621 We find The second EWMA alue is then xl 1 Az 024621 08045 4524 z 1x 1 Az 024573 0804524 4534 173 Statistical Process Control 531 Tablel7 EWMA Calculations for Example 178 Batch i XI UCL LCL 4621 4524 4545 4455 2 4573 4534 4558 4442 3 4437 4515 4565 4435 4 4419 495 4569 4431 5 4373 4471 4571 4429 6 4566 4490 4573 4427 7 4424 4477 4574 4426 8 4448 4471 4575 4425 9 4604 498 4575 4425 10 4404 4479 4575 425 11 4296 4442 4575 4425 12 4602 4474 4575 4425 13 4483 4476 4575 4425 14 4502 481 4576 4424 15 4577 4500 4576 4424 16 4740 4548 4576 4424 17 4755 4590 4576 4424 18 4664 4604 4576 4424 19 4631 4610 4576 4424 20 4482 4584 4576 4424 21 4539 4575 4576 4424 22 4780 4616 4576 4424 23 4669 4627 4576 4424 24 4699 4641 4576 4424 25 4453 4504 4576 4424 The EW1v1A values 3Ie plotted on a control chart along with the upper and lower control Jiito given by Therefore for i 1 UCL 110 LO 1IiJ2iJ V2iJ 17021 45z70841J 110212i 202 LCL Po LOX f1Iij i2ll 1170 0 45 27 084 l2 2 1 1 0211 j 4545 rrX 1 211 LCL Po LOZiJH 1 4455 532 Chapter 17 Statistical QUality Control and Reliability Engineering UCL4576 455 45Q lean 45 445 LCL4424 440 0 5 10 15 20 25 Sample number Figure 1712 EWMAchaforEwnple178 The reIlaining controllimits are calculated similarly and plotted on the control chart giver in Fig 17 12 The conrrollimits end to increase as i increases but then rend to die steady state values given by equations in 1724 UCL 10 La12 A 4527084 02 1202 A LCLuo La 1 21 4527084 102 1202 4424 The EVI1A control chart signals at observation 17 indicating that the process is Ot of concrl The sensitivity of the EWMA control chart for a particular process will depeod on the choices of L and A Various choices of iliese parameters will be presented later in this chap ter when the concept of the average run length is introduced For more details and deyel opments regarding the EWMA see Crowder 1987 Lucas and Saccucci 1990 and Montgomery 2001 1736 Average Run Length In this chapter we have presented controlcharting techniqces for a variety of situations and made some recommendations about the design of the control cbarts In this section we will present the average run length ARL of a control chart The ARL can be used to assess the performance of the control cbart or to detennine the appropriate values of various parame ters for the control charts presented in this chapter r 17 3 Satistical Ptocess Control 533 The ARL is the expected number of samples taken before a control chart signals out of COn trOL In general the ARL is I ARL p where p is the probability of any point exceeding the controillmlts If the process is in con trol and the control chart signals out of control then we say that afalse alarm has occurred To illustrate consider the X control chart with the standard 30limits For this situation p 00027 is the probability that a single point falls outside the limits when the process is in controL The incontrol ARL for the X COltrol chart is ARL I 370 p 00027 In other words evenf the process remains in control we should expect on the average an outofcontrol signal or false alarm every 370 samples In general if the process is actu ally in control then we desjre a large value of the ARL More formally we can define the incontrol ARL as 1 ARLo a where ais the probability that a sample point plots beyond the control limit If on the other hand the process is out of control then a small ARt value is desirable A small value of theARL indicates that the control chart will signal out of control soon after the process has shifted The outofcontrol ARL is 1 113 wbere 13 s the probability of not detecting a shift on the lirst sample after a shift bas occurred To illustrate consider the X control chart with 3 J limits Assume the target or in control meaJ is fto and that the process has shifted to an outof control mean given by fJ1 Ur ka The probability of not detecting this shift is given by 13 PLCL 5 X 5 ucLlu uJl That is 13 is the prooability that the next sample mea plots in control when in fact the process bas shifted out of control Since XNfJdln and LCL fJo Lo and UCLun Lo we can rewrite 13 as 13 Puo Lo 5X 5uo LojJIu ud r fJo LO ul X u uo LoIu 1 Pl I luul 0 vNn 0 J iuo LOuo ko CUo LOuo ko 1 Z l 0 01 J PLk Z5LkFn where Z is a standard normal random variable If we let CI denote the standard normal cumulative distribution fonctlon then 534 Chapter 17 Statistical Quality Control and Reliability Engineering From this 1 fJ is the probability that a shift in the process is detected on the first sample after the shift has occurred That is the process has shifted and a point exceeds the control limitssignaling the process is out of contro1 Therefore ARLl is the expected number of samples observed before a shift is detected The ARLs have been used to evaluate and design control charts for variables and for attributes For more discussion on the use of ARLs for these chms see Montgomery 2001 ARLs for the CUSUM and EWMA Control Charts Earlier in this chapter we presented the CUSlM and EwMA control charts TheARL can be used to specify some of the parameter values needed to design these control charts To implement the tabular CUSUlvl control chart values of the decision interval H and the reference value K must be cbosen Recall that H and K are multiples of the process standard deviation specifically H ha and K ka where k 12 is often used as a standard The proper selection of these values is important The is one criterion that can be used to determine the values of Hand K As stated previously a large value of the ARL when the process is in control is desirable Therefore we can set ARLo to an accept able level and deternine h and k accordingly In addition we would want the control chart to quickly detect a shiftin the process mean This would require values of hand k such that the values of ARL are quite smalL To illustrate Montgomery 2001 provides the ARL for a ClJSLM coritrol chart with h 5 and k 112 These values are given in Table 177 The incontrol average run length ARLo is 465 If a small shift say 050a is important to detect then with h 5 and k 12 we would expect to detect this shift within 38 sam ples on the average after the shift has occurred Hawkius 1993 presents a table of h and k vaJues that will result in an incontrol average run length of ARLo 370 The values are reproduced in Table 178 Design of the EWMA control chart can also be based on the ARLs Recall that the design parameters of the EWMl control chart are the multiple of the standard deviation L and the value of the weighting factor l The values of these design parameters can be cho sen so that the ARL performance of the control charts is satisfactory Several authors discuss the ARL performance of the EWMA control chart including Crowder 1987 and Lucas and Saccucci 1990 Lucas and Saccucci 1990 provide the ARL performance for several combinations of L andt The results are reproduced in Table 179 Again it is desirable to have a large value of the incontrolARL and small values of outokontrolARLs To illustrate if L 28 and A 010 are used we would expect ARLo e 500 while theARL for detecting a shift of 05 aisARL 313 To detect smaller slrifu Table 177 Tabula CUSloM Penornce withh 5 and k 12 Shift in M of 0 0 025 050 075 100 LSD 200 250 300 400 ARL 465 139 380 170 104 575 401 311 257 201 Table 17s Values of h and k Resulting iDARIc 370 Hawkins 1993 k h 025 sol 050 477 075 334 0 252 125 199 L5 161 173 Statistical Process Control 535 Table 179 ARl for Various ElMA Control Schemes Lucas and Saccucci 1990 Shift iL Mean L 3054 L2998 L2962 L2814 L2615 multiple of d 040 025 1020 AO1O 005 0 500 500 500 500 500 025 224 170 150 106 841 050 712 482 4LS 313 288 075 284 201 182 159 164 100 13 111 105 O3 14 150 59 55 55 61 71 200 35 36 31 44 52 250 25 27 29 34 42 300 20 23 24 29 35 400 14 17 19 22 27 in the process mean it is found that small values of I should be used Note that for L 30 and I LO the EWMA reduces to the standard Shewhart control chart with 3sigma lm its Cautions in the Use of ARLs Although the ARL provIdes valuable information for designing and evaluating control schemes there are drawbacks to relying on theARL as a design criterion It should be noted that run length follows a geometric distribution since it represents the number of sarr ples before a success occurs a success being a point falling beyond the control limits One drawback is the sumdard deiation of the run length is quite large Second because the dis tribution of the run length follows a geometric distribuion the mean of the distribution ARL may not be a reliable estimate of the true run length 1737 Other SPC ProblemSolving Tools Vhi1e the control chan is a very powerful tool for investigating the causes of variation in a process it is most effective when used with other SPC problemsolving tools In this section we illustrate some of these toois using the printed circuit board defect da in Example 175 Figure 179 shows a c chart for the number of defects in samples of five printed circuit boards The chart exhibits statistical control but the number of defects must be reduced as the average number of defects per board is 815 16 and this level of defects would require extensive rework The first step in solving this problem is to construct a Pareto diagram of the individual defect types The Pareto diagram shown in Fig 1713 indicates that insufficient solcer and solder balls are the most frequently OCCUrrillg defects accounting for 109fl60IOO 68 of the observed defects Furthermore the fust five defect categories on the Pareto chart are all solderrelated defects This points to the flow solder process as a petential opportunity for improlement To improve the flow solder process a team consisting of the flow solder operator the shop 8upenisor the manufacturing engineer responsible for the process and a quality neer me to study potential causes of solder defects They conduct a brainstonning session and produce the causeandeffect diagram shown in Fig 1714 The causewmdeffect 536 Chapter 17 Statistical Quality Control and Reliability Engneering 75 64 Figure 1713 Pareto diagram for printed circuit board defects diagram is widely used to clearly display the various potential causes of defects in products and their interrelationships It is useful in summari7ing knowledge about the process As a result of the brainstorming session the team tentatively identifies the following variables as potentially influential in creating solder defects lw Flux specific gravity 2 Solder temperature 3 Conveyor speed 4 Conveyor angle 5 Solder wave height 6 Preheat temperature 7 Pallet loading meLod A statistically designed experiment could be used to investigate the effect of these seven variables on solder defects Also the team constructed a defect concentration diagram for the product A defect concentration diagram is just a sketch or drawing of the product with the most frequently occurring defets shown on the part This diagram is used to determine whether defects occur in the same location on the part The defect concentration diagram for the printed circuit board is shown in Fig 1715 This diagram indicates that most of the insufficient solder defects are near the front edge of the board where it makes initial con tact with the solder wave Further investigation showed that one of the pallets used to carry the boards across the wave was bent causing the front edge of the board to make poor con tact with the solder wave I Exhau Conveyorsoeed Conve or an Ie Maintenance Alignment of pallet WfNe height Contact time Wave fluidity Orlertation Contaminated lead 17 4 Relibility Engineering 537 Amount Specifio rav Temperature Figure 1714 Causeandeffect diagram for the printd circuit board flow solder process Sack Figure 17 15 Defect concentration diagram for a printed cireui board When the defective pallet was replaced a designed experiment was used to investigate the seven variables discussed earlier The results of this experiment indicated that several of these factors were influential and could be adjusted to reduce solder defects After the results of the experiment were implemented the percentage of solder joints requiring rework was redueedOOm 1 to under 100 parts per million 001 174 RELIABILITY ENGINEERING One of Ie challenging endeavors of the past three decades has been the design and devel opment of largescale systems for space exploration new generations of commercial and military aircraft and complex electromechanical products such as office copiers and com puters The perlormance of these systems and the consequences of their failure is of vital concern For example the military community has historically placed strong emphasis on equipment reliability This empbasis stems largely from increasing ratios of Illamtenance cost to procurement costs and the strategic and tacticallinplications of system failure In the 538 Chapter 17 Sutistical Quality Control and Reliability Engineering area of consiuner product manufacture high reliability has come to be expected as much as confonnance to other important quality characteristics Reliability engineering encompasses several activities one of which is reliability mod eling Essentially the system survival probability is expressed as a function of a subsystem of component reliabilities survival probabilities Usually these models are fune depend ent but there are some situations where this is not the case A second important activity is that of life testing and reliability esfunation 1741 Basic Reliability Definitions Let us consider a component that has just been manufactured It is to be operated at a stated stress level or within some range of stress such as temperature shock and so On The ran dom variable T will be defined lIS fune to failure and 1ha reliability of the component or subsystem or system at time t is R PT R is called the reliability function The fuil ure process is usually complex consisting of at least three types of failures initial failures wearout failures and those that fail between these A hypothetical composite distribution of tine to failure is shown in Fig 1716 This is a mixeti distribution and 1725 Since for many components or systems the initial failures or time zero failures are removed during testing the random variable Tis conditioned on the event that T 0 so that the failure density is gt jt i pO 0 Thus in tenns of the reliability function R is to otherwise Rt 1 Ft fixdx 1726 1727 The term interval failure rate denotes the rate of failure on a particular interval of time ri t2J and the terms failure rate iflstantaneousjailure rate and hazcrd ill be used syn onymously as a limiting form of thc interval failure rate as 12 t1 The interval fajlure rate FRtt t is as follows t gl o Figure 1716 A compositefaiJure distribution 1728 174 Reliability EngiLeering 539 The first bracketed term is simply PFailure during it tillSurvival to time Id 1729 The second term is for the dimensional characteristic so that we may express the condi tional probability of equation 1729 on a perunit time basis We will develop the instantaneous failure rate as a function of I Let hl be Ie haz ard function Then hll lim RtRIill 1 0 Rrl ill lim RIillRI 1 0 ill Rt or hl RI Jt RI Rt 1730 sinceRt lFr and Kt JI A typical hazard function is shown in Fig 1717 Note that ht dl might be thought of as the instantaneous probability of failure at I given sur vival to t A useful result is that the reliability functionR may be easily expressed in tenns of has R hx H tr e k e where Equation 1731 results from the definition given in equation 1730 ht h Early ag failures Wear Oll and Random failures tfaiiures and rancom random faiures 1allures Figure 1711 A typical hazard function 1731 540 Chapter 17 Statistical Quality Control and Reliability Engineering and the integratiorrof both sides so that r hxdx r RXilx lnRX Ja 0 R x o J hxdx In RI lnRO Since FO 0 we see that In RO 0 and The mean time to failure MITF is A useful alternate form is 1732 Most complex system modeling assumes that only random component failures need be considered This is equivalent to stating that the timetofailure distribution is exponential that is so that t 0 othenvise i l ht 1 RI e is a constant nen all earlyage failures have been removed by bum in and the time to occurrence of wearolit failures is very great as with electronic parts then this assumption is reasonable The normal distribution is most generally used to mode wearout failure or stress fail ure where the random variable under study is StreSS level In situations where most failures are due to wear the nonnal distribution may very well be appropriate The lognormal distribntion has been found to be applicable in describing time to fail ure for some types of components and the literature seems to indicate an increased utilization of this density for this purpose The Weibull distribution has been extensively used to represent time to failure and its nature is such that it may be made to approximate closely the observed phenomena Vthen a system is composed of a number of components and failure is due to the most serious of a large number of defects or possible defects the Weibull distribution seems to do particu larly well as a model The gll1Iltlla distribution frequently resulfll from modeling standby redundancy where components have an exponential timerQfailure distribution We will investigate standby redundancy in Section 1745 174 Reliability Engineering 541 1742 The Exponential TimetoFailure Model In this section we assume that the timetofailure distribution is exponential that is only random failures are considered The density reliability function and hazard functions are given in equations 1733 through 1735 and are shown in Fig 1718 t 0 0 otherwise 1733 0 otherwise 1734 ht ft J Rt 0 otheINise 1735 The constart hazd function is mterpreted to mea1 L1at the failure process has no memory that is 1736 fit a Density function b Reiability function hit c Hazard t mction Figure 17lS Density reUabwry ftUctio and hazard function for the exponential failure model 542 Chapter 17 Statistical Quality Control and Reliability Engineering a quantity that is independent of t Thus if a component is functioning at time t it is as good as new The remaining life has the same density asj Emnple 179 A diode used on a printed creui board has a fated failure rate of 23 x loe failures per hour How eve under an increased temperae stress it is felt that the ate is about t5 x 105 failures per hour The time to failure is exponentially distributed so that we have ftt 15 X 1O5elJhor tO O otherwise RU e L5115I tO 0 othetwise and hf L5x 10 O 0 otherwise To determine the reliability at t 104 and t 1 iY we evaluate R 10 eiJ15 0861 and R 105 150223 1743 Simple Selia Systems A simple serial system is shown in Fig 1719 In order for the system to function all com ponents must function and it is assumed that the components function independenly We let be the time to failure for component Cj for j 1 2 n and let T be system time to failure Tbe reliabiJty model is thus Rt peT t PTI t PT2 f PIT t or RI RI Rt Rt where Example1710 Three components must all iu1ction or a simple system to funetion The random variables T T2 and TJ representing tirJe to failure for the components are independent with the following distributions TrN2XIOJ 4X104 I 1 TWcibUlyO 01 7 TlOgnormal1l 10 14 Figure 1719 A simple serial system 174 ReliabLity Engineering 543 1 follows that so that For exanple if t 2187 hors then R2187 1 I0935eI 11154 017500498J0876 00076 For the simple serial system system reliability may be calculated using the product of the component reliability functions as demonstrated however when all components have an exponential distribution the calculations are greatly simplified since Rt el etf eJI eI AZ 1 or 1738 where Af Lj represent the system failure rate Ve also note that the system reliability function is of the same fonn as the component reliability functions The system failure rate is simply the sum of the component failure rates and this makes application very easy Consider an electronic ciicJit with three integrated ciTCrit devices 12 silicon diodes 8 ceramic capac itors and 15 composition resistors Suppose under given Stress levels of eroperattre shock and so on that each component has failrre rates as shown 1 the following table and the component faillheS independent Therefore and Integrated circuits Diodes Capacitors Resistors Faillhes per Hour 13 x lv 17 X lv 11 X 107 61 x 1U A 3O013 X 10 1217 x 10 812 x 107 15061 x 107 39189 X 10 544 Chapter 17 Statistical Quality Control and Reliability Engineering The circuit mean time to failure is l1TIF ETIX10 255xlOho J 39189 If we wish to determine say ROO we get R104 eo039189 I 096 1744 Simple Active Redundancy A active redundant configuration is sho in Fig 1720 The assembly functioos if k or more of the components function k5 n All components begin operation at time zero thus the term active is used to describe the redundancy Again independence is assumed A general fonnulation is not convenient to work withl and in most cases it is unneces sary When all components hae the same reliability function as is the case when the com ponents are the same type we ret RP r for j 1 2 n so that 1739 Equation 1739 is derived from the definition of reliability Three identical components are arranged in active redundancy operating independently In order for the assembly to funtion at least two of te components must function k 2 The reliability func don for the system is thus 3r11 rlrr j rI 3 2rI1 It is noted tharR is a function of time r Figure 1721 An active redundant configuration 17 A Reliability Engineering S4S When only one of the n components is required as is often the case and the components are not identical we obtain Rt 1I1I Rjt 1 il 1740 The product is the probability that all components fail and obviously if they do not fail the system survives When the components are identical and only one is requited equation 1740 reduces to Rt 1 I rtlJ 1741 where rt Rtj 1 2 When the components have exponential failure laws we will consider two cases First when the components are identical with failure rare t and at least k compocents are required for the assembly to operate equation 1739 becomes 1742 The second case is considered for the situation where the components have identical expo nential failure densities and where only one component must function for the assembly to function Using equation 1741 we get Rtl 1 1 e1 1743 In Example 1712 where thee identical components were arranged in an active redundancy and at east Vo were required for system operation we found Rt rt3 2rtl If the component reliability functions are rtl e then Rt 3 21 3e2M 2e3l If tvo eomponents arc ammgcd in an active reduLdancy as described and only one must function for the assembly to function and furthermore if the timetofailurc densities arc exponential with fail ure rate 1 tben from equation 1742 we obtcin Rt 1 1 e42 2e 1745 Standby Redundancy A common form of redundancy called stllndby redundancy is shown in Fig 1721 The unit labeled DS is a decision switch that we will assume has reliability of 1 for all t The operating rules are as follows Component 1 is initially online and when this component fails the decision switch switches in component 2 which remains online until it fails 546 Chapter 17 Statistical Quality Control and Reliability Engineering Figure 1721 Standby redundancy Standby units are not subject to failure until activated The time to failure for the assem bly is T T1 T Til where T is the time to failure for the ith component and T1 T2 Tn are independent ran dom variables The most common value for n in practice is two so the Central Limit The orem is of little value However we know from the property of linear combinations that ET 2li 11 and YT 2 Vli il We must know the distributions of the random variables Ti in oroer to find the distribu tion of T The most common case occurs when the components are identical and the timeto failure distributions are assumed to be exponential In this case Thas a gamma distribution fturle 10 nl 0 otherwise so that the reliability function is 1 Rt 2e Atk k tO 1744 kO The parameter A is the component failure rate that is ET J ItA The mean time to fail ure and varialce are MTIF E11 nI A 1745 and 1746 respectively 1 174 Reliability Engineering 547 Example i714 Two identical compOllents are assembled in a standby redundant configuration with perfect switch ing The cornponert lives are identically distributed independert randm variables having an expo nertial distribuion with failure rate lOu l The mean time to failure is and the variance is VTl 21100 20000 The reliability function R is 1 Rt 22eJ t100 k 0 or Rt eIOO1 tl00l 1746 Life Testing Ufe tests are conducted for different purposes Sometiroes n units are placed on test and aged until all or most uruts hae failed the purpose is to test a hypothesis about the fann of the tirnetofailure density with certain parameters Both forma1 statistical tests and proba bility plotting are widely used in life testing A secane objective in life testing is to estimate reliability Suppose for example that a manufacturer is interested in estimating RIOOO for a particular component or system One approach to this problem would be to place n units on test and count the number offail ures r occurring before 1000 hours of operation Failed units are not to be replaced in this example An estimate of unreliability isp rln and an estimate of reliability is RIOOO I n 1747 A 1001 a lowercoUidence limi on RlOOO is given by I upper limit onp where p is the unreliability This upper limit on p may be detemrinec using a table of the binomial distribution In tte case where n is large an estimate of the upper limit on pis 1748 Etipi17 One hundred units are placed on life test and the test is run for 1000 Mum There are two failures dur ing test soft 002 ard 1000 098 Using a table ofte binomial Gistribution a 95 uppccon fidence limit on pis 006 so that a lower limit on RlOOO is given by 094 In recent years tere has been much work on the analysis of failureuUe data nclud ing plotting methods for identification of appropriate failuretime models and parameter estimation For a good summary oftbis work refer to Elsayed 1996 S48 Chapter 17 Statistical Quality Control and Reliability Engineering 1747 Reliability Estimation with a Known TimetoFailure Distribution In the case where the form of the reliability function is assumed known and there is only one parameter the maximum likelihood estimator for RI isRt which is formed by sub stituting for the parameter ein the expression for Rt where is the maximum likelihood estimator of O For more details and results for specific timetofailure distributions refer to Elsayed 1996 1748 Estimation with the Exponential TimetoFailure Distribution The most COllUDon case for the oneparameter situation is where the tlmetofailure distri burioo is exponentia Rt eto The parameter e ET is called the mean time to fallure and the estimator for R is Rt where Rt di and j is the maximum likeliliood estimator of e Epstein 1960 developed the maximum likeliliood estimators for e under a number of diEerent conditions and furthermore showed that a 1 OO 1 a confidence interval on Rt is given by 1749 for the twosided case Or 1750 for the lower onesided intervaL In these cases the values e L and eu are the lower and upperconfidence limits on O The following symbols will be used n number of units placed on test at t O Q total test time in unit hours t time at wruch the test is terminated r number of failures accumulated at time t r preassigned number of failures 1 a confidence level X the upper a percentage pomt of the chiwsquare distribution with k degrees of freedom There are four situations to consider according to whether the test is stopped after a preas signed time or after a preassigned number of failures and whether failed items are replaced or not replaced during test For the replacement test the total test time in unit hours is Q nt and for the nonre placement test Q 2 nrt 1751 If items ace censored withdrawn items that have not failed and if failures are replaced whlle censored items are not replaced then c Q 2 nc 1 1752 r 174 Reliability Engineering 549 where c represents the number of censored items and tj is the time of the jrh censorship If neither censored items nor failed items are replaced then c Q 2 2jnrct 1753 11 jl The development of the maximum likelihood estimators for 6 is rather straightforward In the case where the test is nonrep1acement and the test is discontinued after a fixed num ber of items have failed the likeJ11cod function is L TIftiTIW 1754 Then l llnLrlneekti nrt 8 1 and sc1ving awl 0 yields the estimator 2 i n r e il Q 1755 r r It turns out that 1756 is the maximum likelihood estimator of e for all cases considered for the test design and operation The quantity 2y8has a chisquare distribution with 2r degrees of freedom in the case where the test is terminated after a fixed number of failures For fixed termLlation time t l the degrees of freedom becomes 2r 2 Since the expression 2r818 2QO confidence limits on j may be expressed as indi cated in Table 1710 The results presented in the table may be used direcdy with equations 1749 and 1750 to establish oonfidence limits on Rt It should be noted that this testilg procedure does not require that the test be run for the time at which a reliability estimate is required For example 100 writs may be placed On a aoureplacement test for 200 hours the paramorer e estimated and RlOOO calculated In the case of the binomial testing men tioned earlier it would have been necessary to rm the test for 1000 hours Tnble 1710 Confidence Limits on B Nature of Limit FIxed NlIlllber of Failures r Fixed Terminarior Time Two sided lL11its Lower onesided limit 550 Chapter 17 Statistical Quality Control and Reliability Engineering The results are however dependent on the assumption that the distribution is exponential It is sometimes necessary to estimate the time tR for which the reliability will be R For the exponential model this estimate is 177 and confidence limits on R are given in Table 1711 l176 Twenty items are placed on a replacemenr test that is to be OCf3ted until 10 failures occur The tenth failure occurs at 80 hours ard the reliability engineer wishes to estimate he lean time 0 failure 95 twosided limits on 8 R10D and 95 twosided limits on ROOO Finally she wishes 0 esti mate the time for whkh the eliability wil be 08 with point and 95 twosided confdence interval estimates According to equation 1756 and the results presented in Tables 17lO and 1711 iI nt 2080 160 hours r 10 Q m 1600 unit hoUlS r 2Q 2Q c 3200 3200 1 0250 X6YIl r l3417 9591J 936533365J RlOOe100e e1OOf60 0535 According to equation 749 the confidence interval onR100 is eJOOi9J65 eIOO133365 0344 0741J Also The twosided 95 confidence limit is determined from Table 1711 a r 216000223 216000223 1 2097445 L 3417 9591 Table 1711 Confidence Limits on tli Nature of Limit Fixed Number of Failures Fixed Tennination Tune l Twosided limits r 2QlnlR 2qlnIRl 22r2 Xiafl2r2 2QlnlR I J 211 Lower ol1esided limit T I 176 Exises 551 1749 Demonstration and Acceptance Testing It is not uncommon for a purchaser to test incoming products to assure that the vendor is conforming to reliability specifications These tests are destructive tests and in the case of attribute measurement the test design follows that of acceptance sampling discussed eat lier in this chapter A special set of sampling plans that assumes an exponential timelofailure distribution has been presented in a Department of Defense handbook DOD fI108 and hese plans are in vide use 175 SLMMARY This chapter has presented several widely used methods for statistical quality control Con trol charts were introduced and their use as process surveillance deviees discussed The X and R control chatS are used for measurement data tben the quality characteristic is an attribute either the p chart for fraction defective or the cor u chart for defects may be used The use of probability as a modeling technique in reliability analyse was also dis cussed The exponential distribution is widely used as the distribution of time to failure although other plausible models inelude the normal lognormal Weibull and gamma dis tributions System reliability analysis methods were presented for serial systems as weB as for systems having active or standby redundaney Life testing and reliability estimation were also briefly introduced 176 EXERCISES 171 An extrusion die is used to produce aluminum rods The diameter of the rods is a critical quality eharacteristic Below are shown X and R values for 20 samples of five rods eaeh Specifications on the rods are 05035 00010 inch The values given are the last three digits of the measurements that is 342 is read as 050342 Sample X R Sample X R 1 342 3 11 354 8 2 316 4 12 340 6 3 318 4 13 360 4 4 334 5 14 372 7 5 350 4 15 352 3 6 321 2 16 334 10 7 326 7 17 350 4 8 338 9 18 344 7 9 348 10 19 339 8 10 386 4 20 340 4 a Set up the X wd R chats revising the trial control limits if neeessary assuming assignable causes cau be found 0 CalctUae peR and PCRk Interpret these ratios c What pereentage of defectives is being produced by this prOless 17 2 SIppose a process is in control and 3sigma controllimlts are in use on the X chart Let the mean shift by 150 ba is the probability that t1is shift will remain undetected for three consecttive samples Vhat would this probability be if2sigma control lim its are used The saIlple size is 4 113 Suppose that an X chart is used o control a 01 mall distributed process and that samples of size n are taken every h hours and plotted on the chart which has k sigma limits a Find the expected number of samples that will be Laken until a false action signal is generated This is called the incotrol average run length CARL 0 SIppose tha the ptocess srJfts to an outof eontrol state Find the expected number of Sacl pies that will be taken until a false action is generated This is the outofcootrol ARL e Evaluate the incomrolARL fork 3 How does this change if k 27 What do you thilk about the use of 2sigma limits in practice d Evaluate the outofcontrolARL for a shift of one sigma give that n 5 114 Twentyfive samples of size 5 are draWn from a process at regular intervals and the following daa ae obtained 552 Chapter 17 Statistical Quality Control and Reliability Engineering IXi 36275 a Compute the control limits for the X and R charts b Assutning the process is in control and specifica tiorlimits are 1450 050 what conclusions can you draw about the ability of the process to oper ate 1tbin these limits Estimate the percent2ge of defective items ilia will be produced c Calculate peR and peRI Interpret these ratios 115 Suppose an X c1lit for a process is in control with 3sigma limits Samples of size 5 are drawu every 15 minutes on he quarter hour Now suppose me proceos mean stills out of control by 150 10 min utes after the hour If D is the expected number of defectives produced per quarter hour in this outof control st2tefind the expected loss in tenus of defec tive unlt that reults from this control procedure 17i The overall length of a cigar lighter body used in al automobile application is controlled uSing X and R charts The following table gives length for 20 sam ples of size 4 measUements a1 coded from 500 rnm that is 15 is 515 rill Observation 2 3 4 15 10 8 9 2 14 14 0 6 3 9 10 9 11 8 6 9 13 5 14 8 9 12 6 9 0 7 13 7 15 10 12 12 8 14 16 11 10 9 II 7 16 10 10 11 14 11 12 11 13 8 9 5 12 10 15 8 O 13 8 12 14 9 14 15 12 14 6 15 13 16 9 5 16 14 S S 12 17 8 10 16 9 18 8 14 O 9 19 13 15 10 8 20 9 7 15 8 a Set up the X and R charts Is the process in statis tical control b Specifications are 510 005 mm Vbat can you say about process capability 17 7 Montgomery 2001 presents 30 observations of oxide thickness of individual silicon wafers The data are Oxide Oxide Wafer Thickness Weier TIickncss 454 16 584 2 486 17 510 3 495 18 412 4 440 19 471 5 509 20 457 6 552 21 606 7 455 22 510 8 528 23 530 9 453 24 560 10 3 25 472 11 539 26 O 12 498 559 13 469 28 500 14 498 29 479 IS 45 30 534 a Construct a nocnaI probability plot of the data Does the normality assumption seem reasonable b Set up an individuals control chart for oxide thickness Interpret the chart 17s A machine i used to fill bottles Vlith a particu Jar brand of vegetable oil A single bottle is randomly selected every half hour and the weight of the bottle recorded Experience with the process indicates that the variability is quite stable with J 007 oz The process target is 32 oz Twentyfour samples have been recorded in a 12hour time period with the results given below Sample Sample Number x Xumber x 3203 13 3197 2 3198 14 3201 3 3202 15 3193 4 3185 16 3209 5 3191 17 3196 6 3209 18 3188 7 3198 19 3182 8 3203 20 3192 9 3198 21 3181 10 3191 22 3195 11 3201 23 3197 12 3212 24 3194 r I Ii a Construct a normal probability plot of the data Does the norrnality assumption appear to be satisfied b Set up an individuals control chart for the weights Interpret the results 179 The follo1ng are the numbet of defective sol der joints found during successive samples of 500 sol derjollts Day No of Defectives Day No of Defectives 106 11 42 2 116 12 37 3 164 13 25 4 89 14 88 5 99 15 101 6 40 16 64 7 2 17 51 8 36 18 74 9 69 19 71 10 74 20 43 21 80 Construct a fractiondefective control chart Is the process in control 1710 A process is controlled by a p chart using sam ples of size 100 The centerline on the chart is 005 What is the probability th2t the control chart detccts a shift to 008 on the first sample flowing the shift What is the probability that the shift is detected by at least the third sample following the shift 17 11 Suppose a p hart with cCIterline at p with k sigma units is used to control a process There is a critical fraction defective p t3at must be detected with probability 050 on the firs sample followeg the shift to dUs state Derive a general fonuula for the sample size that snould be used on this enart 1712 A normally distributed process uses 667 of the specification band It is ce1tered at the nominal dimension located halfway between the upper and lower specificaon limits a What is tie process capability ratio peR b What fallout level fraction defective is pro duced c Suppose the mean shifts to a distance exactly 3 standard deviations below the upper 11lecification llinil What is the value of PCRk How has peR changed Cd iVhat is the actual faIout experienced after the shift ill the meal 17 6 Exercises 553 17 13 Consider a process where specifications on a qUality characteristic are 180 15 We know that the standard deviation of this quality characteristic is 5 Vbere hou1d we eener the process to minimize the fraction defective produced Now suppOse the mean slifts to 105 ard we atc using a sample size of 4 on an X chan Wh2t is the probability that such a sbift wit be detwed on the first souple following tic shift What sample size would be needed on a p chart to obtain a similar degree of protection 1714 Suppose the following fraction defective had been found L1 successive samples of size 100 read down 009 003 012 010 005 014 013 013 006 008 010 005 014 014 014 009 0D7 011 010 006 009 015 009 013 013 008 012 006 Oll 009 Is the process in control with respect to its fraction defective 1715 The foUoviug represent the nurber of solder defects observed on 24 samples of five printed circuit boards 7 6S 1024 6548 1115 84 6 II 12 86597 14821 Can we conclude that te process is in control using a c chart If not asstne assignabie causes can be found and revise the control limitS 1716 The following represent te number of defects per 1000 feet ir rubbcrcovred wire I 1378 10 5130192469 11 1583674920117 18 10640973 I 8 12 Do the data come from a control1ed process 1717 Suppose te ntmber of deects in a unit is knOVIl1 to be 8 If the number of defects m a unit shifts to 16 what it he probability that it will be detected by the c chart on the Erst sample folloNing the slift 17lS Suppose we are inspectirg disk drives for defects per unit and it is hown that there is an aver age of two defects per urit We decided to make our inspection unit for the c chart five disk drives and we control the tOtal number of defects per inspection unit Describe the new control chart 1719 Consider the data in Exercise 1715 Set tp a u chart for tlrls process Compare it to the c chart in Exercise 1715 554 Chapter 17 Statistica Quality Control and Reliability Engineering 1720 Consider the oxide thickness d3ia given in Exercise 177 Set up an EWMA control chart with 020 and L 2962 Interpret tle chart 1121 Consider the oxide thickness data given in Exercise 177 Construct a CUSUM control chilrt with k 075 and h 334 if the target Lleme is 50 Interpret the chart 1722 Consider the weighs provided in Exereise 178 Set up an EVMA eontro1 chart with 010 and L 27 Interprlt the chart 17kl3 Consider tIe weights provided in Exercise 17g Set up a CUStrM control chart with k 050 and h 40 Interpret the chart 1724 A tituetofailure distribution is givn by a uni form distribution 1 1 fJa 0 otheIVllse Ca Determine the reiability function b Show tha Rtdt tftdt c Determine tle hazard function d Show that whereH is defmed as in eqcation 1731 1725 TIrree units that operate and fail independently form a series configuration as shown in the figure at the bottom of this page The tirretofailure distribution for each UJIit is expo nential with t1e failure rates indicated a Find R60 fur the systeIIL b hat is the mean timetofailurc Cv1TTF for this sytemf 17M26 Five identical units are arranged in an active redundancy to fonn a subsystelXl Unit failure is inde pcadem and a least tNo of the units must survive 1000 hours for the subsystem to perfonn its nission a If the units have cxponeltial timetofailure dis tributions with failure rate 0002 what is tle sub sys reliability 0 Vihatis the reliability if only onc unit is reqcied 1727 If the utrits described in the previous exercise are operated in a standby redundancy with a perfect decision switch and only one unit is required for sub system survival detemine tbe subsystem reliability 1728 One 111mdred unit are paced on test and aged until all units have failed The folloing results are obtained and a mean life oft 160 hours is calculated from the serial data Time Interval Number offallures 0100 50 100200 18 200300 7 300400 8 400500 4 After 500 hours 3 Use the chisquare goodnessoffit test to detennine whether you consider the exponential distribution to represent a reasonable tirnetofai1ure model for thesl data 1729 Fifty uaits are placed on a life test for 1000 hours Eight units fail during the period Estimate RlOOO for these units Determine a lower 95 con fidence interval on RlOOO 1730 In Section 1747 h was noted that for one parttnerer reliability functions Rtfi Rtf1 Rt8 where e and R are the maximum likelihood estima tors Prove this statement for the case Rr8 el 0 tO othenv1se Hint Express the density function fin tenns of R 1731 For a nonreplacement test that is terminated after 200 hours of operation it is noted that failures occur at the following tines 9 21 40 55 and 85 lours The units are aslllned to have an expOThntial tittleTofailure distribution and 100 uoits were on test initially a Estimate the mean time to failure b Construct a 95 lowerconfidence limit on te mean tirte to failure 1732 Use the statement in Exercise 173 2 Estimate R300 and construct a 95 lower confidence limit on R300 b Estinate the time for which the reliability will be 09 and construct a 95 lower limit on teg A13x102 H A26x10G 4 A4x102 H Figure for Exercise 1725 Chapter 18 Stochastic Processes and Queueing 181 INTRODUCfION The term stochastic process Is frequently used in connection with observations from a timeoriented physical process that is controlled by a random mechanism More precisely a stochastic process is a sequence of random variables XI where t E T is a time or sequence index The range space for Xf may be discrete or continuous however in this chapter we will consider only the case where at a particular time t the process is in exactly one of m 1 mutually exclusive and exhaustive stales The states are labeled 0 1 2 3 m The variables Xl might represent the number of customers awaiting service at a ticket booth at times 1 minute 2 minutes and so on after the booth opens Anether example would be daily demands for a certain product on successive days xc represents the initial state of the process The chapter will introduce a special type of stochastic process called a Markov process We will also discuss the ChapmanKolmogorov equations various special properties of Markov chains the birthdearh equations and some applications to waitingline or queue ing and interference problems In the study of stochastic processes certain assumptions are required about the joint probability distribution of the random variables Xl X In the case of Bernoulli trials presented in Chapter 5 recall that these variables were defined to be independent and that the range space state space consisted of two values 0 1 Here we will first consider dis cretetime arkov chains the case where time is discrete and the independence assumption is relaxed ro allow for a onestage dependence 182 DISCRETETThIE MARKOV CHAJNS A stochastic process exhibits the lr1arkovian property if P XrT1 jlX i P Xr l jlXr i Xtl ijXr7 i2 Xo fa 181 for t O 1r 2 and every sequencej i it it This is equivalent to stating that the prob ability of an event at time t 1 given only the outcome at time t is equal to the probability of the event at time t 1 given the entire state history of the system In other words the probability of the event at t 1 is not dependent on the state history prior to time t The conditional probabilities P Xt 1 j1X i Plj 182 are called one step transition probabilities and they are said to be statiorary if for t 012 183 55S 556 Cbapter 18 Stochastic Processes and Queueing so that the transition probabilities remain unchanged through time These values may be displayed in a matrix P Pi called the onestep transitioo matrix Tne matrix P has m 1 rows and m 1 columns and while That is each element of the P matrix is a probability and each row of the matrix surns to one The existence of the onestep stationary transition probabilities implies that p PX jlK i P X jlK i 184 for all t 0 12 The values p are called nstep transition probabilities and they may be displayed in an nstep transition matrix pzn pi where Op I J nO 12 iO 12 m j O 1 2 m and n 012 iOI2 m The Ostep transition matrix is the identity matrix Afinitestate Marrov chain is defined as a stochastic process having a tillite number of states the Markovian property stationary transition probabilities and an initial set of prob bili 0 0 0 0 b P IX a tieL GIn a p ll1 am were ai I I The ChapmanKolmogorov equations are useful in computing nstep transition prob abilities These equations are iO12 m j O12 m Osvn 185 and they indicate that in passing from state i to state j in n steps the process will be in some state say I after exactly v mps v n Therefore pt JJ is the conditional probability that given state i as the starting state the process goes to state 1 in v steps and from 1 to j in n v steps When summed over 1 the sUIIl of the products ie1dspiJ By setting v lor Vn 1 we obtain iO12 m j O12 m nl2 It follows that the nstep transition probabilities pt may be obtained from the onestep probabilities and 186 183 Classification of States and Chains 557 The unconditional probability of being in statej at time tn is where A n IT In aJa l a aO p I I 1 iC j OI2 m n 12 Tnus An A pn Further we note that the rule for matrix mcitiplication solves the total probability law of Theorem 18 so that An AnI p llilir In a eomputing system the probabiHty of an error on each cyele depends on whether or not it was pre ceded by an error We will define a as the error state and 1 as the nonerror stae Suppose the proba bility of an error if preceded by an error is 075 the probability of a error if peceded by a nonerror is 050 he probability of a nonerror if preeded by an error is 025 ard the probability of noneITOr if preceded by nOlerror is 050 Thus r O75 025 p J L 050 050 Twostep threostep sevensep transition matrices arc shoWU below rO688 P LO625 0312 0375 p rO672 LO656 0328 034 jO668 0667 0333 p p 0334 LO664 0666 0667 p 0667 0333 0333 rO667 P 0667 0333J 0333 If we know that initially the system is in the nonerro state thcl ai I d0 O and AII ar A pro Thus forcxample A1 0667 0333J 183 CLASSIF1CATION OF STATES AND CBADlS We will first consider the notion ofJirst passage times The length of time number of steps in discretetime systems for the process to go from state i to state j for the first time is called the first passage time If i j then this is tIe number of steps needed for the process to return to state i for the first time and this is tenned the first return time or recurrence rime for state i FIrst passage times under certain conditions are random variables with an associated probability distribction We let I denote the probability that the first passage time from state i to j is equal to n where it can be shoIJ directly from Theorem 15 that 0 j Pij Pij 1 n fl inll 2 1l2 Ani i Pi J PJJ J Ij Pjj Jj Pjj 188 558 Chapter 18 Stochastic Processes and Queueing Thus recursive computation from the onestep transition probabilities yields the probabil ity that the first passage time is n for given ij taniIZ Using the onestep transition probabilities presented in Exanple 181 the distrlbutloo of the passage time index n for i Oj 1 is determined as f1 POI 0250 f 0312 02505 0187 fi 0328 0250375 018705 0141 f 0332 0250344 01870375 014105 0105 There are four such distribution corresponding to i j value 00 0 I I 0 I 1 If i and are fixed then Itfi s 1 When the sum is equal to one the values It for n 1 2 represent the probability distribution of first passage time for specific ij In the case where a process in state i may never reach state j Ltf L Where i j and Ii 1 the state i is termed a recurrent STate since given that the process is in state i it will always eventually return to i If Pa 1 for some State i then that state is called an absorbing state and the process will never leave it after it is entered The state i is called a transient state if ifS tt 1 n since there is a positive probability that given the process is in state i it will never retum to this state It is not always easy to classify a state as transient or recurrent since it is some times difficult to calculate first passage time probabilities ti for all n1 as was the case in Example 182 Nevertheless the expected first passage time is n I LJj nl 1 a simple conditioning argument shows that Iij 1 IPil Iv j 189 1810 If we take i the expected first passage time is called the expected recurrence time If J4i 00 for a recurrent state itis called null if Jlu 00 it is called narmul or positive recurrent There are no null recurrent states in a finitestate Markov chain All of the states in such chains are either positive recurrent or transient 183 Classification of Staes and Chains 559 A state is called periodic with period 1 1 if a return is possible only in r 2r 3r steps so p 0 for all values of n that are not divisible by 1 1 and ris the smallest integer hav ing this property A state j is termed accessible from state i ifp 0 forsorne n 12 In olexam J pie of the computing system each state 0 and 1 is accessible from the other since p 0 for all i j and all n If state j is accessible from i and state i is accessible from j then the states are said to communicate This is the case in Example 181 We note that any state conununicates with itself If state i communicates withjj also communicates with i Also if i communicates with 1 and I commuricates vithJ then i also communicates with If the state space is partitioned into disjoint sets called equivalence classes of states where COIILYflunicating states belong to the same class then the IvIarkov chain may consist of one or more classes If there is only one class so that at states communicate the Markov chain is said to be irreducible The chain represented by Example 181 is thus also irre ducible For finitestate Markov chains the states of a class are either aU positive recurrent or all transient In many applications the states will all communicate This is the case if there is a value of n for which p 0 for all values of i andj If state i in a class is aperiodic not periodic and if the state is also positive recurrent then the state is said to be ergodic An irreducible Markov chain is ergodic if all of its states are ergodic In the case of such Markov chains the distribution A A P converges as n 7 00 and the limiting disuibution is independent of the initial probabilities AIn Fxamp1e 181 this was clearly observed to be the case and after fiye steps n 5 PIX OJ 0667 and PIX 1 0333 when three significant figures are used In general for irreducible ergodic 11arkov chains lim r lim II nt Pi n aj Pj and furhennore these values Pi are independent of i These steady stare probabilities p satisfy the following state equations m IPj 1 jO m pjLPiPij iO j O12m l8Ha 18Hb l811e Since there are m 2 equations in 1811b and 1811c and since there are m 1 unnowns one of the equations is redundant Therefore j we will use 111 of the 111 1 equations in equa tion 18110 with equation 1811b In the case of the computing system presented in Example 181 we have fro equario1S ISlIb and 1Slle 1 PoPj PoPo 075p050 560 Chapter 18 Stochastic Processes and Queucing or Po 213 and p 113 which agrees with l1e emerging result as n 5 in Exampie 18 1 The steady state probabilities and the mean recurrence time for irreducible ergodic Markov chains have a reciprocal relationship 1 Pj j O12 m In Example 183 lote thatlloo 1ipo 15 and1 lip 3 1812 Tue mood of a corporate president is observed over a period of time by a psychologist in the opera tions research department Being inclined toward mathematica modeling the psychologist classifies mood into three states as follows 0 Good cheerful 1 Fair soso 2 Poor glum and depressed The psychologist observes that mood changes OCClr only overnight thus the data allow estimation of the transition probabilities The equations jO6 02 02 P l03 04 O3J 00 03 07 Po O6pQ a3pt 0P2 PI O2pJ OAp O3P2 1 PoPPz are solved simultaneously for the steady sate probabilities Pa313 p 4113 P2 613 Given that the president is in a bad mood that is state 2 the mean rime reqllired to return to that state is ii where 1 3 Itn days p 6 As noted earlierj ifpkt 1 state k is called an absorbing state and the process remains in state k once that state is reached In this case b is called the absorption probability 184 CQntinuousTime Markoy Chains 561 which is the conditional probability of absorption into state k given state i lathematically we have bik LPl bjl l012 m 1813 iO where and for i recurrent i k 184 CONTINUOUSTIME MARKOV CHAINS If tle time parameter is continuous rather than a discrete index as asslLTIed in the previous sections the Markov chain is called a cOfltinuousparameter chain It is customary to use a slightly different natation for continuousparameter Markov chains namely Xt X where XI J t 0 will be considered 10 bave states 0 I m The discrete nature of the state space range space for XlI is thus maintained and PiPP Xt sjlXs 1 i 012 m jO 1 2t m s 0 tO is the stationary transition probability function It is noted that these probabilities are not cependent on s but only on t for a specified ij pair of states Furthermore at time t 0 the function is continuous with There is a direct correspondence between the discrete time and continuoustime mod els The ChapmanKolmogorov equations become Pijt 2Pev pjtv 1814 10 for 0 v S t and for the specified state pair ij and time t If there are times t1 and 2 such that Pljt 0 and Pjr 0 then states i andj are said to communicate Once agab states that communicate form an equivalence claSs and where the chaL is irreducible all states form a single class pP 0 for t 0 for eacb state pair iJ We also have the property that 562 Chapter 18 Stochastic Processes and Queueing where Pi exists and is independent of the initial state probability vector A The values Pj ae again called the steady state probabilities and they satisfy Pj 0 j 012 m m P v p t I 1 1 fC j O12m t O The intensity of transition given that the state is j is defined as 1815 where the limit exists and is finite Likewise the intensity of passage from state ito statej given that the system is in state i is 1816 again where the limit exists and is finite The interpretation of the intensities is that they rep resent an instantaneous rate of transition from state i to j For a small ill Pijill upt 01 where 0111 0 as 1 0 so that ii is a proportionality constant by which Pij6t is proportional to ill as ill 0 The transition intensities also satisfy the balance equations PI uj LPUjj i7lij j O12 m 1817 These equations indicate that in steady state the rate of transition out of state j is equal to the rate of transition into j An electronic control mechanism for a chemical process is constructed with two identical modUes operating as a parallel active cedurdant pair The function of at least one module is necessary for the mechanism to operate The m3intenance shop has two identical repair stations for these modules and furtilerrnore when a module fails and enters the shop oner work is moved aside and repair work is imrLediately initiated The system here consists of tbe mechanism and repair facility and the states are as fellows 0 Both modules operating 1 One unit operating and one unit itt repair 2 1vo units in repair mechanism down The random variable representing tinle to faillLe for a module has an exponential density say ttl k 1 O 0 0 and the random variable describing repair time at a repair station also has an exponential density say rt j1eJU I 0 0 10 184 ContinuousTlffie Markov Chains 563 Intetiailore and interrepait times are independent axd Xt t can be shown to be a continuousparam eter irredJcible Markov cbain with transitions only from a state to its neighbor states 0 1 1 t 0 1 2 2 1 Of course there may be nO state cbange The transition htensities are Lsing eqaation 1817 ll 2 uO 21 uO uIOp U1AjJ ilL A Ulo 0 i2u 2Jl 2J 1P A ll P J 2Jo 2pp2 and since Po p Pz 1 some algebra gives fJ Po AJl The system availability probability that the mechanism is up in the seady state condition is thus bill 1 t Avaca ty fJt The matrix of transition probabilities for time increment At may be expressed as and where pPijruJ uOru olru u1O6t 1uru UiO At uilAt UmOM Umt6t Pjt U iPit Pijtt iJ 0 uOAt Ullt ulmM uljAt umAt urrjAt 1umDt JO12m pl P Xt jJ From tbejtb equation in tbe m 1 equations of equation 1819 1818 1819 pt ru Pot uoill pt uiill pt1 uill Pmt uc ru 564 Chapter 18 Stochastic Processes am Queueing which may be rewritten as d I PJtbJ pt L pt lim upt u pt at J 6t t 1 IJ I ul Ij The resulting system of differential equations is pjtujpt Luijpt j O124m 1820 1821 which may be solved when m is finite given initial conditions probabilities A and using lhe result that 20 pt L The solution Potpr PmtJ PI 1822 presents the state probabmties as a function of time in the same manner that pt presented state probabilities as a function of the number of trdIlsitions n given an initial condition vector A in the discretetime modeL The solution to equations 1821 may be sornewhatdif ficult to obtain and in general practice transformation tedutiques are employed 185 THE BIRTHDEATH PROCESS IN QUEUEING The maior application of lhe socalled birthdealh process that we will study is in queue ing or waitingline theory Here birth will refer to an arrival and death to a departure from a physical system as shown in Fig 18 L Queueing theory is the mathematical study of queues or waiting lines These waiting lines occur in a variety of problem environments There is an input process or calling pop ulation from which arrivals are drawn and a queueing system which in Fig 181 consists of the queue and service facility The calling population may be finite or infirjte Arrivals occur in a probabilistic manner A common assumption is that the interarrival times are exponentially dis1ributed The queue is generally classified according to whether its capac ity is infinite or finite and the service discipline refers to the order in which the customers in the queue are served The service mechanism consists of one or more servers and the elapsed service time is commonly called the holding time The following notation will be employed Xl Number of customers in system at time t States 0 12 jjr 1 s Number of servers pit PX jlAl p limpt 4 An Arrival rate given that n customers are in the system JJt Service rate given that n customers are in the system The birthdeath process can be used to describe how XI cbanges through time It will be assumed bere that when Xt lhe probability distribution of the time to the next birth arrival is exponential Vith parameter Aj j O 1 2 Furthermore given Xr the remaining rime to the next service completion is taken to be exponential with parameter j1i j 1 2 Poissontype postulates are assumed to hold so tha the probability of mo than one birth or death at me same instant is zero lnpLi process Arrivals 185 The BirthDeaili Process in Queueilg 565 System Service facility 1 o Queue o o o 0 0 I I I I r1 Departlres I I I 0 I I I I I Figure 181 A simple queucing system A transition diagram is shown in Fig 182 The transition matrix corresponding to equation 1818 is AI tJ 0 0 tID 1tJjc AlAr 0 0 tJzD l 0 0 0 16 C po C 0 0 0 Jt 0 0 0 ij u 0 0 0 J1 l Ar 0 0 0 C Ve note that Plj 0 for j i 1 or j i 1 Furthermore the transition intensities and intensities of passage shown in equation 1817 are UjAyu uJ A forj 12 forji I forjil forjilii 1 The fact that the transition intensities and intensities of passage are constant with time is important in the development of this model The nature of ttansiti9D car be viewed to be specified by assumption or it may be considered as a result of the prior assumption about the distribution of time between occurrences births and deatls Figure 182 Transition diagram for the birthdeath process 5jj Chapter 18 Stochastic Processes and Queueing The assumptions of independent exponentially distributed service times and inde pendent exponentially distributed interarrival times yield transition intensities that are con stant in time This was also observed in the development of the Poisson and exponential distributions in Chapters 5 and 6 The methods used in equations 1819 through 1821 may be used to formulate an infi nite set of differential state equations from the transition matrix of equation 1822 Thus the timedependent behavior is described in the following equations t I and iO 12 0 0 0 1 A ao Ill I aj 1823 1824 In the steady state I 7 we have pt 0 so the steady state equations are obtained from equations 1823 and 1824 and L PI L 11p1 AP APo py I 11 p PI iLJP3 A JiJl p 2PI l1j J Il Pj J I1 JPj1 illP j 11Pjl 1825 Equations 1825 could have also been determined by the direct application of equation 1817 which provides a rate balance or intensity balance Solving equations 1825 we obtain If we let 186 Considerations in Queucing Models 567 then and since we obtain C CZ An j J1jJ1jlfl or 1 Pc LPjl 1 PC 1 LC J1 1826 1827 These steady state results assume that the Ah J1J values are such that a steady state can be reached This will be true if Ai for j k so iliat there are a finite number of states It is also true if p NSt 1 where A andt are constant and denotes the number of serverS The steady state will not be reached if Ll I G QQ 186 CONSIDERATIONS IN QUEUEING MODELS Vhen the arrival rate is constant for alij the constant is denoted l SimUarly when the service rate per busy server is cOlstant it will be denoted fl so that 11 sp ifj 2 S and flj j fl if j s 1ne exponential distributions ACt Nk tO 0 tO TTt IlfJil t 0 0 tO for intearrival times and service times in a busy channel produce rateS A and p which are constant The mean interarrival time is II and the mean time for a busy channel to comM plete service is 11 fl A special set of notation bas been widely employed in the steady SAte analysis of queueing systems This notation is given in the following list and L LJ Pi Expected number of customers in be queueing system L LjS Pj Expected queue length W Expected time in the system including sellce time Wq Expected waiting time in the queue excluding service time If A is constant for all j then it bas been shown that LAW LAW 1828 568 Chapter 18 Stochastic Processes and Queueng These resultsare special cases of what is bown as Littles law If the J are not equal I replaces A where 1 LAjPj 1829 iO The system utilization coefficient p Js1 is the fraction of time that the servers are busy In the case where the mean service time is 111 for allj 1 I W I 1830 The birthdeath process rates 1 1 Aj and Jljl JL Jl j may be assigned any positive values as long as the assignment leads to a steady state solution This allows considerable flexibility in using the results given in equation 1827 The specific models subsequently presented will differ in the manner in which Ay and I vary as a function ofj 187 BASIC SINGLESERVER MODEL VITH CONSTAliT RATES We will now consider the case where s 1 that is a single server We will also assume an unlimited potential queue length with exponential mterarrivals having a constant parame ler A So that t AI A Fnrthennore service times will be assumed to be independ ent and exponentially distributed with 11 JL L We will assume 1 As a result of equation 1826 we have j123 1831 and from equation 1827 jI23 1 Ip 1832 Po lLP il Thus the steady state equations are Pj 1 ppi j 0 12 Note that the probability that there XI3 j customers in the systempj is given by a geometric distribution with parameter p The mean number of customers in the system L is deter mined as L Lj1ppJ jO 2 Ip 1834 17 Basic SingleServer Model fith Constant Rates 569 And the expected queue length is LqIUIpj jl LIpo Using equations 1828 and 1834 we find that the expected waiting time ill the system is and the expected waiting time in the queue is W A 1 1111 11 1836 1837 These results could have been developed directly from the distributions of time in the sys tem and time in the queue respectively Since the exponential distribution reflects a mem oryless process an arrival finding j units in the system will wait through j 1 services including its own and thus its waiting time I is the sum ofj 1 independent exponen tially distributed random variables This random variable was shown in Chapter 6 to have a gamma distribution This is a conditional density given that the arrival filds j units in the system Thus if S represents time in the system PSw 2rPTj1 w jO 1 I lppi reIIdl iO wrJl n pleIII PY dt w 10 1838 e uIP wO WO which is seen to be the complement of the distribution function for an exponential random variahle with parameter 11 pl The mean value W IIJl1 p 11jt follows directly If we let S represent time in the queue excluding service time then PSOp Ip 570 Chapter 18 Stochastic Processes and Queueing If we take T as the sum of service times will again have a gamma distribution Then as in the previous manipulations pSqWq 2jP1jWq 11 Llppj PTjw 1 uIp pe q 0 and we find the distribution of time in the queue gwi for IV q 0 to be Thus the probability distribution is gw Ip 41 Ple Wq 0 1839 1840 which was noted in Section 22 as being for amixedtype random variable in equation 22 Gt 0 and Ht 0 The expected wajting timeln the queue Wcould be determined directly from Ibis distribution as W 1 pO Jo w I pVIdWq 4 1841 Vlhen 2 Ji the summation of the terms PI in equation 1832 diverges In this case there is no steady state solution since the steady state is never reached That is the queue would grow without bound 188 SINGLE SERVER WITH LIMITED QUEUE LENGTH H the queue is limited so that at most N units can be in the system and if the exponential service times and exponential interarrival times are retained from the prior model we have 141 11 40 jcN and 1 f1z 11 It follows from equation 1826 that jN 1842 0 jN 188 Single Server with Limited Qteue Iengtl 571 so that and j OI2N N polj 1 0 Ip 1 pNl As a result the steady state equations are given by jO12 N The mean number of customers in the system in this case is The mean number of customers in tlte queue is N Lq LU 1 Pj fool N N LjPj LPj The mean time in the system is found as and the mean time in the queue is where L is given by equaton 1845 1843 1844 1845 1846 18 m 1848 572 Chapter 18 Stochastic Processes and Queueing 189 MULTIPLE SERVERS WUHAN IMITED QUEUE Ve now consider tbe ease where there are multiple servers We also assume that the queue is unlimited and tbat exponential assumptions hold for interarrival times and senrlee times In this case we have and Thus defining Jp we have 11 c j jJ1 j forj s forj s js j5 It follows trom equation 1827 that the state equations are developed as pbipo j ji j 1 where p YS1 Is is the utilization coefficient assuming p 1 1849 1850 1851 The value Lrr representing the mean number of units in the queue is developed as follows Then L W L q I 1852 1853 lS12 Exercises 573 and 1854 so that 1855 1810 OTHER QUEUEING MODELS There are numerous other queueing models that can be developed from Ule birthdeat1 process In addition it is also possible to develop queueing models for situations involving nonexponential distributions One useful result given without proof is for a singleserver system having exponential interanivals and arbitrary service time distribution with mea 1 J1 and variance d If pAi1 I then steady stare measures are given by equations 1856 Po 1 p i2 p2 L q 21p L pLq 1856 In the case where service times are constant at 1IjL the foregoing relationships yield the measures of system performance by taking the variance cJl O 1811 SUlY1MARY This chapter introduced the notion of discretestate space stochastic processes for discrete time and continuoustime orientations The Markov process was developed along with the presentation of state properties and characteristics TIllS was followed by a presentation of the birthdeath process and several important applications to queueing models for the description of waitingtime phenomena 1812 EXERCISES 1s1 A shoe repair shop in a suburban Call has one shoesmith Stoos are brought in for repair and arrive according ro a Poisson process with a constant arrival rate of two pairs per hour The repair time distribution is exponential kith amean of20 minutes and there is independence betwee the reprur and arrival processes Consider a pair of sQoes to be the unit to be served and do the following a In the steady slate find the probability tlJat the number of pairs of shoes in the system eceeds 5 b Find the mean ll1IDber of pairs in Ie shcp and the nean nunber of pairs waiting for serice c Bnd the mea tumaround time for a pair of shoes time in the shop waiting plus repair but exclud ing time waiting to be picked up 1s2 Weather data are analyzed for a particular local it and a Markov chain is employed as a model for weather change as follows The conditional probahil iy of change from rain to clear weather in one day is 03 Likewise the coditional probabili of transition 574 Chapter 18 Stochastie Processes and Qaeueing from elearto rain in one day is 01 The model is to be a discretetime model with transitions occurring only between days a Determine the matrix P of onestep transition probabilities b Find the steady state probabilities c If today is clear find the probability that it will be dear exactly 3 days hence d Find the probability that the first passage from a clear day to a rainy day OCCUiS in exactly 2 days given a clear day is the initial state e Vhat is the mean reCWence time for the rainy day state 193 A communication link transmits binary chamc tets O 1 There is a probability P t1at a trnnsmitted character will be received corredy by a receiver wdch then transmits to another link etc If Xu is the initial character and Xl is the character received after the first transmission X2 after the second etc then ith independence X is a iarkov chain End the onestep and steady state transitinn matrices 184 Consider a twocomponent active redundancy where the components arc identical and the timeto failure distributions are exponential When both units are operating each cames load U1 and each has fail ure rate I However when one unit fails the load car ried by the other component is L and its failure ate under this load is 15A There is only one repair facility available and repair time is exponentially dis tributed Wtth mean lip The system is considered failed when both components are in the failed state Both components are initially operatirg Assume that J1 lS Let the states be as follows 0 No components are failed 1 OJe component is failed and is in repair 2 Two components are failed one is in repair one is waiting and the system is in the failed condition a Detemine the latrix P of transition probabilities associated vith interval ill b Determine the steady state probabilities c Write the system of differential equations that present the transicct or timedependent relation ships for transitioI 18 RS A communication satellite is launched via a booster system that has a dscretetime guidance con trol system Course correction signals form a sequence X where the state space for X is as follows 0 No correction required 1 lvfiuor correction required 2 Major correction required 3 Abort and system destruct If t Xn can be modeled as a Markov chain with one step tranSition matrix as do he following o 16 23 o a Show that states 0 and 1 are absorbing states b If the initial state is state 1 compute the steady state probability Clat the system is in state O c If he utial probabilities are 01121120 com pute the steady state probability Pc d Repeat c ihA lJ41I41I41I4 186 A gambler bets 1 on each hand of blackjack The probability of winning on any hand is p and the probability of losing is 1 P q Tlle gambler will continue to play until either Yhas been accumulated or he has no money left Let XI denote the acctmu lated Winnings on hand t Kate that X I Xt 1 with probability P that Xl 1 Xr 1 with probability q andXH I zX if XI 0 or XY The stochastic process X is a Markov chain a Find the onestep transition matrix P b For Y 4 and p 03 find the absorption proba bilities blO b 14 bJo and by 187 Au ohject moves between fot points on a circle which are labeled 1 2 3 and 4 11 probability of moving one unit to the right is p and the probability of mOing one unit to the left is 1P q Assume that the object starts at 1 and let Xn denote the location on t1c circle after n steps a Find the onestep transition matrix p b FilYJ an expression for the steady stilZe probabili ties Pi c Evaluate the probabilities Pj for p 0 5 and p 08 18 8 For the singieseter queueing model presented in Section 187 sketch the graphs of the following quantities as a function of p Alp for 0 P L a Probability of no units in the system b Mean time in the system c Mean time in the queue 1s9 Interamval times at a telephone booth are expo nential with an average time of 10 minutes The length of a phone call is assumed to be exponentially cEstnouted with a mean of 3 minutes a What is the probability that a person arnrng at the booth vill have to wait b lh31 is the average queue length c The telephone company will irstall a second booth whe4 an arrival would expect to have to wait 3 minutes or more for the phone By how much must the rate of amvals be increased in order to justify a second booth d What is the probability that an arrival VIill have to wait more than 10 minutes for the phone e What is the pobability that it will take a person more than 1 0 nirutes altogether for the phone and to complete the call 0 Estimate the fraction of I day that the phone will be in use 1810 Automobiles arrive at a serviee station in a ran dam manner at a mean rate of 15 per hour This station has ouy one service position with a mean serviciIg rate of2i customers per hour Service times are expo nentially distributed There is space fur only the auto mobile being sencd and two waiting If all three spaces are filled an miving automobile will go on to another station a hat is the average llJmber of units in the station b bat fraction of customers VIill be lost e VbyisLLl 1811 An engineering school has three secretaries in its general office Professors wim jobs for the secre taries arrive at random at an average rate of 20 per 8hour day The amount of time that a secretary spe1ds on a job has an exponential distribution vith a mean of 40 minutes a What fraction of the time aC the secretaries busy 18 12 Eoercises 575 b Hov much time does it take on average for a professor to get his or her jobs completed c If an economy drive reduced the secretarial force to 1o secretaries what wi1 be the new answers to a and b 18 U The mean frequency of arrivals at an airpor is 18 planes per hour and the mean time that a runway is tied up with an arrival is 2 minutes How many run ways VoiD have to be provided so that the probability of a plane having to wait is 0201 Ignore finite popu lation effects and make the assumption of exponential mJera1ival and service times 113 A hotel reservations facility uses inward WATS lines to service customer requests The mear number of ealls that arrive per hour is 50 and the mean serv ice time for a call is 3 minutes Assume that interar rival and service times are exponentially distributee Calls that arrive when all lines are busy obtaln a busy signal and a lost from the systen a Flnd the steady state equations for dlls system b How many WATS Jines must be provided to ensure that the probability of a customer obtain ing a busy signal is 005 c Mat fraetion of the time are all WATS lines busy d Suppose that during the evening hours cal arrivals occur at a mean rate of 10 per hour How does this affect the WATS line utilization e Suppose the estimated mean servce time 3 mill utes is in enor and the true mean service rine is really5 minutes Wha effect INill this have on the probability of a customer finding all lines busy if the number of lines in b are used Chapter 19 Computer Simulation One of the ast widespread application of probability and statitics lies in the use of com puter simulation methods A simulation is simply an imitation of the operation of a real world system for purposes of evaluating that system Over the past 20 years t computer simulation has enjoyed a great deal of popularity in the manufacturing production logis tics service and financial industries to name just a few areas of application Simulations are often used to analyze systems that are too complicated to attack via analytic methods such as queueing theory We are primarHy interested in simulations that are l Dynamicthat is the system state changes over time 2 Discretethat is the system state changes as the result of discrete events such as customer arrivals or departures 3 Stochastic as opposed to deterministic TIle stochastic nature of simulation prompts the ensuing discussion in the text This chapter is organized as follows It begins in Section 191 with some simple moti vational examples designed to show how one can apply simulation to answer interesting questions about stochastic systems These examples invariably involve the generation of random variables to drive the simulation for example customer interarnval times and serv M ice times The subject of Section 19213 the development of techniques to generate random variables Some of these techniques have already been alluded to in previous chapters but we will give a more complete and selfcontained presentation here After a simulation run is completed one must conduct a rigorous analysis of the resulting output a task made dif ficult because simulation output for example customer waiting times is almost never inde pendent or identically distributed The problem of output analysis is studied in Section 193 A particularly attractive feature of computer simulation is its ability to allow the experi menter to analyze and compare certain scenarios quickly and efficiently Section 194 dis cusses methods for reducing the variance of estimators arising from a single scenario thus resulting in moreprecise statements about system performance at no additional cost in simulation run time Ve also extend this work by mentioning methods for selecting the best of a number of competing scenarios Vle point out here that excellent general references for the topic of stochastic simulation are Banks Carson Nelson and Nicol 2001 and Law and Kelton 2000 191 MOTIVATIONAL EXAMPLES 576 This section illustrates the use of simulation through a series of simple motivational exam ples The goal is to show how One uses random variables witlin a simulation to answer questions about the underlying stochastic system 191 Motivational Examples 577 EXam 191 p Coin Flipping We are interested in simulating independent flips of a fair coin Of course his is a trivial sequence of Bernoulli trials with success probability p 112 but this example serves to show how OnC ea use sim ulation to analyze such a system First of all we need to generate realizations of heads H and tails T each with probability 112 Assuming that the simulation can somehow produce a sequence of independent uniform 01 random numbers Vi V2 we will abitrarily designate fllp i as H if we observe Ui 05 and a flip as T if we observe Vi 2 05 HQIN one generates independent uniforms is the subject of Section 192 In any case suppose tha the following uniforrs are observed 032 041 006 093 082 0L9 021 077 071 008 This sequence of uuifonns corresponds to the outcomes HHHTIHHTTH The reader is asked to study this exanple in various ways in Exercise 191 This type of static simulation in which we simply repeat the same type of trials over and over has come to be known as Monte Carlo simtlation in honor of the European citystate where gambling is a populru recreational activity Estimate r In this example we Will estimate 1r using Mone Carlo simulation in conjunction with a simple geo metric relation Referring to Fig 191 consider a unit square with an Llscribed eLde both centered at l212 If one were to tluow darts randomly at the square the probability that a particular dart willlad in the circle is ref4 tle ratio of the circles area to that of the square Hew can we use this simple fact to estimate r We shall use Monte Carlo sirrllation to throw many darts at the square Specifically generae indepehdent pairs of independent Liiform 01 random variables U Ud U2 U12 Tnese pairs will fall randomlY on the square If for pair i itbappens jat 191 tlen that pair will also fall Vlithin the circle Suppose we run the experiment for II pairs darts Let X I if pair i satisfies ineqmlity 191 tltat is if the itb dart falls in the crcle otherwiselet Xi O Now count up the number of dtS X LJX falling in the circe Clearly X has the binoroial disribution with parameters n and p 1r4 Then the proportionp Xlr is the maximum likelihood estimate for p cf4 and so the rnaximum likelihood estimator for cis justj 4ft If for instance 01 r t 10 r t t J Jet I i cpc cc h J c c r I t c It c c I OOj 10 I Figure 191 Throving darts o estimate i 578 Chapter 19 Computer Simulation we conducted n 1000 trials and obscrvedX 753 dart5 in the circle OUI estimate wocld ber 312 We will encounter this estimation technique again in Exercise 192 Monte Curio lnltgratian Another interesting uSc of computer simulation involves Monte Carlo integration Usually the method becomes efficacious only for highdimensional integrals but we will fall back to the basic onedimensiona1 case for ease of exposition To this eld consider the integral 192 As described in Fig 192 we shall estimate the value of this integral by summiog up n rectangles each ohvidth lin cen randorrly at point U on OlJ and of hcightja b aU Then an esti mate for I is 193 One can show see Exercise 193 that is an unbiased estimator for I that is EVJ 1 for all n Tris makes Z an incitive and attractive estimator To illustrate suppose that we wat to estir1ate the integra 31d the followg n 4 numbers are a uniform 01 samplc 0419 0109 0732 0893 y Figure 192 Monte Carlo integration 191 Motivational Examples 579 Plugging into equation 193 we obtain I IO 21co01OU 0896 I wbicb is close to the actual answer of 1 See Exercise 194 for additioa Monte Carlo integration examples A SingleServer Queue Now the goal is to simulate the behavior of a singleserver queueing system Suppose that sx cus tomers arrive at a bank at the following times which have been generated from some appropiac probability distribution 3 4 6 10 15 20 UpOn arrival customers qeue up in front of a single teller and are processed sequentially in a fust comefirstserved manner The erVice times corresponding to the a1iving customers are 7 6 4 6 2 For this example we assume tat the bank opens at time 0 and closes its doors at tme 20 just after custorrer 6 arrives serving any remaicing customers Table 191 and Fig 193 trace the evolution of the system as time prOgTesses The table keeps track of e times at which customers arrive begin semce Ild leave Figure 193 gTaphs the status of the queue as a function of time in particular it graphsLt the number of customersl the system queue service at time t Note that cusromer i can begin service only at tinte ma A D f that is the maximum of his arrival time and the previous customers depanure time The table und figure are quite easy to inter pret For insta4ce the system is empty until time 3 when customer 1 arrives At time 4 customer 2 a1ives but must wait in line until Cusmer 1 finishes service at time lO We see from llc figure hat between times 20 and 26 eustomer 4 is in scrvee while customers 5 and 6 wait in me queue From the table tbe average waiting time for he six customers is ll W16 4416 Further the avcroge num ber of customers it the system is 19 Ltd129 70129 where we have computed the integral by adding up he rectangles in Fig 193 Exercise 195 looks at extensions of be singleserver queue Many simulation software packages provide Simple ways to model und analyze moreeomplicated queueing networks Table 191 Bank Customers in SingleSener Queueing System t eustomer Ai arrivai time Bi begin servcc St service time D Gepart time 1 wait 3 3 7 10 0 2 4 10 6 16 6 3 6 16 4 20 10 4 0 20 6 26 10 5 15 26 I 27 11 6 20 27 2 29 7 580 Chapter 19 Conputer Simulation Queu e i 1 I 1E I 2 1 3 4 6 Customerl 3 I i I 2 i I i 10 4 3 2 r i 5 i 4 5 6 3 1 4 5 6 121 3 I 4 5 6 I 15 16 20 2627 29 Figure 193 Number of customers Lt in singleserver queueing system EXamJilei s S Inventory Policy Customer orders for a particular good arrive at a store every day DurIDg a certain oneweek period the quantities ordered are 10 6 11 3 20 6 8 The store starts the week off with an initiai stock of 20 If the stock falls to 5 or below the owner orders enough from a central warehouse to replenish the stock to 20 Such replenishment orders are placed only at the end of the day and are receved before the store opens the next day There arc ro cusmmer back orders so any custorter orders that are DOt filled immediatelY ae lost This is called an s S inventory sYStem where the inventory is replenisbed to S 20 whenever it hits level s 5 The fonowing is a history for this system Initial Stock Customer Order End Stock Reorder Lost Orders 20 10 10 No 0 2 10 6 4 Yes 0 3 20 11 9 No 0 4 9 3 6 No 0 5 6 20 0 Yes 14 6 20 6 14 No 0 7 14 8 6 No 0 We see that at the end of days 2 and 5 replenishment orders were made In particular on day 5 the store ran out of stock and lost 14 orders as a result See Exercise 196 192 GEJERATION OF RANDOM VARIABIES All the exampJes described in Section 191 required random variables to drive the simula tion In Examples 19 1 through 193 we needed uniform 01 random variables Example 194 and 195 used morecomplicated random variables to model customer arrivals serv 192 Generation of Random Variables 581 ice times and order quantities This section discusses methods to generate such random variables automatically The generation of uniform 01 random variables is a good pI to start especially since it turns out that uniform al generation forms the basis for the generation of all other random variables 1921 Generating Uniform 01 Random Variables There are a variety of methods for generating uniform 01 random variables among them are the following L Sampling from certain physical devices such as an atomic clock 2 Looking up predetermined random nllIlbers from a table 3 Generating pseudorandom numbers PRNs from a deterntinistic algorithm The most widely used techniques in practice all employ the latter strategy of generating PRNs from a deterministic algorithm Altiougb by definition PRNs ae not truly random there are many algorithms available that produce PRNs that appear to be perfectly random Fu1her these algorithms have the advantages of being computationally fast and repeatablespeed is a good property to have for the obvious reasons while repeatability is desirable for experimenters wbo want to be able to replicate their simulation results when the runs are conducted under identical conditions Perhaps the most popular method for obtaining PRNs is the linear congruential gener ator LeG Here we start with a nonnegative seed integer Xo use the seed to generate a sequence of nonnegative integers X Xz and then convert the Xi to PRNs UI Uz The algorithm is simple 1 Specify a nonnegative seed integer Xj 2 For i 1 2 letX aX c mod m wbere a c andm are appropriately cho sen integer constants and where umod H denotes the modulus function for example 17 mod 5 2 andI mod 5 4 3 For i 1 2j let Ui Xm Eiiii6 Consider the toy generator Xi SXf1 1 mod 8 vith seed Xo O This produces the nteger sequence X 1X2 6 X 7 X4 4Xs 5 X6 2 X 3 Xl O whereupon things start repeat ing or cycling The PRlis corresponding to the sequence stating with seedXo 0 are therefore VI li8 Uz 618 UJ 78 U4 48 U 58 U6 28 U 3fS Us O Since any seed evenJualy pr0 duces all integers 0 1 7 we say that this is ajttUcycle orfull period generator Pleljt Not all generators are full period Consider another toy generator Xi 3Xi1 1 mod 7 With seed Xc O This ptoeuces the integer sequence X 1Xz 4 X3 6 X 5Xj 2X6 0 whereupon cycling ensues Further notice that for this generator a seed of Xc 3 produces the sequence Xl 3 Xl X3 not very random looking The cycle length ofte generator from ExampJe 197 obviously depends on the seed chosen which is a disadvantage Fullperiod generators such as that studied in Example 1967 obviously avoid this problem A fullperiod generator with a long cycle length is given in the following example 582 Chapter 19 Computer Simulation TiIIJ The generator XI 16807 Xl mod 231 1 is full period Since c 0 this generator is termed a mul tipiicative LeG and must be sed with a seed XO O This generator is used in many rea1world appli cations and passes most satistical ests for uniformity and randomness Lrt order to avoid integer overllow and realarithmetic roundoff problems Bratley Fox and Schrage 1987 oner the follow ing Fortran implementation scheme for this algorithm FmCTION ltrJIFIX Kl IXl27773 IX 16807IX K1127773 K12836 p IXLTOIX IX 2147483647 UNIF IX 46566l2S75ElO uRN END L1 the above prognun we input aI integer seed IX and receive a PRN UNIF The seed IX is auto rratical2y updated for the next calL ote t1at b Fortran integer division results in truncation for example 154 3 thus IJ is an wtege 1922 Gimerating Nonuniform Random Variables The goa1 now is to generate random variab1es from distributions other than the uniform The methods we will use to do so always start with a PRN and then apply an appropriate trans formation to the PRN that gives the desired nonuniform random variable Such nonunifonn random variables are important in simulation for a number of reasons for example cus tomer arrivals to a service facility often follow a Poisson process service times may be nor mal and routing decisions are usually cha3cterized by Bernoulli random variables Inverse Transform Methojl for Random Variate Generation The most basic technique for generating random variables from a uniform PRN relies on the remarkable Inverse Transfonn Theorem Theorem 191 If X is a random variable with cumulative distribution function CDF Fx then the random variable Y FCX has the unifOffil 01 distribution Proof For ease of exposition suppose thatX is a continuous random variable Then the CDF of fis GyPYy PFX y PX F y the inverse exists since FCx is continuous FPIy y Since Gy y is the CDF of the unifonn 01 distribution we are done With Theorem 191 in hand it is easy to generate certain random variables All one has to do is the following 1 Find the CDF of X say Fx l I I l 192 Generation of Random Variables 583 2 Set FX U where U is a uniform 01 PRN 3 Solve for X F1 U We illustrate this technique wth a series of examples for both continuous and discete dislributions Here we generate an exponential random variable with rate A fol1owig the ecipe outlined above 1 The COP is Fx le 2 Se FX 1 eX U 3 Solving for X we obtain X rU 1nl UIJ Thus if one supplies a unifoIm 01 PRN U we seehat X Inl U is an exponential ran dom variable vith parameter 1 Now we try to generate a standad normal random variable cal itZ Using the special notation D for the standard nor 01 COP we set 12 U so that Z IIU tfortunate1y he verse COP does not exist in closed fonn so one must reso to the use of standard Doana tables or other approxi mations For instan if we have U 072 then Table II Appendix yields ZplO72 0583 I1ii Ve can exend the previous example to generate any normal random variable that is one with ahi crary lean and variance This fQilows easily since ifZ is standard normal then X 1 aZ is llormcl with mean 1 and variance 52 For inst3nce suppose we are interested in generating a rorroal Briate X Rith mean1 3 and variance J 4 Then if as in the previous example U 072 we obtainZ 0533 and asa consequence X o 3 20583 4166 EpIeQI We C3f ilio use the ideasfrom Theorem 19 1 to generate realizations from discrete random variables SUppose that the discrete random variable X has probability function rO3 ifxl to6 ifx23 Pb 01 ifx o otherwise To generate variates frore this distribution we set uphe following table where Fx is the associated CDF and U denotes the set ofunifonn 01 PRs coresponding to each xvalue x Fx U 1 03 D3 003 23 06 09 0309 7 01 10 0910 To generate a realization of X we first generate a PRN U and then read the corresponding xvalue from the table For instance if U 043 then X 23 584 Chapter 19 Computer Simulation Other Random Variate Generation Iethods Although the inverse transform method is intuitively pleasing to use its reallife application may sometimes be difficult to apply in practice For instance closedform expressions for the inverse CDF r U might not exist as is the case for the normal distribution or appli cation of the method might be unnecessarily tedious We now present a small potpourri of interesting methods to generate a variety of random wables BoxMillier Method The BoxMuller 1958 method is an exact technique for generating independent and iden tically distributed lID standard normal 01 random variables The appropriate theorem stated without proof is Thwrem 192 Suppose that U and U are lID uniform 01 random variables Then Z j2lnU cos2nU and are IID standard normal random variates Kote that the sine and cosine evaluations must be carried out in radians Suppose that Vt 035 and U 065 are two Ill PRNs Using the BoxMilllermethod to generate two normal 01 random variates we obtain Z J2JnO35 cos2rO65 08517 Z 21nO35 sin2rO65 1172 Central Limit Theorem One can also use the Central Limit Theorem CLT to generate quickanddirty random variables that are approxillUltely normal Suppose that U U U are lID PIUs Then for large enough r the CLT says that ZU ElU c IV ar lZ U I I71Ui IilEUd r lvarU 171UI n2 In12 101 192 Generation of Random Variables 585 In particular the choice n 12 which turns out to be large enough yields the conven ient approximation 12 2Ui 6 N 01 jl Suppose we have the following PRNs 028 087 044 049 010 076 065 098 024 029 077 090 Then 12 2 U 6 077 i is a realization from a distribution that is aproximately standard nOrIlat Convolution Another popular trick involves the generation of random variables via convolution indi cating that some sort of sum is involved Suppose that X Xl X are lID exponential random variables with rate A Then Y LIX1 is said to have an Erlang distribution with para1leters n and A It turns out that this distribution has proba bility density function 194 whid readers may recognize as a special caqe of the gamma distribution see Exercise 1916 This distributions CDF is too difficult to invert directly One way that comes to mi1d to gener ae a tion from the BrIang is simply to generate and then add up n TID cxponcntia1 random variables The following scheme is an efficient way to do precisely that Suppose that Vj Vz Ur are TID PRs From Example 199 wc know thatXrxlnl UI i 1 2 n are lID exponer tial random variables I1erefore we can rite This iu1plcmematiou is quite efficient sirce it requires only one execution of a natural log operation In fact we can even do slightly better from an efficiency point of vicwsimpy note that both I ad 1 U ae uniform 01 Then is also Erlang 586 Chapter 19 Computer Simularion To illustrate suppose that we have three lID PRNs at our disposal VI 023 U 097 and U3 048 To generate an Erlang realization V1th parameters n 3 and 1 2we simply take Y InU UU InO23O91O4S 1117 AcceptanceRejection One of the most popular classes of random variate generation procedures proceeds by sam pling PRlis until some appropriate acceptance1 criterion is met Jiitnt9i An easy example of the acceptancerejection technique involves the generation of a geometric taIl dom variable with snccess probability p To this end consider a sequence of PRNs UI U Our aim is to genet3te a geometric realizatior X that is one that has probability function pxi1P p lfx 12 l O othenvise In words X represents the number of Bernoulli trials nntil the first success is obse ed This English characterizatior immcdiately suggests an elemenTary acceptancerejection algorithm 1 Initialize i O 2 Leti ti 1 3 Take a Bemoullip observation 11 I 0 ifUi po otherwise 4 If Y I then we have our first snccess Md We StOp in which case We accept X i Other wise if YO then we reject and go back to step 2 To illustrate let US generate a geometric variate having success probablity p 03 Suppose we have ore at our disposal the fonowing PRNs 038 067 024 089 010 071 Since U 038 cp we have Y10 and so we reject X 1 Since U2 067 cp we have 12 0 and so we reject X 2 Since 13 024 p werurve Y3 1 and so we acceptX3 193 OOTPUT ANALYSIS Simulation output analysis is one of the most important aspects of any proper and complete simulation study Since the input processes driving a simulation are usually random vari ables eg interarrival times service times and breakdown times we must also regard the output from the simulation as random Thus runs of the simulation only yield estimates of measures of system performance eg the mean customer waiting time These estimators are themselves random variables and are therefore subject to sampling errorand sampling error must be taken into aCCOlllt to make valid inferences concerning system performance 193 Output Analysis 587 The problem is that simulations almost never produce convenient raw output that is ffi normal data For example consecutive customer waiting times from a queueing system are not independenttypically they are serially correlated if one customer at the post office waits in line a long time then the next customer is also likely to wait a longtime are not identically distributed customers showing up early in the morning might have a much shorter walt than those who show up just before closing time are not normally distributedthey are usually skewed to the right and are certainly never less than zero The point is that it is difficult to apply classical statistical techniques to the analysis of simulation Output Our purpose here is to give methods to perform statistical analysis of output from discreteevent computer simulations To facilitate the presentation we identify two types of simulations with respect to output analysis Terminati1g and steady state simulations 1 terminating or transient simulations Here the nature of the problem explicitly defines the length oftbe simulation run For instance we might be interested in sim ulating a bank that closes at a specific time each day 2 Nollterminating steady stare simulations Here the longrun behavior of the sys tem is studied Presumedly this steady state behavior is independent of the simula tions initial conditions ln example is that of a continuously running production line for which the experimenter is interested in some longrun performance measure Techniques to analyze output from tenninating simulations are based on the method of independenlreplications discussed in Section 1Y3 I Additional prob1ems arise for steady state simulations For instance we must now worry about the problem of starting the simulationhow should it be initialized at time zero and how long must it be run before data representative of steady state can be collected Initialization problems are considered in Section 1932 Finally Section 1933 deals with point and confidence interval estima tion for steady state simulation performance parameters 1931 Terminating Simulation Analysis Here we are interestedin simulating same system of interest over a finite time horizon For nowj assume we obtain discrete simulation output y Yt H Ymt where the number of observations m can be a constant or a random variable For example the experimentcr can specify the number m of customer waiting times Yl Yt Ym to be taken from a queueing simulation Or m could denote the random number of customers observed during a speci fied time period 0 TJ Alternatively we might observe continuous simulation output YtIO t T over a specified interval 0 TJ For instance if we are interested in estimating the timeaveraged number of customers waiting in a queue during 0 1 the quantity Yt would be the num ber of customers in the queue at time I The easiest goal is to estimate the expected value of the sample mean of the observations e EYmJ where the sample mean in the discrete case is 1 m YmIr m 1 588 Chapter 19 Computer Sirculation with a similar expression for the continuous case For instance we might be interested in estimating the expected average waiting time of all customers at a shopping cemer during the period 10 aill to 2 pm Although Y m is an unbiased estimator for e a proper statistical analysis requires that we also proide an estimate of VarYm Since the Y are not necessarily IID random variables it may be that VarYm i VarYm for any i j a case not covered in elementary statistics courses For this reason the familiar sample variance 2 1 m 2 S yy 1 t m m il is likely to be highly biased as an estimator of m Varim Thus one sbould not use 1m to estimate Varim The way around the problem is via the method of independent replications IR IR estimates Varim by conducting b independem simulation runs replications of the system under study where each replication consists of m observationsl is eruy to make the repli cations independentsimply reinltialize each replication with a different pseudorandom number seed To proceed denote the sample mean from replication i by where YiJis observationj from replication i for i 1 2 b andj 1 2 m If each run is started under the same operating conditions eg all queues empty and idle then the replication sample means Zlj Zb are ill random variables Then the obvious point estimator for Varlm VarZ is 1 b 2 VR 2Z Z bl 1 where the grand mean is defined as Notice how dosely the forms of lin and Slim resemble each other But silce the replicate sample means are IID VR is usually much less biased for VarY m than is S2m In light of the above discussion we see that VIb is a reasonable estimator for VarZ Further if the number of observations per replication m is large enough the Central Limit Theorem tells us that the replicate sample means are approximately lID normal Then basic statistics Chapter 10 yields an approximate 1001 a twosided con fidence interval eI for e 195 where is the 1 cd2 percentage point of the t distribution with b 1 degrees of freedom 193 OutpUt Analysis 589 Suppose we want to estimate the expected avcage waiting time for the first 5000 custOmerS ii a ccr tam queueing system We wiil make five indepedent replications of th system vitl each run ini tialized enpty and idle and consisting of 5000 waiting times The resulting replicate means are 2 3 4 5 Z 32 43 51 42 46 Then Zs 428 andV 0487 For level a 005 we have tom14 272 and equation 195 gives 3415153 as a 95 CI for the expeced average waiting time for the first 5000 customers Independent replications can be used to calculate variance estimates for statistics other tian sample means Then the method can be used to obtain CIs for quantities oilier than EYm for example quantiles See any of the standard simulation teXts for additional uses of independent replications 1932 Initialization Problems Before a simulation can be run one must provide initial values for all of tle simulationS state variables Since the experimenter may not know what initial values are appropriate for the state variables these values might be chosen soewhat arbitrarily For instance we might decide that it is most convenient to initialize a queue as empty and idle Such a choice of initial conditions cae have a significant but unrecognized impact on the simula tion runs outcome Thus the initialization bias problem can lead to errors particularly 11 steady state output analysis Some examples of problems concerning simulation initialization are as follows 1 Visual detection of initialization effects is sometimes difficutespecially in he case of stochastic pocesses having high intrinsic variance such as queueing systems 2 How should he simulation be initialized Suppose that a machine shop doses at a certain time each day even if there are jobs waiting to be served One mllst there fore be careful to 1 each day with a dand that depends on the number of jobs remaining from he previous day 3 Initialization bias can lead tO point estimators for steady state parameters haring high mean squared error as well as for CIs having poor coverage Since initialization bias raises important concerns how do we detect and a1 with it We first list methods to detect it 1 Attempt 10 detect the bias visually by scanning a realization of the simulated process This might not be easy since VLUal analysis can nllSS bias FurJ1er a visual scan can be tedious To make the visual analysis more efficient one might transform he data eg take logs or square roots smooth it aveage it across sev eral independent replications or construct moving average plots 590 Chcpter 19 Computer Simulation 2 Conduct statistical testsJor initialization bias Kelton and Law 1983 give anintu itively appealing sequential procedure to detect bias Various omer tests check to see whether the initial portion of the simulation output contains mOre variation than lat ter portions If initialization bias is detected one may want to do something about it There are tvo simple methods for dealing with bias One is to truncate the output by allowing the simu ation to warm up before data are retained for analysis The experimenter would then hope that the remaining data are representative of the steady state system Output truncation is probably the most popular method for dealing with initialization bias and all of the major simulation languages have builtin truncation functions But how can one find a good truncation point If the output is truncated too early significant bias might still exist in the remaining data If it is truncated too late then good observations might be wasted Unfor tunately all simple rules to determine trmcation pomts do not perfortl well in general A common practice lS to average observations across several replications and then visually choose a truncation point hased on the averaged run See Welch 1983 for a good visualgraphical approach TIle second method is to rrake a very long mn to overwhelm the effects of initializa tion bias This method of bias control is conceptually simple to carry out and may yield point estimators having lower mean squared errors than the analogous estimators from truncated data see g Fisbman 1978 However a problem with his approach is that it can be wasteful with observations for some systems an excessive run length might be required before the initialization effects are rendered negligible 1933 Steady State Simulatiou Analysis Sow assume that we have on hand stationary steady state simulation output YI Y 1 YIl Our goal is to estimate some parameter of interest possibly the mean customer waiting time or the expected proft produced by a certain factory configuration As in the case of termi nating simulations a good statistical analysis must accompany the value of any point esti mator wiL a measure of its variance A number of methodologies have heen proposed in the literature for conducting steady state output analysis batch means independent replications t standardized time series spec tral analysis regeneration time seies modellilg as well as a host of others We will examine the two most popular batch means and independent replications Recall As discussed ear lier confidence intervals for terminating simulations usnally use independent replications BatchMcam The method of batch means is often used to estimate 11lrYn or calculate as for the steady state process mean JL The idea is to divide one long simulation ron into a number of con tiguous batches and then to appeal to a Central Limit Theorem to assume that the resulting batch sample means are approximately IID normal In particular suppose that We partition Yt Y j Y into b nonoverlapping contiguous batches eacb consisting of m observations assume that n bm Thus the itb hatcb i 1 2 b consists of the randoIl variables The ith batch mean is siIDply the sample mean of the mobservations from batch i i 12 b 1 m Z LHmr m 11 194 Comparison of Systems 591 Similar to independent replications we define the batcb means estimatorfor VaiZl as 1 b 2 VB blIZz l wbere 1 b y Zb IZ 1 is the grand sample mean If m is large then the batcb means are approxixnately IID nor mal and as for IR we obrain an approximate 1001 a Clfar f1 JL E Zb ta2blJVBb This equation is very similar to equation 195 Of course the difference here is that batch means divides one long run into a number of batches wbereas independent replica tions uses a number of independent sborter runs Indeed consider the old IR example from Section 1931 with the understanding that the ZI must now be regarded as batch means instead of replicate means then the same numbers carry through the example The technique of batch means is intuitively appealing and easy to understand But problems can come up jf the 1 are not stationary eg if significal1t initialization bias is present if the batcb means are not nonnal or if the batch means are not independent If any of these assumption violations exist poor confidence interval coverage may resultun be knOJ1St to the analyst To ameliorate the initialization bias problem the user can truncate some of the data or make a long run as discussed in Section 1932 In addition the lack of independence or normality of the batch means can be countered by increasing the batch size In Independent Replications Of the difficulties encountered when using batch means the possibility of correlation among the batch means might be the most troublesome This problem is explicitly avoided by the method of IR described in the context of terminating simulations in Section 1931 In fact the replicate means are independent by their construction Unfortunately since each of the b replications has to be started properly initialization bias presents more trou ble when using IR than when using batch means The usual recommendation in the context of steady state analysis is to use batch means over IR because of the possible initialization bias in each of the replications 194 COMPARISON OF SYSTEMS One of the most important uses of simulation output analysis regards the comparison of competing systems or alternative system configurations For example suppose we wish to evdiuate two different restart strategies that an airline can evoke foll01oing a major traf fic disroption such as a snowstorm in the Northeast Vlhich policy minimizes a certain cost function associated with the restart Simulation is uniquely equipped to belp the experi menter conduct this type of comparison analysis There are many techniques available for comparing systemS among them i classical statistical CIs ii common random numbeIS t iii antithetic variates and iv ranking and selection procedures 592 Chapter 19 COrlputer Simclation 1941 Oassical Confidence lntervals With our airline example in mind let 2J be the cost from thejth simulation replication of strategy i i 1 2 j 1 2 bi Assume that Zil Zil Zb are ITO normal with unknown mean I1r and unknown variance i 1 2 an assumption that can be justified by arguing that we can do the following 1 Get independent data by controlling the random numbers between replications 2 Get identically distributed costs beeen replications by performing the replications under identical conditions 3 Get approximately normal data by adding up or averaging many subcosts to obtain overall costs for both strategies The goal here is to calculate a 1001 a CI for the difference 11 jl To this end suppose that the 2 are independent of the zJ ane define and 1 hi z 2 111 b 1 I 1 2 1 L b 2 S 2 Z j b 1 tj 11 I j An approximate 1001 a CI is where the approximate degrees of freedom v a function of the sample variances is given in Chapter 10 Suppose as in airline example that small cost is good Jf the interval lies entirely to the left rigbt of zero then system I 2 is better if the interval contains zero then the two systems must be regarded in a statistical sense as about the same An alternative classical strategy is to use a C1 that is analogous to a paired Hest Here we take b replications from both strategies and set the differences Dj ZJ zJ forj I 2 b Then we calculate the sample mean and variance of the differences 1b 1 b 2 DbLD and SLDjDb b il bl Jl The resulting 1001 a CI is 111 J1z E Db tX2blSb These paired tintervals are very efficient if CorrZJ 2zJ Oj I 2 b where we still assume that 2 Z Zb are lID and zl z z are lID In that case it rums out that v15 V 2j V zj b b Jf ZlJ and zJ hed been simulated independently then we would bave equality in the above expression Thus the trick may result in relatively small S and hence small C1 length So how do we evoke the trick 195 Summrl 593 1942 ConunOll Random Numbers The idea behind the above trick is to use cOmmon rantlom numbers that is to use the same pseudorandom numbers in exactly the same ways for corresponding runs of each of the competing systems For example we might Use precisely the same customer arrival times when simulating different proposed configurations of a job shop By subjecting the alter native systems to identical experimental conditions we hope to make it easy to distinguish which systems are best eVen though the respective estimators are subject to sampling error Consider the case in which we compare two queueing systems A and B on the basis of their expected customer transit times fJA and 88 where the smaller 8value corresponds to the better system Suppose we have estimators A and B for eA and es respectively We will declare4 as the beer system ileA If e and e are simulated independently then the variance of their difference could be very large in which case our declaration might lack conviction H we could reduce V9A fiB then We could be much more confident about our declaration eRN sometimes induces a high positive correlation between the point estimators A and 88 Then we have VA 1Il VA Vils 2eavllA e VA VD and we obtain a savings in variance 1943 Antithetic Random Numbers Alternatively if we can induce negative correlation between two unbiased estimators at and for some parameter e then the unbiased estimator ill 22 might have low variance Most simulation texts give advice on bow to run Lfe simulations of the competing sys tems so as to induce positive or negative correlation between them The consensus is that if conducted properly common and antithetic random numbers can lead to tremendous vari ance reductions 1944 Selecting the Best System Ranking selection ard multiple comparisons methods form another class of statistical techniques used to compare alternative systems Here the experillenter is interested in seiecting the best of a number of competing processes Typically one specifies the desired probability of correctly selecting the best process especially if the best process is signifi cantly better than its competitorS These methods are simple to use airly general and irtu itively appealing See Bechhofer Santner and Goldsman 1995 for a synopsis of the most popular procedures 195 SUMMARY This chapter bega1 vrith SOme simple motivational examples illustrating arious simulation concepts After this the discussion tlrned to the generation of pseudorandom numbers that is numbers that appear to be lID unifOD1 01 PRNs are important because they drive the generation of a number of other important random variables for example normal expo nential and Erlang We also spent a great deal of discussion On simulation output analy sissimulation output is almost never IID so special care must be taken if we are to make 594 Chapter 19 Computer Sillulation statistically valid conclusions about the simulations results We concentrated on output analysis for both terminating and steady state simulations 196 EXERCISES 191 Extension of Example 19 1 a Fup a coin 100 times How nmy heads to do you observe b How many times do you observe two heads in a row T1me in a row Four Five c Find 10 friends and repeat a and b based on a rotal of 1000 flips d Now simulate eoin flips via a spreadsheet pro gram Flip the simulated coin 10000 times and answer a and h 192 Extension ofRxampie 192 Throw 11 datsran domly at a unit square cootaining an inscribed circle Use the results of your tosses to estimate 1C Let r 2 for k 1 2 15 and graph your estimates as a function of k 193 Extension of Example 193 Show thd t defined in equation 193 is urbiased for the integral I defined in equation 192 194 Other cx1ensions ofRxample 193 a Use Monte Carlo ntegration with n10 observa tions to estimate r2 etl2dx Now use r J 2n 1000 Compare to the answer tat you ca1 obtain via nonna tables b Vthat would you do if you had to estimate r 1QJe12dx Jo 21r c Use Monte Carlo integration withn 10 observa tions to estimate I COS21CC dx Now use n 1000 Compare to the actual answer 195 Extension of Example 194 Suppose that 10 customers arrive at a post office at the following times 3 4 6 7 13 14 20 25 28 30 Upon arrival eustomers queue up in ont of a single clerk and are processed in a firstcomewfirstserved JJ1amer The service times corresponding to the arriv ing customers are as follows 60 55 40 10 25 20 20 25 40 25 Assume that the post office opens at time 0 and closes its doors at time 30 Gust after customer 10 arrives SIDing any remaining customers a wnen does the last customer fillally leave the system b What is the average waitiIlg time for he 10 CJstomers c Vlhat is the maximum number of customers in the system When is this maximum achieved d What is the average number of customers in line during the first 30 minutes e Now repeat parts ad assuming ttlat the serv ices are performed lastinmstout 196 Repeat Example 195 whichdea with an s S imentory pOlicy except now use order level sc 6 197 Consider the pseudorandom number generator X 5X I mOO 16 with seed A O a Calculate Xl andX1 along with the eorresponding PR1Ifs U and Uz b Is this afullwperiod generator c What is XJ5I 198 Consider the recoIUTlended pseudorandom nJJLber geoerator XI 16807 XiI mod 231 1 with seed Xc 1234567 a Calculate X andXz along with the corresponding PR1s Vi and V h WhatisXtooooo 199 Show how to use the inverse transform method to generate an exponential ralldom vaiable with rate A 2 Demonsnte your technique using he PRN V075 1910 Consider the inverse transformrnethod to gen erate a standard normal 0 1 random variable 1 DemOllStn1te your technique using the PRN VO25 b Using your answer in a generate aN 09 ran dom variable 1911 SUpPose thatXhas probability density funetion jx xl41 2 x 2 a Develop an inverse transfonn technique to gener ate a realization of X b DemoJstrate yQWtechuque usiDg U 06 c Sketch outex and see if you can come up with another method to geoerate X 1912 Suppose thathe discrete random variable X has probability function 035 px 1025 1 0 40 O if x 25 ifx 10 ifx 105 otherwise As ia Exarple 19 2 set up a table X generate real izations from this distribution Illustrate yOGC tech nique with the PRN U 086 1913 The Weibull a 3 distributiou popular inreli ability theory and oher applied statistics disciplines has CDP ifx 0 otherwise a Show how to use the inverse transform method to generate a realization from the Weibull distribution b Demonstrate your technique for a reibull 1520 random vaiable using the PRN U066 1914 Suppose that VJ 045 and U2 012 are two TID PRs vse the BoxMiiller method to generate two N 01 variates 1915 Consider the following Pls 088 087 033 069 020 079 021 096 011 042 091 070 Use the Central Limit theOrem method to generate a realization that is approximately standard normal 1916 Prove equation 194 from the text Tnis 11OWS that the sum of n ITO exponential tandom variables is Erlang Hint Find the momentgenerating function of y and compare it to fuat of the ganrla distribution 1917 Using ro PRls U 073 and U 011 generate a realization from an Erlang distibuti011 with n2andi3 1918 Suppose that UI U2 Un are PRl4s a Suggest an easy inverse transform method to gen erate a sequence of IID Bernoulli random vari ables each with success pardlIleter p b Show how to use your answer to a to generate a billomial rmdom variate with parameters n and p 1919 Use thc aceeptancerejeetion technique to gen erate a georeetric random variable wiTl success prob ability 025 Use as many of the Pls from Exercise 19 15 as necessary 1920 Suppose that Z 3 z 5 and z 4 are hree batch means resting frm a long ation run Find a 90 tvosided confidence lnterYfh for the mean 1921 Suppose that 11 E 2535 is a 90 confi dence interval for the mean eost incurred by a eenain inventory policy Further suppose that this interval was based on five independent replications of the underlying inventory system Unfortunately the boss has decided that she wants a 95 confidenee interval Can you supply it 196 Eercises 595 1922 The yearly unemployment rates for Andorra daring the past 15 years are as follows 69 83 88 99 92 123 11A 118 121 106 139 92 82 89 110 Use the method of batch means on the above data to obtain a twosided 95 confidence interval for the mean unemployment Use five batches each consist ing of tbJee years data 1923 Suppose that we are interested in steady state confidence intervals for the mean of simulation ouut Xl X2 j X1COOO You can preend hat tiese are wail ing times We have conveniently divided the run up into five batches each of size 2000 supose that the resulting batch means are as follows 100 80 90 110 120 Use the method of batch means on the above data to obtain a twosided 95 confidence interval for the mean 1924 The yearly total snowfall figures for Siberacuse NY during the pat 15 years are as follows 100 103 162 123 88 72 98 121 106 139 92 142 169 110 99 a Use the mehod ofbatcb means on the above data to obtain a twosided 95 confidence interval for the mean yeary snowfaL Use eve batches each consisting of three years data b The corresponding yearly Oml snowfall figures for Buffoonalo IY which is down the road from Siberacuse are as fo11O11s 90 95 72 68 95 110 112 90 75 144 110 123 81 130 145 How does Buffoonalos snowfall compare to Siberacuses Just give an eyeball answer c Now find a 95 confidence inerval for tle dif ference in means beWeen the two cities Hint Think common random numbers 1925 Antithetic variates Suppose hat Xl Xz X are IID with mean 11 and yatce cr Frrthe sup pose that fl Y YIf are also no With meal f1 and Variance 01 The triek here is that we will also asscme that CovX Y 0 for all 1 So in other words the observations within one of the tNO seqences are nD but they are negatively oonelared between sequences a Here is an eKmlpie showing how can we end up wih the above scenario using simulations Let Xi mUi and nO U where the Ui are the usual IID uniform 01 random varib1es i hat is be distribtion of Xi Of Y iL What is CovU U 5 Chapter 19 Computer SirmLation iii Would you expect that CovX 11 07 Answer Yes b LetI and f denote the sample means of theX aed Y resectively each based on n observations Nihout actualy calculating CovXi Yi state how VO 12 compares 0 VX k other words should we do two negativelY correlated runs each consisting of n observations or just one run con sisting of 2r observations e What if you tried to use tills trick when using Monte Carlo simulation to estimate J sinnx dx 19U Another varianee ceduction technique Sup pose that our goal is to estimate the mean jl of some steady state simulation output process Xl Xi XII Suppose we SOIThhow know the expected value of some other RV Y and we also know that CovX 1 0 where X is the sample mell Obviously X is the usual estimator for jL Let us look at another estblaM tor for j namely the controlvariate estimator cX kiY El where k is some consmnt a Show that C is unbiased for jl b Find an expression for VC Commems c Minimize Vee with respect to k 1927 A miscellaneous computer exercise Make a histogram of XI InU for i I 2 20000 where the Ui ae IID uniform 01 Vvhat ki1d of dis tribution does it look like 1928 Another miscellaneous computer exercise Let us see if the Central Limit Theorem works In Exercise 1927 you generated 20000 exponcntiall observations Now form 000 averages 020 observa tiollS each fror theoriginal20QOO More precisely let I Y X 20iJ JJ 11 Make a histogram of the Yr DQ they look approxi M nately nornal 1929 Yet another nUsccllaneous computer exec cise Let us generate some Donnal observations via the BoxMtl1cr method To do so fust generate 1000 pairs of liD uniform 01 flndoID nurnbers UJ Ul l 0 U22 UUlJo U2JOIX Set r XI r21nUljcos2rrUiJ and for i 12 1000 Make a hisrogram of the result ingX The XlS areNOl Now gtaphX vs Y Any commcns Appendix Table I Cumulative Poisson Distribution Table II Cumulative Standard Normal Distribution Table m Percentage Points of the X2 Distribution Table IV Percentage Points of the t Distribution Table V Percentage Points of the F Distribution Chart VI Operatiug Characteristic Curves Chart VII Operatiug Chara teristic Curves for the FixedEffects Model Analysis of Variance Chart vm Operatiug Characteristic Curves for the RandomEffects Model Analysis of Variance Table IX Critical Values for the Wilcoxon TwoSample Test Table X Critical Values for the Sign Test Table XI Critical Values for the Wilcoxon Signed Rank Test Table XII Percentage Points of the Studentized Range Statistic Table xm Factors for QualityControl Charts Table XIV k Values for OneSided and TwoSided Tolerance Intervals Table XV Random Numbers 597 598 Appendix Table Cumulative Poisson Distribution CN x om 005 010 020 030 040 050 060 0990 0951 0904 0818 07l 0670 0606 0548 0999 0998 0995 0982 0963 0938 0909 0878 2 0999 0999 0998 0996 0992 0985 0976 3 0999 0999 0999 0998 0996 4 0999 0999 0999 0999 5 0999 0999 c iJ x 070 080 090 LOO LlO 120 130 lAO 0 0496 0449 0406 0367 0332 0301 0272 0246 1 0844 0808 0772 0735 0699 0662 0626 0591 2 0965 0952 0937 0919 0900 0879 0857 0833 3 O99L 0990 0986 0981 0974 0966 0956 0946 4 0999 0998 0997 0996 0994 0992 0989 0985 5 0999 0999 0999 0999 0999 0998 0997 0996 6 0999 0999 0999 0999 0999 0999 0999 7 0999 0999 0999 0999 0999 8 0999 0999 Co Ar x 150 160 170 180 190 200 210 220 0 0223 0201 0182 0165 0149 0135 0122 0110 0557 0524 0493 0462 O33 Ol6 0379 0354 2 0808 0783 0757 0730 0703 0676 0649 0622 3 0934 0921 0906 0891 0874 0857 0838 0819 4 0981 0976 0970 0963 0955 0947 0937 0927 5 0995 0993 0992 0989 0986 0983 0979 0975 6 0999 0998 0998 0997 0996 0995 0994 0992 7 0999 0999 0999 0999 0999 0998 0998 0998 8 0999 0999 0999 0999 0999 0999 0999 0999 9 0999 0999 0999 0999 0999 0999 10 0999 0999 Appecdix 599 Tablel Cumuative Poisson Distribution continued CN x 230 240 250 260 270 280 290 300 0 0100 0090 0082 0074 0067 0060 0055 0049 0330 0308 0287 0267 0248 0231 0214 0199 2 0596 0569 0543 0518 093 0469 0445 0423 3 0799 0778 0757 0736 0714 0691 0669 0647 4 0916 0904 0891 0877 0862 0847 0831 0815 5 0970 0964 0957 0950 0943 0934 0925 0916 6 0990 0988 0985 0982 0979 0975 0971 0966 7 0997 0996 0995 0994 0993 0991 0990 0988 8 0999 0999 0998 0998 0998 0997 0996 0996 9 0999 0999 0999 0999 0999 0999 0999 0998 10 0999 0999 0999 0999 0999 0999 0999 0999 II 0999 0999 0999 0999 0999 0999 12 0999 0999 c jt x 350 400 450 500 550 600 650 700 I030 0018 0011 0006 0004 0002 0001 0000 1 0135 0091 0061 0040 0026 0017 0011 0007 2 0320 0238 0173 Q24 0088 0061 0043 0029 3 0536 0433 0342 0265 0201 0151 0111 0081 4 0725 0628 0532 0440 0357 0285 0223 0172 5 0857 0785 0702 0615 0528 0445 0369 0300 6 0934 0889 0831 0762 0686 0606 0526 0449 7 0973 0948 0913 0866 0809 0743 0672 0598 8 0990 0978 0959 0931 0894 0847 0791 0729 9 0996 0991 0982 08 0946 0916 0877 0830 10 0998 0997 0993 0986 097L 0957 0933 0901 11 0999 0999 0997 0994 0989 0979 0966 0946 12 0999 0999 0999 0997 0995 0991 0983 0973 13 0999 0999 0999 0999 0998 0996 0992 0987 4 0999 0999 0999 0999 0998 0997 0994 15 0999 0999 0999 0999 0998 0997 16 0999 0999 0999 0999 0999 17 0999 0999 0999 0999 18 0999 0999 0999 9 0999 0999 20 0999 continues 600 Appendix TabId Cumulative Poisson Distribution Cconfinued c At x 750 800 850 900 950 100 150 200 c 0 0000 0000 0000 0000 0000 0000 0000 0000 1 0004 0003 0001 0001 0000 0000 0000 0000 2 0020 0013 0009 0006 0004 0002 0000 0000 3 0059 042 0030 0021 0014 0010 0000 0000 4 0132 0099 0074 0054 OMO 0029 0000 0000 5 0241 0191 0149 0115 0088 0067 0002 0000 6 0378 0313 0256 0206 0164 0130 0007 0000 7 0524 0452 0385 0323 0268 0220 ODlS 0000 8 0661 0592 0523 0455 0391 0332 0037 0002 9 0776 0716 0652 0587 0521 0457 0069 0005 10 0362 0815 0763 0705 0645 0583 0118 0010 11 0920 0888 0848 0803 0751 0696 0184 0021 12 0957 0936 0909 0875 0836 0791 0267 0039 13 0978 0965 0948 0926 0898 0864 0363 0066 14 0989 0982 097Z 0958 0940 0916 00465 OIM 15 0995 0991 0986 0977 0966 0951 0568 0156 16 0998 0996 0993 0988 0982 0972 0664 0221 17 0999 0998 0997 0994 0991 0985 0748 0297 18 0999 0999 0998 0997 0995 0992 0819 0381 19 0999 0999 0999 0998 0998 0996 0875 0470 20 0999 0999 0999 0999 0999 r 0998 0917 0559 21 0999 0999 0999 0999 0999 Ql99 0946 0643 22 0999 0999 0999 0999 0999 0967 0720 23 0999 0999 0999 0999 0980 0787 24 0999 0999 0988 0843 25 0999 0993 0887 26 0996 0922 27 0998 0947 28 0999 0965 29 0999 0978 30 0999 0986 31 0999 0991 32 0999 0995 33 0999 097 34 0998 x Enries in the table are ucs of Fx PX s x Lecc1i B1auic spaces below the last cnllin any 0 column may be rend as 10 I I iOonCil o J or ilpendix 601 1 f 0 sQrff Table II Cumulative Standard Normal Distribution v r lOUT r Iz L 1 eildu 2 z 000 00 0Q2 om 004 z 00 050000 050399 0507 98 051l 97 051595 00 01 053983 054379 054776 OS5l72 055567 01 02 057926 0583 17 058706 059095 059483 02 OJ 061791 062172 062551 062930 063307 03 04 005542 065910 066276 066640 067003 04 05 069146 069497 069847 070194 070540 05 06 072575 072907 073237 073565 073891 06 07 075803 0761 15 076424 036730 077035 07 08 0788 14 079103 079389 079673 0799 54 08 09 081594 081859 082121 082381 082639 09 l0 084134 084375 0846 13 084849 085083 LO L1 0864 33 086650 086864 0S70 76 0872 85 11 12 088493 088686 088877 089065 0892 51 12 13 090320 090490 0906 58 090824 0909 SS 13 14 0924 092073 092219 092364 092506 14 15 93lJ9 093448 cr9i5JD 093699 093822 15 16 cr 094520 0946 30 094738 094845 094950 16 17 095543 095637 095728 0958 18 095907 17 18 0964 07 096485 096562 0966 37 0967 II 18 19 097128 097193 0972 57 097320 0973 81 19 26 iXin is 0977 78 097831 097882 097932 20 21 098214 098257 098300 098341 098382 21 22 098610 098645 098679 0987 i3 098745 22 23 098928 098956 098983 09900 099036 23 24 099180 099202 099224 0992 45 099266 24 25 099379 099396 0994 3 0994 30 099446 25 26 099534 099547 099560 099573 099585 26 27 099653 099664 099674 099683 099693 27 28 099744 099752 099760 099767 09977d 28 29 099813 099819 099825 099831 099836 29 30 099865 099869 099874 099878 099382 30 31 099903 099906 099910 0999 13 099916 31 32 099931 0999 34 099936 099938 09994 32 3J 0999 52 099953 099955 0999 57 099958 33 34 0999 66 0999 68 099969 099970 0999 71 34 35 0999 77 0999 7S 099978 099979 099980 35 36 099984 0999 85 099985 099986 099986 36 37 099989 0999 90 099990 099990 099991 37 38 099993 099993 0999 93 099994 099994 38 39 099995 099995 099996 099996 099996 39 continues 602 Appendix Table II Cumulative Standard Normal Distribution continued J 1 uu Iz e du 21t Z 005 006 007 008 009 z 00 051994 052392 052790 053188 053586 00 01 055962 056356 056749 057142 057534 01 02 059871 060257 060642 061026 061409 02 03 063683 0641J 58 0644 31 064803 065173 03 04 067364 067724 068082 0684 38 068793 04 05 070884 071226 071566 071904 072240 05 06 074537 074857 075175 075490 D742 1 06 07 0773 37 077637 077935 078230 078523 07 08 0802 34 080510 080785 0810 57 0813 27 08 09 08 083141 083397 083646 083891 09 10 6853 14 085543 085769 085993 086214 10 11 8iOb876 97 087900 088100 088297 11 12 089435 089616 089796 089973 090147 12 13 091149 0913 08 091465 091621 091773 13 14 0926 47 0927 85 092922 093056 093189 14 15 093943 094062 094179 094295 0944 08 15 16 095053 095154 095254 095352 0954 48 16 17 095994 0960 80 096164 096246 096327 17 18 096784 096856 096926 096995 097062 18 19 097441 097500 097558 097615 097670 19 20 097982 098030 098077 098124 098169 20 21 098422 098461 098500 098537 098574 21 22 098778 098809 098840 098870 098899 22 23 099061 0990 86 099111 099134 099158 23 24 099286 099305 099324 099343 099361 24 25 099461 099477 0994 92 099506 099520 25 26 099598 099609 099621 099632 099643 26 27 099702 099711 099720 099728 0997 36 27 28 099781 099788 099795 099801 099807 2S 29 099841 099846 099851 099856 099861 29 30 099886 099839 099893 099897 099900 30 31 099918 099921 0999 24 099926 0999 29 31 32 099942 099944 0999 46 099948 099950 32 33 0999 60 0999 61 099962 099964 099965 33 34 099972 099973 099974 099975 099976 34 35 099981 099981 099982 099983 0999 83 35 36 099987 099987 0999 88 099988 099989 36 37 0999 91 0999 92 099992 0999 92 09999l 37 38 099994 099994 099995 099995 099995 38 39 099996 099996 099996 099997 0999 97 39 Appendix 603 Table ill Percentage Points of the Xl Distribution 0995 0990 0975 0950 0900 0500 0100 0050 0G25 0010 0005 1 2 3 4 5 6 7 9 10 11 r 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 4J 50 60 70 80 000 om 007 021 0041 068 099 34 173 216 260 307 357 407 460 514 570 626 684 743 g03 864 926 989 JO52 ILl6 1181 1246 1312 1379 2071 2799 3553 4328 5117 000 002 011 030 055 087 124 165 209 256 305 357 411 466 523 581 641 701 763 826 890 954 1020 1086 1152 1220 1288 1357 1426 1495 2216 2971 3748 4544 5354 000 005 022 048 083 24 69 218 270 325 382 440 51 563 627 691 756 823 891 959 1028 1098 1169 1240 1312 1384 1457 1531 1605 1679 2443 3236 4048 4876 5715 000 010 035 071 Ll5 164 217 273 333 394 457 523 589 657 726 796 867 939 1012 1085 1159 1234 1309 1385 1461 1538 1615 1693 771 1849 2651 3476 4319 574 6039 002 045 021 139 058 237 106 336 161 435 220 535 283 635 558 1034 630 1134 704 1234 779 1334 855 1434 931 1534 1009 1634 1087 1734 1165 1834 124 1934 1324 2034 1404 2134 1485 223 1566 2334 1647 243 1729 2534 1811 2634 1894 2734 1977 2834 2060 2934 2905 3934 3769 4933 4646 5933 5533 6933 6428 7933 271 384 461 599 625 781 778 949 924 1107 1065 1259 1202 1407 1336 1551 1468 1692 1599 r JU 1728 1968 1855 2103 1981 2236 2106 2368 2231 2500 2354 2630 2477 2759 2599 2887 2720 3014 2841 3141 2962 3267 3081 3392 3201 3517 3320 3642 3428 3765 3556 3889 3674 4011 3792 4134 3909 4256 4026 4377 51815576 6317 6750 7440 7908 8553 9053 9658 10l88 502 738 935 1114 1283 1445 1601 1753 1902 2048 2192 2334 2474 2612 2749 2885 3019 3153 3285 3417 3548 3678 3808 3936 4365 4192 4319 4446 4572 4698 663 921 1134 1328 1509 1681 1848 2009 2167 2321 2472 2622 2769 2914 3058 3200 3341 3481 3619 3757 3893 4029 4164 4298 4431 4564 4696 4828 4959 5089 788 1060 1284 1486 1675 1855 2028 2196 2359 2519 2676 2830 2982 3132 3280 3427 3572 3716 3858 4000 4140 4280 4418 4556 4693 4829 4965 5099 5234 5367 5934 6369 6677 742 765 7949 8330 8838 9195 9502 10042 10422 10663 11233 11632 90 5920 6175 6565 6913 7329 8933 10757 1134 11814 12412 12830 100 6733 7006 722 7793 8236 9933 11850 1234 12956 13581 14i17 v cegreos offecdorn 604 Appendix Table IV Percentage Points of the t DistDDution I O IJ2 lo 005 002500 1 0005O025O0005 1 0325 1000 3078 6314 12706 31821 63657 12732 31831 63662 2 0289 0816 1886 2920 4303 6965 9925 14089 23326 31598 3 0277 0765 1638 2353 3182 4541 5841 7453 10213 12924 4 0271 0741 1533 2132 2776 3747 4604 5598 7173 8610 5 0267 0727 1476 2015 2571 3365 4032 4773 5893 6869 6 0265 0718 1440 1943 2447 3143 3707 4317 5208 5959 7 S 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 40 60 120 0263 0262 0261 0260 0260 0259 0259 0258 0258 0258 0257 0257 0257 0257 0257 0256 0256 0256 0256 0256 0256 0256 0256 0256 0255 0254 0254 0253 0711 0706 0703 0700 0697 0695 0694 0692 0691 0690 0689 0688 0688 0687 0686 0686 0685 0685 0684 0684 0684 0683 0683 0683 0681 0679 0677 0674 1415 1397 US3 1372 1363 1356 1350 1345 1341 1337 1333 1330 1328 1325 1323 1321 1319 DI8 1316 1315 1314 1313 1311 1310 1303 1296 1289 1282 1895 1860 1833 1812 1796 1782 1771 1761 1753 1746 1740 1734 1729 1725 1721 1717 1714 1711 1708 1706 1703 1701 1699 1697 1684 l671 1658 1645 2365 2306 2262 2228 2201 2179 2160 2145 2131 2120 2110 2101 2093 2086 2080 2074 2069 2064 2060 2056 2052 2048 2M5 242 2021 2000 1980 1960 2998 3499 4029 4785 5408 28 3355 3833 4501 5041 2821 3250 3690 4297 4781 2764 3169 3581 4144 4587 2718 3106 3497 4025 4437 2681 3055 3A28 3930 4318 2650 3012 3372 3852 4221 2624 2977 3326 3787 4140 2602 2947 3286 3733 4073 2583 2921 3252 3686 4015 2567 2898 3222 3646 3965 2552 2878 3197 3610 3922 2539 2861 3174 3579 3883 2528 2845 3153 3552 3850 2518 2831 3135 3527 3819 2508 2819 3119 3505 3792 2500 2807 3104 3485 3767 2492 2797 3091 3467 3745 2485 2787 3078 3450 3725 2479 2779 3067 3435 3707 2473 2771 3057 3421 3690 247 2763 3047 3408 3674 2462 2756 3038 3396 3659 2457 2750 3030 3385 3646 2423 2390 2358 2326 2704 2971 3307 3551 2660 2915 3232 3460 2617 2860 3160 3373 2576 2807 3090 3291 Source TIlls table is adapted from Biometrika Tables for SraristiciAns VoL t 3rd ediiOD 1966 by permissioI of dle Biometrika Trustees Table V Percentage Points of th F Distribution 1 1l il i v 2 3 4 5 6 9 ill 11 12 8 M li 16 n U 19 20 21 n D 24 e U D U B 30 60 120 v 583 257 202 181 169 162 LS7 154 LSI 149 147 146 145 144 143 142 142 141 141 140 140 140 139 139 139 138 138 138 138 138 136 135 134 132 2 750 300 228 200 LaS 176 170 166 162 160 158 156 LS5 LS3 152 LSI 151 LSO 149 149 148 148 147 147 147 146 146 146 145 145 144 142 140 139 820 315 236 205 188 178 172 167 163 160 LS8 156 LS5 153 152 151 LSO 149 149 148 148 147 147 146 146 145 145 145 145 144 142 141 139 137 4 858 323 239 206 19 179 172 166 163 159 157 155 LS3 LS2 151 LSO 149 148 147 147 146 145 145 144 144 144 143 143 143 142 140 138 137 135 882 328 241 217 19 179 171 166 162 159 LS6 154 152 LSI 149 148 147 146 146 145 144 144 143 143 142 142 142 141 141 141 139 137 135 133 6 898 331 2042 208 19 178 171 165 161 15 155 153 LSI 150 104 147 146 lAS 144 144 143 142 142 141 141 141 lAO lAO lAO 139 37 US 133 131 F025 Vl Degrees of fredom for the numerator VI 7 8 9 to 12 9lD 334 243 208 189 178 170 IM 16 157 154 152 150 149 147 146 145 144 143 143 142 141 141 140 140 139 139 139 138 138 136 133 131 129 919 335 244 208 189 178 170 164 160 LS6 153 LSI 149 148 146 145 144 143 142 142 141 lAO lAO 139 139 13 138 138 137 137 135 132 130 12 926 337 244 20 189 177 170 163 LS9 LS6 LS3 LSI 149 147 146 144 143 142 141 141 140 139 139 138 138 137 137 137 16 136 134 131 129 127 932 338 244 208 189 177 169 163 159 LS5 152 150 1048 146 lAS 144 143 142 141 140 139 139 138 138 137 137 136 136 135 135 133 130 128 125 941 339 245 20 189 177 168 162 158 154 LSI 149 147 145 144 143 141 140 140 139 138 137 137 136 136 135 135 134 134 134 131 129 126 124 15 949 3041 2046 208 189 176 168 162 157 153 LSO 148 146 144 143 141 lAO 139 13 l37 137 LJ6 135 135 134 134 133 133 132 132 130 127 124 122 20 958 343 246 208 88 176 167 161 U6 152 149 147 lAS 143 141 140 139 138 137 136 135 134 134 133 133 132 132 131 131 130 128 125 122 119 24 963 343 246 208 188 175 167 160 LS6 LS2 149 146 144 142 141 139 138 137 136 135 134 133 133 132 132 131 131 130 130 129 126 124 12 118 30 967 344 247 208 188 175 166 16 LS5 151 148 145 143 141 140 138 137 136 135 134 133 132 132 131 131 130 130 129 129 128 125 122 119 116 40 971 345 247 208 188 175 166 159 LS4 LSI 147 145 142 141 139 137 136 135 134 133 132 131 131 130 129 129 128 128 127 127 124 121 LI8 114 Source Adapted with permhiion from Blomerikn Tllbfes illr Swtisticillns Vol 1 3rd edition by E S PearsOD and H O HlITtIlY Cambridge Univenity Press Cambridge 1966 60 976 346 247 208 187 174 165 159 154 LSO 147 144 142 140 138 136 135 134 133 132 131 130 130 129 128 128 127 127 126 126 122 119 Ll6 Ll2 120 980 347 2047 208 187 174 165 158 153 149 146 143 141 139 137 135 134 133 132 131 130 129 128 128 127 126 126 125 125 124 121 117 L13 108 985 348 247 208 187 174 165 15 153 148 145 142 140 138 136 134 133 132 130 129 128 128 127 126 125 125 124 124 123 123 Ll9 LIS 110 100 ContiHlles g g Table V Percentage Points of he F Distrihution cmltinlJed l ll a I Y 2 3 4 5 6 7 9 10 II I 13 14 IS 16 17 19 20 21 22 23 24 25 U 27 28 30 40 60 10 v 31l86 it3 54 454 406 378 359 146 336 329 3lB 314 310 307 305 303 301 299 297 21l5 94 293 292 291 90 289 29 2B8 ll4 79 275 271 2 49JO 5A6 42 378 146 126 311 301 286 281 26 273 270 267 64 162 261 259 257 256 255 254 253 252 251 250 250 249 244 219 235 230 3 5359 916 539 419 362 329 11l7 292 28 273 266 21 256 252 249 246 244 242 2M 238 136 235 21 233 232 31 2JO 229 228 22 223 118 213 203 4 55B 924 534 411 352 118 2J6 281 9 21 254 2048 243 239 2 233 211 229 227 221 213 222 221 219 218 217 217 216 21 2l4 M 199 194 5724 929 53 4l5 t45 311 28 73 261 252 2A5 239 235 231 22 224 222 218 216 214 213 211 210 2J9 208 1JfJ 206 06 un 2 95 100 185 6 5820 933 528 4Ot AD 305 233 255 2A6 39 233 228 224 121 218 215 213 2ll J9 208 206 OS 2M 2Jl 201 00 200 199 198 193 181 182 177 FOI Dgrees of from for the numerator VI 7 8 9 10 U 5891 5944 5tS6 60l9 6071 U5 937 93 939 941 527 525 524 523 522 198 395 394 392 39 317 134 332 UO 27 301 298 296 294 290 278 275 272 270 267 262 259 256 254 2JO B1 2A7 244 242 238 2A1 238 235 212 2Ut 234 210 227 225 221 226 224 221 219 215 223 220 Uti 214 210 219 215 212 2tO 205 216 212 209 206 202 213 209 2M 203 199 210 206 203 200 196 20B 204 200 198 193 206 202 198 191i 191 204 2JJO 196 194 189 202 198 195 192 lB7 201 197 L93 190 L86 199 195 192 189 184 19 194 191 188 U3 197 un L89 187 182 196 192 U Uti 181 195 191 187 ISS 180 194 90 187 U4 179 191 189 L86 un 178 191 UIS U15 182 177 L87 183 L79 176 l71 U2 177 174 171 166 177 L72 168 165 160 172 167 163 160 U5 15 6122 942 520 387 24 In 263 246 234 224 217 210 205 201 197 194 191 Ul9 186 IIl4 183 LSI UlU l8 171 176 175 174 171 172 166 160 155 149 20 6174 44 5IR 384 121 84 39 242 230 220 212 2J16 201 196 192 189 186 U4 L81 179 US 176 174 173 172 171 170 169 168 167 U1 154 lAg JA2 24 6200 945 518 Sl 319 258 2040 228 Vi 210 198 194 100 L87 IIl4 181 179 171 L75 173 172 170 169 168 167 166 16 164 LS7 151 iA5 U8 30 6226 9A1i 517 1S2 317 280 256 238 225 216 20Il 201 196 191 187 Lll4 L31 Ug 176 174 172 170 169 167 166 165 11 163 162 111 154 lAS 41 134 40 6253 947 516 BO 316 278 254 26 221 2l3 205 199 193 Ul9 135 l81 178 175 173 171 169 167 166 164 163 161 U50 159 US LS7 151 144 137 130 60 6179 947 515 379 14 276 234 221 211 203 196 100 186 lS2 178 U5 172 170 16 166 11 162 161 LSI 158 157 156 155 154 147 lAO 132 124 120 63JJ6 948 514 178 312 24 219 232 218 20 00 193 La l83 179 175 172 169 167 164 162 160 159 157 156 154 153 152 151 l50 1042 US L26 Ll7 6133 JA9 513 376 310 247 229 216 206 197 100 185 L80 Uti 172 169 Jj6 If 16J 19 157 155 153 152 150 149 1048 IA7 lA6 US 129 U9 100 l 6 g If Table V Penentage Points of the FDistrlbution comitmed FilMv Vt Degrees of freedom for the numerlltOr VI 3 4 5 6 7 8 9 10 12 15 20 24 30 40 6J 120 j j l i Il 2 3 4 5 6 7 8 9 IO 11 12 13 4 15 16 lJ 18 19 20 21 23 24 27 30 40 60 120 iol4 1995 l1S1 2246 2302 2340 2368 2389 2405 2119 2439 2459 2480 2491 2501 2Ll 85l 1900 1916 1925 1930 1911 1935 1937 1933 1940 J941 1943 1945 1945 1946 1941 013 955 923 912 901 894 3R9 lUiS ItSl 819 874 870 866 864 862 859 U6 UI M om D 599 514 476 453 419 428 421 415 410 4M 400 394 387 184 31 171 41 1m D 512 426 l8j 161 148 337 329 121 18 3l4 307 301 294 290 26 un 410 1M I UI 8 J 1M D9 n W UJ W Uri ill n I UI J 461 UH 341 318 103 292 283 277 27l 267 260 253 246 242 238 234 460 314 334 311 2 285 26 270 2iiS 260 253 246 239 235 23 22 I m B 445 359 320 2 Hi 270 261 25 249 24S 23g 231 221 219 215 2H 441 355 316 293 277 266 253 751 246 241 234 227 219 215 2ll 206 438 352 1J 3 290 2J4 261 254 2AS 242 238 231 223 216 2l1 2111 200 W IO 10 DI I 432 317 307 284 268 27 249 242 237 212 225 2J3 210 205 201 196 430 344 305 28 266 255 246 AO 234 230 223 2 t5 207 203 198 194 1M In 110 WI 1M 1m 426 lAO 301 278 262 251 242 236 230 225 218 fll 203 198 19 189 424 l39 299 276 260 249 240 24 2211 224 216 209 201 196 J92 187 423 337 298 274 259 247 239 232 227 222 215 2JJ7 loY 195 191 185 421 335 2 271 257 246 237 211 225 220 213 206 L97 193 UHs U4 v U9 12 M m I ill 2JO I 1M 417 t32 292 269 253 242 233 227 221 216 209 7 Ol 191 U9 184 L79 408 371 284 261 245 214 225 718 212 20S 200 192 184 119 174 169 400 315 276 251 237 22 217 210 UM 199 L92 184 175 170 165 159 192 to 268 245 229 217 2J9 202 196 191 1 83 175 66 16l 15 155 384 tOo 260 211 22l 210 201 194 LSS 1113 175 167 157 152 146 139 2522 t948 851 569 441 74 30 301 279 262 249 2Jf 230 222 216 211 21l6 ill 19 195 192 189 UIIi 184 182 LSO L79 117 US 174 164 143 132 233 2543 1949 1950 8SS 853 566 S63 4AD 436 370 167 311 323 297 293 2S 211 258 254 245 240 234 230 225 221 218 213 211 207 206 201 201 L un 192 193 JIg L9 UN 187 181 1M L7 181 176 L79 173 177 J7l us 169 173 167 171 LG 110 141 t68 162 158 151 IA7 119 US 125 122 LOa cOnjimlfS 0 g 13 Table V Percenfage POints of lhe FDistribulion colltr1H1ed FOf5Y vIr lJgTCS of freedom for lb mlmt13lOr Y 1 2 3 4 5 6 7 9 10 12 15 20 24 30 40 60 120 1 5 q t i I 647 7995 8647 3996 9218 Sml 9432 9567 9633 9636 9767 949 9931 9972 100 1006 lOW lOJ4 HHB 2 I 3851 3900 3917 3925 3930 1933 1936 l931 3939 3940 3M 3941 394 lAo 3946 3947 194 3949 3950 3 1744 16H l5A4 15l0 148 1473 1462 144 1441 1442 J434 415 417 i4J2 140 114 1399 1395 1390 D 9 w m 1 W UI U g 5 HUll 43 776 739 7l5 69 65 676 66 662 652 643 633 62 613 61 612 607 602 m m n J2 m 7 n n W W W W W W un I n 9 721 511 SAil 472 448 432 420 4f 403 396 387 371 367 361 356 351 345 339 333 10 12 11 I 15 17 9 10 1 22 II 26 I 29 I 30 694 672 655 6Al 630 62Q 612 64 598 592 587 583 579 515 572 69 566 5613 561 559 557 5042 519 515 502 W 1n D UI m M oM W w n n 10 4A7 412 389 313 361 351 344 337 328 31 3J7 301 2 291 285 279 272 497 435 400 377 360 JAS 39 331 125 115 305 295 289 284 21 27J 266 260 g U 10 W 477 415 380 358 141 329 320 312 t06 296 286 216 110 164 259 252 246 2AO 469 408 373 350 331 322 312 305 299 289 2J9 268 263 257 21 2A5 23 232 U 116 W D UI 1m 1U 2 UI 2 2JI 446 3116 151 329 313 301 291 284 217 HiS 257 216 24J 235 229 222 216 209 142 32 3A8 35 3M 297 27 280 73 264 253 242 237 231 215 218 11 t 204 43 17 144 122 305 2m 24 276 210 260 150 139 133 221 221 214 20 200 41 115 3Al 318 302 290 281 213 261 251 2A7 236 130 224 2JR UJ 204 197 132 172 38 311 2911 2jn 278 270 264 254 244 231 27 221 215 208 201 194 429 69 U5 313 297 85 275 268 261 251 AI 230 121 218 212 205 198 L9J 427 361 333 30 244 22 273 2M 259 249 239 228 222 216 209 OJ 195 188 W UI m W n I 422 363 329 106 290 8 269 261 t55 245 234 223 117 211 205 198 191 lB3 420 36 317 34 23S 276 267 259 253 243 232 221 215 209 Hjl 196 189 LSI LIS 359 32 303 2B7 275 265 257 251 241 2H 220 214 207 20t L94 un 119 405 346 313 2W 274 262 253 2A5 239 229 U8 207 2JH 194 til LO 172 61 393 34 301 279 263 2SI 241 233 227 217 206 194 18 182 174 un 158 148 3go 323 289 267 l52 239 230 222 216 205 194 12 L76 169 L61 153 lA3 131 369 312 279 257 241 229 219 211 205 194 U3 L71 164 157 118 139 127 100 I g f l Table V Percentage Points of the FDislrlbutlon continued I e J v 2 3 4 5 6 7 g 9 10 11 J2 13 14 lS 16 17 l 19 20 21 22 24 26 27 2 30 40 60 120 3 4 6 4052 4m5 5403 5625 5764 5859 9850 900 9917 9925 9930 9933 3412 3032 29A6 2lt71 2S24 2791 2120 1616 D75 1225 1L26 lQS6 1OQI 965 933 907 886 868 853 840 829 818 810 802 795 788 782 777 7n 78 764 760 756 731 7OS 635 6i 1800 1121 1092 955 865 g2 756 721 693 670 651 636 6n 611 601 5m 5S5 513 5n 566 561 551 553 5A9 545 542 539 513 493 479 4til 1669 1206 978 845 73 699 655 622 595 574 56 542 529 51 509 Di 494 487 482 476 4n 46 464 460 457 41 451 431 413 3 118 lt98 1139 9l5 U5 701 642 599 567 AI 521 504 489 4f1 467 458 450 443 437 431 426 422 4l8 414 411 407 401 402 313 365 3AS 331 1552 1091 K75 746 663 600 564 532 506 486 469 436 444 434 425 417 410 404 399 391 390 385 3J2 378 375 373 370 351 34 3n 301 1521 1067 847 719 637 580 539 507 482 462 446 432 420 410 401 3S14 87 381 376 371 367 363 3 356 353 350 317 329 312 2 230 Flm Degrelis of frdorn fer the namenter v1 7 8 9 10 12 15 20 23 9936 27G7 1498 10A6 M9 68 561 520 489 464 444 428 414 401 193 384 377 370 364 359 354 350 JAG 342 339 336 333 31 312 295 279 264 WR2 6022 6056 6W6 6j7 6209 9937 9939 9940 9942 9943 9945 ZiA9 2735 2723 nos 2687 2i9 140 1029 IUO 614 603 5A7 506 474 450 430 414 400 389 179 l71 363 356 35J 34 341 336 332 329 3a6 321 UO 3J7 299 282 266 251 1466 1016 793 672 591 535 494 463 439 419 403 389 378 368 360 352 lA6 lAO 335 330 326 322 118 315 312 3l9 07 219 272 256 241 1455 1005 77 662 581 526 485 454 430 410 394 3W 369 359 351 J43 337 33J 326 321 317 313 309 306 301 300 298 lJlO 263 247 232 143 989 7n 647 561 511 471 440 416 3 3W 367 155 146 337 330 313 317 312 307 303 2 2 293 290 287 284 266 250 234 218 1420 972 756 611 2 496 456 425 401 382 366 352 34 331 323 315 3m 3m 298 291 289 285 281 278 275 273 270 251 35 19 204 1402 955 710 616 16 481 411 410 386 166 151 337 326 316 108 300 294 2JPl 233 23 214 Uo vS6 263 Via 257 55 217 220 203 L88 24 613 9946 2600 1393 9A7 731 607 528 473 433 402 59 343 329 31 3OS 300 292 2116 280 2 270 266 262 258 23 22 249 247 229 212 195 19 30 6261 9947 2650 1384 938 723 99 520 465 423 3 310 31 135 321 310 300 292 214 218 272 267 262 258 254 250 247 244 241 239 220 203 LS6 170 40 6287 99A7 2641 117 929 714 591 512 417 417 386 362 JA3 327 113 302 292 214 276 269 264 258 21 219 245 242 238 235 233 230 21 1 194 L76 159 60 6313 994S 2632 11M 920 706 582 SUJ 448 4OR 378 354 331 3111 10 293 2JiJ 275 267 261 2iS 250 245 2A0 236 U3 229 226 223 221 202 184 166 117 120 6339 9949 2622 1156 911 69 574 495 440 400 369 345 325 300 296 284 215 266 258 252 lAG lAO 25 231 227 223 220 21 214 211 192 173 153 132 6366 9950 2613 1346 9m 68g 565 486 431 39l HiO 336 317 300 2117 2J5 265 257 2 242 236 231 226 221 217 211 210 206 203 201 LSD 160 138 100 i g 610 Appendix Chart VI Operating ChMacteristic Curves J i r 2 3 4 5 d 0 OC curves for different values of n for the tvfosided normal test for a leve of significance O05 I I 000 l I 060 L d U i O4C D 6 020 0 d b OC curves fot ciifferent values of n for the twosided nonnal rest for a level of significance OOL SO4rce Ciat VIa e k r md q are reproduced Vlifr pe1Dissior from Operating ChaaeristiC for the Common StJistical Tests of Significance by C L Ferris F E Grebbs and C L WaverA1w11s of 14athematicai Statistics June 1946 Cha1s Vlb c d g k ij I n 0 p and r are reproduceC 1m pemissio1 from Ergineering Srartics 21d edi non by A H Bowker and G 1 Liebernan PreJoceHal Englewood Cliffs NJ 1972 Appendix 611 Chart VI Operating Characteristic Curves continued 100 rlllIiilli 080 f F Co 060 rrIiIicfII c 040 020 o 100 050 00 050 100 150 200 250 300 d c OC curves for different values of n for the onesided normal test for a level of significance a 005 100 I 080 I F rn 060 8 E 040 e 020 I 0 100 050 00 050 100 150 200 250 300 d d OC curves for different values of n for the onesided normal test for a level of significance aOOL continues 612 Appendix Chart VI Operating Characteristic Curves continued 10 08 08 1l 5 04 e 02 d e OC curves for diffent values of n for the tltosided r test for a level of significance ct 005 f OC curves for different values of n for the Nosided t test for a level of significance 0 001 Appentli 613 Chart VI Operating Characeristie Curves comirtlled t 0 5 D 2 10C iFfl1liIircTrI iTT1 090 080 070 Ofie 05C 04 030 g OC curves for different values of II for the oaesided t test for a level of significance Ct 005 1 i 090 0 f i ilixfcIj r j f fiI H 030 t t 020 r C10 I d h OC ctne for differeat values cf n for the onesided t test for a level of significaacc a 001 614 Apendix Chart VI Operating Characteristic Curves conrnued iOOTTrrTTT 070 g I O0UiJjJjJJJjrj CAO 200 24C 2SO 320 360 4CO 1 0 OC CJVCS for different values of r for the tVosided crusquare test for a level of sgnificance a 005 100 i 070 a j60 c II 0 050 I OAO i OM II I 080 iOO 120 160 200 1 2M 260 320 36 400 01 OC cUVcs for different values of r for the twoMsided chisquare test for a level of signillcance a 001 Appendix 615 Chart VI Operating Characteristic Curves continued 100 c yr 080 i c 060 n 8 u 0 if D 040 D e Q 020 0 iO 20 30 AO k OC curves for diffLent values of r for me onesided upper tail chlsquae test fo a leve of significance a 005 100 C80 1 0 n 060 8 l 15 040 D 75 e 50 020 20 5 0 10 20 0 40 50 60 10 80 90 l OC curves for different values of n for the onesided upper tail chisquare test for a level of significance a OOL continues 616 Appendix Chart 1 Operating Characteristic Curves cohtfnued A m ac curves for different values of n for the onesided lower taU chisquare test for a level of siguificaucc t 005 f I 60 I 8 II z 5 i GAOl E a 2l 11 oc C4cves for different vucs of n for the onesided lower taU chisquare test for a level of signifcance aO01 Appendix 617 ChartVI Operating Characteristic Cuves continued 100 rrr 080 OBO E 0 5 OLD 5 i L 020 0 OC curves for diferen va2ues of n for the twosided Ftest for a level of significance cc 005 DC 080 E 060 d 5 040 L 020 0 0 p OC curves for different values of n for the twosided Ftest for a level of sigificance a 001 corrinues 618 Appendix ChartVl Operating Characteristic Curves continued q OC curves for different values of n for tbe onesided Ftest for a level of significance a 005 100 080 if E 06Q 15 s 040 n t o o 100 200 400 500 800 1000 1200 1400 1600 r OC CUlVes for different values of n for the onesided Ftest for a level of significance a 001 Appcndix 619 Chart VII Operating Characteristic Curves for the FixedEffects Model Analysis of Variance 100c 5 0 0 B E e ora 0Q12 3 4 5 100 040 030 020 010 008 0Q7 006 005 004 003 002 001 1 2 fJfora 0Q1 1 3 fora 005 2 3 1 l1umorutor degroos olroodom 2 donomlnator dogroas 01 lroodom T J 4 5 Source Chart vn is adapted with permission from Biometrika Tablesor Srarisricians Vol 2 by E S Pearson ilIld H O Hartley Cambridge Univenity Press Cambridge 1972 continues 620 Appendi ChartVll Opemtiug Characteristic Curves for the FIxedEffects Model Analysis of Variance continued i 030 1 I 020 i g 010 005 M7 i 0 t t 005 I LJt i lS i t oror 01 tproreiC1 2 3 4 100 oen 070 060 050 i 02C 10 I 006 il eC7 000 C 05 1 01 oosl Appendix 621 Chart vn Operating Characteristic Curves for the FixerlEffects Model Analysis of Variance continued I I I 5 00 00 070 050 050 OAO 00 O03C00 0111 O011LlLlLlSLLLLLlJ 2 4 continues 622 Appendix Chart vn Opeating Characteristic Curves for the FixedEffects Model Atalysis of Variance continued 100 T OllIJj 10 I I 001 l31ao0S cra OOl1 2 4 Appendix 623 Chart VIII Operating Olaractcristic CureS ror the RandomEffects Model Analysis ofVaiance 4 I g S u 15 g jj D e 00 O1ll O1C 000 05C OA 030 020 010 Q08 0Q7 006 OOS 004 om M2 000 00 000 OSO 040 020 i 020 s g K 010 008 0 0C7 C 006 f a JCS E 304I U Mai 190 fw l 005 110 tOO ooL 1LJL 3 5 1 9 11 i3 15 17 19 21 23 25OlorculOSj 5 7 11 3 15 17 19 21 23 25 Sorce Reproduced with pennission from Ettgineern8 Statistics 2nd edition by A H Bowker and G Lieberman PnticeHalJ Englewood Cliffs NJ 1972 continues 624 Appendix Chart VDI Operating Characteristic CtUYCS for the RandomEffects Model Analysis of Vaiance continued 003 002 1 I I 00 t I 1 t 3 4 5 6 7 0 10 11 fort 005 lforXOOl 1 3 4 5 6 7 8 9 10 12 2 4 5 0 13 I 1 Appendix 625 Chart vm Operating Characteristic Curves for the RandolEffects Mode Analysis of Tariance coLd 00 C80 70 60 cso C40 cso l c2C Ii i i 3 010 i 000 i f t Jl C04 003 C02 001 L 25LjlJ tfpra vOl1 2 3 4 5 bra C051 6 1 continues 626 Appendix Chart vm Operating Characteristic Curves for the RandomEffects Model Analysis of Variance contirued 00 c CTrr 0 070 000 i C50 00 I O3Q I 1 02 i 7 tOO Yr Tr j 083 070 CuD i 050 i GAO OOSO i 020 I l 000 007 2 OAiS 0C5 il i 002 for ti 005 1 7 I r I Appendix 627 Table IX Critical Values for the Wl1coxon TwoSample Test R05 n 2 3 4 5 6 7 8 9 10 11 12 l3 14 15 4 10 5 6 11 17 6 7 12 18 26 7 7 13 20 27 36 8 3 8 14 21 29 38 49 9 3 8 15 22 31 40 51 63 10 3 9 15 23 32 42 53 65 78 1J 4 9 16 24 34 44 55 68 81 96 12 4 10 17 26 35 46 58 71 85 99 115 3 4 10 18 27 37 48 60 73 88 103 119 137 14 4 I 19 28 38 50 63 76 91 106 123 141 160 15 4 11 20 29 40 52 65 79 94 110 127 145 164 185 16 4 12 21 31 42 54 67 82 97 114 131 150 169 17 5 12 21 32 43 56 70 84 100 l7 135 154 18 5 13 22 33 45 58 72 87 103 121 139 19 5 13 23 34 46 60 74 90 107 124 20 5 14 24 35 48 62 77 93 110 21 6 14 25 37 50 64 79 95 22 6 15 26 38 51 66 82 23 6 15 27 39 53 68 24 6 16 28 4l 55 25 6 16 28 42 26 7 17 29 27 7 17 28 7 oune ReproduCed witt permission from The Use of Ranks in A Test of Significance for Comparing Two Treatments by C Nlllte Biomerrics1952 Vol 8 p 37 For large n and 1 R is ltorimarely normaly ditributed with mean nln n 112 and variance 1l1n1t1 11 112 coninUes 628 Appendix ubi IX Critical Values for the Wilcoxon TwoSample Test continued 1 nl 2 3 4 5 6 7 8 9 10 II 12 13 14 15 5 15 6 10 16 23 7 10 17 24 32 8 II 17 25 34 43 9 6 11 18 26 35 45 56 10 6 12 19 27 37 47 58 71 11 6 12 20 28 38 49 61 74 87 12 7 13 21 30 40 51 63 76 90 106 13 7 14 22 31 41 53 65 79 93 109 125 14 7 14 22 32 43 54 67 81 96 112 129 147 15 8 15 23 33 44 56 70 84 99 115 133 151 171 16 8 15 24 34 46 58 72 86 102 119 137 155 17 8 16 25 36 47 60 74 89 105 122 140 18 8 16 26 37 49 62 76 92 108 125 19 3 9 17 27 38 50 64 78 94 111 20 3 9 18 28 39 52 66 81 97 21 3 9 18 29 40 53 68 83 22 3 10 19 29 42 55 70 23 3 10 19 30 43 57 24 3 10 20 31 44 25 3 il 20 32 26 3 11 21 27 4 11 28 4 Appendix 629 Table X Critical Values for the R Z 010 005 001 IX 010 005 om n n 5 0 23 7 6 4 6 0 0 24 7 6 5 7 0 0 25 7 7 5 8 0 0 26 8 7 6 9 0 27 8 7 6 10 0 28 9 8 6 11 2 0 29 9 8 7 12 2 2 30 10 9 7 13 3 2 31 10 9 7 14 3 2 32 10 9 8 15 3 3 2 33 II 10 8 16 4 3 2 34 II 10 9 17 4 4 2 35 12 II 9 18 5 4 3 36 12 11 9 19 5 4 3 37 13 12 0 20 5 5 3 38 13 12 10 21 6 5 4 39 13 12 II 22 6 5 4 40 14 13 II For n 40 R is approximately normally distributed with mean nI2 and variance nl4 630 Appendix Table XI Critical Values for the Wilcoxon Signed Rank Test EI 010 005 002 OO I 00 005 002 001 4 I 28 130 116 101 9 5 I 0 29 140 126 110 100 6 2 0 30 151 137 120 109 7 3 2 0 31 163 147 130 118 8 5 3 0 32 175 159 140 128 9 8 5 3 1 33 187 170 151 138 10 10 8 5 3 34 200 182 162 148 11 13 10 7 5 35 213 195 173 159 12 17 13 9 7 36 227 208 185 171 13 21 17 12 9 37 241 221 198 182 14 25 21 15 12 38 256 235 211 194 15 30 25 19 15 39 271 249 224 207 16 35 29 23 19 40 286 264 238 220 17 41 34 27 23 41 302 279 252 233 18 47 40 32 27 42 319 294 266 247 19 53 46 37 32 43 336 310 281 261 20 60 52 43 37 44 353 327 296 276 21 67 58 49 42 45 371 343 312 291 22 75 65 5 48 46 389 361 328 307 23 83 73 62 54 47 407 378 345 322 24 91 81 69 61 48 426 396 362 339 25 100 89 76 68 49 446 415 379 355 26 110 98 84 75 50 466 434 317 373 27 119 107 92 83 Source Adapted with peonission from Extended Table of the Wilcoxon Matched Pair Signed Rank Statistic by Robert L McComnck Joumw of the American Statistical AssocatiOIl Vol 60 September 1965 If n 50 R is approximately normally distributed with mean nn 1 14 and varianoe rtr lX2Jt 1 1124 Thhe XII Percentage Point of the Studentized Range StJlisli QO01Pf p f 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 1 900 135 164 186 202 ll 227 237 246 253 260 266 272 272 282 286 290 294 298 2 140 190 223 247 266 282 295 307 317 326 334 314 348 354 3M 365 370 375 379 3 826 106 122 133 142 150 156 162 167 171 175 179 182 185 188 191 193 195 198 4 651 812 917 996 106 ILl ll5 119 123 126 128 131 ll3 135 137 J39 141 142 144 5 570 697 780 842 891 932 967 997 lO24 WAS 1070 1089 1108 1121 1110 1155 lL68 lLKl lL93 6 524 633 703 756 797 832 861 887 910 930 949 965 981 995 lO08 1021 JO32 1013 1054 7 495 592 654 701 737 768 794 S17 837 855 B7l 886 900 912 924 935 916 955 965 8 474 563 620 663 696 724 747 768 787 803 818 831 844 855 866 876 885 894 903 9 460 543 596 635 666 691 713 732 749 765 778 791 803 813 823 832 841 849 857 10 4A8 527 577 614 643 667 687 705 721 736 748 760 771 781 791 799 807 815 822 11 439 514 562 597 625 648 667 684 699 713 725 736 746 756 765 773 781 7R8 795 12 432 504 550 584 610 632 651 667 681 64 70 717 726 736 744 752 759 766 773 13 426 496 540 573 598 619 637 653 667 679 690 701 710 719 727 734 742 748 755 14 421 489 532 563 588 608 26 641 654 666 677 687 696 705 712 720 727 733 739 15 417 483 525 556 580 599 616 631 644 655 666 676 684 693 70 U7 714 720 726 16 413 478 519 549 572 592 608 622 635 646 656 666 674 682 690 697 703 709 715 17 410 474 514 5 3 566 585 601 615 627 638 618 657 666 673 680 687 694 700 705 18 407 470 509 538 560 579 594 608 620 631 611 650 658 665 672 679 685 691 696 19 405 467 505 533 555 571 589 602 614 625 634 643 651 658 665 672 678 684 689 20 402 464 502 529 551 569 584 597 609 619 629 637 645 652 659 665 671 676 682 24 396 454 491 517 537 554 569 581 592 602 611 619 626 633 639 645 651 656 661 30 389 445 480 505 524 540 554 565 576 585 593 601 60S 614 620 626 631 636 641 40 382 437 470 493 51l 527 539 550 560 569 577 84 590 596 602 607 612 617 621 60 376 428 460 120 370 420 450 364 412 440 482 499 471 487 460 476 536 545 553 560 567 573 579 584 589 593 598 602 521 530 538 5A4 551 556 561 566 571 575 579 583 508 516 523 529 535 540 545 549 554 557 561 565 513 525 501 512 488 491 i f degress of freedom lrom J M May lileuded and Corrected Tables of tbe Upper Peenllge Points of tbe Slldellized Rang JJiomelrika Vbl 39 pp 192193 1952 Reproduced by permission of the truseM of Biowetrlka contllIues el Table 1I Percentage Points of the Studcnlized Range StatisticacQIIilwed f I 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 24 30 4Q 60 120 qpf p 2 3 4 5 6 7 8 9 10 II 12 13 14 15 16 11 18 19 20 181 609 450 393 267 328 372 405 431 828 980 1089 1173 1243 588 683 751 804 847 500 576 631 673 706 454 473 491 506 519 532 543 554 563 572 580 588 596 1303 1354 1399 1439 1475 1508 1538 1565 1591 1614 1636 1657 1677 885 918 946 972 995 1016 1035 1052 JO69 1084 1098 1112 1124 735 760 783 803 821 837 852 867 880 892 903 914 924 364 460 522 567 603 633 658 680 699 717 732 747 760 772 783 346 434 490 531 563 589 612 632 649 665 679 692 704 714 724 334 416 468 506 535 559 580 599 615 629 642 654 665 75 684 326 404 453 489 517 54Q 560 571 592 605 618 629 639 6A8 651 320 395 442 476 502 524 543 560 574 587 598 609 619 628 636 315 388 433 466 491 512 530 546 560 572 583 593 603 612 620 311 382 426 458 482 503 520 535 549 561 571 581 590 598 606 308 377 420 451 475 495 512 527 SAO 551 561 571 580 588 595 306 373 415 446 469 488 505 519 532 543 553 563 571 579 586 303 370 411 441 464 483 499 513 525 536 546 556 564 572 579 301 367 408 437 459 478 494 508 520 531 540 549 557 565 572 300 365 405 434 456 474 490 503 515 526 535 544 552 559 566 298 362 402 431 452 470 486 499 511 521 531 539 547 555 561 297 361 400 428 449 467 483 496 507 517 527 535 543 550 557 296 359 398 426 447 464 479 4n 504 514 523 532 539 546 553 295 358 396 424 445 462 477 490 501 511 520 528 536 543 550 292 353 390 417 437 454 468 481 492 501 510 518 525 532 538 289 348 384 411 430 446 460 472 4B3 492 500 508 515 521 521 286 344 379 404 423 439 452 463 474 4B2 490 498 505 511 517 283 340 374 398 416 431 444 455 465 473 481 488 494 500 506 280 336 369 392 410 424 436 447 456 464 471 478 484 490 495 277 332 363 386 403 411 429 439 447 455 462 468 474 480 484 793 803 734 143 693 701 665 673 644 651 627 634 614 620 602 609 593 600 586 592 579 585 573 579 568 574 563 569 559 565 556 561 544 550 533 538 522 527 51 I 515 500 504 498 493 812 751 708 680 658 641 627 615 606 598 591 584 579 574 570 566 555 543 532 520 509 497 821 759 716 687 665 647 633 621 611 603 596 590 584 579 575 571 559 548 536 524 513 501 li f t Appendix 633 Tablxm Factors for QualityCon1rol oarts x Chart R Chart Factors for Factors for Factorsfo Control Limits Central Line Control Limits n 2 3760 1880 1128 0 3267 3 2394 1023 1693 0 2575 4 1880 0729 2059 0 2282 5 1596 0517 2326 0 2115 6 1410 0483 2534 0 2004 7 1277 0419 2704 0076 1924 8 1175 0373 2847 0136 1864 9 1094 0337 2970 0184 816 10 1028 0308 3078 0223 1717 11 0973 0285 3173 0256 1744 12 0925 0266 3258 0284 1716 13 0884 0249 3336 0308 1692 14 0848 0235 3407 0329 1671 15 0816 0223 3472 0348 1652 16 0788 0212 3532 0364 1636 17 0762 0203 3588 0379 1621 18 0738 0194 3640 0392 1608 19 0717 0187 3689 0404 1596 20 0697 0180 3735 0414 1586 21 0679 0173 3778 0425 1575 22 0662 0167 3819 0434 1566 23 0647 0162 3858 0443 1557 24 0632 0157 3895 0452 1548 25 0619 0153 3931 0159 1541 7t 25 AI 3fh n nl1ITber of obserYlticns in sample 634 Appendix Table XIV k Values for OTleSidcd and TwoSided Tolerance Intervals OneSided mlerance Intervals Confidence Level 090 095 099 Percent Coverage 090 095 099 090 095 099 090 095 099 2 10253 13090 18500 20581 26260 37094 103029 131426 185617 3 4258 5311 7340 6155 7656 10553 13995 17370 23896 4 3188 3957 5438 4162 5144 7Q42 7380 9083 12387 5 2742 3400 4666 3407 4203 5741 5362 6578 8939 6 2494 3092 4243 3006 3708 5062 4411 5406 7335 7 2333 2894 3972 2755 3399 4642 3859 4728 6412 8 2219 2754 3783 2582 3187 4354 3497 4285 5812 9 2133 2650 3641 2454 3031 4143 3240 3972 5389 10 2066 2568 3532 2355 2911 3981 348 3738 5074 11 2011 2503 3443 2275 2815 3852 2898 3556 4829 12 1966 2448 3371 2210 2736 3747 2m 3410 4633 13 1928 2402 3309 2155 2671 3659 2677 3290 4472 14 1895 2363 3257 2109 2614 3585 2593 3189 4337 15 1867 2329 3212 2068 2566 3520 2521 3102 4222 16 1842 2299 3l72 2033 2524 3464 2459 3028 4123 17 L819 2272 3137 2002 2486 3414 2405 2963 4037 18 1800 2249 3105 1974 2453 3370 2357 2905 3960 19 1782 2227 3077 1949 2423 3331 2314 2854 3892 20 1765 2028 3052 1926 2396 3295 2276 2808 3832 21 1750 2190 3028 1905 2371 3263 2241 2766 3777 22 1737 2174 3007 1886 2349 3233 2209 2729 3727 23 1724 2159 2987 1869 2328 3206 2180 2694 3681 24 1712 2145 2969 1853 2309 3181 2154 2662 3640 25 1702 2132 2952 1838 2292 3158 2129 2633 3601 30 1657 2080 2884 Im 2220 3064 2030 2515 3447 40 1598 2010 2793 1697 2125 2941 1902 2364 3249 50 1559 1965 2735 1646 2065 2862 1821 2269 3125 60 1532 1933 2694 1609 2022 2807 1764 2202 3038 70 1511 1909 2662 1581 1990 2765 1722 2153 2974 80 1495 1890 2638 1559 1964 2733 1688 2114 2924 90 1481 1874 2618 1542 1944 2706 1661 2082 2883 100 1470 1861 2601 1527 1927 2684 1639 2056 2850 Appendix 635 ThbleXIV k Values for OneSided and TwoSided Tolerance Intervals continued TwoSided Tolerance Intervals Confidence Level 090 095 099 Percent Coverage 090 095 099 090 095 099 090 095 099 2 15978 18800 24167 32019 37674 48430 160193 18849 242300 3 5847 6919 8974 8380 9916 12861 18930 22401 29055 4 4166 4943 6440 5369 6370 8299 9398 ILl50 14527 5 3949 4152 5423 4275 5079 6634 6612 7855 10260 6 3131 3723 4870 3712 4414 5775 5337 6345 8301 7 2902 3452 4521 3369 4007 5248 4613 5488 7187 8 2743 3264 4278 3136 3732 4891 4147 4936 6488 9 2626 3125 4098 2967 3532 4631 3822 4550 5966 10 2535 3018 3959 2839 3379 4433 3582 4265 5594 11 2463 2933 3849 2737 3259 4277 3397 4045 5308 12 2404 2863 3758 2655 3162 4150 3250 3870 5079 13 2355 2805 3682 2587 3081 4044 3130 3727 4893 14 2314 2756 3618 2529 3012 3955 3029 3608 4737 15 2278 2113 3562 2480 2954 3878 2945 3507 4605 16 2246 2676 3514 2437 2903 3812 2072 3421 4492 17 2219 2643 3471 2400 2858 3154 2808 3345 4393 18 2194 2614 3433 2366 2819 3702 2753 3279 4307 19 2112 2588 3399 2331 274 3656 2703 3221 4230 20 2152 2564 3368 2310 2752 3615 2659 3168 4161 21 2135 2543 3340 2286 2723 3577 2620 3121 4100 22 2118 2524 3315 2264 2697 3543 2584 3078 4044 23 2103 2506 3292 2244 2673 3512 2551 3040 3993 24 2089 2489 3270 2225 2651 3483 2522 3004 3947 25 2071 2474 3251 2208 2631 3457 2494 2972 3904 30 2025 2413 3170 2140 2529 3350 2385 2841 3733 40 1959 2334 3066 2052 2445 3213 2247 2677 3518 50 1916 2284 3001 1996 2379 3126 2162 2576 3385 60 1887 2248 2955 1958 2333 3066 2103 2506 3293 70 865 2222 2920 1929 2299 3021 2060 2454 3225 80 1848 2202 2894 1907 2272 2986 2026 2414 3173 90 1834 2185 2872 l889 2251 2958 1999 2382 3130 100 1822 2172 2854 1874 2233 2934 1977 2355 3096 636 Appendix ThbleXV Random Numbers 1G480 15011 01536 02011 81647 91646 69179 14194 62590 22368 46573 25595 85393 30995 89198 27982 53402 93965 24130 48360 22527 97265 76393 64809 15179 241130 49340 42167 93093 06243 61680 07856 16376 39440 53537 71341 37570 39975 81837 16656 06121 91782 6G468 81305 49684 77921 06907 11008 42751 27756 53498 18602 70659 90655 99562 72905 56420 69994 98872 31016 71194 18738 44013 96301 91977 054i3 Y1972 18876 20922 94595 56869 69014 89579 14342 63661 10281 17453 18103 57740 84378 25331 85475 36857 53342 53988 53060 59533 38867 62300 08158 28918 69578 88231 33276 70997 79936 56865 05859 90106 63553 40961 48235 03427 49626 69445 18663 72695 52180 09429 93969 52636 92737 88974 33488 36320 17617 30015 10365 61129 87529 85689 48237 52267 67689 93394 01511 07119 97336 7lG48 08178 77233 13916 47564 81056 97735 51085 12765 51821 51259 77452 16308 60756 92144 49442 02368 21382 52404 60268 89368 19885 55322 44819 01188 own 54092 33362 94904 31273 G4146 18594 29852 71585 52162 53916 46369 58586 23216 14513 83149 98736 23495 07056 97628 33787 09998 42698 06691 76988 13602 51851 48663 91245 85828 14346 09172 30168 90229 04734 59193 54164 58492 22421 74103 47070 25306 764i8 26384 58151 32639 32363 05597 24200 13363 38005 94342 28728 35806 29334 27001 87637 87308 58731 00256 45834 15398 46557 02488 33062 28834 07351 19731 92420 60952 61280 50001 81525 72295 04839 96423 24878 82651 66566 14778 76797 29676 2591 68086 26432 46901 20849 89768 81536 86645 00742 57392 39064 66432 34673 40027 32832 61362 98947 05366 G4213 25669 26422 44107 44048 37937 639G4 45766 91921 26418 64117 94305 26766 25940 39972 2229 71500 00582 04711 87917 77341 4226 35126 74087 99547 81817 00725 69884 62797 56170 86324 88072 76222 36086 84637 69011 65795 95876 55293 18988 27354 26575 08625 40801 25976 57948 29888 886G4 67917 48708 18912 82271 65424 09763 83473 73577 12908 30883 18317 28290 35797 05998 91567 42595 27958 30134 04024 86385 29880 99730 55536 17955 56349 90999 49127 20044 59931 06115 20542 18059 4i503 18584 18845 49618 023G4 51038 20655 58727 28168 92157 89634 94824 78171 34610 82834 09922 25417 44137 14577 62765 35605 81263 367 47358 56873 56307 61607 98427 07523 33362 64270 01638 W477 669 98420 04880 34914 63976 88720 82765 34476 17032 87589 40836 32427 70060 20277 39475 46473 23219 53416 94970 25832 69975 53976 54914 06990 67245 68350 82948 11398 42878 80287 76072 29515 40980 07391 58745 25774 22987 80059 39911 90725 52210 83974 29992 65831 38857 50490 83765 55657 64364 67412 33339 31926 148S3 24413 59744 92351 97473 08962 00358 31662 25388 61642 34072 81249 35648 56891 95012 68379 93526 70765 10592 G4542 76463 54328 02349 15664 10493 2G492 38391 91132 21999 59516 81652 27195 References Agresti A and B Coull 1998 Approximate is Better than Exact for Jrterval Estimation of Bnomiai Proportions The Awerican Statistician 522 t derson V L ardR A McLean 1974 Design of Experiments A Realistic Approach 1arcel Delker New York Banks J S Carsoo BL Nelson andD MNicol 2001DiscreteElent System Simulation 3rd edi tion PrenticeHall Upper Saddle River NJ Bartlett M S 1947 The Use ofTransforma tions Biometrics Vol 3 pp 3952 Bechhofer R E T 1 Santner and D Goldsman 1995 Design and Analysis of Experiments for Statistical Selection Screening and Multiple Com parnaM lohn Wiley and Sors New York BelsleY D A E Kuh and R E Welsch 1980 Regression Diagnostics John Wiley Sons New York Berrettoni J M 1964 Practical Applications of the Weibull Distribution Industrial Quality Con I Vol 21No 2pp 7179 Box G E P and D R Cox 1964 AllA1alysis of Ttansfonlations Journal of the Royal Statistical Society B VoL 26 pp 211252 Box G E p and M F Mdller 1958 A Note on the Generation of Normal RandgID Deviates An1tals of Mathematical Statistics Vol 29 pp 610011 Bradey P B L Fox and 1 E Schrage 1987A Guide to Simulation 2nd edition SpringerVerlag New York Cheng R C 1977 The Generation of Gamma Variables with NoninJcgra1 Shape Parameters Applied Statistics VoL 26 No I pp 7175 Cochran W G 1947 Some CoSeqees ben the Assumptions for the Analysis of Variance Are Not Satisfied Biometrics Vol pp 2238 CochrllJ1 W G 1977 Sampling Techniques 3rd edition Joha Wl1ey Sons New York Cochrm W G and G M Cox 1957 Experimen tal Designs John Wiley Sons NeVYork Cook R D 1979 1Dlluential ObselV1loons in Lin ear Regression Joumal oftheAmencan Staisti cal Association Vol 169174 Cook R D 1977 Detection of1Dlluential Ooser vations in Linear Regression Technoffetrics Vot 19 pp 1518 Crowder S 1987 A Simple Method for Studying RunLeIlctrth Distributions of Exponentially Weighted Moving Average Olarts Technometncs VoL 29 pp 401407 Daniel C and F S Wood 1980 Fitting Equations to Daw 2nd edition John Wiley Sons New York Davenport W B and W 1 Root 1958 An Intro duction to the Theory ofRarulom Signals and Noise McGrawHill ewYork Draper N R and W G Hanter 1969 Transfor mations Some Examples Revisited Technomet rics Vol 11 pp 234 Draper N R and H Smith 1998 Applied Regres sion Atudysts 3rd edition John Wiley Sons NewYotk Dmcan A I 1986 Quality Control and Industrial Statistics 5th edition Richard D Irwin Home wood IL Duncan D B 1955 Multiple Range and Multipe FTests Biometrics Vol III PP 142 Efron B andR TIbshi1mi1993AnlnrrotiwctionlO the Bootstrap an ard Hall ew lOCk Elsayed E 1996 Reliability Engineering Addison Wesley Longman Reading MA Epstein B 1960 Estimation from Ufe Test Data IRE Transactions on Reliability Vol RQC9 Feller W 1968 An Introduction to Probability Theory and Its Application 3rd edition John Wiley Sans New York Fishman G S 1978 PrinCiples of Discrete Event Simulation John Wtiey Sons C York FumiYll G 1 and R W Wilson Jr 1974 Regression by Leaps and Bounds Tecrmomet nus Vol 16 pp 499512 Hahn G and S Shapiro 1967 Statistical Models in Engineering John Wliey Sons New York HaldA 1952 Statistical Dutory wiJI Engineering Applications John Wiley Sons New York Hawkins S 1993 Cumulative Sum Control Chartillg All underntilized SPC Tool Quality Engineering Vol 5 pp 463477 637 638 References Hocking R R 1976 The Analysis and Selection of Variables in Linear Regression Biometrics Vo 32 pp 149 Hocking R R F M Speed and M J Lynn 1976 A Class of Biased Estimators in Linear Regres SiOD TecntUmetrics Vol 18 pp 425437 HOed A E and R W Kennard 1970a Ridge Regression Biased Estimation for NonOrthogonal Problems Technometdcs Vol 12 pp 5567 Hoed A E andR W Kennard 197Gb Ridge Regression Application to NonOrthogonal Prob leInS TechtUmetrics Vol 12 pp 6982 Kelton W D and A M Law 1983 A New Approach for Dealblg with the Startup Problem in Discrete Event Simulation Naval Research Logistcs Qtarterly O 30 pp 641558 Kendall M G and A Suart 1963 The Advanced Theory of Statistics Hafner Publishing Conpany lew York Keuls M 1952 The Use of me SUdentized Range in Connection with aD Analysis of Variance Euphytics Vol 1 p 112 Law A M and W D Kelton 2000 Simulation Modeling andAnalysis 3rd edition McGrawHill New York LloydD K aDdM Lipow 1972 Reliability Managerrent Methods arul Mathenulrics P1entice Hall Englewood ClilIs NJ Lucas 1 and M Saccucd 1990 Exponentizlly Weighted Moving Average Control Schemes Properties and Enhancements Technometrics Vol 32 pp 112 Marquardt D W and R D Snee 1975 Ridge Regression in Practice The American Statistician Vol 29 pp 320 Montgomery D C 2001 Design and Analysis of Experimeras 5th edition 10hn Wiley Sons New York fontgomery D C 2001lntroduction to Statisti cal Quality Control 4th edition John Wt1ey SOIlS New York Montgomery D C E A Peck and O G vining 2001 Introduction to Linear Regressicrn Analysis 3rd edition John Wiley Sons New York Montgomery DC and GC Runger 2003 Applied Statistics and Probability for Engineers 3rd edtion John Wiley SODS New York Mood A M F A Graybill and D C Bees 1974 IrtlroducMn to roe Theory 0 Statistics 3rd edi ion McGraw Hill New York Neterl M Kutner C Nachtsheim and W Wasser man 1996 Applied Lirear Statistical Mrxiels 4th edition Irwin Press HomewoodIL lewman D 1939 The Distribution of the Range in Samples from a NoID13l Population Expressed in Terns of an Iodependent EstiJruIte of Standard Deviation Biometrika Vol 31 p 20 Odeh R and D Owens 1980 Tablesor Nonral Tolerance Limits Sampling Plans ami Screening Marcel Dekker ew York Owen D B l962 Handbook of Statistical Tables AddisonWesley Publishing CompllIlY Reading Mass Page E S 1954 Continuous Inspection Schemes Biometrika Vol 14 pp 100115 Roberts S 1959 Control ChattTests Based on Geometric Moving Averages Technometrics Vol I pp 239250 Scbeff6 H 1953 A Memod for Judging All Con trasts in the Analysis of Variance Biometrika VoL 4 pp87104 Snee R D 1977 Validation of Regression Mod eIs Methods and Examples Technometrics VoL 19 No4 pp 415428 Tucker Ii G 1962 An Introduction to Probability and Mathematical Statistics Academic Press ewYork Tukey J W 1953 The Problem of Multiple Comparisons unpublished notes Princeton University Tukey J W 1977 Exploratory Data Analysis AddisonWesley Reading MA United States Department of Defense 1957 Mili tary Sttmdard Sampling Procedures and Tablesfor Inspection by Variables for Percent Defective MlLSTD414 Government Printing Office Washingon DC Welch P D 1983 The Statistical Analysis of Simulation Results in The Corrputer Perfor mance Modeling Handbook ed S Lavenberg Academic Press Orlando FL i Answers to Selected Exercises Chapter 1 11 a 075 h 018 13 a it B 5 h it c B I 34 5 6 7 8 91O e A0i 2 3 4 5 d Ul23456789lOj e ABC 1256789IO 15 9 W r t 0 I 0 A t tt 0 t 0 l 12 S 015 B l 2 0 0 max t t 015 e t tl l 0 t 0 It rjt2 006 17 9 NNNNN NNNND NNNDN NNNDD NNDIN lCIDND NNDD IDlW NDtNIJ NDND NDD DNNNN DNNND DNND miD DD 19 a N not defective D defective 111 115 117 121 123 125 9 NNN NIJ NDN NDD DNN DND DDN DDD h 9 NNNN NNND NNDN NDiN DNNN 30 tomes 113 560560 ways 3OOP1300IpYi t i x l IOx I PAccept I p L 0 31 28 comparisons 119 4039 1560 tests a 25 ways h 100 ways R 1 020101 1 0201 09 0880 S Siberia U Ural PS 06 PU 04 peElS PFlS 05 PFU03 PSIF X 0714 0604 04 03 1 1 1 ml m 2 ml 127 129 PwomenI6 003226 131 l4 m1 m m m m2ml 135 pli 365364365 I 3651 n 21 22 30 40 PCB 0444 j 0476 0706 iJ891 137 8 MmO 139 0441 Chapter 2 21 PxXx xo L 2 3 4 60 0994 639 640 Answers to Selected Questions 3SlYes b No eJ Yes 27 a b 29 PXs 29 0978 211 k I b 11 2 l e 7x 0 0 8 Ox2 x lxS 2Sx4 x4 213 b 2 14 212 14 2J 215 a k b u cri e Fxx 0 l kl Sx2 rf 2 5x3 1x3 217 k 10 and 2 kW 83 days 221 FxOxO I9Ox3 1x23 b 11 2 cr e 113 3 d mcc 2 223 FlevO 225 kII11 OxO Chapter 3 31 a y pfy b El 14 Vel 564 0 06 20 03 80 01 Qthervise 00 33 al 0221 b SI5580 35 liz er Z 2 0 0 othervise 37 93S cgal 39 a fJ kfr1f1 trOflill y D 0 otherwise b Mv 2 v 0 0 otherwise c fuefuuO 0 otherNise 311 s 513 x 10 313 al Y14yrOyS3 Q otherwise b Jyy J e5 y C 0 othGINisc Answers Q Selected Questions 641 6 315 Mxt Ite EXMOt VX M0M0J2 317 ECl I VCl I ECX 616 VCX 0027 319 Mjt 1 tnt ECX MCO I VCX MCO MCOJ 321 MJt Ee IY EeraX J e1b EealX erb Mxat 323 a MJt ie t e2t ic l EX MxO VX O f Cb FyYOyO OY1 1y4 1y4 Chapter 4 41 Ca x 0 I pjx 2750 II50 Y 0 I pyy 2050 1550 b Y 0 I PnoCy llf27 827 Ce x 0 I PoCx llnO 420 43 Ca k 1 Cb IxCx 100 0 x 100 0 otherwise IxCx TI 0 x 10 0 otherwise 2 650 2 1050 2 4f27 2 2120 c IXIxxI 0 if xl or Xz 0 3 4 350 li50 3 4 450 150 3 4 327 In7 3 4 120 Ino 0x l 100 0Xz10 frm a XI 100 10 ti XI 100 O 10 I x 100 x 10 45 Ca b ior Ce JCw 2w 0 w I 0 otherwise 5 150 5 120 47 49 ECXix l ECXi 415 ECl 80 VCl 36 419 P 421 X and Y are not independent p 0135 423 Ca Ixx3cIx 1xI 0 otherwise frYJIy OyI 0 otherwise 642 Answers to Selected Questions 423 h fx Iy XIy 2Iy 427 0 otherwise 1 C iJ1xY r 0yVIx1 i1X2 0 otherwise e EXiyO r EYlxtlx 23y a EXly 3h2Y Oyl 429 a k n 1n 2 h Fx y I 1 x 1 y 1 x YP 0 y O 431 a Independent b Not independent c Not independent 433 a i b i 435 aFzFxalb b FzIFilh e Fz Fxi d Fz FJn z Chapter 5 51 PXx p 1p4 x 0 1234 53 Assuming independence PW 4 105 0927 55 50 pX2IL x 0Q2098 0078 rl 57 PCu S 003 098 59 pX 5 00407 511 p 08 513 EX Ix 0 1 EX MO 1 EXEX 3 p p P 515 PX 36 00083 517 PX 4 0077 PX 4 0896 519 EX 625 VeX 15625 521 p3 0 0 pO 3 0 PO 0 3 DllS 523 p4 I 3 2 0005 525 PX 2 098 Binomial approx PX S 2 097 527 529 PX I 095 Binomialppro n 9 PxIO11388SXIOyf25Y 531 PX5 0215 xd xl 533 Poisson model c 30 PXS3e 30 0 xl PX5Ie 3 r XI 535 Poisson model c 25 PX 2 0544 537 X number of errors on n pages Bin 5 nJ200 a n 50 Px 1 0763 h PX 3 090 ifn 15 539 PX 2 00047 Chapter 6 61 63 n fy l 5 y 9 0 otherwise Answers to Selected Questions 643 65 EX MO f3 al2 VeX MO MOJ f3 0112 67 y Fyiy y 1 0 ly2 03 2y3 05 3y4 09 y4 10 Generate realizations ui uniform 0 1 as random numbers as is described in Section 66 use these in the inverse as Y Fy1u i 1 2 69 EX MO 11 VeX MO MOJ 1 611 1 e1I6 0154 613 1 e1f3 0283 615 C1 C X IS ClI 3C x 15 CZx515 3CZ x 15 ECI Ce C 21 e31J C 045122 EC 3Ce3n 3C 21 eJJ 3C 034862 Process J if C 005132 619 08305 623 08488 627 ForI r2 fxtx 3 xOIX2IX0x1 r 1 r 2 0 otherwise 631 1 eI 063 633 024 635 a 022 b 4800 637 03528 Chapter 7 71 a 04772 b 06827 e 09505 d 09750 e 01336 I 09485 g 09147 h 09898 73 a c 156 b c 196 e c 257 d c 1645 75 a 09772 b 050 e 06687 d 06915 e 095 77 3085 79 237663 fc 713 a 00455 b 00730 e 03085 d 03085 715 B if cost of A 01368 717 7 719 a 06687 b 4 e 6018 723 0616 725 000714 727 a 0276 b At 120 729 a 0552 b 0100 e 0758 d 009 730 n 139 736 249724 737 EX e623 VeX e125oI mdn eso mode 0 741 09788 742 04681 Chapter 8 81 x 13130 11385 s 1067 821 x 74002 6875 x 10 s 00026 823 a The sample average will be reduced by 63 644 Answers to Selected Questions b The sample mean and standard deviation will be 100 units larger The sample valiance viJl be 10000 units larger 825 a x 829 a x 12022 s 566 s 238 b x 120 mode 121 831 For 829 cv 00198 for 830 CY 972 833 x 2241 s 20825 x 2281 mode 2364 Chapter 9 91 jx x x 1I2nO5len L x Il 93 jx x x xJ 1 95 N500O00125 97 Use S Ir 99 z IT 5 0 ThestandatderrorofXXis iL2 047 911 NOI in r 25 30 913 sepjIIN seppHn 915 Jl uO2u 917 921 923 925 2n2mft2 ForFIlwehaveunn2rorn2anda forn4 rnn24 f 1 etN F t 11 eJAlt JX1l I Iii al 273 b 1134 e 3417 d 2048 a 163 b 285 el 0241 d 0588 Chapter 10 101 Both estinators are unbiased ow VX1 Jllln whereas VX2 a 2111 Since VX VXJ XI is a more efficient estimator han 2 103 l because it would have a smalier MSE A fox 107 IX X 109 lit il n 1025 1021 1029 1031 1035 r 6f 11 nX JJc 1 2 fuJxxZxCJ 22rtl12Jexp Jl Zf i l I 2 c a 00 I L j j n 1 wMre C a 0 The posterior density for p is a beta distribution with paraneters a n and b L Xi n The posterior density fer is gamma with parameters r m Lx 1 and 0 n m ll 0967 1033 03783 f jxe 2x al jelx 22 b J x e xl 112 1037 at a cd2 is shorter 1039 a 7403533 17403666 b 740356 u 10 41 al 323211 i 326789 b 100480 Jl 1043 150 Or 151 1045 a 00723 Jl Ilz 326789 b 0049 Il Ilz 033 e i Ilz 03076 Answers to Selected Questions 645 1041 368 S 1 iLl S 212 1049 1830 S IS 2566 1051 13 1053 94282 S it 111518 1055 D839 u iLl D679 1057 0355 1 iLl 0455 1059 a 64960 a 285369 0 71456 s a e a 246062 1061 00039 a 00124 1063 0514 3614 1065 Oll S 01 086 1067 0088 p S 0152 1069 165i7 1011 D0244 p p 00024 1013 2038 1 iLl 37148 1075 315291 iLl 01529 19015 1 iLl S 09015 D1775 S1 iLl S 21775 Chapter 11 111 aJz 333 do not reject H 0 005 U3 a z 1265 reject lID b 3 115 z 250 reject II 117 alz 1349 do not rject H b 2 e L 119 Z 2656 reject H n11 z 725 reject H 1113 10 1842 do not reject Iio 1115 tod47 do not rejectHo at a 005 1117 3 1119 a 0 849 reject Ii b 0 235 do not reject II e 1 d 5 1121 Fe 08832 do not reject He 1123 a Fo 107 do not reject He b 015 e 15 1125 to 056 do not reject Hey 1127 a 4375rejectH b 03078 x 10 e 030 d 17 1129 aj X 228 eject H b 058 1131 Fo 3069 rejeet H P 065 11 33 24465 do not rBect Hr 1l5 to 521 reject HD 11 37 Zo 1333 do not reject HQ 1141 Zr2023donotrejectHo 1147 295donotrejctHrr 1149 4724 do not reject Ho 1153 00331 do not rejectHc 1155 2A65 do not reject Hrr 11s 34896 rejectHD 1159 2206 reject He Chapter 12 121 aj Fe 317 123 a Fo 1213 bj fixing technique 4 is different from I 2 and 3 125 a Fe 262 b jl 2170 0023 T D 166 i 0029 4 0059 127 a Fe40L b Mean 3 differs from 2 e S524633 d 088 129 aJ Fe 238 b None 1211 n 3 1215 aJ l 2047 033 T 173 i 207 b T i lAO Chapter 13 131 Source DF S5 MS J P CS 2 00317805 00158903 1594 0000 DC 2 00271854 00135927 1364 0000 CSDC 4 00006873 00001718 017 09S0 Error 18 00179413 00009967 Total 26 00775945 Main efiects are significant interaction is not significant 133 2393 i fL 515 135 No change in conclusions 137 Source DJ S5 MS F P glass 1 144500 144500 27379 0000 phos 2 9333 4667 884 0004 glassphos 2 1333 667 126 0318 Error 12 6333 528 Total 17 161500 Significant main effects 646 Answers to Selected Questions 139 Source DP SS MS F P Cone 2 77639 38819 1062 0001 Freeness 2 193739 96869 2650 0000 Time 1 202500 202500 5540 0000 ConcllFreeness 4 60911 15228 417 0015 ConcTime 2 20817 1 0408 285 0084 EreenessTime 2 21950 1 0975 300 0075 ConcFreenessTirne 4 19733 04933 135 0290 Error 18 65800 03656 Total 35 663089 ConcentrationlIDle Freeness and the interaction TlllleFreeness axe significant at 005 1315 Main effects A B D E and the interaction AB are significant 1311 Block 1 1 abo ac be Block 2 a b c abe 1319 Block 1 ll abo bed lCd Block 2 a b cd abed Block 3 c abc bd ad Block 4 d abd be ac 1321 A and C are significant 1325 a D ABC b A is significant 13M27 2 witi two replicates 13 29 252 design Estimates for A E and AB are large Chapter 14 141 a y 104391 000156 b F 2052 e 00038 fl 000068 d 7316 143 aJYJ 1656 0041x b Fo 57639 c 8159 d 1937421388 147 a y 933399 156485 b Lack of fit not significant regression significant e 7997 8 s 23299 d 74828 flo s 111852 e 126012138910 149 a y 63378 920836x b Regression is significant e to 2341 reject He d 5255852991 e 52122 53428 1411 a y 777895 118634x b Lack of fjt not signifi regression is significruL e 03933 d 45661 191607 1413 Ca y 396 000169x b Rcgrcsiion is significant e 0001500019 d 952 1415 Ca y 691044 04194 b 7735 e t 585 reject lit Cd ZO 161 e 0551308932 Chapter 15 151 Ca Y73000183xl0399x b Fo 1519 153 0802400044 155 y 1808372 000359x O1939360x 004l15x IS7 159 1513 Ca 102713 0605x 8924 1437 0014 b Fe 5106 c fJ Fa 0361 fJFo OOOOl Cal y 13729 1050 018954 a y 4459 1384 1467x b Significant lack of fit 1515 to 17898 1521 vIF VIF 14 Chapter 16 1il R 2 165 R 2 169 R 885 1613 R 75 1615 Z 2117 1617 K 4835 Chapter 17 171 a 3432 R 565 b PeR 1228 e 0205 e F 1668 175 DI2 177 LCL 3455 CL 4985 UCL 6514 179 Process is not in control Answers to Selected Questions 647 1713 01587 n 6 or 7 1715 R 1sed cOlltrollimits LCL O UCL 1732 1717 UCL16485 0434 1719 LCL om UCL 4378 1725 alR60lO0105 b 1316 1727 098104 1729 084085 1731 Cal 3842 b 91363 OOJ Chapter 18 181 al 0088 b L 2 L 133 el W 1 h 183 187 189 1811 lPJ 121 12 I P al p 0 Ip P 0 AlOOol o p o 1 P 9 a 0 bii e 3 d 003 e 010 I 10 a 0555 b 56378 min e 24418 min 1813 a Pjllili fjPof012 s b s6pG417 0 otherwise I Po i li j j e p 0354 d From4L6 to 833 el 417 p 0377 Chapter 19 193 Ell b tfabaV11J baEJabaV bo t fabaudu I 195 a 35 b 475 e 5 at time 14 197 Ca XI V 1116 Xi 6 V 616 b Yes e Xo 2 199 al XII1 tnl U b 0693 1911 a X212V ifOV12 b XO894 2 if 112 V 1 1913 a X atnl vp b 1558 1915 E V6107 1917 Xl1mVVO84L 1919 X 5 trials 1921 341441 1923 8041196 1925 EJlllelltial with parameter 1 V V 112 Index 22 factorial design 373 23 factorial design 379 2 factorial designs 373 379 2kl fractional factorial design 394 2k fractional factorial design 400 3sigma control limits 511 A Absorbilg stae for a Markov cbain 558 560 Acceptancerejection method of random number generation 586 Active redundaLcy 544 Actual process capabilit 517 Additivity theorem of chl square 207 Adjusted R 453 476 Aliasing of effects in a fractional factrlal395400 All possible regressions 474 Alternate fraction 396 Alternative hypothesis 266 Analysis of variance 321 323 324331342 Analysis of variance model 323 328337341359366367 Analysis of variance tests in regression 416 448 Analytic study 172 AiOVA see analysis of variance Altithetic variates 591 593 595 Approximate coniidence intervals in maumum likelihood estimation 251 Approximation to the mean and variance 62 Approxintations to Cistributions 122 binomial approximation to hypergeometric 122 Poisson approximation to binomial 122 nonnal approximation to binomial 155 Assignable cause of variation 509 Asymptotic prOperties of the maximum likelihood estimator 223 Asymptotic relative effiCiency 499501 Attribute data 170 Attocorrelarion in regression 472 Average ron length ARL 532 534535 B Bacbvard elimination of variables in regression 483 Batcb means 590 Bayes estimator 228 230 Bayes theorem 27 226 Bayesian confidence intervals 252 Bayesian inference 226227 252 Bernoulli distribution 106 124 Bernoulli process 106 Bernoulli rlIDdom variable S82 Bernoulli traJs 106555 Best linear unbiased estimator 260 Beta distribution 141 Biased estimator 220 467 588 589 Binomial approximation to hypergeometric 122 Binomial distribution 40 46 66 106 O8 Ill 124492 melID and variance of 109 cumulative tlmomial distribution 110 Birtha equations 555 564 Bivariate normal distribution 160427 Blocking Plcpe 341 390 Bootstrap 231 Bootstrap confidence intervals 253 Bootstrap standard etror 231 Box plot 179 BOAMill1ermethod of generating normal random numbers 54 Burnin 540 c c chart 523 Caresian product 5 71 Cauchy distribution 168 Causeandeffect diagram 509 535 Censoredife teSt 548 549 Census 172 Centerline on control chart 510 Central limit theorem 152 202 233546584588 Central moments 48 58 189 Chance cause of variation 509 ChapmanKolmogorov equations 555 556 561 Characteristic function 66 Chebysbevs incqHhlty 48 Check sheet 509 Chisquare distribution 137 168206238300549 mean and variance of 206 additivity theorem of 207 percentage points of 603 Chisquare goodnessoffit test 300 Class intervals for histogram 176 Cochrans Theorem 325 Coefficient of deternrination 426 Coefficient of multipie determination 453 Combinations 16 Common random numbers irl simulation 59 593 Comparison of sign t aJd Itest 496 649 650 Index Comparison ofilooxon rank sum test and ttest 501 Comparison ofW1ooxon signed rank test and Itest 499 Complement set complement 3 Completely randomized design 359 Components of variance modei 323 Conditional distributions 71 79 80128162 Conditional expectation 71 82 84 Conditional probzbLity 1921 Confidence coefficient 232 Confidence interal 232 233 Coiidence interval on the difference in means for paired observations 247 Confidence interai on the difference in means of two 1ormal distnDutions variances known 242 Confidence interval on the difference in means of two ormal distributions variances unknown 244 246 Confidence interval on the difference in two proportions 250 Confidence interval on mean response in regression 418 Confidence interval on the mean of a nor1al distribution variance lcnOvn 233 236 Confidence interval on the mean of a normal distribution variance unknown 236 Confidence interval on a proportion 239 241 Confidence interval On me ratio ofvariancC oftwn normal distributiors 248 Confidence i1tervaLs on regression coefficients 47 444 Confiderce interVal on simllation output 588 591 592 Confidence iterval OIl thc variance of a normal distribution 238 Confidence level 234 235 Confidence limitS 232 Confounding 390 394 Consistent estimator 220 223 Contingency table 307 Continuity corrections 155 Continuous data 170 Continuous function of a contirtuous random variable 55 Continuous random aiable 41 55 Continuous sample space 6 Continuous simIliation output 587 Continuous uniform distribution 128140 mean ard Rriance of 129 Continuoustime Markov chain 561 Contour plot of response 355 358 Contrast 374 Control chart 509 ContrOl charts for attributes 520 522523524 Control chart for individuals 518 Control charts for measurements 510518522 Control limits 510 511 Convolution 585 CODYohtion method of random number generation 585 Cooks distlltlce 470 Correlation coefficient 190 Correlation matrix of egressor variables 461 Correlation 71 87 88 101409 427 Covariance 71 87 88161 Covariance matrix of regression coefficients 3 Cp Statistic in regression 475 CramerRao lower bound 219 223251 Critical region 267 Critical values of a test statistic 267 Cumulative distributiQn function CDF see distribution function CUllUlaovc nornal distribution 145 Currulative sun ClSLM control chart 525 526 534 Cycle length of a random DllIlber generator 581 D Data 173 Decision intTIal for the CUSUM527 Defect concentration diagram 536 Defects 523 Defining contTIlSts 391 Defining relation for a desig1 395400 Degrees offreedom 188 Delta method 62 Demonstration and acceptance testing 551 Descriptive statistics 172 Design generator 395 400 Design of experimentS 353 354 Design resolution 399402 Designed experiment 321 322 536 Determnistic versus nondererrrinistic systems 1 Discrete data 170 Discrete distributions 106 Discrete random variable 38 54 Discrete sample space 6 Discrete simulation output 587 Discreteevent simulation 576 Distribution free statistical methods see nonpararnetric nethods Distribution function 363738 91110 Dot plots 173 175 DurbinWatson test 472 Dynamic simulation 576 E Enumerative study 172204 Equally likely outcomes 10 Equivalent events 34 52 Ergodic property of a Markov chain559 Erlang distributiou 55 Error sum of squares 325 413 Estiroable functions 329 Estinated standard error 203 230240 Estirration of r in regression 414444 EStiInationofvariance components 338 367 368 Events 8 Expectation 58 Epected JJie 62 Expeeted mean squares 326 338360361366368 Expected ecurrence time in a Markov chaw 558 Expected value of a random variable 58 77 Experimental design 508 Exponential distribution 42 46 130140540541548 546564583 relationship of exponeIltia and Poisson 131 mean and variance of 131 memoryless property 133 Exponentially weighed mOving average EViMA control ehart 525 526 529534 EXtra sum of squares method 450 Extrapolation 447 F Factor effects 356 Factorial experiments 355 359 369 Failure rate 62 also see bazard function Fdistribution 211 mean and vaianec of 212 percentage points of 605 606607608609 Finite population 172 204 Finite sample spaces 14 Finitestate Markov chain 556 First passage tiJGe 557 Fintorder autoregressive model 472 Fixed effects ANOVA 323 Fixed effects model 359 Forward selection of variables in regression 482 Fraction efective or nonconforming 520 Fractional factorial design 394 Frequency distnbution175 176 FuUcyele random number generator 581 Function of a discrete tandom variable 54 Function of a random variable 52 53 Functions of two random variables 92 G Gamma distribution 67134 140540546 relationship of gamma and exponential 135 mean and variance of 135 relationship to chisquare distribution 137 threeparameter gamma 141 Gamma funetion 134 General regression signifieance eslS450 Generalized irteraction 393 Generation of random variables 580582 Generation of realizations of random variables 123 138 164 Geometric distribution 06 112 124535586 H mean and variance of 113 memoryless property 114 Halfinterval corrections 155 Halfnonnal dlstribution 168 Hat matrix in regression 471 Hazard function 538 539 541 Histogram 175 177 509 514 Hypergeometric distribution 40 106117124 mean and variance of lIS Hypothesis testing 216 266 267 Hypothesis tests in the correlation model 429 Hypothesis tests in multiple linear regression 447 Hypothesis tests in simple linear regression 414 Hypothesis tests on a proportion 283 Hypothesis testS on individual coefficients in multiple regression 450 Hypothesis tests on the equality of two variances 295 296 Index 651 Hypothesis tests on the mean of a norrna1 distribution varianee known 271 Hypothesis teStS on the mean of a normal distribution variance urknown 278 Hypothesis tests on the means of tvlo normal distributions variances knmvn 286 Hypothesis tests on the means of two normal distributions variances unknmvn 288 290 Hypothesis tests on the vatianee 0 a nol1llal distribution 281 Hypothesis tests on tvlO propoctions 297299 I Idealized experimentS 5 ldentiy element 381 Incontrol process 509 Independent events 20 23 Independent expetments 25 Independent ra1rlom variables 71868895 Indiearor variables 458 Inferential statistics 170 Infilential observations in regression 470 Initialization bias in simulation 589 Instantaneous failure rate see hazard function Intcnsty of passage 562 Intensity of transition 562 Interaction 356 374438 Interaction term in a regression model 438 Intersection set intersection 3 Interval failure rate 538 IntriJsieally linear egression model426 InvarianCc property of the maimum likelihood estimator 223 Inventory system 580 Inverse transfonn method for generating andom nll1tlbers 582 Inverse transform theorem 582 Inverse transformation method 64 Ineduco1e Markov chai 559 652 Index J Jitter 174 Joint probability distributions 71 727394 Judgmentsample 198 K KrJSkalWallis tesr SOl 503 504 Kurtosis 189 307 L Lack of fit test in regression 422 Largesample confidence interval 236 Law oflargerumbers 71 99 101 Law of the urconscious statistician 58 Least squaes estimation 328 410438 Least squaCs nonnal equations 328410439440 Life testing 547 Likelihaud function 221 226 Linear combinations of random variables 96 99151 Linear congruential random number generator LeG 581582 Linear regression DJJXlcl 409 437 Littles law 568 LognormaJdistribution 157 159 mean and variance of 158 Loss function 227 Lower control1imit 51 0 M Main effects 355 374 MannVlhitney test see VlUcoXQO ranksum test Marginal distribution 71 75 76 161 Markov chain 555 558 559 560561 Markov process 555 573 MlLkov property 555 Maximum likelihood estimator 221223230251428 Mean of a random variable 44 58 Mean per unit estimate 204 Mean square error of an estiuJator 218 Mean squares 325 342 360 Mean time to failure MTIF 62540 Measurement data 170 Median of the population 185 Median 158492 Memoryless property of the exponential distribution 133541 Memoryess property of the geometric distributioo 114 Method of least squares see least squares estimation Method of maximum likelihood see maximum lielihood estimator 1finimum varianc unbiased estimator 219 Mixed mudd 367 Mude 158 Model adequacy checking 330 345364384414421 452 454 Moment estimator 224 Moment generating function 65 99107110114 116121 129 132 135 145 ISO 202 Moment 44 47 48 58 65 MonteCarlo integration 578 MonteCarlo sintulation 577 Moving range 518 mpl see mean per unit estimator Multicollinearity 464 Multicollinearity diagnostics 466 Multinomial distribution 106 116 Multiple comparison procedures 593 Multiple regression model 437 Multiplication principle 14 22 Multiplicative linear congruential random oumber generator 582 Mutually exclusive 22 24 N Natural tolerance limits of a process 516 Negative binomial distribution 106124 Noncentral F distribution 347 Nonccntral tdistribution 279 Nonparametric ANOVA see Kruska1 Wallis test Nonparametric confidence interval 237 Nonparametric methods 237 491 Nonterminating steady state sUtxilations587590 NoIJDal apprOximation for the sign test 493 Normal approximation for the Wilcoxon ranksum test 501 Normal approximation for the Wilcoxon signed rank test 497 Nonnal approximation to the binomial 155241 Normal distribution 143583 mean and variance of 144 cumulative distribution 145 reproductive property of 150 Normal probability plot 304 305 Normal probability plot of effects 387 398 Normal probability plot of residuals330 Null hypothesis 266 o One factor at a time expericent 356 Onehalf fraction 394 Onesided alternative hypothesis 266269271 Onesided confidence interval 233236 Onestep transition matrix for a Murkov chain 556 563 Oneway classification ANOVA 323 Operating charactutistic OC curve 274 275280 347 charts for 610626 Optimal estimator 220 Order statistics 214 Origin moments 4758224 Orthogonal contrasts 332 Orthogonal design 381 Outlier 180421 Output analysis of simulation models 586 587 591 p p cnart 520 lurcd data 292 494 498 Paired rtest 292 Parameter estimation 216 Pareto char 181509535 Partial regression coefficients 437 Partition of je sample space 25 Pascal tUStribUtiOD 106 115 124 mean and variance of 115 Pearson correlation coefficient see oorrelatlon coefficient PutatioDS 15 19 Piecewise linear regression 490 Point estimate 216 Point estimator 216 Poisson approximation to binorrial 122 Poisson distribution 41 106 118 120 124523 mean and variance of 120 cumu1ative probabilities for 598599600 Poisson process 119 582 Polynomial regression model 438456 Pooled estimator of variance 289 Pooled rtest 289 PopUlation 170 171 Popclarion mean 184 Population mode 185 Positive reClIIent Markov chain 558 Posterior distribution 226 Potential process capability 517 Power of a statistical test 267 347 Practical versus statistical significance iI hypothesis tests Tl7 Precision of estimation 234 235 Prediction in regression 420 446 Prediction interval 255 Prediction interval in regression 420446 Principal block 392 Principal medon 396 Prior distribution 226 Probability density function 42 Probability distribution 39 42 Probability mass function 39 Probability plotting 303 Probability sampling 172 Probability 1 8 9 10 1112 13 ProCess capability 514 516 Process capability ratio 516 58 Projection of 2k designs 384 Projection of2K1 designs 398 Properties of estimated regression coefficients 412 413443 Properties of probability 9 Pseudorandom numbers PRNs 581 Q QuaLitative regressor variables 458 Quality improvement 507 Quality of conformance 507 Quality of design 507 Quetting 555 564 568 570 572573579 R Robart 510 512 R426429453475 R1adit see adjusted R2 Random effects A10VA 323 337 Random effis model 366 Random experiments 5 Ra1dom number see pseudorandom number Random sample 198 199 Random sampling 172 Random variable 33 38 Random vector 71 Randomization 322359 Randomized block ANOVA 342 Randomized block design 341 Rank transformation in ANOV A 504 Ranking and selection procedures 591 593 Ranks 496 498 499 502 504 Rational subgroup 510 Rayleigh distnlludon 168 ReCUI1ence state for a Markov chain 558 Recurrence time 557 Redundant systems 544 545 Regression analysis 409 Regression model 437 Index 653 Regression of thc mean 71 85 163 Regression sum of squares 416 Regressor vaiable 409 Rejection region see critical region Relationship between hypothesis teSts and confidence intervaJs276 Relationship of exponential and Poisson random variables 131 Relationship of gamma and exponential ratldom variables 135 Relative efficiency of an estimator 218 Relative frequency 9 10 Relative range 511 Reliability 507 538 Reliability engineering 24 507 537 Reliability estimation 548 Reliability funcdon 538 539 Reliability of serial systems 542 Replication 322 Reproductive property of the normal distribution 150 151 Residual analysis 330 345 364 384414421454 R dlWs 330 364 377 421 Resolution m design 399 Resolution N design 399 Resolution V design 400 Response variable 409 Ridge regression 466 4f7 Risk 227 S Sample correlation coefficient 428 Smlple mean 184 Sample median 184 185 Sample mode 185 Sample range 188 Sample size fur ANOVA 347 Sample size for confulence intervals 235 237 241 24 Sample size for hypothesis tests 273279282284287 291 296 298 Sample spaces 5 6 8 Sample standard deviation 181 654 Inde Sample variance 186 Sampling distribution 201 202 Sampling fraction 204 Sampling with replacement 199 204 Sampling without replacement 172 199204 Saturated fractional facrorial 402 Scatter diagram see scatter plot Scatter plot 173 174 175 41 1 412509 Seed for a random nU0100r generator 581 Selecting the form of a distribution 306 Set operations 4 Sets 2 Shewhart control charts 509 525 Sign rest 491 492 493 494 496 critical values for 629 Sign test for paied samples 494 Significance level of a statistical test 267 Significance levels in the sign test 493 Significance of regression 415 447 Simple correlation coefficient see correlation oocfficient Simple linear regression model 4Q9 Simulation 576 Simultaneous confidence intervals 252 Single replicate of a factorial experiment 364 386 Skewness 189203306307 Sparsity of effects principle 386 Specification limits 514 Standard deviation 45 Standard error 203 Standard error of a point estimator 230 Standard error of factor effect 383 Standard Dormal distribution 145152233 cumulative probabilitie for 60602 Standardized regression coefficients 462 Staudardized residual 421 Sandardizing 146 Standby redundancy 545 Sate equations for a Markov chain 559 Statistic 201 216 Statistical control 509 Statistical hypotheses 266 Statistical inference 216 Statistical process control SPC 508 Statistical quality control 507 Statistically based pling plans 508 Statistics field of 169 Stem and leaf plot 178 Stepwise regression 479 Stirling fomula 145 Stochastic process 555 Stochastic simulation 576 Strata 200 204 Stratified ra1dom sample 200 204 Strong versus weak conclusions in hypothesis testing 268 Studentized residual 471 Sum ofPoiSSn random variables 121 Summaj statistics for grouped dam 191 System failure rate 543 T Tabular form of the CUSWM 526 Taylor series 62 tlistcibution208236 percentage points at 604 Terminating transient simulations 587 Three factor analysit of variance 366 Three factor factoria1 369 1hfeedimensional scatter plot 174176 TIer chart see tolerance chart Tws in the KruskalWallis test 503 Ties in the sign test 493 Ties in the Wilcoxon signed t3Ilk test 497 lime plots 183 Titrlbrofailure distribution 53S 540541 Tolerance chan 514 TQlerance interval 257 Total robability Law 26 Transfonnaon in regression 426 Transformation of the response 330 Transient state for a Markov chain 558 Thmsition probabilities 555 556563 Treatment 321 Treatment sum of squares 325 Tree diagram 14 Ilial control limits 512 Triangular distribution 43 TriInIned mean 195 ttestS on individual regression coefficients 450 Tukeys test 335 336 344 363 Twofacto factorial 359 Twofactorrando effects ANOYA366 Twofactor mixed model kiOVA367 Twrsided alternative hypothesis 266 269271 Twosided confidence interval 233 Typc 1 error 267 Type U error 267 494 lYpe II error for the sign test 494 U u chart 524 Vnbalanced design 331 Unbiased estimator 217219 220 Uncorrelated random variables 88 Uniform 0 1 1lIldom numb 581582 Union of sets 3 Universal set 3 Universe 170 Upper controlliroit 510 Y Variability 169 Variable selection in regression 474479482483 Variance components 337 366 368 Variance inflation factors 464 Variance of a random variable 4558 Variarce of an estimator 218 Vaiance reduction JlleUods in simulation 591 593 596 Venn diagam 4 W Waitingline tbcoiy see queuing Weibull distribution 137 140 540 mean and variance of 137 Veighted least squares 435 Vllcoxon rantsum test 499 critical values for 627 628 V1lcoxon sgned rank test for paired s3JLples 498 Index 655 Wllcoxon signed rank test 496 critical values for 630 X X chart 510 511 y Yates algorithm fot the2k 385