PREVIEW OF THE BOOK Here is a preview of key points in the book: 1) One cannot avoid making judgments; the process of statistical inference cannot ever be perfectly routinized or objectified. Even in science, "fitting of the model to experience" (close to Braith- waite's phrase in his 1953 book) requires judgment. 2) In statistical inference as in all sound thinking, one's purpose is central. All judgments should be made relative to that purpose, and to costs and benefits. (This is the spirit of the Neyman- Pearson approach). 3) The best ways to infer are different in different situations. The differences are especially great in science versus decision-making, and among the different fields of economics, psychology, history, business, medicine, engineering, physics. 4) The logically and historically primary task in most scientific situations is to decide whether two (or more) sets of observations should be considered similar or different, while the secondary task is to decide how big is a difference (or other magnitude). Hypothesis tests are intended to serve the primary task, and confidence intervals are intended to serve the secondary task; this distinction helps sort out the roles of the two main modes of inference, which in the past have been tangled. 5) The different tools should be used as situations call for them - sequential vs fixed sampling, Neyman-Pearson vs. Fisher, and so on. 6) In statistical inference, when data and procedures are ambiguous, the best wisdom is not to argue about the proper conclusion. Instead, whenever possible one should go back and get more data, hence lessening the importance of the efficiency of statistical tests. In some cases, however - especially in biostatistics, taking as an example unusual cases of cancer - one cannot easily get more data, or even conduct an experiment. And with respect to the past one cannot produce more historical data. But one can gather more and different kinds of data, e.g. as was done in the history of the inquiry into the connection between smoking and lung cancer. The book does not deal with the valuational aspects of decision-making - neither the expected values nor the allowances for risk. My book on managerial economics (1975) discusses ways of embedding the probabilistics element into the larger framework of decision theory. Though Chapter IV-2 discusses the relationship of statistical correlations to causality, and the overall concept of causality, the book also does not deal with the practical aspects of determining causality such as the search for hidden third factors. One may consult texts on research methods (e. g. Simon and Burstein, 1985) for discussion of this topic. PREFACE With luck, the arguments in this book are written sufficiently clearly and simply that anyone willing to make an effort can understand them. And I hope you will find the overall argument integrated and without any logical gaps from one part of the discussion to another. But even if I have succeeded in doign that, it does not imply that the subject is simple. . Just the opposite: The ideas underlying statistical inference are, in my opinion, as subtle and difficult as any set of fundamentals in any field. This inherent difficulty explains why, after centuries and millenia of struggling for understanding of these matters, it was not until the past century or so that these ideas have finally begun to be understood, and there still is massive controversy about the foundations of the science. (See the Introduction) Writers such as Fisher, Neyman, Pearson, Jeffreys, and others have made great discoveries by focusing laser beams into distant uncharted regions. But they have left much of the firmament unmapped. I hope that this book provides a broad map to help one pass from one of those discovered areas to another. The book also proposes some particular ways of addressing the subject that are intended to promote this integrated understanding, such as treating the concept of probability with the concept of operational definition and hence side-stepping and obviating the centuries-long struggle between the frequentists and the subjectivists. The greatest strength, from my point of view, of the book's treatment of statistical inference also surely wil be considered its greatest weakness by many others: its non-axiomatic non- deductive pragmatic approach oriented toward operational definitions rather than the property definitions that are necessary for axiomatic treatment. This non-formulaic approach is also at the heart of the resampling (Monte Carlo) method, which is the second focus of the book. Both these matters are illuminated by a a delightful and revealing scenario by John Barrow wrote about what might happen if we receive a response from Martians to our extra-terrestrial messages, which depend heavily upon mathematics, on the assumption that that will be the easiest for the Martians to decode. (The scenario is reproduced in Chapter IV-3.) Barrow imagines that the arriving Martians present stupendous and exciting new mathematical ideas which are not found to be false, but which they arrive at by induction - and therefore are rejected by earthly mathematicians because they do not employ the earthly method of deductive proof. Our earthling mathematician are disappointed, in Barrow's story. That is accurate: Terrestrial mathematicians are not excited by a method that simply offers answers or solutions. The method must also meet esthetic tests to be acceptable. It is here that resampling faces its greatest obstacle; it lacks the esthetic appeal of proof-based mathematical findings. Whether the approach to understanding inference presented here will eventually be accepted, I do not know. But I think it safe to predict that eventually resampling must win its way into the center of statistical practice, because its practical advantages are enormous. That future seemed evident to me in 1967 when I first began to practice, teach, and write about resampling, and indeed, by now the concept is entirely accepted theoretically by mathematical statisticians. But just how soon resampling will become the standard tool of first recourse in everyday work continues to be unclear. The situation of resampling has something in common with the famous book Calculus Made Easy by Sylvanus Thompson, whose front motto is "What one fool can do, another can, too". A motto for resampling might be "What one mathematical dolt can invent, another mathematical dolt can understand". Though Calculus Made Easy is damned by academic mathematicians, after almost a century it is still available in paperback and selling briskly in college bookstores - simply because it makes quite clear, using a system of approximation, what is extraordinarily difficult to comprehend - after all, it required Newton and Leibniz to invent it - using the mathematician's elegant method of limits. Resampling has come in for the same damnation and neglect for the last quarter century. But its chances of lasting forever are even better than Thompson's method, because resampling is actually a better tool than the conventional formulaic method, rather than just a better pedagogical method. One of the joys of writing this book is that it integrates ideas that I have been chewing on for a quarter of a century. Now the links between them appear - not only the links of substance, but also the links of points of view such as the emphasis on open rather than closed systems, and the need for judgment in all our statistical and scientific work. Much material concerning the actual practice of resampling statistics is shared between this book and the text Resampling: The New Statistics. I believe that the two treatments benefit from there being both applied and philosophical points of view. I hope that no no reader is troubled by this overlap. Peter Bruce has greatly assisted all my work in this book in a variety of ways. With respect to this chapter, his help in clarifying these ideas by discussing them with me, along with teaching them jointly with me, has been very great.