One More Walk to the Devil

Books on Probability and Statistics

This is a list of some books that I have read on probability and statistics, along with my personal opinion of their contents. Some were texts, but others were found at used booksales (which is why some are a bit out of date).

Introductory Books

Introductory books on statistics are often ugly compromises, written with the intent of introducing some rather subtle ideas to students who may not have a sufficient background in mathematics to understand them fully. Great strides have been made in the last decade in removing excessive math from introductory books, and in introducing the analysis of real data. Unfortunately, many introductory texts are hideously ugly and and badly typeset, making it a real chore for students to read them. Orange is NOT an appropriate colour for all of the graphics in a text.

• The Cartoon Guide to Statistics by Gonick and Smith. This is a very good book for reviewing the basic ideas of statistics. It is light on math, but clearly presents many central ideas. While I feel that Gonick's artistic style is a bit loose, this book is a very good example of how illustrated books can be used to convey complex ideas most efficiently.
• Probability and Statistics for Scientists and Engineers by Devore. This book is very comprehensive and clearly written, and assumes a basic knowledge of calculus. It introduces all basic statistical and probabilistic ideas with plenty of examples, and is not too hard on the eyes.
• Statistics by Freedman, Pisani, Purves, and Adhikari. This is one of the best-written texts on basic statistical ideas that is essentially math-free. Many examples show clearly how statistics is used, and the strengths and weaknesses of basic techniques. Unfortunately, the book also uses non-standard terminology (`chance' for probability), and is perhaps best used as a supplementary text to complement a more mathematical introductory book. Nicely typeset.
• The Basic Practice of Statistics by DS Moore. This book use terminology that is standard, but again is fairly math-free. It has good sections on regression, and contains very little probability.
• Introduction to the Practice of Statistics by DS Moore and GP McCabe. This book requires calculus, and contains more probability than does the previous entry. As with all of Moore's books, it is clearly written and very applications-oriented.
• How To Lie With Statistics by Huff. This little booklet has been in print from the 30s, and is an excellent guide to how NOT to interpret the results from statistical procedures.

Probability

• Introduction to Probability Theory by S Ross. An excellent introductory book (i.e. the word `measure' does not arise), clearly written for the most part with a nice balance between mathematical content and practical examples. The examples in the book are wonderful, ranging from the clear and simple to the complex and relevant; many of the more complex examples are related to everyday situations which can be easily stated, but which require long and detailed answers. Also, the solutions to numerical problems are almost all correct, a notable achievement. One minor irritation: some results are introduced earlier than they need to be, then solved using very involved methods that emphasize the use of basic ideas, but which are far harder to understand than what comes later.
• An Introduction To Probability Theory by W Feller. The two volumes of this classic are well worth looking at, but carefully. Wiley had serious editorial problems with this book, and you must be sure to use the corrected editions rather than first printings, as almost all the answers to exercises are wrong in the first printing. Volume 1 is an excellent introduction to discrete probability and discrete Markov processes, while Vol 2 uses somewhat idiosyncratic notation to cover measure-theoretic probability. The ordering and priority of topics may also seem odd to those used to later texts (e.g. Billingsley, Chung).
• Probability and Measure by P Billingsley. A very good introduction to measure-theoretic probability, especially for those who intend to make use of it in statistics. The book is broken down into short chapters that clearly discuss basic ideas in the field, and there are many good problems and examples. This is a very good book to learn from. Trivial note: if you watch Brian De Palma's film The Untouchables carefully, the author has a small part as a bailiff in a courtroom scene.
• Probability Theory: An Analytic View by DW Stroock. A beautifully written book that puts all the basic ideas of measure-theoretic probability and stochastic processes together in a very elegant and mathematical package. While this would be useless as a first book to anyone not on their way to a Fields Medal, it would be an excellent book to read for anyone planning on putting a course together in probability and stochastic processes.
• Real Analysis and Probability by RB Ash. A good book to learn from, combining basic foundational ideas (i.e. real analysis) with basic measure-theoretic probability.
• Counterexamples in Probability Theory by JP Romano & AF Siegel. This is a very useful book to look at, especially if you really want to be sure that you understand probability theory. It contains many traps for the intuitive, and many curiosities that help explain why probability didn't get straightened out as a mathematical discipline until the 1930s.
• A Course in Probability Theory, by K-L Chung. This is one of my favorite books, and the first one I turn to as a reference. It is quite concise and has a very mathematical flavour (more so than Billingsley or Feller), and is perhaps not good as a first book to learn from. It is organized around mathematical ideas rather than statistical ones, and I especially enjoy its sections on convergence.

Theoretical Statistics

These books are intended to present the theoretical techniques useful in evaluating the quality of statistical techniques. These books are best approached after you have taken at least one course in applied statistics involving real data.

• Theories of Probability by TL Fine. This is a book on probability, but is of greatest use to those who are interested in statistical inference. It goes over all the major ways of connecting probability to real world events, at a fairly high level. It also has copious references to the historically important papers in which many key ideas were introduced. Primarily for researchers into the history and foundations of the subject, but also good for sorting out all the different definitions of probability that are floating about in the ether.
• Comparative Statistical Inference by V Barnett. This is an excellent guide to sorting out all the different ways that statistics have been built over the probabilities built in Fine. It also suggests that there is no one way of constructing statistics that cannot be made to look ridiculous in a situation of practical importance. Even so, it does clearly lay out the philosophical ideas that are often overlooked in more mathematical treatments of statistics.
• Mathematical Statistics by RV Hogg & AT Craig. A very good introductory book, even if it is a bit abstract and lacking in examples. If it were to be merged with the examples in Cox and Hinkley, a near ideal book would emerge. The later editions are good, as they admit that Bayesian statistics exists.
• Mathematical Statistics and Data Analysis by JA Rice. A book that tends to evoke strong reactions from those who use it. On the plus side, it motivates theoretical problems with a great deal of data analysis. On the minus, the problems are apparently quite confounding. Again, may be good as a secondary reference beside Hogg & Craig.
• Probability and Statistics by DAS Fraser. For a self-published book, this is really quite good. While non-measure theoretic, it contains many challenging exercises and is at the approximate level of Hogg and Craig.
• Statistical Inference by G Cassella & RL Berger. This book assumes that you know measure theoretic probability, then goes on from there. It has a strongly mathematical flavour, but is also quite comprehensive.
• Linear Statistical Inference and Its Application by CR Rao. This book is very heavily centered on inference involving the multivariate normal distribution, and familiarity with matrices is essential. Not as comprehensive as Cassella and Berger, but may be of use to econometrists who make heavy use of the normal distribution.
• Theoretical Statistics by DR Cox & DV Hinckley. A books that consists of a huge number of examples, and not very much mathematics. It is a difficult book to use as an intro test, but would be an essential supplement to any of the mathematically-flavoured advanced texts.
• Mathematical Methods For Statistics by H Cramer. This is a very old book, but contains some interesting results that it may be hard to find proofs for elsewhere. The typesetting is appallingly ugly.
• Testing Statistical Hypotheses and Theory of Point Estimation by EL Lehmann. These are very mathematical presentations of the basic theory of estimation and testing. TSH is more sophisticated that TPE, but both require a good knowledge of measure theoretic statistics. Both are clearly written, though the early editions lack discussions of computer-assisted methods such as the jackknife and the bootstrap.
• Asymptotic Statistics by AW Van Der Vaart. Fantastic book on asymptotic methods that goes from parametric to semi-parametric methods, illustrating many useful results on the way. Requires a high level of mathematical sophistication, but is rewarding.

Applied Statistics

Good books in applied statistics have lots of data sets, and are organized around using statistical methods to provide good answers to questions that need to be answered. While elegance and pretty theorems have their place, any first attempt to learn statistical techniques that emphasizes theory over practice is a dangerous waste of time.

• Applied Statistics by DR Cox & L Snell. This book contains many data sets of varying size and complexity, and turns a wide variety of statistical techniques loose on them. This would be a good book to go through before an applied statistics exam, but should be used as an adjunct to a book that says more about the theory.
• Applied Linear Regression by S Weisberg. This book is brief, but quite comprehensive. It clearly discusses all of the major ideas in linear regression, introducing theory to deal with problems that can easily arise in the analysis of real data. It is an excellent first book to learn from.
• Generalized Linear Models by P McCullagh. Basically, an introduction by the inventors of the technique. At such, it is strongly not recommended to anyone trying to learn about these techniques for the first time. Instead, Dobson's An Introduction To Generalized Linear Models can give a much better idea of what the techniques are for, and why they are important. After trying a few data sets from those books, then read McC and get an idea of the real power of the techniques.
• Statistical Methods by GW Snedecor & HG Cochran. This is a classic, showing all of the estimation methods of 50 years ago in use on a wide range of data sets. All examples are chosen so that you can easily compute solutions using your desktop Burroughs adding machine. While it is a good collection of examples, a modern edition of this book involving computer-based data handling techniques would be welcome.

Useful Math For Statistics

If you are planning to make a career of doing stochastic calculus for investment bankers, it helps to know what a Brownian Motion really is. If you are going to find out, you will need a good background in analysis. These books are useful, though mathematicians might consider them to be a bit out of date.

• Real Analysis by G Folland. This is the one that I learned from. It is very complete and has good exercises, but the typesetting is appalling, designed to minimize the total page count. Has a chapter on probability theory, and a rather frightening but necessary proof of the Marcinkiewicz Interpolation Theorem.
• Real & Complex Analysis by W Rudin. Nice typesetting, but the attempt to combine Real and Complex analysis ends up forcing out a lot of things found in Folland. Trivial note: Rudin once lived in a Usonian house designed by Frank Lloyd Wright.
• Counterexamples In Analysis by BR Gelbaum & JMH Olmstead. This is a very useful book, in the sense that it keeps you humble about your abilities as an analyst.
• Functional Analysis by W Rudin. Better than his real analysis book, it still is a bit terse and also contains a lot of very hard exercises. Still, it is comprehensive and gives lots of pagespace to Fourier transforms.
• Functional Analysis by K Yosida. This is another old classic. It is less of a narrative than Riesz-Nagy, but is almost an encylcopedia of the classical results in functional analysis. Great to use as a reference.
• Functional Analysis by A Wilansky. This is comparable to Rudin in terms of usefulness, and is good as a book to learn from. It is also, like all the other Blaisdell math texts, beautifully designed.
• Mathematical Physics by R Geroch. From the title, it is clear that this is primarily a book about functional analysis. No, really. It is intended to bring physics grads up to speed on the math that underlies QM and relativity, and while not all that rigorous, it provides tons to interesting examples showing how functional analysis can actually be made use of. Also, it is more modern than most of the books listed here, since it makes use of categories and definition via diagram.
• Functional Analysis by F Reisz & B Sz.-Nagy. This is out of date, but beautifully written. It manages to supply a lot of the intuitive detail missing in more modern texts, which makes it useful for those who are trying to learn about the subject.
• Fourier Analysis by TW Koerner. This is one of the best texts in mathematics I've ever read. While rigorous, it also lays out the history of the techniques, the problems that motivated their introduction, and the uses to which they can be put. A second volume builds on the first, but I haven't read it.

Go to the Stat 420 page.

Return to my home page.

© Jeffrey D Picka, January 4, 2001.

다음에 사용할 구문은 \( \)으로 묶어준 다음 \1 \2 식으로 참조.

파일이나 문자열 내에 포함되어 있는 특별한 패턴(또는 특별한 조건을 만족하는 문자열)을 검색하기 위해
미리 정의된 다양한 특수 문자들의 조합을 정규식(regular expression)이라 한다.
정규식에서의 특수 문자(special character)는 다음과 같다.