Information Theory, Inference and Learning Algorithms

JNC Academic Books Verified independent store

→

David J. C. MacKayCambridge University PressEdition: Sixth Printing 2007, 9/25/2003EAN 9780521642989, ISBN10: 0521642981Hardcover, 640 pages, 25.4 x 19.5 x 3.4 cmLanguage: EnglishI nformation theory and inference, taught together in this exciting textbook, lie at the heart of many important areas of modern technology - communication, signal processing, data mining, machine learning, pattern recognition, computational neuroscience, bioinformatics and cryptography. The book introduces theory in tandem with applications. Information theory is taught alongside practical communication systems such as arithmetic coding for data compression and sparse-graph codes for error-correction. Inference techniques, including message-passing algorithms, Monte Carlo methods and variational approximations, are developed alongside applications to clustering, convolutional codes, independent component analysis, and neural networks. Uniquely, the book covers state-of-the-art error-correcting codes, including low-density-parity-check codes, turbo codes, and digital fountain codes - the twenty-first-century standards for satellite communications, disk drives, and data broadcast. Richly illustrated, filled with worked examples and over 400 exercises, some with detailed solutions, the book is ideal for self-learning, and for undergraduate or graduate courses. It also provides an unparalleled entry point for professionals in areas as diverse as computational biology, financial engineering and machine learning.1. Introduction to information theory2. Probability, entropy and inference3. More about inferencePart I. Data Compression4. The source coding theorem5. Symbol codes6. Stream codes7. Codes for integersPart II. Noisy-Channel Coding8. Dependent random variables9. Communication over a noisy channel10. The noisy-channel coding theorem11. Error-correcting codes and real channelsPart III. Further Topics in Information Theory12. Hash codes13. Binary codes14. Very good linear codes exist15. Further exercises on information theory16. Message passing17. Constrained noiseless channels18. Crosswords and codebreaking19. Why have sex? Information acquisition and evolutionPart IV. Probabilities and Inference20. An example inference taskclustering21. Exact inference by complete enumeration22. Maximum likelihood and clustering23. Useful probability distributions24. Exact marginalization25. Exact marginalization in trellises26. Exact marginalization in graphs27. Laplace's method28. Model comparison and Occam's razor29. Monte Carlo methods30. Efficient Monte Carlo methods31. Ising models32. Exact Monte Carlo sampling33. Variational methods34. Independent component analysis35. Random inference topics36. Decision theory37. Bayesian inference and sampling theoryPart V. Neural Networks38. Introduction to neural networks39. The single neuron as a classifier40. Capacity of a single neuron41. Learning as inference42. Hopfield networks43. Boltzmann machines44. Supervised learning in multilayer networks45. Gaussian processes46. DeconvolutionPart VI. Sparse Graph Codes47. Low-density parity-check codes48. Convolutional codes and turbo codes49. Repeat-accumulate codes50. Digital fountain codesPart VII. AppendicesA. NotationB. Some physicsC. Some mathematicsBibliographyIndex.'This is an extraordinary and important book, generous with insight and rich with detail in statistics, information theory, and probabilistic modeling across a wide swathe of standard, creatively original, and delightfully quirky topics. David MacKay is an uncompromisingly lucid thinker, from whom students, faculty and practitioners all can learn.' Peter Dayan and Zoubin Ghahramani, Gatsby Computational Neuroscience Unit, University College, London Show more