Markov Processes and Applications

Markov chains are a fundamental class of stochastic processes. They are widely used to solve problems in a large number of domains such as operational research, computer science, communication networks and manufacturing systems. The success of Markov chains is mainly due to their simplicity of use, the large number of available theoretical results and the quality of algorithms developed for the numerical evaluation of many metrics of interest. The author presents the theory of both discrete-time and continuous-time homogeneous Markov chains. He carefully examines the explosion phenomenon, the Kolmogorov equations, the convergence to equilibrium and the passage time distributions to a state and to a subset of states. These results are applied to birth-and-death processes. He then proposes a detailed study of the uniformization technique by means of Banach algebra. This technique is used for the transient analysis of several queuing systems. Contents 1. Discrete-Time Markov Chains 2. Continuous-Time Markov Chains 3. Birth-and-Death Processes 4. Uniformization 5. Queues About the Authors Bruno Sericola is a Senior Research Scientist at Inria Rennes – Bretagne Atlantique in France. His main research activity is in performance evaluation of computer and communication systems, dependability analysis of fault-tolerant systems and stochastic models.

An introduction to micro and nano replication processes and applications Micro/Nano Replication: Processes and Applications provides an overview of the fundamentals, processes, and applications involved in micro and nano replication in the manufacturing of product parts. A major field of nanotechnology, the study of micro/nano replication is sure to become one of increasing importance as the construction of completely new devices based on innovative concepts and crafted at the molecular level increases. Designed to help the reader understand and learn to work with the growing number of tools for molding plastic components, the book covers the key topics related to replication, including patterning technology, the modification of mold surface properties, and much more. In addition, it addresses the strengths and weaknesses of different molding processes. With a strong focus not only on how micro/nano replication works, but also the broader implications for the industry, the book is packed with examples of real world applications. These are drawn from a variety of fields, including information storage devices, optoelectronic elements, optical communication, and biosensors, in order to provide a complete view of the importance of micro and nano processes. A valuable introduction to a new but fast-growing field, Micro/Nano Replication is an essential resource for anyone looking to get a head start on understanding this emerging discipline.

This book serves as a comprehensive source of asymptotic results for econometric models with deterministic exogenous regressors. Such regressors include linear (more generally, piece-wise polynomial) trends, seasonally oscillating functions, and slowly varying functions including logarithmic trends, as well as some specifications of spatial matrices in the theory of spatial models. The book begins with central limit theorems (CLTs) for weighted sums of short memory linear processes. This part contains the analysis of certain operators in Lp spaces and their employment in the derivation of CLTs. The applications of CLTs are to the asymptotic distribution of various estimators for several econometric models. Among the models discussed are static linear models with slowly varying regressors, spatial models, time series autoregressions, and two nonlinear models (binary logit model and nonlinear model whose linearization contains slowly varying regressors). The estimation procedures include ordinary and nonlinear least squares, maximum likelihood, and method of moments. Additional topical coverage includes an introduction to operators, probabilities, and linear models; Lp-approximable sequences of vectors; convergence of linear and quadratic forms; regressions with slowly varying regressors; spatial models; convergence; nonlinear models; and tools for vector autoregressions.

Piecewise-deterministic Markov processes form a class of stochastic models with a sizeable scope of applications: biology, insurance, neuroscience, networks, finance… Such processes are defined by a deterministic motion punctuated by random jumps at random times, and offer simple yet challenging models to study. Nevertheless, the issue of statistical estimation of the parameters ruling the jump mechanism is far from trivial. Responding to new developments in the field as well as to current research interests and needs, Statistical inference for piecewise-deterministic Markov processes offers a detailed and comprehensive survey of state-of-the-art results. It covers a wide range of general processes as well as applied models. The present book also dwells on statistics in the context of Markov chains, since piecewise-deterministic Markov processes are characterized by an embedded Markov chain corresponding to the position of the process right after the jumps.

Probability, Random Variables, and Random Processes is a comprehensive textbook on probability theory for engineers that provides a more rigorous mathematical framework than is usually encountered in undergraduate courses. It is intended for first-year graduate students who have some familiarity with probability and random variables, though not necessarily of random processes and systems that operate on random signals. It is also appropriate for advanced undergraduate students who have a strong mathematical background. The book has the following features: Several appendices include related material on integration, important inequalities and identities, frequency-domain transforms, and linear algebra. These topics have been included so that the book is relatively self-contained. One appendix contains an extensive summary of 33 random variables and their properties such as moments, characteristic functions, and entropy. Unlike most books on probability, numerous figures have been included to clarify and expand upon important points. Over 600 illustrations and MATLAB plots have been designed to reinforce the material and illustrate the various characterizations and properties of random quantities. Sufficient statistics are covered in detail, as is their connection to parameter estimation techniques. These include classical Bayesian estimation and several optimality criteria: mean-square error, mean-absolute error, maximum likelihood, method of moments, and least squares. The last four chapters provide an introduction to several topics usually studied in subsequent engineering courses: communication systems and information theory; optimal filtering (Wiener and Kalman); adaptive filtering (FIR and IIR); and antenna beamforming, channel equalization, and direction finding. This material is available electronically at the companion website. Probability, Random Variables, and Random Processes is the only textbook on probability for engineers that includes relevant background material, provides extensive summaries of key results, and extends various statistical techniques to a range of applications in signal processing.

A comprehensive and accessible presentation of probability and stochastic processes with emphasis on key theoretical concepts and real-world applications With a sophisticated approach, Probability and Stochastic Processes successfully balances theory and applications in a pedagogical and accessible format. The book’s primary focus is on key theoretical notions in probability to provide a foundation for understanding concepts and examples related to stochastic processes. Organized into two main sections, the book begins by developing probability theory with topical coverage on probability measure; random variables; integration theory; product spaces, conditional distribution, and conditional expectations; and limit theorems. The second part explores stochastic processes and related concepts including the Poisson process, renewal processes, Markov chains, semi-Markov processes, martingales, and Brownian motion. Featuring a logical combination of traditional and complex theories as well as practices, Probability and Stochastic Processes also includes: Multiple examples from disciplines such as business, mathematical finance, and engineering Chapter-by-chapter exercises and examples to allow readers to test their comprehension of the presented material A rigorous treatment of all probability and stochastic processes concepts An appropriate textbook for probability and stochastic processes courses at the upper-undergraduate and graduate level in mathematics, business, and electrical engineering, Probability and Stochastic Processes is also an ideal reference for researchers and practitioners in the fields of mathematics, engineering, and finance.

As the range of feedstocks, process technologies and products expand, biorefineries will become increasingly complex manufacturing systems. Biorefineries and Chemical Processes: Design, Integration and Sustainability Analysis presents process modelling and integration, and whole system life cycle analysis tools for the synthesis, design, operation and sustainable development of biorefinery and chemical processes. Topics covered include: Introduction: An introduction to the concept and development of biorefineries. Tools: Included here are the methods for detailed economic and environmental impact analyses; combined economic value and environmental impact analysis; life cycle assessment (LCA); multi-criteria analysis; heat integration and utility system design; mathematical programming based optimization and genetic algorithms. Process synthesis and design: Focuses on modern unit operations and innovative process flowsheets. Discusses thermochemical and biochemical processing of biomass, production of chemicals and polymers from biomass, and processes for carbon dioxide capture. Biorefinery systems: Presents biorefinery process synthesis using whole system analysis. Discusses bio-oil and algae biorefineries, integrated fuel cells and renewables, and heterogeneous catalytic reactors. Companion website: Four case studies, additional exercises and examples are available online, together with three supplementary chapters which address waste and emission minimization, energy storage and control systems, and the optimization and reuse of water. This textbook is designed to bridge a gap between engineering design and sustainability assessment, for advanced students and practicing process designers and engineers.

Generally, books on mathematical statistics are restricted to the case of independent identically distributed random variables. In this book however, both this case AND the case of dependent variables, i.e. statistics for discrete and continuous time processes, are studied. This second case is very important for today’s practitioners. Mathematical Statistics and Stochastic Processes is based on decision theory and asymptotic statistics and contains up-to-date information on the relevant topics of theory of probability, estimation, confidence intervals, non-parametric statistics and robustness, second-order processes in discrete and continuous time and diffusion processes, statistics for discrete and continuous time processes, statistical prediction, and complements in probability. This book is aimed at students studying courses on probability with an emphasis on measure theory and for all practitioners who apply and use statistics and probability on a daily basis.

A mathematical and intuitive approach to probability, statistics, and stochastic processes This textbook provides a unique, balanced approach to probability, statistics, and stochastic processes. Readers gain a solid foundation in all three fields that serves as a stepping stone to more advanced investigations into each area. This text combines a rigorous, calculus-based development of theory with a more intuitive approach that appeals to readers' sense of reason and logic, an approach developed through the author's many years of classroom experience. The text begins with three chapters that develop probability theory and introduce the axioms of probability, random variables, and joint distributions. The next two chapters introduce limit theorems and simulation. Also included is a chapter on statistical inference with a section on Bayesian statistics, which is an important, though often neglected, topic for undergraduate-level texts. Markov chains in discrete and continuous time are also discussed within the book. More than 400 examples are interspersed throughout the text to help illustrate concepts and theory and to assist the reader in developing an intuitive sense of the subject. Readers will find many of the examples to be both entertaining and thought provoking. This is also true for the carefully selected problems that appear at the end of each chapter. This book is an excellent text for upper-level undergraduate courses. While many texts treat probability theory and statistical inference or probability theory and stochastic processes, this text enables students to become proficient in all three of these essential topics. For students in science and engineering who may take only one course in probability theory, mastering all three areas will better prepare them to collect, analyze, and characterize data in their chosen fields.

Praise for the First Edition «. . . an excellent textbook . . . well organized and neatly written.» —Mathematical Reviews «. . . amazingly interesting . . .» —Technometrics Thoroughly updated to showcase the interrelationships between probability, statistics, and stochastic processes, Probability, Statistics, and Stochastic Processes, Second Edition prepares readers to collect, analyze, and characterize data in their chosen fields. Beginning with three chapters that develop probability theory and introduce the axioms of probability, random variables, and joint distributions, the book goes on to present limit theorems and simulation. The authors combine a rigorous, calculus-based development of theory with an intuitive approach that appeals to readers' sense of reason and logic. Including more than 400 examples that help illustrate concepts and theory, the Second Edition features new material on statistical inference and a wealth of newly added topics, including: Consistency of point estimators Large sample theory Bootstrap simulation Multiple hypothesis testing Fisher's exact test and Kolmogorov-Smirnov test Martingales, renewal processes, and Brownian motion One-way analysis of variance and the general linear model Extensively class-tested to ensure an accessible presentation, Probability, Statistics, and Stochastic Processes, Second Edition is an excellent book for courses on probability and statistics at the upper-undergraduate level. The book is also an ideal resource for scientists and engineers in the fields of statistics, mathematics, industrial management, and engineering.