Quantitative and Statistical Data in Education. From Data Collection to Data Processing

A pragmatic approach to Big Data by taking the reader on a journey between Big Data (what it is) and the Smart Data (what it is for). Today’s decision making can be reached via information (related to the data), knowledge (related to people and processes), and timing (the capacity to decide, act and react at the right time). The huge increase in volume of data traffic, and its format (unstructured data such as blogs, logs, and video) generated by the “digitalization” of our world modifies radically our relationship to the space (in motion) and time, dimension and by capillarity, the enterprise vision of performance monitoring and optimization.

A comprehensive introduction to statistical methods for data mining and knowledge discovery. Applications of data mining and ‘big data’ increasingly take center stage in our modern, knowledge-driven society, supported by advances in computing power, automated data acquisition, social media development and interactive, linkable internet software. This book presents a coherent, technical introduction to modern statistical learning and analytics, starting from the core foundations of statistics and probability. It includes an overview of probability and statistical distributions, basics of data manipulation and visualization, and the central components of standard statistical inferences. The majority of the text extends beyond these introductory topics, however, to supervised learning in linear regression, generalized linear models, and classification analytics. Finally, unsupervised learning via dimension reduction, cluster analysis, and market basket analysis are introduced. Extensive examples using actual data (with sample R programming code) are provided, illustrating diverse informatic sources in genomics, biomedicine, ecological remote sensing, astronomy, socioeconomics, marketing, advertising and finance, among many others. Statistical Data Analytics: Focuses on methods critically used in data mining and statistical informatics. Coherently describes the methods at an introductory level, with extensions to selected intermediate and advanced techniques. Provides informative, technical details for the highlighted methods. Employs the open-source R language as the computational vehicle – along with its burgeoning collection of online packages – to illustrate many of the analyses contained in the book. Concludes each chapter with a range of interesting and challenging homework exercises using actual data from a variety of informatic application areas. This book will appeal as a classroom or training text to intermediate and advanced undergraduates, and to beginning graduate students, with sufficient background in calculus and matrix algebra. It will also serve as a source-book on the foundations of statistical informatics and data analytics to practitioners who regularly apply statistical learning to their modern data.

Guides readers through the quantitative data analysis process including contextualizing data within a research situation, connecting data to the appropriate statistical tests, and drawing valid conclusions Introduction to Quantitative Data Analysis in the Behavioral and Social Sciences presents a clear and accessible introduction to the basics of quantitative data analysis and focuses on how to use statistical tests as a key tool for analyzing research data. The book presents the entire data analysis process as a cyclical, multiphase process and addresses the processes of exploratory analysis, decision-making for performing parametric or nonparametric analysis, and practical significance determination. In addition, the author details how data analysis is used to reveal the underlying patterns and relationships between the variables and connects those trends to the data’s contextual situation. Filling the gap in quantitative data analysis literature, this book teaches the methods and thought processes behind data analysis, rather than how to perform the study itself or how to perform individual statistical tests. With a clear and conversational style, readers are provided with a better understanding of the overall structure and methodology behind performing a data analysis as well as the needed techniques to make informed, meaningful decisions during data analysis. The book features numerous data analysis examples in order to emphasize the decision and thought processes that are best followed, and self-contained sections throughout separate the statistical data analysis from the detailed discussion of the concepts allowing readers to reference a specific section of the book for immediate solutions to problems and/or applications. Introduction to Quantitative Data Analysis in the Behavioral and Social Sciences also features coverage of the following: • The overall methodology and research mind-set for how to approach quantitative data analysis and how to use statistics tests as part of research data analysis • A comprehensive understanding of the data, its connection to a research situation, and the most appropriate statistical tests for the data • Numerous data analysis problems and worked-out examples to illustrate the decision and thought processes that reveal underlying patterns and trends • Detailed examples of the main concepts to aid readers in gaining the needed skills to perform a full analysis of research problems • A conversational tone to effectively introduce readers to the basics of how to perform data analysis as well as make meaningful decisions during data analysis Introduction to Quantitative Data Analysis in the Behavioral and Social Sciences is an ideal textbook for upper-undergraduate and graduate-level research method courses in the behavioral and social sciences, statistics, and engineering. This book is also an appropriate reference for practitioners who require a review of quantitative research methods. Michael J. Albers, Ph.D., is Professor in the Department of English at East Carolina University. His research interests include information design with a focus on answering real-world questions, the presentation of complex information, and human–information interaction. Dr. Albers received his Ph.D. in Technical Communication and Rhetoric from Texas Tech University.

Ecologists are increasingly tackling difficult issues like global change, loss of biodiversity and sustainability of ecosystem services. These and related topics are enormously challenging, requiring unprecedented multidisciplinary collaboration and rapid synthesis of large amounts of diverse data into information and ultimately knowledge. New sensors, computers, data collection and storage devices and analytical and statistical methods provide a powerful tool kit to support analyses, graphics and visualizations that were unthinkable even a few years ago. New and increased emphasis on accessibility, management, processing and sharing of high-quality, well-maintained and understandable data represents a significant change in how scientists view and treat data. These issues are complex and despite their importance, are typically not addressed in database, ecological and statistical textbooks. This book addresses these issues, providing a much needed resource for those involved in designing and implementing ecological research, as well as students who are entering the environmental sciences. Chapters focus on the design of ecological studies, data management principles, scientific databases, data quality assurance, data documentation, archiving ecological data and information and processing data into information and knowledge. The book stops short of a detailed treatment of data analysis, but does provide pointers to the relevant literature in graphics, statistics and knowledge discovery. The central thesis of the book is that high quality data management systems are critical for addressing future environmental challenges. This requires a new approach to how we conduct ecological research, that views data as a resource and promotes stewardship, recycling and sharing of data. Ecological Data will be particularly useful to those ecologists and information specialists that actively design, manage and analyze environmental databases. However, it will also benefit a wider audience of scientists and students in the ecological and environmental sciences.

The world has become digital and technological advances have multiplied circuits with access to data, their processing and their diffusion. New technologies have now reached a certain maturity. Data are available to everyone, anywhere on the planet. The number of Internet users in 2014 was 2.9 billion or 41% of the world population. The need for knowledge is becoming apparent in order to understand this multitude of data. We must educate, inform and train the masses. The development of related technologies, such as the advent of the Internet, social networks, «cloud-computing» (digital factories), has increased the available volumes of data. Currently, each individual creates, consumes, uses digital information: more than 3.4 million e-mails are sent worldwide every second, or 107,000 billion annually with 14,600 e-mails per year per person, but more than 70% are spam. Billions of pieces of content are shared on social networks such as Facebook, more than 2.46 million every minute. We spend more than 4.8 hours a day on the Internet using a computer, and 2.1 hours using a mobile. Data, this new ethereal manna from heaven, is produced in real time. It comes in a continuous stream from a multitude of sources which are generally heterogeneous. This accumulation of data of all types (audio, video, files, photos, etc.) generates new activities, the aim of which is to analyze this enormous mass of information. It is then necessary to adapt and try new approaches, new methods, new knowledge and new ways of working, resulting in new properties and new challenges since SEO logic must be created and implemented. At company level, this mass of data is difficult to manage. Its interpretation is primarily a challenge. This impacts those who are there to «manipulate» the mass and requires a specific infrastructure for creation, storage, processing, analysis and recovery. The biggest challenge lies in «the valuing of data» available in quantity, diversity and access speed.

Few books on statistical data analysis in the natural sciences are written at a level that a non-statistician will easily understand. This is a book written in colloquial language, avoiding mathematical formulae as much as possible, trying to explain statistical methods using examples and graphics instead. To use the book efficiently, readers should have some computer experience. The book starts with the simplest of statistical concepts and carries readers forward to a deeper and more extensive understanding of the use of statistics in environmental sciences. The book concerns the application of statistical and other computer methods to the management, analysis and display of spatial data. These data are characterised by including locations (geographic coordinates), which leads to the necessity of using maps to display the data and the results of the statistical methods. Although the book uses examples from applied geochemistry, and a large geochemical survey in particular, the principles and ideas equally well apply to other natural sciences, e.g., environmental sciences, pedology, hydrology, geography, forestry, ecology, and health sciences/epidemiology. The book is unique because it supplies direct access to software solutions (based on R, the Open Source version of the S-language for statistics) for applied environmental statistics. For all graphics and tables presented in the book, the R-scripts are provided in the form of executable R-scripts. In addition, a graphical user interface for R, called DAS+R, was developed for convenient, fast and interactive data analysis. Statistical Data Analysis Explained: Applied Environmental Statistics with R provides, on an accompanying website, the software to undertake all the procedures discussed, and the data employed for their description in the book.

The field of data mining lies at the confluence of predictive analytics, statistical analysis, and business intelligence. Due to the ever-increasing complexity and size of data sets and the wide range of applications in computer science, business, and health care, the process of discovering knowledge in data is more relevant than ever before. This book provides the tools needed to thrive in today’s big data world. The author demonstrates how to leverage a company’s existing databases to increase profits and market share, and carefully explains the most current data science methods and techniques. The reader will “learn data mining by doing data mining”. By adding chapters on data modelling preparation, imputation of missing data, and multivariate statistical analysis, Discovering Knowledge in Data, Second Edition remains the eminent reference on data mining. The second edition of a highly praised, successful reference on data mining, with thorough coverage of big data applications, predictive analytics, and statistical analysis. Includes new chapters on Multivariate Statistics, Preparing to Model the Data, and Imputation of Missing Data, and an Appendix on Data Summarization and Visualization Offers extensive coverage of the R statistical programming language Contains 280 end-of-chapter exercises Includes a companion website for university instructors who adopt the book

Praise for the First Edition «An indispensable addition to any serious collection on lifetime data analysis and . . . a valuable contribution to the statistical literature. Highly recommended . . .» -Choice «This is an important book, which will appeal to statisticians working on survival analysis problems.» -Biometrics «A thorough, unified treatment of statistical models and methods used in the analysis of lifetime data . . . this is a highly competent and agreeable statistical textbook.» -Statistics in Medicine The statistical analysis of lifetime or response time data is a key tool in engineering, medicine, and many other scientific and technological areas. This book provides a unified treatment of the models and statistical methods used to analyze lifetime data. Equally useful as a reference for individuals interested in the analysis of lifetime data and as a text for advanced students, Statistical Models and Methods for Lifetime Data, Second Edition provides broad coverage of the area without concentrating on any single field of application. Extensive illustrations and examples drawn from engineering and the biomedical sciences provide readers with a clear understanding of key concepts. New and expanded coverage in this edition includes: * Observation schemes for lifetime data * Multiple failure modes * Counting process-martingale tools * Both special lifetime data and general optimization software * Mixture models * Treatment of interval-censored and truncated data * Multivariate lifetimes and event history models * Resampling and simulation methodology

The main purpose of this book is to investigate, explore and describe approaches and methods to facilitate data understanding through analytics solutions based on its principles, concepts and applications. But analyzing data is also about involving the use of software. For this, and in order to cover some aspect of data analytics, this book uses software (Excel, SPSS, Python, etc) which can help readers to better understand the analytics process in simple terms and supporting useful methods in its application.

A practical, one-stop reference on the theory and applications of statistical data editing and imputation techniques Collected survey data are vulnerable to error. In particular, the data collection stage is a potential source of errors and missing values. As a result, the important role of statistical data editing, and the amount of resources involved, has motivated considerable research efforts to enhance the efficiency and effectiveness of this process. Handbook of Statistical Data Editing and Imputation equips readers with the essential statistical procedures for detecting and correcting inconsistencies and filling in missing values with estimates. The authors supply an easily accessible treatment of the existing methodology in this field, featuring an overview of common errors encountered in practice and techniques for resolving these issues. The book begins with an overview of methods and strategies for statistical data editing and imputation. Subsequent chapters provide detailed treatment of the central theoretical methods and modern applications, with topics of coverage including: Localization of errors in continuous data, with an outline of selective editing strategies, automatic editing for systematic and random errors, and other relevant state-of-the-art methods Extensions of automatic editing to categorical data and integer data The basic framework for imputation, with a breakdown of key methods and models and a comparison of imputation with the weighting approach to correct for missing values More advanced imputation methods, including imputation under edit restraints Throughout the book, the treatment of each topic is presented in a uniform fashion. Following an introduction, each chapter presents the key theories and formulas underlying the topic and then illustrates common applications. The discussion concludes with a summary of the main concepts and a real-world example that incorporates realistic data along with professional insight into common challenges and best practices. Handbook of Statistical Data Editing and Imputation is an essential reference for survey researchers working in the fields of business, economics, government, and the social sciences who gather, analyze, and draw results from data. It is also a suitable supplement for courses on survey methods at the upper-undergraduate and graduate levels.