7.48. Exploratory data analysis (Elective)

7.48. Exploratory data analysis (Elective)

Figure 7.48: Connection Map. CS253 Exploratory data analysis

7.48.1. Justification ↑ Back to top

Exploratory Data Analysis (EDA) is a critical stage in any data science project. This course provides the tools to understand the structure, detect anomalies, and extract initial patterns from data before formal modeling. It focuses on the use of descriptive statistics, data cleaning, and preliminary visualization to validate assumptions and guide the selection of complex models, ensuring the integrity and quality of scientific findings.

7.48.2. Generales Goals ↑ Back to top

  1. Apply data cleaning and preprocessing techniques to ensure data quality.
  2. Use descriptive statistical methods to summarize complex datasets.
  3. Identify patterns, outliers, and relationships between variables through visual analysis.
  4. Formulate hypotheses based on the initial exploration of the data.
  5. Communicate preliminary findings effectively through technical reports.

7.48.3. Contribution to Outcomes ↑ Back to top

ABET-3) An ability to develop and conduct experiments or test hypotheses, analyze and interpret data, and use scientific judgment to draw conclusions. (Usage)
ABET-2) An ability to formulate or design a system, process, procedure, or program for the intended purpose. (Usage)

7.48.4. Content ↑ Back to top

7.48.4.1. Foundations of Exploratory Data Analysis (8 hours) [Skills ABET-2,ABET-3] ↑ Back to top

Bibliography: (Tukey, 1977; Peng, 2016)

Topics

  1. Philosophy and objectives of exploratory analysis.
  2. Data types: structured, semi-structured, and unstructured.
  3. Measurement scales: nominal, ordinal, interval, and ratio.
  4. Workflow in data exploration.

Learning Outcomes

  1. Identify the importance of EDA in the data life cycle [Familiarity].
  2. Classify variables according to their measurement scale and type [Usage].
7.48.4.2. Data Cleaning and Preparation (8 hours) [Skills ABET-2,ABET-3] ↑ Back to top

Bibliography: (Wickham, 2014; McKinney, 2022)

Topics

  1. "Tidy Data" concept and table manipulation.
  2. Handling missing values: imputation versus removal.
  3. Detection and treatment of outliers.
  4. Normalization and standardization of numerical variables.

Learning Outcomes

  1. Implement cleaning routines for noisy datasets [Usage].
  2. Evaluate the impact of different imputation strategies [Assessment].
7.48.4.3. Univariate Descriptive Statistics (8 hours) [Skills ABET-2,ABET-3] ↑ Back to top

Bibliography: (Healy, 2018; Tukey, 1977)

Topics

  1. Measures of central tendency, dispersion, and shape.
  2. Frequency distributions and density functions.
  3. Skewness and kurtosis analysis in real variables.
  4. Univariate visualization: Histograms, Boxplots, and Kernel Density Estimation (KDE).

Learning Outcomes

  1. Numerically summarize the distribution of a variable [Usage].
  2. Interpret density plots to identify biases in the data [Assessment].
7.48.4.4. Bivariate and Multivariate Exploration (8 hours) [Skills ABET-2,ABET-3] ↑ Back to top

Bibliography: (Healy, 2018; Wickham and Grolemund, 2016)

Topics

  1. Pearson, Spearman, and Kendall correlation.
  2. Contingency tables and measures of categorical association.
  3. Scatter plots, correlation matrices, and heatmaps.
  4. Conceptual introduction to dimensionality reduction (PCA).

Learning Outcomes

  1. Quantify the relationship between multiple variables in a dataset [Usage].
  2. Use heatmaps to detect collinearity in datasets [Assessment].
7.48.4.5. Computational Tools for Exploration (8 hours) [Skills ABET-2,ABET-3] ↑ Back to top

Bibliography: (McKinney, 2022; Wickham and Grolemund, 2016)

Topics

  1. Computational ecosystems for analysis (Python/Pandas or R/Tidyverse).
  2. Efficient manipulation of large-volume DataFrames.
  3. Automatic generation of data profiling reports.

Learning Outcomes

  1. Implement automated EDA processes through programming [Usage].
  2. Optimize data processing for efficient analysis [Assessment].
7.48.4.6. Exploratory Hypothesis Testing (8 hours) [Skills ABET-2,ABET-3] ↑ Back to top

Bibliography: (Peng, 2016; Healy, 2018)

Topics

  1. Normality tests and variance assumptions.
  2. Group comparison: T-test and ANOVA in an exploratory context.
  3. Non-parametric methods and resampling (Bootstrapping).

Learning Outcomes

  1. Validate statistical assumptions necessary for subsequent modeling [Usage].
  2. Design exploratory experiments to confirm visual findings [Assessment].
7.48.4.7. Data Ethics and Responsibility (8 hours) [Skills ABET-2,ABET-3] ↑ Back to top

Bibliography: (Healy, 2018)

Topics

  1. Identification of selection and confirmation biases in data.
  2. Data privacy: Anonymization and masking techniques.
  3. Ethics in communicating preliminary findings and associated risks.

Learning Outcomes

  1. Identify potential ethical biases in real-world datasets [Familiarity].
  2. Apply basic anonymization techniques during analysis [Usage].
7.48.4.8. Result Communication and Data Storytelling (8 hours) [Skills ABET-2,ABET-3] ↑ Back to top

Bibliography: (Wickham and Grolemund, 2016; Healy, 2018)

Topics

  1. Structure of a technical data quality report.
  2. Storytelling based on exploratory discovery.
  3. Selection of key visualizations for decision-making.

Learning Outcomes

  1. Write effective technical reports on data quality [Assessment].
  2. Perform presentations of exploratory findings to technical audiences [Assessment].

7.48.5. Bibliography ↑ Back to top

Tukey, J. W. (1977). Exploratory Data Analysis. Addison-Wesley.

Peng, R. D. (2016). Exploratory Data Analysis with R. Leanpub.

Wickham, H. (2014). Tidy data. Journal of Statistical Software, 59(10):1–23.

McKinney, W. (2022). Python for Data Analysis. O'Reilly Media, 3rd edition.

Healy, K. (2018). Data Visualization: A Practical Introduction. Princeton University Press.

Wickham, H. and Grolemund, G. (2016). R for Data Science. O'Reilly Media.

Spotted a typo, an outdated course, a broken link, or have a suggestion? Let us know.

Scan to open on your phone