Different Types of Predictive Models:

  • A key aim of machine learning is to build predictive models1 from data.

  • For statistical models, we assume a particular functional relationship between two or more variables (such as a straight line relating two variables in simple linear regression).

    • Example: Explaining the relationship between study hours and test scores.
  • In mechanistic models, we develop a model for a process that enables us to relate different variables to each other. These process-based models often involve a simulation based upon rules of physics, chemistry, economics, or some other domain.

    • Example: Modeling how a virus spreads through a population using infection and recovery rates.

Diagram comparing statistical models and mechanistic models with labeled examples showing functional relationships

 

  • Machine learning models introduce a more algorithmic approach driven by data, without assuming a particular form for the relationships among variables. ML models are often black boxes, able to make predictions after being trained, but with little insight to how those predictions are made.
    • Example: Determine whether an incoming email is spam or not through classification.

Diagram illustrating machine learning as a black box model trained on data to produce predictions


Introduction to Machine Learning:

  • ML aims to automate the processes of pattern discovery and statistical modeling of data.
  • Supervised Learning: Works with a set of labeled examples to begin with. There are two broad sub types of problems:
    • Classification: Predicting a categorical label (e.g., spam or not spam).
    • Regression: Predicting a numerical value (e.g., price of a house).

Diagram of supervised learning divided into Classification (categorical labels) and Regression (numerical values)

  • Unsupervised Learning: Only the data themselves are provided. Examples include:
    • Feature extraction: Finding a smaller set of features that can be used to represent the data and capture semantic meaning.
    • Clustering: Grouping similar data points together (e.g., customer segmentation).

Strategy For Evaluating Supervised Learning Models:

  • Train-test split: Split the data into a training set and a test set. The training set is used to train the model, while the test set is used to evaluate its performance.

Diagram showing data split into training set and test set for evaluating supervised learning model performance

Footnotes

  1. Breiman, Leo. Statistical modeling: The two cultures. Statistical science 16.3 (2001): 199-231. ↩