4. Machine Learning#

At its core, machine learning (ML) is about the idea of utilizing computers to learn from data. Instead of explicitly programming a computer with a fixed set of rules, we provide it with examples, and the computer figures out patterns to make decisions and/or predictions. You can think of it like showing a toddler how to recognize shapes: instead of explaining every single detail, you give them examples until they understand what a square or circle looks like.

In today’s day and age, machine learning is used everywhere! From self-driving cars, recommendation systems, chatbots, language translation, and even diagnosing diseases. But this wasn’t always the case. The idea of machines “learning” from data started long before the powerful computers we have today.

4.1. A Brief History#

The roots of machine learning trace back to the mid-20th century. In 1959, computer scientist Arthur Samuel coined the term “machine learning” while working on a program that could play checkers. His program didn’t just follow fixed strategies—it improved its gameplay over time by learning from past matches. Samuel described this as giving the computer the “ability to learn without being explicitly programmed.”

Over the decades, advancements in computing power and data availability have transformed machine learning from a niche academic pursuit to a cornerstone of modern technology.

4.2. Starting with Simple Linear Regression#

For this course, we will primarily focus on Simple Linear Regression (SLR). To truly appreciate machine learning, we need to start small. Do not be deceived though, Linear Regression is not an “easy” concept to understand by any means. If you really dive deep into the math behind it, it can get pretty complicated. Feel free to look at some of the materials with courses like DATA 140 or COMPSCI 189 to get a feel for the math derivations. However, since this is not a math class, we will try out best to explain this concept intuitively.

As a brief overview, SLR focuses on answering the key question: “Can we use one variable to predict another?”

Consider the example:

  • Can the number of hours a student studies predict their test score?

  • Can the size of a house predict its price?

4.3. Expectations#

In this chapter, we will introduce the fundamental concepts of Machine Learning, focusing on the basics that form the foundation for more advanced topics. Specifically, we will explore how ML systems learn from data by diving into concepts like training, testing, and validation. These processes ensure that the models we create not only perform well on the data they learn from but also generalize effectively to new, unseen data.

To ground our understanding, we will start Linear Regression. In particular, practicing how to predict ____ from ____, as well as _____ from _____. Through this, you will learn how to use data to predict outcomes, evaluate model performance, and understand what makes one model better than another. Along the way, we’ll explore key techniques like feature engineering, which involves selecting and transforming the inputs (features) of your model to improve its accuracy and reliability.

Finally, to make implementing these concepts easier and more efficient, we’ll introduce you to a popular Python library called Scikit-learn and Statsmodels. This library simplifies the process of building, training, and evaluating machine learning models while providing powerful tools for data preprocessing and analysis. By the end of this chapter, hopefully you will have a strong foundation to use these tools and techniques in your own projects.