A two part tutorial series implementing the gradient descent algorithm without the use of any machine learning libraries.

Gradient descent is a very commonly used optimization method in modern machine learning. This two part series is intended to help people gain a better understanding of how it works by implementing it without the use of any machine learning libraries.

A basic understanding of calculus, linear algebra and python programming are required to get the most out of these tutorials.

Jupyter notebooks that contain explanations of underlying concepts followed by code that can be run from within the notebook.

Part 1 - Intoduction to gradient descent on a simple linear regression problem

Part 2 - Training a neural network to classify handwritten digits

Code is written for readability and is heavily commented to aid beginners. Not an exemplar of production code.

Jupyter Notebook
Python 3x
Numpy, matplotlib

• Clone repository
• Download mnist here https://drive.google.com/file/d/1b3vmpg0J7IABO-FUTLVT0uEHHBDzI0Mb/view?usp=sharing
• Put mnist_normalized.pkl into local repository
• Open 'part_1' with jupyter-notebook.

Any constructive feedback on how this could be improved is welcome.