Building a Neural Network From Scratch

*NOTE: This guide assumes you have some experience with programming (knowing python is definitely helpful too). For a guide on getting started with programming, go here (link coming soon) *

Build Your Own Neural Network

When I first wanted to dive into AI and machine learning, I had no idea where to start. One of my favorite youtube channels, 3blue1brown, had released a video on neural networks a few months back, and I had internalized the video as being pretty cool, but I didn’t take the time to really learn everything being presented. I remember thinking that the video was awesome, but that there was no way I could actually work with this technology myself.

Once I heard that Mr. Johnson was going to be offering the AI course this year, I got a burst of motivation to rewatch the aforementioned video. This time, when I watched the video, I took notes. After taking my notes, I went into my python IDE of choice, and I started making a neural network from scratch. This process ended up taking me a week on the first attempt, but my result was very poor. It worked, but it felt like I had hacked together a bunch of quick fixes near the end of its development. The next weekend, I set aside an entire day to redo my network, and make it much better. This time, I had more confidence since I had already constructed one before, and I ended up with a very good result - a modular neural network that I could adjust the size of very easily. By doing this, I came out with a very strong understanding of neural networks, as I was forced to debug all the small issues along the way. However, his process wasn’t easy. It involved hours and days of frustration trying to get all the components of the network to function together. But even though it took a lot of time and energy, I can honestly say that the first time I saw successful results come out of my network I was ecstatic.

Undertaking a project like this won’t be easy as an introductory task for learning about machine learning, but I strongly recommend it if you want to truly immerse yourself into this field. Throughout the process, you will surely hit roadblocks and get stuck, but at these moments I encourage you to ask questions: whether it be to the internet, Mr. Johnson, your classmates, or even to me. Below, I will link some resources, along with a very rough outline for how to go about creating the network (no specifics).

3Blue1Brown Neural Network Series
Some Helpful Code (if you get stuck)
Anaconda (Easy to install version of python along with various important ML libraries)
MNIST (Sample data, definitely worth reading about)

*NOTE - I would reccommend doing this project in Jupyter Notebook for easy prototyping. For more information on Jupyter Notebook, click here (link coming soon)*

Steps:

1. Install Anaconda

2. Watch the 3Blue1Brown series and write down the key terms and ideas (these will be very important concepts for all of machine learning)

3. Start with the skeleton of a 2 layer network - Going from the input layer to the output layer

4. Decide a fixed size for this 2 layer network (maybe 3 inputs to 2 outputs)

5. Implement the 'feed-forward' process of the network

6. Research how to implement back propagation (arguably the most difficult step in this entire process), and then put it in your network. The video series should give you an understanding of how back propagation works, but in order to implement it in python you will likely have to look through various other resources. This process took me a couple weeks before I really understood how to properly implement back propagation.

7. Put together the feed-forward and back-prop components to form a functioning network

8. Implement a way to calculate and print the cost/error

9. Test your network on sample data to ensure that it is working (cost/error decreases over training iterations)

10. Scale up your network (still 2 layers) so that it works with variable input and output sizes
11. Take note of what works well in your network, what doesn't work so well, and make plans for improving certain components.

Now, if you made it this far, congratulations! However, the worst may still be yet to come...

12. Create a new jupyter notebook, and start over! This time, with the idea of creating a 'deep' neural network (3 layers or more). Decide on a fixed amount of layers (maybe 3), but keep in mind that you will be wanting to eventually scale this up to a variable number of layers.

HINT:

13. Test your completed network on sample data and ensure it is still working

14. Now, scale up your network to work with a variable number of layers specified at the time of creation



You should do this project spaced out over the course of a couple weeks, or even over the course of a month. It will take a lot of time and patience, but when you make it through, I can promise that you will have a thorough understanding of how neural networks function. Again, if you have any questions throughout the process (and you surely will! The resources I linked above will not provide you with every bit of information that you will need to finish this project) feel free to reach out to Mr. Johnson or myself. Good luck!