Hacker School, Monday, June 30th, 2014

Machine Learning

On Monday, I had deliberately put aside the design decisions about Protagonist that I brought up the day before. My plan was a one-line meta-plan: work with my machine learning partner on developing a project plan.

She and I decided we wanted to focus on ML algorithms, not the pre-processing work involved in finding data and cleaning it up. I was reminded of Julia Evans' recent post, Machine learning isn't Kaggle competitions, and realised that Kaggle competitions were exactly what we wanted. They have the Digit Recognition Task, which is a canonical problem in machine learning, and we thought we would try learning how to write a neural network from scratch to tackle it.

(I also noticed an interesting problem called Billion Word Imputation, in which they have taken sentences and removed exactly one random word from each (not the first or last), and the task is to replace it, with partial credit given by character-level Levenshtein distance on the whole string. This is closely related to some of my previous work, and I am putting it on the ever-growing list of fun things to do.)

I realised some of my conceptions of neural nets were a bit vague, particularly the equations for backpropagation. So after a while, we broke off to study by ourselves. We both started studying from Michael Nielsen's partially complete book, Neural Networks and Deep Learning.

Later, we got together and studied his code, line by line, until we were confident in how it worked. Then we separated again to study the backpropagation part in more detail.

Eventually, we got a bit tired, and having realised the scope of the task, decided to do it as an ongoing project, rather than working on it and nothing else until it is done.


In the evening, the group attended an inspiring talk: Write An Excellent Programming Blog, the text of which was posted on the speaker, A. Jesse Jiryu Davis's blog.

As readers of this series probably know, I have a tough time writing these posts, because they are specifically NOT Excellent Programming Posts, and I am reluctant to fill my blog, which I had previously considered a professional face, which such quotidian stuff. The solution Davis suggested was to reorganise the site so that the log posts go on in the background, but never appear on the front page. I have thought of this, too, and the only hurdle I face with it is that I use Blogger to host, and Blogger is for blogs, not websites. I would like to change that, and switch to Pelican or Nikola.

Relatedly, an alum was kind enough to tell me that he is dissatisfied with the Hacker School Blog Aggregator being lately more populated with feeds of this type. Such posts are weakening the signal-to-noise for him; he is interested in technical writing from Hacker Schoolers, but not daily reports. On the other hand, other Hacker Schoolers subsequently said that these posts are very meaningful for them. So, I am not sure whether it is more appropriate to withdraw the feed or not.


Fasting was harder yesterday than the previous day. I'm not sure why. Hypotheses include:

  1. Unlike Sunday, the space was filled with lunch smells for hours on end, and this may have upped the challenge.
  2. Sunday night, I think I was inadvertently exposed to carbs, as I had some uncomfortable carb-like symptoms after dinner. Perhaps this threw off my ketosis for the day.
  3. Maybe it's just harder the second day?

In any case, a shot of humility is healthy now and then.