What are computers for? Historically, different answers to this question – that is, different visions of computing – have helped…
Read More
Sequence Modeling with CTC
Introduction Consider speech recognition. We have a dataset of audio clips and corresponding transcripts. Unfortunately, we don’t know how the…
Read More
Feature Visualization
There is a growing sense that neural networks need to be interpretable to humans. The field of neural network interpretability…
Read More
Why Momentum Really Works
Step-size α = 0.02 Momentum β = 0.99 We often think of Momentum as a means of dampening oscillations and…
Read More
Research Debt
Achieving a research-level understanding of most topics is like climbing a mountain. Aspiring researchers must struggle to understand vast bodies…
Read More
