Summer project

Over the summer, I decided to research how to use LLM’s for learning. I have been teaching economics, econometrics and data science for more than a decade, but never got any formal training on teaching or pedagogy. My first realization was that to use technology effectively, I needed to learn about the science of learning.

I came across Math Academy. One of the main forces behind this company is Justin Skycak. He has an in depth review of the scientific literature on learning that caught my eye. It laid out the scientific literature around learning, and how Math Academy leverages it’s main findings to optimize learning. While there is no LLM’s in MA, it was very much what I was looking for, so I dove right in.

The first thing I like about MA is that is designed to optimize learning, based on what we know about how the brain works. Other technologies like schools, books, youtube videos and online courses are not.

As a long time teacher and casual reader of the scientific literature on learning, a lot of the science resonates with me and what I naturally do in the classroom. Others principles were quite new to me, so that was very useful too! The dirty secret of education systems is that some of the principles that emerge from the literature are very hard to put into practice in a normal classroom setting, so most teachers ignore them.

A brief note about me, learning and math

In my early twenties, I discovered I loved learning. I started listening to podcasts and lectures , mostly about literature, history and economics. That way I learned a lot and discovered great teachers that inspired me to become a teacher myself. The best teachers were the ones that kept you engaged and were able to hold your attention for an hour, so that was what I tried to do in the classroom.

I didn’t realize that I was falling into the trap of passive learning. In this mode, I enjoyed listening to lectures a great deal, got inspired by teachers who could talk for hours about their area of expertise and even acquired some useful general knowledge. However, it is almost impossible to develop real skills this way.

As I got older, I became more interested in math and science, and also started worrying about learning something that would make some money. That was about the time where Coursera came out. I learned how to program with a few project based courses from Coursera that revolutionized the way I thought about learning. This made me realize that in order to learn how to do something, you need to spend as much time as possible practicing that skill.

I was always ok at math. I studied very little in high school, but managed to escape it learning a few derivative formulas and the formula for the roots of a quadratic polynomial. As I advanced in my university courses, I got more interested and managed to get through Calculus, Linear Algebra, Probability and Statistics. I even took a few classed that required Differential Equations and Control Theory. After getting my degree, I taught econometrics for a few years, so I had to learn statistics. I got pretty good at using results but knew that the building of my math knowledge had very shaky foundations. I had never seen complex numbers, trigonometry or geometry in formal course, so had zero knowledge on those topics. Throughout the years, I have always had a growing interest in math, and realized that it underpins all the things I want to learn more about and get good at. I pushed to fill in those gaps, and even expand my knowledge a bit further, but was pretty inconsistent.

Back to MA

So what are the main findings of the literature on learning? What is the best way to use technology to leverage these findings? Here are the main principles MA is built on.

Neuroscientists agree that we can keep a limited amount of information in our head at the same time. This works like RAM on a computer. To solve a problem, we need to keep all the relevant pieces of a puzzle in working memory at the same time. This type of memory is very limited, and almost impossible to expand.

On the other hand, the human brain can store practically unlimited amounts of information in long term memory. This is like the hard drive of a computer, it has longer access times, but it is cheap and easy to expand.

In this model, you enconde knowledge by solving problems putting the pieces of the puzzle in working memory. As you do this repeatedly, your brain stores the knowledge in long term memory, and you get better at recalling it.

Think of math a a huge puzzle. When you open a math book, you have thousands of puzzle pieces in front of you. Experts, who know what the final picture looks like, find it easy to find their way around a big fat textbook. They can look at a random example in the middle of the book and make sense of it because they have the structure of the subject encoded in their working memory.

A novice, however, looks at a textbook and feels overwhelmed by the amount of information. Since they have no representation of the information in their long term memory, all this material quickly fills up their working memory and makes them feel overwhelmed. Think of a computer that runs out of RAM and starts to lag.

An amazing feature of MA is that it has a detailed model that records your math knowledge. It has many bite sized lessons that are exactly at the level of difficulty that you need. This allows the app to present you with puzzles that are missing just a few pieces, so you keep getting the rewards of putting the last couple of pieces into place, instead of being overwhelmed by sifting through mountains of puzzle pieces and not having a clue of how they fit together.

By scaffolding the problems this way, you never get stuck or overwhelmed, and most of your time is spent reinforcing the patterns in your brain that allow you to solve math problems. This is not to say that studying at MA is easy. Learning math is hard, that is why it is so much fun. MA is a ladder with small steps, but you still need to climb it.

Learning consists of 2 parts: encoding and retrieving. Encoding is when you take information in. You pay attention and try to take in as much as you can. After that, you need to retrieve the information from your brain. This is quite simple, but it is surprising how much you can get out of keeping this simple model in mind. Both parts of learning are crucial, and you can’t have one without the other. So, if you don’t test what you learn, your brain will forget it. In fact, you need to spend a lot more time testing yourself retrieving what you have learned than consuming information. Another interesting thing I realized lately is that redundance in encoding is pretty inefficient, but redundance in retrieval is totally ok. For example, listening to a lecture on a topic I know well feels like a waste of time, but solving problems I find easy because I solved similar problems in the past is totally ok.

A few years ago I wanted to learn Linear Algebra, and found Gilbert Strang’s course on youtube. 36 videos of 45 minutes, I could do one or two a day and get through it in a month. Right? Wrong! I knew I needed to spend more time doing practice problems than watching lectures, so I bought the book. I did a lot of problems, but still lacked the structure. MA gives you a nice structure you can navigate, and makes sure you are moving towards a well defined goal.

You don’t get to choose which topic to learn. You enroll in a course and the system makes sure to move you along the path by serving lessons, reviews and quizzes This are basically problems you need to solve to acquire XP points. Lessons and reviews are pretty much the same, but quizzes are timed. This is quite fun, and it also highlights that simply knowing how to do a problem is not enough. You need to be fluent and solve them with a certain degree of automaticity.

The system is very well calibrated to get a good sense of how well you know the material. That is one of the strengths of MA. An interesting pattern I fell into was bombing on a quiz, getting served a few review problems, try the quiz again and ace it. According to my reading of the literature, spending an hour on this is the best use of your time when learning math. Of course you could spend that hour in the sacred halls of MIT via youtube and feel like Will Hunting but your brain would discard that information pretty fast.

I figure I have about a year of material on Math Academy. They are working on more content, so it is likely that it will take longer. I wish they added courses on Game Theory and Information Theory and Physics. Math Academy is the best education technology out there, if you want to learn math and can afford it, I would definitely recommend it.