As a student — or any busy professional — can attest, keeping track of tasks can be a nightmare. Besides simply tracking when tasks are due, it is often helpful to track the priority of tasks, different categories and labels, or even tracking dependencies between tasks. The possibilities are endless — and most organisational systems (including your memory) can’t handle this. Having a fully fledged, customisable dashboard for tracking and organising your tasks is now possible with software such as Notion. With Notion we can treat all our tasks as items in a database — like an Excel spreadsheet —…

In the previous blog post we developed some ideas and theory needed to discuss a causal approach to reinforcement learning. We formalised notions of multi-armed bandits (MABs), Markov Decision Processes (MDPs), and some causal notions. In this blog post we’ll finally get to developing some causal reinforcement learning ideas. The first of which is dubbed *Task 1*, for CRL can help solve. This is *Generalised Policy Learning*. Let’s begin.

- Causal Reinforcement Learning
- Preliminaries for CRL
- Task 1: Generalised Policy Learning
- Coming Soon: Task 2

Reinforcement learning typically involves learning and optimising some policy about how to interact in an environment…

In the previous blog post we discussed and motivated the need for a causal approach to reinforcement learning. We argued that reinforcement learning naturally falls on the interventional rung of the ladder of causation. In this blog post we’ll develop some ideas necessary for understanding the material covered in this series. This might get quite technical, but don’t worry. There is still always something to takeaway. Let’s begin.

- Causal Reinforcement Learning
- Preliminaries for CRL
- Task 1: Generalised Policy Learning
- Coming Soon: Task 2

As you probably recall from high school, probability and statistics are almost entirely formulated on the idea…

In this series of posts I will break down the emerging field of Causal Reinforcement Learning (CRL) into digestible blog chucks. This is an exciting field which is being spearheaded by Elias Bareinboim and Judea Pearl, among others. I will try to present this in such a way as to satisfy those craving some mathematical detail whilst also trying to paint a broader picture as to why this is generally useful and important. Each of these blog posts will be self contained in some way. Perhaps it will be about a specific idea or research paper. In this case, it…

*Author: Hi, I’m St John and I write blogs about modern technologies and interesting things for my personal blog **stjohngrimbly.com**. I’m currently interested in machine learning and causality among other things. I hope you enjoy this quick read!*

In the last episode we developed the first tools we need to develop the theory needed to formalise interventions and counterfactual reasoning. In this article we’ll discuss how we can go about learning such a causal model from some observational data, and what constraints are required for doing this. Note, this is an active area of research and so we really don’t…

*Author: Hi, I’m St John and I write blogs about modern technologies and interesting things for my personal blog **stjohngrimbly.com**. I’m currently interested in machine learning and causality among other things. I hope you enjoy this quick read!*

Last time we discussed and motivated the need for a modern theory of causal inference. We developed some of the basic principles necessary to develop this theory, but we have yet to strictly define what exactly a causal model entails. In this ‘episode’ we’ll briefly introduce the notion of a structural causal model and give some examples and implications of how this…

*Author: Hi, I’m St John and I write blogs about modern technologies and interesting things for my personal blog **stjohngrimbly.com**. I’m currently interested in machine learning and causality among other things. I hope you enjoy this quick read!*

What’s the first thing a statistician will say when you dare say the word *cause*? If you’ve ever taken a statistics class, I have little doubt it was the classic anachronism, *Correlation does not imply causation*. The classic anachronism seems to imply that we can never state that A causes B from analysis of data. R.A. Fisher, one of the fathers of…

*stjohngrimbly.com*

The World Models (Ha et al., 2018) paper presented at NIPS in 2018 exploits the idea of having an agent train entirely within its latent representation of the world it is in — its world model — and apply this learned knowledge to the real world. This paper distills many years and areas of research into a highly efficient framework for model…

Hi, I'm St John! I'm an applied math student at the Univ. of Cape Town. I maintain my blog, stjohngrimbly.com, as a way to keep track of anything interesting.