Embodying MDPs / by Siobhán Cronin

When I had my cognitive revival in 2016, I thought I was closing the door on the period of my life where my ideation would be rooted in somatic understanding of abstract concepts. While leading me to depths of empathy and artistic practice, the primacy of my embodied mode of thinking had prevented me from accessing other forms of abstraction and meaning-making. And then, all of a sudden, I had access to number, and could relate to the immaterial concepts of set theory, graph theory, network theory, computer science, calculus, and linear algebra. And yet...

My mind processes abstract concepts in concert with the nerves that run through my muscles, this rich estuary of embodied experience. Is my mind looking for data from my lived history to validate the abstract concept? Lately I have noticed this most when falling asleep, my mind flashing with snippets of the theorems and concepts I'm studying, and my body twitching in a wild fireworks display. On evenings of days when I've studied many new ideas, the twitching is so active my partner eventually gives up and moves to the couch. 

The twitching continues throughout the night, as my dreams unfold possibilities and dead ends. Sometimes the dead ends are benign, and sometimes they jolt me awake, such as last night when a figure in my dream leaned in and whispered 'Java is a prime number'. To a mind still forming around what is possible within computer science and math, such a concept equates to a nightmare.

But last night was dominated by a single thread of inquiry, as I begin to understand the RL and personal implications of Hindsight Experience Replay (HER), and that is how does my mind overlay reward functions and policies in the continuous time-space of my life? How does my body understand Markov Decision Processes (MDPs)? When have I seen individual layers, or layer clusters, exposed in meditation? When did policies that served my early development die off?

Perhaps the biggest gift of all I was given in my exile and in my long apprenticeship in the somatic research tribe is the ability to ask questions as research prompts, not as urgent requests for answers. And, so I will continue to ask.