Priming the canvas / by Karl Cronin

I am starting to learn more about deep reinforcement learning, and am interested in seeing how swarm intelligence might stitch up with RL policy algorithms. I began studying particle swarm optimization (PSO) last fall, and have been looking for applied contexts to test out the myriad variations I've encountered. My hunch is that tuning deep learning systems could be a fruitful area to explore, yet I'll need to poke around a bit and see.

To prime the canvas, I'm kicking things off with PSO Baselines, a repo where I'm sussing out the differences between various PSO variations by implementing some papers. My hope is that by implementing a range of PSOs, I'll start to get a sense of the various dials researchers are turning, and perhaps in time I'll get some hint of an idea as to where I might be able to contribute some fresh thinking. 

This morning I hit an interesting snag, which prompted me to stop and consider the architecture of what I'm writing. In the back of my mind, I've been hoping to implement a few multi-objective PSO papers to see if there might be some universal multi-objective class I could write and add to PySwarms. I mentioned to LJ (PySwarms' lead developer) that I would attempt to do this, and perhaps I still can. The snag is that it seems each PSO algorithm I pick up is different from other PSOs in ways that require different base structure, so it not as straightforward as I originally thought to build out a generic multi-objective class.

My strategy was to start with a functional approach, but there are cases where it seems it would be easier to have objects (for instance, in local best, where we want to compare each particle to all other particles to determine neighbors, and then update each particle - this challenges my initial list of list data structure in my functional approach). Since several of the algorithms build on local best, it seems I need to tackle this directly up front, rather than skirting around the issue by implementing global best variations.