I wanted to dip my toes to machine learning, and at the time, I was having fun playing a simple web game called Universal Paperclips based on the famous Paperclip Maximizer thought experiment in which an AI designed to make paperclips eventually destroys humanity in its pursuit of more paperclip making resources. So I decided to make a literal paperclip maximizing AI that would attempt to play the game. I admit the irony of the idea was a major factor in my decision to pursue it. Besides, what could go wrong?
At first I set it up so it was only playing one game at a time, but quickly realized that training would take much too long. So I refactored my project to run 20 instances of the game at a time in the same browser window. I spent the next week leaving it running for a day, and then tweaking inputs parameters to try to get the AI’s to learn. I eventually succeeded in getting one of the AI’s to learn how to manage the price it was selling its paperclips for so that it could afford to buy more wire instead of running out of resources. A bit short of my lofty goals of converting the entire digital universe into paperclips, but I was satisfied with my results anyways.
One of the biggest mistakes I made early on was having far too many inputs. I instead did some basic manipulations in the inputs to decrease how many things the AI needed to pay attention to. For example, instead of including the current paperclip price, number produced per second, number sold per second, and whether or not the increase / decrease buttons were available, I performed a pre-calculation to determine the net amount of paperclips. Now the AI only had to figure out that if the value was negative it should increase prices, and if it was positive, it should decrease prices.
I’m excited to try more machine learning projects in the future. Next time I’d like to work on something more precise, rather than randomly stirring a pile of linear algebra until it can play a game.