Computer Science

Using Unity Game Engine and MLAgents package, created a soccer setting and set training parameters such as rewards to iterate on the training process and quickly train agents to push objects together. Fastest training for this process was 5 hours, with 128 hidden layers, split between two training sessions. A 30 minuet training session where the agent was rewarded for touching the blue block, and an four and a half hour session where he was rewarded only for bringing the two blocks together. This agent achieved 98% success rates after this time. 

Further expanded this project to 3D, and iterated on the training parameters to reach a relatively fast training with low end hardware. Started with a base case of scoring = reward, but then moved on to a curriculum approach as the task was to hard to build on its own. Transitioned from curriculum to having all the rewards pre baked into the code, with staying in bounds for a full episode == .01 reward, touching the ball == 1 reward, and scoring == 100 reward for that episode. Scoring was not cumulative in these episodes, so the only 4 score outputs possible from a session were 0, .01, 1 and 100. Have a x100 scale would create a large incentive to learn the new behaviors. Originally I tried this process with a 5x scale, but the Bee would spend far longer just hitting the ball randomly, drastically slowing that training. I assume my approach is not novel, but it works for this project better then the curriculum based training runs, and I have not seen it elsewhere. 

The end results show extremely similar results between my process and a standard curriculum process, but mine seems easier to implement. Both processes reached higher scoring (above 60% success) much faster than a "raw" process that only rewards the end goal. However, over the long term, the raw process did achive higher results overall. 

TenserBoard Outputs