Predicting effect of force using game engine synthesized dataset

Abstract

This research aims to solve the problem of predicting the long-term sequential movements of an object in an image given a force vector applied to the object. To approach this topic using deep learning method, a dataset rich in quality and quantity is desired. In this thesis, a method of generating the dataset is proposed and evaluated. A game engine (Unity 3D) is used to generate a dataset with scene screenshots, object mask, force representation and resulting movements. With one scene, a total of 370,440 sequences can be generated. The result shows that, with a larger and more precise dataset, the accuracy can increase significantly.