Investigating Cold-Start Failure in Active Learning for Images

Abstract

Active learning is a machine learning strategy which seeks to achieve the best possible results with the fewest labeled examples. When successful, active learning improves model performance at a lower labeling cost than labeling randomly or uniformly. However, if these active learning strategies are employed too early, active learning may perform worse than random selection, a condition known as cold-start failure. This thesis first characterizes the problem of cold-start failure in image classification, examining the training conditions under which cold-start failure occurs using the MNIST dataset. Following this, behaviors and selections of active learning strategies under cold-start are analyzed and compared to training behavior in both uniform sampling and successful active learning situations. Finally, self-supervision strategies are introduced to generate new features from the images within the unlabeled pool in an attempt to alleviate cold-start failure and allow active learning training to successfully begin earlier. We did not find evidence that this additional feature extraction was useful in alleviating cold-start failure for our dataset.