Detecting and adapting to new situations is tough for AI in open-world environments because novelties come in many forms. To measure how hard it is for AI to handle different types of novelties, a method was created using spatial relations, learning algorithms, and statistical measures. By applying this method to novelties in Angry Birds, it was shown that AI struggles with certain types of novelties.