Filtered datasets improve AI generalization, revealing overestimated machine performance.
Large neural models may perform well on standard tasks but struggle with new or tricky samples. This could be because they're too focused on quirks in the data rather than the real task. A new method called AFLite helps filter out these biases, making models better at handling unexpected challenges. By using AFLite, models perform better on different tasks, even ones they weren't trained on. However, this filtering process can significantly reduce model accuracy, showing that models rely heavily on dataset biases. This means that filtered datasets can create new challenges for improving model performance.