Thanks for your interest in our work. We've played around a little bit with the particular choice of classifier, but our focus has so far mainly been on the representations themselves.

Going forward, we've actually started looking into using *simpler* classifiers to do the screening, so that we can evaluate more representations per unit time, and find representations that don't rely as heavily on having a strong classifier backend. Meanwhile, we're also doing some work looping in other, fancier ensemble methods at the validation stage (after screening) to squeeze every last drop of performance out of one, or a collection of representations. There are lots of interesting options to explore in both directions.