You can create a dataset with only one class, or with only 1 instance in each class. 

When you click 'Evaluate', the tool takes you to a page where you can choose some settings and submit the dataset (e.g. http://localhost:8080/datasets/b5c4d6d3-a5eb-4b84-be0b-372429c4899f/evaluate)

Only after submitting for evaluation do you receive an error message "Cannot add this dataset because of a validation error: Dataset needs to have at least 2 classes."

We should validate this dataset on the page previously and show the user the warning at this stage. The validate_dataset method at https://github.com/metabrainz/acousticbrainz-server/blob/d7913b44165304c82712280ff002c279db1c240e/db/dataset_eval.py#L81 should be called before the user has the option to submit a dataset.

Note that this is partially related to AB-148 - because validate_dataset checks the structure as well as the contents of the dataset, this step could take a long time to complete.

For now, we should split this method into validate_dataset_structure and validate_dataset_contents and call _structure before submit, and _contents after.