Problem
Emotion in speech is a hard label: it is subjective, it varies by speaker, clips differ in length and pitch, and the classes are imbalanced. A model has to make something consistent out of a signal that isn't.
Method
Audio clips were normalised and padded or truncated to a fixed length, then passed through a VGG16 network pretrained on images and fine-tuned for emotion classes with cross-entropy loss. Built in PyTorch with Librosa for audio processing and a custom dataset class so the pipeline could be extended. The trade-offs were made on purpose: a CNN over a sequence model for speed, fixed length over full clips for training feasibility, a pretrained backbone over a scratch one for a small dataset.
Evaluation
Just over 70% accuracy on the held-out set across ten classes (five emotions × two genders), against a 10% chance baseline; gender alone is separated at 100%. Compared against MLP and LSTM baselines, which fell well short. Tested on our own recorded voices, clips nothing like the training data, and it still labelled an angry male speaker correctly. Error analysis showed the confusions cluster between similar emotions; the model's mistakes were the same ones people make. Built during the Sabudh Foundation internship.
The decision it drives
What it taught: CNNs on fixed-length audio throw away time, and time is where most of the emotion lives. The next version would use spectrograms with a transformer, augmentation, and a check on cross-dataset generalisation.