Training a Transformer model to discriminate spam emails from legitimate ones.
A good example of a Transformer model's effectiveness as applied to the classification of spam emails. The model is constructed as outlined in Google's paper Attention Is All You Need (Vaswan, et al., 2017).
A Transformer model is constructed and trained (from scratch!) against 3000 emails from the Email Spam dataset from Kaggle. As usual, a portion of the dataset is used for training, validation and testing.
Standard statistical methods are used to evaluate the performance of the model against previously unseen emails.
The trained model achieved a 99% prediction accuracy against the test data set.