Why should you care?

AIPredict ticket classifiers fail when emoji, HTML, and PII pollute training text.

See it live — copy this example

Use a .NET console or Web API project with Microsoft.ML. Run dotnet run after pasting.

var clean = ml.Transforms.Text.NormalizeText("Norm", "RawTicket")
    .Append(ml.Transforms.Text.TokenizeIntoWords("Tokens", "Norm"))
    .Append(ml.Transforms.Text.RemoveDefaultStopWords("Filtered", "Tokens"))
    .Append(ml.Transforms.Text.FeaturizeText("Features", "Filtered"));

What happened?

• Chain normalize → tokenize → stop words → featurize.
• Fit once; saved model applies same steps at predict.

Practice next

1. NormalizeText on tickets.
2. Tokenize and drop stop words.
3. FeaturizeText → train.
4. Add regex replace for order ids.
5. Try char n-grams via options.

Remember

Normalize pipeline. Train=serve chain. Drop noise early.

AIPredict ticket clean

Noisy support text normalized before classify.

Outcome: F1 stable between train and API.