Review process
How can I test an AI detector on writing with a known origin?
The short answer
Gather human and AI samples with a documented origin. Leave the inputs unchanged and keep every result, so the exercise is repeatable. It can show you how the tool handles those examples; a few successful checks cannot establish its overall accuracy.
Choose the text before checking it
For human writing, use drafts with a reliable history. For AI writing, save outputs straight from the generating conversation. Choose a language, approximate length and kind of writing that fit your intended use. Do not call an online passage AI-generated just because a different detector flagged it.
Make the procedure repeatable
RAID covers varied domains, generators and decoding conditions. Against that range, testing one familiar prompt is a narrow exercise. Record your own test conditions and keep that distinction clear, rather than presenting the exercise as an equivalent benchmark.
- Note the source, language and length for each input.
- For AI samples, keep the model name and generation date where available.
- Keep the scan mode the same and write down the date.
- Keep mistaken classifications along with correct ones.
Source: Dugan et al. (ACL 2024): the RAID evaluation benchmark
Explain the limits of the result
Count the human samples marked as AI apart from the generated ones the detector missed. Say how many texts you checked. If they came from one genre or model, make that clear; do not extend the finding to every kind of document.