Review process

How can I test an AI detector on writing with a known origin?

The short answer

Gather human and AI samples with a documented origin. Leave the inputs unchanged and keep every result, so the exercise is repeatable. It can show you how the tool handles those examples; a few successful checks cannot establish its overall accuracy.

Choose the text before checking it

For human writing, use drafts with a reliable history. For AI writing, save outputs straight from the generating conversation. Choose a language, approximate length and kind of writing that fit your intended use. Do not call an online passage AI-generated just because a different detector flagged it.

Make the procedure repeatable

RAID covers varied domains, generators and decoding conditions. Against that range, testing one familiar prompt is a narrow exercise. Record your own test conditions and keep that distinction clear, rather than presenting the exercise as an equivalent benchmark.

  1. Note the source, language and length for each input.
  2. For AI samples, keep the model name and generation date where available.
  3. Keep the scan mode the same and write down the date.
  4. Keep mistaken classifications along with correct ones.

Source: Dugan et al. (ACL 2024): the RAID evaluation benchmark

Explain the limits of the result

Count the human samples marked as AI apart from the generated ones the detector missed. Say how many texts you checked. If they came from one genre or model, make that clear; do not extend the finding to every kind of document.

Put the guidance into practice

Choose the passage you want to review and keep its context alongside the result.

Check your textWork with documented model samples