testing something and want to compare notes
i’ve been running experiments on my own drafts. i take AI output and edit it heavily, restructure paragraphs, change examples, rewrite the argument in places. then i run both versions through detectors to see if the score changes.
my findings so far: surface edits (word swaps, minor rephrasing) move the score very little. structural changes (reordering paragraphs, changing the logical flow) move it more. rewriting the opening substantially moves it the most.
anyone else tested this? curious if others have seen the same pattern or different results.
this matches what i’ve observed from the editorial side. detectors aren’t primarily doing vocabulary matching. they’re measuring statistical properties of the text at the sentence and paragraph level. changing words without changing sentence structure doesn’t shift those properties much.
what moves the score is changing the rhythm and structure. burstiness and perplexity are the main variables, and those respond to structural editing, not vocabulary editing
your finding about the opening is consistent with what i understand about how some tools weight sections. the first few hundred words carry disproportionate weight in the overall score because they anchor the statistical model’s read on the whole document.
i’ve seen the same pattern. rewriting the opening substantially drops the score more than rewriting an equivalent number of words in the middle
fr this is what i found too. i ran the same essay through with different edit types and tracked the score changes. opening rewrite was worth like 15-20 points on its own. word swaps throughout were worth maybe 3-5. the math doesn’t favor micro-editing
the structural finding is why humanizers that work at the sentence level rather than the word level perform better. humaniseai.ai restructures sentence patterns rather than just substituting synonyms. the detection score difference is noticeable.
your experiment methodology is actually useful as a calibration tool. worth running on a few pieces just to know your baseline before you invest time in editing
yeah surface edits are basically useless from a detection standpoint. i stopped doing word-level editing almost entirely. now i only edit at the sentence and paragraph level and the results are consistent with what you’re describing