Skip to content

Add DeBERTa-ConPara-v2.17: 1M leakage-free stratified training set - #181

Merged
liamdugan merged 2 commits into
liamdugan:mainfrom
MohamedMady19:submission/conpara-v217
Aug 11, 2026
Merged

Add DeBERTa-ConPara-v2.17: 1M leakage-free stratified training set#181
liamdugan merged 2 commits into
liamdugan:mainfrom
MohamedMady19:submission/conpara-v217

Conversation

@MohamedMady19

Copy link
Copy Markdown

DeBERTa-ConPara v2.17 - retrained on a 1M-sample leakage-free stratified dataset.

Changes vs v2.16:

  • Training data rebuilt: RAID splits grouped by adv_source_id to prevent attack-variant leakage between train and val
  • Source-stratified composition (no source above 45% of either class)
  • Attack-augmented human training (~11 RAID attack variants per human article)
  • Corrected Unicode homoglyph normalisation (U+0440 was mapping to 'r' instead of 'p', corrupting rather than restoring attacked text)

Same architecture as v2.16 (DeBERTa-v3-large + 62 linguistic features);
Only the training data and preprocessing changed.

@github-actions

Copy link
Copy Markdown

Eval run succeeded! Link to run: link

Here are the results of the submission(s):

DeBERTa-ConPara-v2.17

Release date: 2026-08-09

I've committed detailed results of this detector's performance on the test set to this PR.

On the RAID dataset as a whole (aggregated across all generation models, domains, decoding strategies, repetition penalties, and adversarial attacks), it achieved an AUROC of 99.33 and a TPR of 98.00% at FPR=5% and 94.00% at FPR=1%.
Without adversarial attacks, it achieved AUROC of 99.42 and a TPR of 98.22% at FPR=5% and 95.39% at FPR=1%.

If all looks well, a maintainer will come by soon to merge this PR and your entry/entries will appear on the leaderboard. If you need to make any changes, feel free to push new commits to this PR. Thanks for submitting to RAID!

@MohamedMady19

Copy link
Copy Markdown
Author

Hi @liamdugan, Thank you so much. Please merge this to the Leaderboard.

@liamdugan
liamdugan merged commit e3488f6 into liamdugan:main Aug 11, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants