The Hidden DNA of LLM-Generated JavaScript: Structural Patterns Enable High-Accuracy Authorship Attribution
2510.10493v1
cs.CR, cs.LG
2025-10-16
Авторы:
Norbert Tihanyi, Bilel Cherif, Richard A. Dubniczky, Mohamed Amine Ferrag, Tamás Bisztray
Abstract
In this paper, we present the first large-scale study exploring whether
JavaScript code generated by Large Language Models (LLMs) can reveal which
model produced it, enabling reliable authorship attribution and model
fingerprinting. With the rapid rise of AI-generated code, attribution is
playing a critical role in detecting vulnerabilities, flagging malicious
content, and ensuring accountability. While AI-vs-human detection usually
treats AI as a single category we show that individual LLMs leave unique
stylistic signatures, even among models belonging to the same family or
parameter size. To this end, we introduce LLM-NodeJS, a dataset of 50,000
Node.js back-end programs from 20 large language models. Each has four
transformed variants, yielding 250,000 unique JavaScript samples and two
additional representations (JSIR and AST) for diverse research applications.
Using this dataset, we benchmark traditional machine learning classifiers
against fine-tuned Transformer encoders and introduce CodeT5-JSA, a custom
architecture derived from the 770M-parameter CodeT5 model with its decoder
removed and a modified classification head. It achieves 95.8% accuracy on
five-class attribution, 94.6% on ten-class, and 88.5% on twenty-class tasks,
surpassing other tested models such as BERT, CodeBERT, and Longformer. We
demonstrate that classifiers capture deeper stylistic regularities in program
dataflow and structure, rather than relying on surface-level features. As a
result, attribution remains effective even after mangling, comment removal, and
heavy code transformations. To support open science and reproducibility, we
release the LLM-NodeJS dataset, Google Colab training scripts, and all related
materials on GitHub: https://github.com/LLM-NodeJS-dataset.
Ссылки и действия
Дополнительные ресурсы: