CN116842140B - Method and system for detecting machine-generated text - Google Patents
Method and system for detecting machine-generated text Download PDFInfo
- Publication number
- CN116842140B CN116842140B CN202311075839.7A CN202311075839A CN116842140B CN 116842140 B CN116842140 B CN 116842140B CN 202311075839 A CN202311075839 A CN 202311075839A CN 116842140 B CN116842140 B CN 116842140B
- Authority
- CN
- China
- Prior art keywords
- detection
- text
- sample
- machine
- generated
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 24
- 238000001514 detection method Methods 0.000 claims abstract description 235
- 238000009826 distribution Methods 0.000 claims abstract description 16
- 230000006870 function Effects 0.000 claims abstract description 10
- 238000004364 calculation method Methods 0.000 claims description 44
- 238000006467 substitution reaction Methods 0.000 claims description 29
- 238000012360 testing method Methods 0.000 claims description 28
- 238000012545 processing Methods 0.000 claims description 21
- 238000010276 construction Methods 0.000 claims description 18
- 239000013598 vector Substances 0.000 claims description 18
- 238000007689 inspection Methods 0.000 claims description 11
- 238000003860 storage Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 description 7
- 238000012549 training Methods 0.000 description 5
- 239000002131 composite material Substances 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000000528 statistical test Methods 0.000 description 3
- 238000013145 classification model Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 235000002198 Annona diversifolia Nutrition 0.000 description 1
- 241000282842 Lama glama Species 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001627 detrimental effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000013179 statistical model Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3346—Query execution using probabilistic model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/194—Calculation of difference between files
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a detection method and a detection system for a machine-generated text, wherein disturbance is implemented on a target sample through the replacement of a paraphrasing, and a probability difference value is calculated and generated so as to realize disturbance difference detection; calculating the prediction probability, the prediction probability ranking and the prediction distribution cross entropy of the sample according to the target generation model, and realizing statistical detection based on the three statistical indexes; comprehensively considering the results of the two detection methods, and outputting a comprehensive detection result. The invention also provides a detection method aiming at machine-generated text description, and text restoration is realized while the description content is identified. The invention realizes the detection function of the machine-generated text, can be used for solving the abuse problem of the text generated by the current language model, and provides a feasible and effective scheme for the supervision of the text sources.
Description
Technical Field
The invention relates to the technical field of file detection and identification, in particular to a method and a system for detecting a machine-generated text.
Background
The pre-training model is taken as an important breakthrough in the field of natural language processing, is superior to the traditional statistical model in almost all text tasks, and is widely applied in the industry. Recently, the ability to generate a pre-training language model is rapid, GPT3 can answer the complex questions such as science, history and current affairs, and the scientific papers generated by ChatGPT and GPT4 cannot be accurately distinguished even by senior specialists. The rise of such large language models provides great convenience for personal work and life and business production operations, but misuse of text generation tools has had a socially detrimental effect. In the field of paper writing, some students use ChatGPT to generate research background and meaning introduction, and the method belongs to academic inequality and compromises academic fairness. In the field of news reporting, some bad molecules generate and release unqualified news by using a pre-training language model, and a public opinion wind wave is triggered, so that social order is greatly disturbed, and bad influence is caused. The advent of these behaviors has increased the difficulty of society in supervising the sources of text, and has also created a new need for automatic detection of machine-generated text. Therefore, how to implement automatic detection of machine-generated text for the characteristics of machine-generated text has become an important research content at present.
At present, a common detection model of machine-generated text is generally a classification model, the machine and manually-generated text are regarded as different categories, and text classification is carried out by training a machine learning model, so that machine-generated text detection is realized. This detection method requires extensive amounts of labeling data to supervise the model, and non-generic features of the selected text corpus may be captured by the model, resulting in overfitting. In addition, the machine-generated text detection method based on the classification model does not utilize the unique characteristics of the machine-generated text, so that the accuracy of detection results is low.
Disclosure of Invention
Therefore, the embodiment of the invention provides a method and a system for detecting a machine-generated text, which are used for solving the problem of low accuracy of a detection result in the prior art.
In order to achieve the above object, the present invention provides the following technical solutions:
the first aspect of the invention provides a detection method of machine-generated text, which comprises disturbance difference detection, statistical detection and detection result summarization, wherein,
the disturbance difference detection is to perform paraphrasing substitution in the target sample, generate a substitution sample, calculate the generation probability of the candidate target sample and the substitution sample in the target generation model, and mark the candidate target sample and the substitution sample as machine-generated text when the difference exceeds a preset threshold significance level;
the statistical detection is to calculate the prediction probability, probability ranking and information entropy of the words at the current position when the text is given based on the target generation model, and the three parts of statistical calculation result indexes are integrated to give judgment to the text source;
and summarizing the detection results, namely summarizing and integrating the results of the disturbance difference detection and the statistical detection, and outputting a comprehensive detection result.
Preferably, the disturbance difference detection comprises the steps of text disturbance processing and difference detection calculation, wherein the input of the text disturbance processing step is a target sample and a word list, and a replacement sample set is output; the input of the difference detection calculation step is a target sample, a replacement sample set and a target generation model, and a disturbance difference detection result is output;
the text disturbance processing comprises the steps of obtaining character vector representation by using a pre-provided universal word list, describing the similarity relationship between characters by the cosine similarity of the character vector, for a designated character, finding out the character with the highest cosine similarity of the character vector corresponding to the character from the pre-provided word list as the near-sense character of the current character, randomly selecting k characters in a target sample, and respectively replacing the k characters with the near-sense characters to obtain k replacement samples to form a replacement sample set;
and (3) performing difference detection calculation, namely comparing the generation probability of the disturbed sample with the difference of the target sample by applying micro disturbance to the text, so as to realize the detection of the machine generated text.
Preferably, the text perturbation processing step's near-word substitution operation perturbs the target sample by calculating a regularized probability difference value, and determining that the machine-generated text is when the difference value exceeds a predetermined threshold value by:
(1) Calculating a generation probability mean value of the replacement sample:
(2) Calculating a probability difference value:
(3) Probability difference regularization:
(4) Comparison of significance differences:
if it isDetermining that the machine-generated text;
wherein x is the target sample, and the sample is obtained,to replace the sample set, p θ A model is generated for the target, e is the saliency difference threshold.
Preferably, the statistical test comprises two stages of statistical test index calculation and comprehensive index construction; the input of the statistical detection index calculation stage is a word list, a target sample and a target generation model, and the statistical detection index is output; the input of the comprehensive index construction stage is a statistical detection index, and a statistical detection result is output;
the statistical detection index calculation judges whether the text is generated by a machine by detecting whether the current word is positioned at the head of the distribution;
building comprehensive indexes, namely building three inspection indexes respectively, and then integrating the inspection indexes:
first test indexWherein I (x) is an oscillometric function, and is 1 when x is true, 0 when x is false, alpha 1 As a probability reference value, beta 1 Is a first proportional threshold;
second test indexWherein alpha is 2 For ranking reference value, beta 2 Is a second proportional threshold;
third test indexWherein->α 3 For information entropy reference value, beta 3 For the third proportional threshold, the arithmetic mean value of the three test indicators is +.>As a composite index, if T < 0, the target sample is marked as machine-generated.
Preferably, the method for detecting machine-generated text further comprises a step of sample restoration, wherein the method for detecting text identifies the remarked content in the machine-generated text and restores the remarked content to obtain the original text, specifically:
sequentially performing disturbance difference detection, statistical detection and detection result summarization steps by taking the initialized modified sample as a target sample to obtain a comprehensive detection result;
if the artificial writing is judged, generating an L-th word by the target generation model, and updating the current sample by using the model generation word to obtain the current sample; if the machine generation is judged, updating the current sample by using the L-th word of the original sample to obtain the current sample;
if the length of the current sample is equal to the length of the modified sample, the current sample is restored to the obtained original text.
The invention also provides a detection system of the machine-generated text, which comprises a disturbance difference detection unit, a statistics detection unit and a detection result summarization unit,
the disturbance difference detection unit is used for performing paraphrasing substitution in the target sample, generating a substitution sample, calculating the generation probability of the candidate target sample and the substitution sample in the target generation model, and marking the candidate target sample and the substitution sample as machine-generated text when the difference exceeds a preset threshold significance level;
the statistical detection unit is used for calculating the prediction probability, probability ranking and information entropy of the words at the current position when the text is given on the basis of the target generation model, and comprehensively judging the text source by integrating the three parts of statistical calculation result indexes;
and the detection result summarizing unit is used for summarizing and integrating the results of the disturbance difference detection and the statistical detection and outputting a comprehensive detection result.
Preferably, the disturbance difference detection unit comprises a text disturbance processing module and a difference detection calculation module, wherein the input of the text disturbance processing module is a target sample and a word list, and a replacement sample set is output; the input of the difference detection calculation module is a target sample, a replacement sample set and a target generation model, and a disturbance difference detection result is output;
the text disturbance processing module acquires character vector representation by using a pre-provided universal word list, and characterizes the similarity relationship between characters by the cosine similarity of the character vector, and for a designated character, a character with the highest cosine similarity of the character vector corresponding to the character is found out from the pre-provided word list to be used as a near-meaning character of the current character, then k characters are randomly selected from a target sample, and the k characters are respectively replaced by the near-meaning characters to obtain k replacement samples, so that a replacement sample set is formed;
and the difference detection calculation module is used for comparing the generation probability of the disturbed sample with the difference of the target sample by applying micro disturbance to the text, so that the detection of the machine generated text is realized.
Preferably, the statistical detection unit comprises a statistical detection index calculation module and a comprehensive index construction module; the input of the statistical detection index calculation module is a word list, a target sample and a target generation model, and the statistical detection index is output; the input of the comprehensive index construction module is a statistical detection index, and a statistical detection result is output;
the statistical detection index calculation module judges whether the text is generated by a machine by detecting whether the current word is positioned at the head of the distribution;
the comprehensive index construction module is used for respectively constructing three inspection indexes and then integrating the inspection indexes:
first test indexWherein I (x) is an oscillometric function, and is 1 when x is true, 0 when x is false, alpha 1 As a probability reference value, beta 1 Is a first proportional threshold;
second test indexWherein alpha is 2 For ranking reference value, beta 2 A second ratio threshold;
third test indexWherein->α 3 For information entropy reference value, beta 3 For the third proportional threshold, the arithmetic mean value of the three test indicators is +.>As a composite index, if T < 0, the target sample is marked as machine-generated.
The present invention also provides an electronic device including: at least one processor and at least one memory;
the memory is used for storing one or more program instructions;
the processor is configured to execute one or more program instructions to perform the machine-generated text detection method described above.
The present invention also provides a computer readable storage medium having one or more program instructions embodied therein for performing the machine-generated text detection method described above.
The embodiment of the invention has the following advantages:
the invention provides a detection method of a machine-generated text, which comprises the steps of firstly, disturbing a target sample through the replacement of a paraphrase, and calculating a generated probability difference value to realize disturbance difference detection; calculating the prediction probability, the prediction probability ranking and the prediction distribution cross entropy of the sample according to the target generation model, and realizing statistical detection based on the three statistical indexes; and finally, comprehensively considering the results of the two detection methods, and outputting a comprehensive detection result. In particular, the invention also proposes a detection method for machine-generated text reformulations based on the aforementioned detection method, which implements text reduction while recognizing the reformulation content. The invention realizes the detection function of the machine-generated text, can be used for solving the abuse problem of the text generated by the current language model, and provides a feasible and effective scheme for the supervision of the text sources by related departments.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It will be apparent to those of ordinary skill in the art that the drawings in the following description are exemplary only and that other implementations can be derived from the drawings provided without undue effort.
The structures, proportions, sizes, etc. shown in the present specification are shown only for the purposes of illustration and description, and are not intended to limit the scope of the invention, which is defined by the claims, so that any structural modifications, changes in proportions, or adjustments of sizes, which do not affect the efficacy or the achievement of the present invention, should fall within the scope of the invention.
FIG. 1 is a flow chart of a method for detecting machine-generated text according to an embodiment of the present invention;
FIG. 2 is a flow chart of disturbance difference detection according to an embodiment of the present invention;
FIG. 3 is a flow chart of statistical detection provided in an embodiment of the present invention;
FIG. 4 is a sample recovery flow chart provided in an embodiment of the present invention;
fig. 5 is a schematic diagram of a detection system architecture of machine-generated text according to an embodiment of the present invention.
Detailed Description
Other advantages and advantages of the present invention will become apparent to those skilled in the art from the following detailed description, which, by way of illustration, is to be read in connection with certain specific embodiments, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Aiming at the detection problem of machine-generated texts, the invention provides a detection method of machine-generated texts, which is used for detecting text paragraphs by two methods of disturbance difference detection and statistic detection and judging whether the texts belong to machine generation or not according to the detection results of the two methods. The disturbance difference detection needs to implement the paraphrase replacement on the text, and the unique characteristics of the machine-generated text are effectively utilized to realize the text detection by calculating the probability difference value of the replacement sample; the statistical test involves three parts of statistical calculations, determining whether the sample is machine-generated by a probability distribution. In particular, the present invention also contemplates a detection scheme for generating a reformulation of text that is capable of identifying the reformulation portion in the text and restoring the text. Compared with a general machine-generated text detection model, the method provided by the invention does not need to perform supervised training of the model, combines the specific characteristics of the machine-generated text, and solves the machine-generated text detection problem more pertinently.
The embodiment of the invention provides a detection method of a machine-generated text, which comprises the following steps:
wherein, disturbance difference detects: performing paraphrasing substitution in the target sample to generate a substitution sample, calculating the generation probability of the candidate target sample and the substitution sample in the target generation model, and marking the candidate target sample and the substitution sample as machine-generated text when the difference exceeds a preset threshold significance level;
and (3) statistical detection: calculating the prediction probability, probability ranking and information entropy of the words at the current position when the text is given on the basis of the target generation model, and comprehensively judging the text source by the three statistical calculation result indexes;
summarizing detection results: and summarizing and integrating the disturbance difference detection and the statistical detection results, and outputting a comprehensive detection result.
For generating a description of the text, the method further includes sample reduction.
Referring to fig. 1, the method for detecting machine-generated text provided by the embodiment of the invention specifically includes:
step S01, disturbance difference detection, namely performing paraphrasing substitution in a target sample, generating a substitution sample, calculating the generation probability of candidate target samples and substitution samples in a target generation model, and marking the candidate target samples and the substitution samples as machine-generated texts when the difference exceeds a preset threshold significance level;
step S02, statistical detection, namely calculating the prediction probability, probability ranking and information entropy of the current position word when the text is given on the basis of a target generation model, and comprehensively judging the text source by integrating three parts of statistical calculation result indexes;
and S03, summarizing detection results, summarizing and integrating the results of the disturbance difference detection and the statistical detection, and outputting a comprehensive detection result.
In particular, for the reformulation of the generated text, the sample reduction is to identify the reformulation content in the generated text by the text detection method and restore the reformulation text.
The detection method of the machine-generated text provided by the embodiment of the invention comprises disturbance difference detection, statistical detection, result summarization and output. In particular, for generating a description of the text, the method further includes sample reduction.
In the embodiment of the invention, the disturbance difference detection comprises two operations of text disturbance processing and difference detection calculation. The input of the text disturbance processing stage is a target sample and a vocabulary, and a replacement sample set is output. The input of the difference detection calculation stage is a target sample, a replacement sample set and a target generation model, and a disturbance difference detection result is output.
A specific flow of disturbance difference detection is shown in fig. 2.
And (3) performing text disturbance processing, namely firstly obtaining word vector representation by using a pre-provided universal word list. The method comprises the steps of describing the similarity relation between characters by cosine similarity of character vectors, finding out the character with highest cosine similarity of the character vector corresponding to the character in a word list for a specified character as a near-meaning character of a current character, randomly selecting k characters in a target sample, and replacing the k characters with the near-meaning characters to obtain k replacement samples to form a replacement sample set. Recording the set of substitution samples asX＝{x 1 ，...，x k }，x i To replace the samples, i=1, …, k.
The difference detection calculation, taking into account the characteristics of the machine-generated text, comes from the generation model p θ Is usually located at p θ In the negative curvature region of the logarithmic probability function, therefore, the detection of the machine-generated text can be realized by applying a small disturbance to the text and comparing the generation probability of the disturbed sample with the difference of the target sample.
The invention performs perturbation on the target sample through the near-word substitution operation of the text perturbation processing stage, calculates regularized probability difference values through the following steps, and marks machine-generated text when the difference values exceed a preset threshold significance level. The significance difference threshold value epsilon=0.3 is adopted in the invention. The target generation model may be a common autoregressive language model including GPT, T5, LLaMA, and the like.
Input: target sample x, replacement sample setObject generation model p θ Significance difference threshold e.
Calculating a generation probability mean value of the replacement sample:
calculating a probability difference value:
probability difference regularization:
comparison of significance differences:
if it isThe true is output, otherwise the false is output.
In the embodiment of the invention, the statistical detection comprises two operations of statistical detection index calculation and comprehensive index construction. The input of the statistical detection index calculation stage is a word list, a target sample and a target generation model, and the statistical detection index is output. The input of the comprehensive index construction stage is a statistical detection index, and a statistical detection result is output.
A specific flow of statistical detection is shown in fig. 3.
Statistical detection index calculation, since machine-generated text is typically sampled from the head of the probability distribution, it can be determined whether the text is machine-generated by detecting whether the current word is located at the head of the distribution. Let x be 1 ，...，x j-1 For the first j-1 words of the target sample,for the j-th word in the text, x j And D represents a random variable corresponding to the character at the j-th position, and D represents a character set corresponding to the vocabulary. Generating a model p based on a target θ
The following three statistical detection indexes are calculated:
predictive probability p of jth word j ，
Predictive probability ranking r of jth word j ，
r j Is p j Predicting probability sets in all wordsRanking of (3);
information entropy e of jth word predictive distribution j ，
Wherein p is j And r j For checking whether the j-th word of text comes from the head of the target-generated model probability distribution, e j For checking text x based on the above 1:j-1 For the current word x j The degree of confidence in the prediction, j=1,..n, n is the total number of words of the target sample.
And (3) constructing comprehensive indexes, namely respectively constructing three inspection indexes, and then integrating the inspection indexes.
First test index construction, if the prediction probability of a word in a target sample is closer to 1, the sample is more likely to be machine-generated, and calculation is performedAs a first test index, wherein I (x) is an indication function, and is 1 when x is true, 0 when x is false, and alpha 1 As a probability reference value, beta 1 For the first proportional threshold, α is taken in the present invention 1 ＝0.1，β 1 ＝0.5。
Second test index construction, if the predictive probability of a word in a target sample ranks higher, then the sample is more likely to be machine-generated, calculatedAs a second test index, wherein alpha 2 For the ranking reference value beta 2 For the second proportional threshold, α is taken in the present invention 2 ＝100，β 2 ＝0.5。
Third test index construction, if the greater the entropy of the word prediction distribution in the target sample, the greater the uncertainty representing the prediction, the less likely the sample is machine-generated, and the calculationAs a third test criterion, wherein->α 3 For information entropy reference value, beta 3 For the third proportional threshold, α is taken in the present invention 3 ＝0.1，β 3 =0.1. Taking the above three test indexes into consideration, taking their arithmetic mean +.>As a composite index, if T < 0, the target sample is marked as machine-generated.
And the detection result assembly always collects and integrates the results of the disturbance difference detection and the statistical detection, and a final judgment of whether the target sample is generated by the machine is given. The input of the result summarizing and outputting step is the disturbance difference detection result and the statistic detection result, and the comprehensive detection result is output. Considering the detection results comprehensively, if the target sample does not pass the disturbance difference detection or the target sample does not pass the statistical detection (namely, the disturbance difference detection result marks the target sample as machine-generated or the statistical detection result marks the target sample as machine-generated), outputting the comprehensive detection result as machine-generated, otherwise, outputting the comprehensive detection result as artificial writing.
The sample reduction is to identify the modified part in the machine-generated text and reduce the original text.
The specific flow is shown in fig. 4.
The input of the sample reduction step is to reform the sample and the target generation model, and output the original sample. By x *
Representing a change of the sample to be described, represents x * L=1, …, m, p θ Representing the object generation model. The text detection method is used for checking the modified sample word by word, and the specific steps are as follows:
input: description of sample x * Target generation model p θ 。
Initializing l=2, initializing the original samples to be
(1) If L.ltoreq.m:
sequentially executing disturbance difference detection, statistical detection and detection result summarization steps by taking s as a target sample to obtain a comprehensive detection result;
if the comprehensive detection result is generated by the machine, turning to the step (2);
if the comprehensive detection result is written manually, turning to the step (3);
otherwise, go to step (6)
(2) Adding words to the end of string s
Then go to step (5)
(3) Generating an L-th word using the object generation model:
then go to step (4)
(4) Adding words to the end of string s
Then go to step (5)
(5) L+.L+1, go to step (1)
(6) The original sample s is output.
Referring to fig. 5, an embodiment of the present invention also provides a detection system 500 for machine-generated text, the system including a disturbance difference detection unit 510, a statistical detection unit 520, a detection result summarization unit 530,
the disturbance difference detection unit 510 is configured to perform paraphrasing substitution in the target sample, generate a substitution sample, calculate a generation probability of the candidate target sample and the substitution sample in the target generation model, and mark the candidate target sample and the substitution sample as machine-generated text when a difference value of the candidate target sample and the substitution sample exceeds a predetermined threshold significance level;
the statistics detection unit 520 is configured to calculate, based on the target generation model, a prediction probability, a probability ranking, and an information entropy of the word at the current position when the text is given, and integrate the three statistics calculation result indexes to provide a judgment for the text source;
and a detection result summarizing unit 530, configured to summarize and integrate the results of the disturbance difference detection and the statistical detection, and output a comprehensive detection result.
In the detection system of the machine-generated text, a disturbance difference detection unit 510 comprises a text disturbance processing module 51a and a difference detection calculation module 51b, wherein the input of the text disturbance processing module 51a is a target sample and a vocabulary, and a replacement sample set is output; the input of the difference detection calculation module 51b is a target sample, a replacement sample set and a target generation model, and a disturbance difference detection result is output;
the text disturbance processing module 51a acquires a word vector representation by using a pre-provided universal word list, characterizes a similarity relationship between words by cosine similarity of the word vector, finds out a word with highest cosine similarity of the word vector corresponding to a specified word from the pre-provided word list as a near-sense word of the current word, randomly selects k words in a target sample, and respectively replaces the k words with the near-sense words to obtain k replacement samples to form a replacement sample set;
the difference detection calculation module 51b compares the generation probability of the disturbed sample with the difference of the target sample by applying a micro disturbance to the text, thereby realizing the detection of the machine-generated text.
In the machine-generated text detection system, the statistical detection unit 520 includes a statistical detection index calculation module 52a and a comprehensive index construction module 52b; the input of the statistical detection index calculation module is a word list, a target sample and a target generation model, and the statistical detection index is output; the input of the comprehensive index construction module is a statistical detection index, and a statistical detection result is output;
the statistical detection index calculation module 52a determines whether the text is machine-generated by detecting whether the current word is located at the head of the distribution;
the comprehensive index construction module 52b is configured to construct three test indexes respectively, and then integrate the test indexes:
first test indexWherein I (x) is an oscillometric function, and is 1 when x is true, 0 when x is false, alpha 1 As a probability reference value, beta 1 Is a first proportional threshold;
second test indexWherein alpha is 2 For ranking reference value, beta 2 Is a second proportional threshold;
third inspection fingerLabel (C)Wherein->α 3 For information entropy reference value, beta 3 For the third proportional threshold, the arithmetic mean value of the three test indicators is +.>As a comprehensive index, if T<0, marking the target sample as machine-generated.
The embodiment of the invention provides electronic equipment, which comprises: at least one processor and at least one memory;
the memory is used for storing one or more program instructions;
the processor is configured to execute one or more program instructions to perform the detection method as described above.
Embodiments of the present invention also provide a computer readable storage medium having one or more program instructions embodied therein for performing the foregoing detection method.
Technical effects
According to the method for detecting the machine-generated text, firstly, disturbance is implemented on a target sample through the replacement of a paraphrasing, and a probability difference value is calculated and generated so as to realize disturbance difference detection; then, calculating the prediction probability, the prediction probability ranking and the prediction distribution cross entropy of the sample according to the target generation model, and realizing statistical detection based on the three statistical indexes; and finally, comprehensively considering the results of the two detection methods, and outputting a comprehensive detection result. In particular, the invention also proposes a detection method for machine-generated text reformulations based on the aforementioned detection method, which implements text reduction while recognizing the reformulation content. The invention realizes the detection function of the machine-generated text, can be used for solving the abuse problem of the text generated by the current language model, and provides a feasible and effective scheme for the supervision of the text sources by related departments.
While the invention has been described in detail in the foregoing general description and specific examples, it will be apparent to those skilled in the art that modifications and improvements can be made thereto. Accordingly, such modifications or improvements may be made without departing from the spirit of the invention and are intended to be within the scope of the invention as claimed.
Claims (10)
1. A method for detecting machine-generated text is characterized by comprising disturbance difference detection, statistical detection and detection result summarization, wherein,
the disturbance difference detection is to perform paraphrasing substitution in the target sample, generate a substitution sample, calculate the generation probability of candidate target samples and substitution samples in the target generation model, and mark the candidate target samples and the substitution samples as machine-generated texts when the difference exceeds a preset threshold significance level;
the statistical detection is used for calculating the prediction probability, probability ranking and information entropy of the words at the current position when the text is given based on the target generation model, and judging whether the text is generated by a machine or not by detecting whether the current word is positioned at the head of the probability distribution or not: further calculating a first detection index T according to the prediction probability, probability ranking and information entropy calculated by the current position words 1 Second detection index T 2 Third detection index T 3 Combining the three detection indexes to give judgment to the text source, and marking the text as machine-generated text when the arithmetic average value of the three detection indexes exceeds a preset threshold significance level;
and summarizing the detection results, namely summarizing and integrating the results of the disturbance difference detection and the statistical detection, and outputting a comprehensive detection result.
2. The method for detecting machine-generated text of claim 1,
the disturbance difference detection comprises the steps of text disturbance processing and difference detection calculation, wherein the input of the text disturbance processing step is a target sample and a word list, and a replacement sample set is output; the input of the difference detection calculation step is a target sample, a replacement sample set and a target generation model, and a disturbance difference detection result is output;
the text disturbance processing comprises the steps of obtaining character vector representation by using a pre-provided universal word list, describing the similarity relationship between characters by the cosine similarity of the character vector, for a designated character, finding out the character with the highest cosine similarity of the character vector corresponding to the character from the pre-provided word list as the near-sense character of the current character, randomly selecting k characters in a target sample, and respectively replacing the k characters with the near-sense characters to obtain k replacement samples to form a replacement sample set;
and (3) performing difference detection calculation, namely comparing the generation probability of the disturbed sample with the difference of the target sample by applying micro disturbance to the text, so as to realize the detection of the machine generated text.
3. The method for detecting machine-generated text according to claim 2, wherein the near-word substitution operation of the text perturbation processing step perturbs the target sample by calculating a regularized probability difference value, and determining the machine-generated text when the difference value exceeds a predetermined threshold value:
(1) Calculating a generation probability mean value of the replacement sample:
(2) Calculating a probability difference value:
(3) Probability difference regularization:
(4) Comparison of significance differences:
if it isDetermining that the machine-generated text;
wherein x is the target sample, and the sample is obtained,for replacing sample sets +.> To replace an element in a sample set, p θ A model is generated for the target, e is the saliency difference threshold.
4. The method for detecting machine-generated text of claim 1,
the statistical detection comprises two stages of statistical detection index calculation and comprehensive index construction; the input of the statistical detection index calculation stage is a word list, a target sample and a target generation model, and the statistical detection index is output; the input of the comprehensive index construction stage is a statistical detection index, and a statistical detection result is output;
the statistical detection index calculation judges whether the text is generated by a machine by detecting whether the current word is positioned at the head of the probability distribution;
building comprehensive indexes, namely building three inspection indexes respectively, and then integrating the inspection indexes:
first test indexWherein I (x) is an oscillometric function, and is 1 when x is true, 0 when x is false, alpha 1 As a probability reference value, beta 1 Is a first proportional threshold;
second test indexWherein alpha is 2 For ranking reference value, beta 1 Is a second proportional threshold;
third test indexWherein->α 3 For information entropy reference value, beta 3 For the third proportional threshold, the arithmetic mean value of the three test indicators is +.>As a comprehensive index, if T<0, marking the target sample as machine-generated.
5. The method for detecting machine-generated text according to claim 1, further comprising the step of sample restoration, wherein the method for detecting text identifies the remarked content in the machine-generated text and restores the original text, specifically:
sequentially performing disturbance difference detection, statistical detection and detection result summarization steps by taking the initialized modified sample as a target sample to obtain a comprehensive detection result;
if the artificial writing is judged, generating a first word by the target generation model, and updating a current sample by using the model generation word to obtain the current sample; if the machine generation is judged, the current sample is updated along with the first word of the original sample, and the current sample is obtained;
if the length of the current sample is equal to the length of the modified sample, the current sample is restored to the obtained original text.
6. A machine-generated text detection system is characterized in that the system comprises a disturbance difference detection unit, a statistics detection unit and a detection result summarization unit,
the disturbance difference detection unit is used for performing near meaning word replacement in the target sample, generating a replacement sample, calculating the generation probability of the candidate target sample and the replacement sample in the target generation model, and marking the candidate target sample and the replacement sample as machine-generated text when the difference exceeds a preset threshold significance level;
the statistical detection unit is used for calculating the prediction probability, probability ranking and information entropy of the word at the current position when the text is given based on the target generation model, and judging whether the text is generated by a machine or not by detecting whether the current word is positioned at the head of the probability distribution or not: further calculating a first detection index T according to the prediction probability, probability ranking and information entropy calculated by the current position words 1 Second detection index T 2 Third detection index T 3 Combining the three detection indexes to give judgment to the text source, and marking the text as machine-generated text when the arithmetic average value of the three detection indexes exceeds a preset threshold significance level;
and the detection result summarizing unit is used for summarizing and integrating the results of the disturbance difference detection and the statistical detection and outputting a comprehensive detection result.
7. The machine-generated text detection system of claim 6,
the disturbance difference detection unit comprises a text disturbance processing module and a difference detection calculation module, wherein the input of the text disturbance processing module is a target sample and a word list, and a replacement sample set is output; the input of the difference detection calculation module is a target sample, a replacement sample set and a target generation model, and a disturbance difference detection result is output;
the text disturbance processing module acquires character vector representation by using a pre-provided universal word list, characterizes the similarity relationship between characters by the cosine similarity of the character vector, finds out the character with the highest cosine similarity of the character vector corresponding to the character from the pre-provided word list as the near-meaning character of the current character for the appointed character, randomly selects k characters in a target sample, and respectively replaces the k characters with the near-meaning characters to obtain k replacement samples to form a replacement sample set;
and the difference detection calculation module is used for comparing the generation probability of the disturbed sample with the difference of the target sample by applying micro disturbance to the text, so that the detection of the machine generated text is realized.
8. The machine-generated text detection system of claim 6,
the statistical detection unit comprises a statistical detection index calculation module and a comprehensive index construction module; the input of the statistical detection index calculation module is a word list, a target sample and a target generation model, and the statistical detection index is output; the input of the comprehensive index construction module is a statistical detection index, and a statistical detection result is output;
the statistical detection index calculation module judges whether the text is generated by a machine by detecting whether the current word is positioned at the head of the probability distribution;
the comprehensive index construction module is used for respectively constructing three inspection indexes and then integrating the inspection indexes:
first test indexWherein I (x) is an oscillometric function, and is 1 when x is true, 0 when x is false, alpha 1 As a probability reference value, beta 1 Is a first proportional threshold;
second test indexWherein alpha is 2 For ranking reference value, beta 1 Is a second proportional threshold;
third test indexWherein->α 3 For information entropy reference value, beta 3 For the third proportional threshold, the arithmetic mean of the three test indexes
As a comprehensive index, if T<0, marking the target sample as machine-generated.
9. An electronic device, the electronic device comprising: at least one processor and at least one memory;
the memory is used for storing one or more program instructions;
the processor is configured to execute one or more program instructions to perform the method of any of claims 1-5.
10. A computer readable storage medium, comprising one or more program instructions embodied therein for performing the method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311075839.7A CN116842140B (en) | 2023-08-25 | 2023-08-25 | Method and system for detecting machine-generated text |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311075839.7A CN116842140B (en) | 2023-08-25 | 2023-08-25 | Method and system for detecting machine-generated text |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116842140A CN116842140A (en) | 2023-10-03 |
| CN116842140B true CN116842140B (en) | 2024-01-26 |
Family
ID=88163741
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311075839.7A Active CN116842140B (en) | 2023-08-25 | 2023-08-25 | Method and system for detecting machine-generated text |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116842140B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113590803A (en) * | 2021-02-24 | 2021-11-02 | 腾讯科技（深圳）有限公司 | Data processing method and device, storage medium and computer equipment |
| WO2022105083A1 (en) * | 2020-11-19 | 2022-05-27 | 平安科技（深圳）有限公司 | Text error correction method and apparatus, device, and medium |
| CN115309898A (en) * | 2022-07-27 | 2022-11-08 | 方盈金泰科技(北京)有限公司 | Word granularity Chinese semantic approximate countermeasure sample generation method based on knowledge enhanced BERT |
| CN115688697A (en) * | 2022-09-09 | 2023-02-03 | 武汉中科华研科技有限公司 | Text rewriting method and system for near meaning item replacement based on deep learning |
| CN116108154A (en) * | 2023-02-08 | 2023-05-12 | 京东科技信息技术有限公司 | Sample generation method and device, model training method and system and storage medium |
-
2023
- 2023-08-25 CN CN202311075839.7A patent/CN116842140B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022105083A1 (en) * | 2020-11-19 | 2022-05-27 | 平安科技（深圳）有限公司 | Text error correction method and apparatus, device, and medium |
| CN113590803A (en) * | 2021-02-24 | 2021-11-02 | 腾讯科技（深圳）有限公司 | Data processing method and device, storage medium and computer equipment |
| CN115309898A (en) * | 2022-07-27 | 2022-11-08 | 方盈金泰科技(北京)有限公司 | Word granularity Chinese semantic approximate countermeasure sample generation method based on knowledge enhanced BERT |
| CN115688697A (en) * | 2022-09-09 | 2023-02-03 | 武汉中科华研科技有限公司 | Text rewriting method and system for near meaning item replacement based on deep learning |
| CN116108154A (en) * | 2023-02-08 | 2023-05-12 | 京东科技信息技术有限公司 | Sample generation method and device, model training method and system and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116842140A (en) | 2023-10-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113688631B (en) | Nested named entity identification method, system, computer and storage medium | |
| CN112966259B (en) | Operation and maintenance behavior security threat assessment method and equipment for power monitoring system | |
| CN111222981A (en) | Credibility determination method, device, equipment and storage medium | |
| Ishihara | A likelihood ratio-based evaluation of strength of authorship attribution evidence in SMS messages using N-grams. | |
| CN116127953B (en) | Chinese spelling error correction method, device and medium based on contrast learning | |
| CN112613293A (en) | Abstract generation method and device, electronic equipment and storage medium | |
| CN116245139B (en) | Training method and device for graph neural network model, event detection method and device | |
| CN116842140B (en) | Method and system for detecting machine-generated text | |
| CN109977400B (en) | Verification processing method and device, computer storage medium and terminal | |
| CN116384388B (en) | Method, device, equipment and medium for reverse identification AI intelligent writing | |
| CN115310449A (en) | Named entity identification method and device based on small sample and related medium | |
| CN110059180B (en) | Article author identity recognition and evaluation model training method and device and storage medium | |
| CN113688633A (en) | Outline determination method and device | |
| CN114254622A (en) | Intention identification method and device | |
| Noyori et al. | Deep learning and gradient-based extraction of bug report features related to bug fixing time | |
| CN110705597A (en) | Network early event detection method and system based on event cause and effect extraction | |
| AlSaif | Large scale data mining for banking credit risk prediction | |
| CN117171653B (en) | Method, device, equipment and storage medium for identifying information relationship | |
| Anh et al. | Kwa: A new method of calculation and representation accuracy for speech keyword spotting in string results | |
| CN117648921B (en) | Cross-theme composition automatic evaluation method and system based on paired double-layer countermeasure alignment | |
| Siva Rama Rao et al. | Named Entity Recognition Using Stanford Classes and NLTK | |
| Xu et al. | The Study of NER Methods Based on Bi-LSTM+ CRF Model | |
| Jain et al. | Attention-based Deep Learning Model for Accurate Detection of Misleading News. | |
| CN115146079A (en) | User tag generation method, question recommendation method, device, equipment and medium | |
| Iqbal et al. | Authorship Verification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: No. 401-1, 4th floor, podium, building 3 and 4, No. 11, Changchun Bridge Road, Haidian District, Beijing 100089 Patentee after: Beijing Xinghe Zhiyuan Technology Co.,Ltd. Country or region after: China Patentee after: Zhiguagua (Tianjin) Big Data Technology Co.,Ltd. Address before: No. 401-1, 4th floor, podium, building 3 and 4, No. 11, Changchun Bridge Road, Haidian District, Beijing 100089 Patentee before: Beijing Zhiguagua Technology Co.,Ltd. Country or region before: China Patentee before: Zhiguagua (Tianjin) Big Data Technology Co.,Ltd. |
|
| CP03 | Change of name, title or address | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240511 Address after: No. 401-1, 4th floor, podium, building 3 and 4, No. 11, Changchun Bridge Road, Haidian District, Beijing 100089 Patentee after: Beijing Xinghe Zhiyuan Technology Co.,Ltd. Country or region after: China Address before: No. 401-1, 4th floor, podium, building 3 and 4, No. 11, Changchun Bridge Road, Haidian District, Beijing 100089 Patentee before: Beijing Xinghe Zhiyuan Technology Co.,Ltd. Country or region before: China Patentee before: Zhiguagua (Tianjin) Big Data Technology Co.,Ltd. |
|
| TR01 | Transfer of patent right |