CN117435910A - Abnormal data detection method and device and computer equipment - Google Patents
Abnormal data detection method and device and computer equipment Download PDFInfo
- Publication number
- CN117435910A CN117435910A CN202311248655.6A CN202311248655A CN117435910A CN 117435910 A CN117435910 A CN 117435910A CN 202311248655 A CN202311248655 A CN 202311248655A CN 117435910 A CN117435910 A CN 117435910A
- Authority
- CN
- China
- Prior art keywords
- data
- classification
- operation result
- features
- vector machine
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 78
- 238000001514 detection method Methods 0.000 title claims abstract description 15
- 230000003993 interaction Effects 0.000 claims abstract description 93
- 230000004927 fusion Effects 0.000 claims abstract description 66
- 238000000034 method Methods 0.000 claims abstract description 58
- 230000006870 function Effects 0.000 claims description 34
- 238000012549 training Methods 0.000 claims description 32
- 238000004590 computer program Methods 0.000 claims description 23
- 238000000605 extraction Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 230000002547 anomalous effect Effects 0.000 description 5
- 238000010276 construction Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000007796 conventional method Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3692—Test management for test results analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2433—Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- General Factory Administration (AREA)
Abstract
The application relates to a method and a device for detecting abnormal data and computer equipment. The method comprises the following steps: acquiring resource interaction data and operation result data corresponding to the resource interaction data, wherein the operation result data comprises abnormal data and correct data; extracting the characteristics of the resource interaction data and the operation result data to obtain resource interaction characteristics and operation result characteristics; based on weight coefficients corresponding to different features, carrying out feature fusion on the resource interaction features and the corresponding operation result features to obtain fusion features; and inputting the fusion characteristics into a pre-trained classification vector machine to obtain abnormal data in the operation result data. By adopting the method, the detection efficiency of the abnormal data can be improved.
Description
Technical Field
The present invention relates to the field of big data technologies, and in particular, to a method and an apparatus for detecting abnormal data, and a computer device.
Background
The low code is a group of digital technology tool platforms, and can realize rapid construction, data arrangement, ecological connection, middle service and the like based on more efficient modes such as graphic dragging, parameterized configuration and the like. Compared with a code-free development platform, the operation difficulty is high, and errors in operation are difficult to discover in time.
In the related art, the correctness of the operation result can be judged by adopting a manual screening mode, and if the operation result is wrong, the resource interaction data is judged, so that the working efficiency is lower.
Disclosure of Invention
Based on the above, it is necessary to provide a method for detecting abnormal data, by collecting the existing resource interaction data and operation result data, classifying them based on an artificial intelligence algorithm, and performing regular derivation to obtain a final model, and correcting the error by the existing prompt information.
In a first aspect, the present application provides a method for detecting abnormal data. The method comprises the following steps:
acquiring resource interaction data and operation result data corresponding to the resource interaction data, wherein the operation result data comprises abnormal data and correct data;
extracting the characteristics of the resource interaction data and the operation result data to obtain resource interaction characteristics and operation result characteristics;
based on weight coefficients corresponding to different features, carrying out feature fusion on the resource interaction features and the corresponding operation result features to obtain fusion features;
and inputting the fusion characteristics into a pre-trained classification vector machine to obtain abnormal data in the operation result data.
In one embodiment, the generating manner of the classification vector machine includes:
and inputting the training samples into a classification vector machine, and determining a classification hyperplane and a classification decision function which are output by the model when the error between the result output by the classification vector machine and the training samples is smaller than an error threshold value, so as to obtain the trained classification vector machine.
In one embodiment, the training samples are obtained as follows:
acquiring different types of resource interaction data, wherein the operation result data corresponds to the resource interaction data;
extracting the characteristics of the resource interaction data and the operation result data to obtain different types of resource interaction characteristics and operation result characteristics;
carrying out feature fusion on the resource interaction features of the same type and the corresponding operation result features to obtain fusion features;
and splicing the fusion features to obtain spliced features, wherein the spliced features are used as training samples for training the classification vector machine.
In one embodiment, after the fused features are input to a pre-trained classification vector machine, the method comprises:
obtaining a classification hyperplane and a classification decision function which are output by the classification vector machine;
and inputting the fusion characteristics in the classification hyperplane distance threshold value into a classification decision function, and determining the type of the abnormal data according to the classification hyperplane when the output result accords with the classification decision function.
In one embodiment, the method further comprises:
and (3) taking the original abnormal data and the new abnormal data output by the classification vector machine as data for optimizing the classification vector machine, and redefining a classification hyperplane and a classification decision function of the classification vector machine.
In a second aspect, the present application further provides a device for detecting abnormal data, where the device includes:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring resource interaction data and operation result data corresponding to the resource interaction data, and the operation result data comprises abnormal data and correct data;
the extraction module is used for extracting the characteristics of the resource interaction data and the operation result data to obtain the resource interaction characteristics and the operation result characteristics;
the fusion module is used for carrying out feature fusion on the resource interaction features and the corresponding operation result features based on the weight coefficients corresponding to the different features to obtain fusion features;
and the detection module is used for inputting the fusion characteristics into a pre-trained classification vector machine to obtain abnormal data in the operation result data.
In one embodiment, the generating manner of the classification vector machine includes:
and inputting the training samples into a classification vector machine, and determining a classification hyperplane and a classification decision function which are output by the model when the error between the result output by the classification vector machine and the training samples is smaller than an error threshold value, so as to obtain the trained classification vector machine.
In one embodiment, the training samples are obtained as follows:
acquiring different types of resource interaction data, wherein the operation result data corresponds to the resource interaction data;
extracting the characteristics of the resource interaction data and the operation result data to obtain different types of resource interaction characteristics and operation result characteristics;
carrying out feature fusion on the resource interaction features of the same type and the corresponding operation result features to obtain fusion features;
and splicing the fusion features to obtain spliced features, wherein the spliced features are used as training samples for training the classification vector machine.
In one embodiment, after the inputting the fusion features into a pre-trained classification vector machine, the apparatus comprises:
obtaining a classification hyperplane and a classification decision function which are output by the classification vector machine;
and inputting the fusion characteristics in the classification hyperplane distance threshold value into a classification decision function, and determining the type of the abnormal data according to the classification hyperplane when the output result accords with the classification decision function.
In one embodiment, the apparatus further comprises:
and (3) taking the original abnormal data and the new abnormal data output by the classification vector machine as data for optimizing the classification vector machine, and redefining a classification hyperplane and a classification decision function of the classification vector machine.
In a third aspect, the present disclosure also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the steps of the method for detecting abnormal data when the processor executes the computer program.
In a fourth aspect, the present disclosure also provides a computer-readable storage medium. The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of a method of detecting anomalous data.
In a fifth aspect, the present disclosure also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of a method of detecting anomalous data.
The method for detecting the abnormal data at least comprises the following beneficial effects:
according to the embodiment scheme provided by the disclosure, the resource interaction data and the operation result data can be subjected to feature extraction to obtain the resource interaction features and the operation result features, the resource interaction features and the corresponding operation result features are subjected to feature fusion based on the weight coefficients corresponding to the different features to obtain fusion features, and the fusion features are input into the classification vector machine to determine abnormal data in the operation result data, so that the abnormal data detection efficiency is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments or the conventional techniques of the present disclosure, the drawings required for the descriptions of the embodiments or the conventional techniques will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present disclosure, and other drawings may be obtained according to the drawings without inventive effort to those of ordinary skill in the art.
FIG. 1 is an application environment diagram of a method for detecting anomalous data in an embodiment;
FIG. 2 is a flow chart of a method for detecting abnormal data according to an embodiment;
FIG. 3 is a flow chart illustrating a method for detecting abnormal data according to an embodiment;
FIG. 4 is a block diagram of an apparatus for detecting abnormal data in one embodiment;
FIG. 5 is a block diagram of an apparatus for detecting abnormal data in one embodiment;
FIG. 6 is an internal block diagram of a computer device in one embodiment;
fig. 7 is an internal structural diagram of a server in one embodiment.
Detailed Description
In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the foregoing figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the disclosure described herein may be capable of operation in sequences other than those illustrated or described herein. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims. The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, it is not excluded that additional identical or equivalent elements may be present in a process, method, article, or apparatus that comprises a described element. For example, if first, second, etc. words are used to indicate a name, but not any particular order.
The embodiment of the disclosure provides a method for detecting abnormal data, which can be applied to an application environment as shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices, and portable wearable devices, where the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart vehicle devices, and the like. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers.
In some embodiments of the present disclosure, as shown in fig. 2, a method for detecting abnormal data is provided, and the method is applied to the server in fig. 1 to process the abnormal data for illustration. It will be appreciated that the method may be applied to a server, and may also be applied to a system comprising a terminal and a server, and implemented by interaction of the terminal and the server. In a specific embodiment, the method may include the steps of:
s202: and acquiring resource interaction data and operation result data corresponding to the resource interaction data, wherein the operation result data comprises abnormal data and correct data.
The low-code platform shields a large number of codes, a user can conduct transaction construction in a graphical dragging and text information configuration mode, wherein resource interaction data can be data configured through graphical dragging and text information, operation result data can be a final presentation result of the resource interaction data on a page, and the operation result can comprise abnormal data and correct data.
In some embodiments of the present disclosure, a user may complete the construction of a service application by dragging components onto an application development interface according to a certain logic sequence, and setting internal parameters of the components, where different components correspond to different service attributes, and final operation result data are different. The component information may be understood as related information of each component of the building target application, may include component self information of each component, such as component identification, internal parameters set by the component, and the like, and may also include component environment information of each component, such as component identifications of a previous component and a next component of each component, and the like.
S204: and extracting the characteristics of the resource interaction data and the operation result data to obtain the resource interaction characteristics and the operation result characteristics.
The resource interaction data may include different types of resource interaction data, such as text, video, etc., and the operation result data corresponding to the resource interaction data is changed accordingly. The feature extraction is performed on the resource interaction data and the operation result data, which may include the number of times the user uses the component, the feature of the component, and the like, the resource interaction feature and the operation result feature may be obtained, the resource interaction information may include the number of times the user uses the component, the frequency, and the like, and the operation result feature may refer to the final feature of the resource interaction information.
S206, carrying out feature fusion on the resource interaction features and the corresponding operation result features based on the weight coefficients corresponding to the different features to obtain fusion features.
Different features have different influences on the detection of the final abnormal data, so that corresponding weight coefficients are formulated according to the detection influence degree of the features on the abnormal data. The platform can be composed of a plurality of resource interaction data, each resource interaction feature can be a local feature of the construction platform, and then feature fusion is carried out on each local feature so as to obtain a fusion feature corresponding to the construction platform, namely a global feature.
S208, inputting the fusion features into a pre-trained classification vector machine to obtain abnormal data in the operation result data.
And inputting the fusion features into a classification vector machine to determine abnormal data in the operation result data, and carrying out feature analysis on the resource interaction data and the operation result data corresponding to the resource interaction data to improve the accuracy of feature extraction and further improve the accuracy of abnormal data detection.
In the method for detecting abnormal data, the feature extraction can be performed on the resource interaction data and the operation result data to obtain the resource interaction feature and the operation result feature, the feature fusion is performed on the resource interaction feature and the corresponding operation result feature based on the weight coefficients corresponding to different features to obtain the fusion feature, and the fusion feature is input into the classification vector machine to determine the abnormal data in the operation result data, so that the abnormal data detection efficiency is improved.
In some embodiments of the present disclosure, the generation manner of the classification vector machine includes:
and inputting the training samples into a classification vector machine, and determining a classification hyperplane and a classification decision function which are output by the model when the error between the result output by the classification vector machine and the training samples is smaller than an error threshold value, so as to obtain the trained classification vector machine.
The vector machine is a machine learning algorithm for solving the classification problem, and can separate samples of different classes by finding a partition hyperplane in a sample space, and simultaneously, the minimum distance from two point sets to the plane is maximized, and the distance from the edge point in the two point sets to the plane is maximized, so that the samples are classified. The training samples are input into a classification vector machine, when the error between the result output by the classification vector machine and the training samples is smaller than an error threshold, namely, the situation that the classification error of the vector machine possibly occurs in the classification process, the error threshold can be set, when the error is smaller than the error threshold, the classification hyperplane and the classification decision function output by the model are determined, the number of the classification hyperplanes can represent the number of types of the samples, and the classification decision function can determine the shape of the classification hyperplane.
In some embodiments of the present disclosure, the training samples are obtained by:
acquiring different types of resource interaction data, wherein the operation result data corresponds to the resource interaction data;
extracting the characteristics of the resource interaction data and the operation result data to obtain different types of resource interaction characteristics and operation result characteristics;
carrying out feature fusion on the resource interaction features of the same type and the corresponding operation result features to obtain fusion features;
and splicing the fusion features to obtain spliced features, wherein the spliced features are used as training samples for training the classification vector machine.
The platform can comprise different types of resource interaction data, the different types of resource interaction data correspond to different operation result data, the same type of resource interaction data and the corresponding operation result data can be subjected to feature extraction, the same type of resource interaction features and the corresponding operation result features are subjected to feature fusion to obtain fusion features, and abnormal data can be detected by combining the fusion features with the resource interaction features and the corresponding operation result features. And the fusion characteristics of different types can be spliced, and the fusion characteristics are uniformly judged by combining the interaction characteristics of various types of resources and the corresponding operation result characteristics, so that the abnormal data detection efficiency is improved.
In some embodiments of the present disclosure, after the inputting the fusion features into a pre-trained classification vector machine, the method comprises:
obtaining a classification hyperplane and a classification decision function which are output by the classification vector machine;
and inputting the fusion characteristics in the classification hyperplane distance threshold value into a classification decision function, and determining the type of the abnormal data according to the classification hyperplane when the output result accords with the classification decision function.
The classification vector machine can detect the abnormal data in the platform, and after the abnormal data is obtained, the abnormal data can be input into the classification vector machine, and finally the type of the abnormal data can be obtained.
In some embodiments of the present disclosure, fig. 3 is a flow chart of a method for detecting abnormal data in one embodiment, where the method further includes:
s302, the original abnormal data and the new abnormal data output by the classification vector machine are used as data for optimizing the classification vector machine, and the classification hyperplane and the classification decision function of the classification vector machine are redetermined.
According to the fusion characteristics, the abnormal data in the operation result data are input into a pre-trained classification vector machine, the original abnormal data and the new abnormal data output by the classification vector machine can be used as data of an optimized classification vector machine, the data are input into the classification vector machine again, and the classification hyperplane and the classification decision function are optimized, so that the optimized classification vector machine is obtained.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the present disclosure also provides a detection apparatus for abnormal data for implementing the above-mentioned detection method for abnormal data. The implementation scheme of the device for solving the problem is similar to that described in the above method, so the specific limitation in the embodiment of the device for detecting abnormal data provided below can be referred to the limitation of the method for detecting abnormal data hereinabove, and will not be repeated here.
The apparatus may comprise a system (including a distributed system), software (applications), modules, components, servers, clients, etc. that employ the methods described in the embodiments of the present specification in combination with the necessary apparatus to implement the hardware. Based on the same innovative concepts, embodiments of the present disclosure provide for devices in one or more embodiments as described in the following examples. Because the implementation scheme and the method for solving the problem by the device are similar, the implementation of the device in the embodiment of the present disclosure may refer to the implementation of the foregoing method, and the repetition is not repeated. As used below, the term "unit" or "module" may be a combination of software and/or hardware that implements the intended function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
In one embodiment, as shown in fig. 4, an apparatus 400 for detecting abnormal data is provided, where the apparatus may be the foregoing server, or a module, a component, a device, a unit, etc. integrated with the server.
The apparatus 400 may include:
an obtaining module 402, configured to obtain resource interaction data and operation result data corresponding to the resource interaction data, where the operation result data includes abnormal data and correct data;
the extracting module 404 is configured to perform feature extraction on the resource interaction data and the operation result data to obtain resource interaction features and operation result features;
the fusion module 406 is configured to perform feature fusion on the resource interaction feature and the corresponding operation result feature based on the weight coefficients corresponding to the different features, so as to obtain a fusion feature;
and the detection module 408 is configured to input the fusion feature to a pre-trained classification vector machine, so as to obtain abnormal data in the operation result data.
In one embodiment, the generation mode of the classification vector machine includes:
and inputting the training samples into a classification vector machine, and determining a classification hyperplane and a classification decision function which are output by the model when the error between the result output by the classification vector machine and the training samples is smaller than an error threshold value, so as to obtain the trained classification vector machine.
In one embodiment, the training samples are obtained as follows:
acquiring different types of resource interaction data, wherein the operation result data corresponds to the resource interaction data;
extracting the characteristics of the resource interaction data and the operation result data to obtain different types of resource interaction characteristics and operation result characteristics;
carrying out feature fusion on the resource interaction features of the same type and the corresponding operation result features to obtain fusion features;
and splicing the fusion features to obtain spliced features, wherein the spliced features are used as training samples for training the classification vector machine.
In one embodiment, after the inputting the fusion features into a pre-trained classification vector machine, the apparatus comprises:
obtaining a classification hyperplane and a classification decision function which are output by the classification vector machine;
and inputting the fusion characteristics in the classification hyperplane distance threshold value into a classification decision function, and determining the type of the abnormal data according to the classification hyperplane when the output result accords with the classification decision function.
In one embodiment, fig. 5 is a block diagram of an apparatus for detecting abnormal data in one embodiment, where the apparatus further includes:
the optimization module 502: and (3) taking the original abnormal data and the new abnormal data output by the classification vector machine as data for optimizing the classification vector machine, and redefining a classification hyperplane and a classification decision function of the classification vector machine.
The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.
The respective modules in the above-described detection device for abnormal data may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 6. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the user behavior of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used to store exception data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program, when executed by a processor, implements a method of detecting anomalous data.
In one embodiment, a computer device is provided, which may be a terminal, and the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the user behavior of the operating system and computer programs in the non-volatile storage media. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program, when executed by a processor, implements a method of detecting anomalous data. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the structures shown in fig. 6 and 7 are merely block diagrams of partial structures related to the disclosed aspects and do not constitute a limitation of the computer device on which the disclosed aspects are applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, implements the method of any of the embodiments of the present disclosure.
In one embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method described in any of the embodiments of the present disclosure.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided by the present disclosure may include at least one of non-volatile and volatile memory, among others. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided by the present disclosure may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors involved in the embodiments provided by the present disclosure may be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic, quantum computing-based data processing logic, etc., without limitation thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples have expressed only a few embodiments of the present disclosure, which are described in more detail and detail, but are not to be construed as limiting the scope of the present disclosure. It should be noted that variations and modifications can be made by those skilled in the art without departing from the spirit of the disclosure, which are within the scope of the disclosure. Accordingly, the scope of the present disclosure should be determined from the following claims.
Claims (13)
1. A method for detecting abnormal data, the method comprising:
acquiring resource interaction data and operation result data corresponding to the resource interaction data, wherein the operation result data comprises abnormal data and correct data;
extracting the characteristics of the resource interaction data and the operation result data to obtain resource interaction characteristics and operation result characteristics;
based on weight coefficients corresponding to different features, carrying out feature fusion on the resource interaction features and the corresponding operation result features to obtain fusion features;
and inputting the fusion characteristics into a pre-trained classification vector machine to obtain abnormal data in the operation result data.
2. The method of claim 1, wherein the generating means of the classification vector machine comprises:
and inputting the training samples into a classification vector machine, and determining a classification hyperplane and a classification decision function which are output by the model when the error between the result output by the classification vector machine and the training samples is smaller than an error threshold value, so as to obtain the trained classification vector machine.
3. The method according to claim 2, wherein the training samples are obtained by:
acquiring different types of resource interaction data, wherein the operation result data corresponds to the resource interaction data;
extracting the characteristics of the resource interaction data and the operation result data to obtain different types of resource interaction characteristics and operation result characteristics;
carrying out feature fusion on the resource interaction features of the same type and the corresponding operation result features to obtain fusion features;
and splicing the fusion features to obtain spliced features, wherein the spliced features are used as training samples for training the classification vector machine.
4. The method of claim 1, wherein after the inputting the fused features into a pre-trained classification vector machine, the method comprises:
obtaining a classification hyperplane and a classification decision function which are output by the classification vector machine;
and inputting the fusion characteristics in the classification hyperplane distance threshold value into a classification decision function, and determining the type of the abnormal data according to the classification hyperplane when the output result accords with the classification decision function.
5. The method according to claim 1, wherein the method further comprises:
and (3) taking the original abnormal data and the new abnormal data output by the classification vector machine as data for optimizing the classification vector machine, and redefining a classification hyperplane and a classification decision function of the classification vector machine.
6. An apparatus for detecting abnormal data, the apparatus comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring resource interaction data and operation result data corresponding to the resource interaction data, and the operation result data comprises abnormal data and correct data;
the extraction module is used for extracting the characteristics of the resource interaction data and the operation result data to obtain the resource interaction characteristics and the operation result characteristics;
the fusion module is used for carrying out feature fusion on the resource interaction features and the corresponding operation result features based on the weight coefficients corresponding to the different features to obtain fusion features;
and the detection module is used for inputting the fusion characteristics into a pre-trained classification vector machine to obtain abnormal data in the operation result data.
7. The apparatus of claim 6, wherein the means for generating the classification vector machine comprises:
and inputting the training samples into a classification vector machine, and determining a classification hyperplane and a classification decision function which are output by the model when the error between the result output by the classification vector machine and the training samples is smaller than an error threshold value, so as to obtain the trained classification vector machine.
8. The apparatus of claim 7, wherein the training samples are obtained by:
acquiring different types of resource interaction data, wherein the operation result data corresponds to the resource interaction data;
extracting the characteristics of the resource interaction data and the operation result data to obtain different types of resource interaction characteristics and operation result characteristics;
carrying out feature fusion on the resource interaction features of the same type and the corresponding operation result features to obtain fusion features;
and splicing the fusion features to obtain spliced features, wherein the spliced features are used as training samples for training the classification vector machine.
9. The apparatus of claim 6, wherein after the inputting the fused features into a pre-trained classification vector machine, the apparatus comprises:
obtaining a classification hyperplane and a classification decision function which are output by the classification vector machine;
and inputting the fusion characteristics in the classification hyperplane distance threshold value into a classification decision function, and determining the type of the abnormal data according to the classification hyperplane when the output result accords with the classification decision function.
10. The apparatus of claim 6, wherein the apparatus further comprises:
and (3) taking the original abnormal data and the new abnormal data output by the classification vector machine as data for optimizing the classification vector machine, and redefining a classification hyperplane and a classification decision function of the classification vector machine.
11. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 5 when the computer program is executed.
12. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 5.
13. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311248655.6A CN117435910A (en) | 2023-09-26 | 2023-09-26 | Abnormal data detection method and device and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311248655.6A CN117435910A (en) | 2023-09-26 | 2023-09-26 | Abnormal data detection method and device and computer equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117435910A true CN117435910A (en) | 2024-01-23 |
Family
ID=89547048
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311248655.6A Pending CN117435910A (en) | 2023-09-26 | 2023-09-26 | Abnormal data detection method and device and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117435910A (en) |
-
2023
- 2023-09-26 CN CN202311248655.6A patent/CN117435910A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114861172B (en) | Data processing method and system based on government affair service system | |
| CN114549849A (en) | Image recognition method and device, computer equipment and storage medium | |
| CN108875901B (en) | Neural network training method and universal object detection method, device and system | |
| CN114692889A (en) | Meta-feature training model for machine learning algorithm | |
| CN117540915A (en) | Big data technology-based selection scheme generation method, device, equipment and medium | |
| WO2021236423A1 (en) | Identifying claim complexity by integrating supervised and unsupervised learning | |
| CN117631682A (en) | Method and system for determining inspection route of unmanned aerial vehicle of power grid | |
| CN116932935A (en) | Address matching method, device, equipment, medium and program product | |
| CN116894721A (en) | Index prediction method and device and computer equipment | |
| CN111049988A (en) | Intimacy prediction method, system, equipment and storage medium for mobile equipment | |
| CN113010687B (en) | Exercise label prediction method and device, storage medium and computer equipment | |
| CN117435910A (en) | Abnormal data detection method and device and computer equipment | |
| CN115758271A (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN114510592B (en) | Image classification method, device, electronic equipment and storage medium | |
| CN115965856B (en) | Image detection model construction method, device, computer equipment and storage medium | |
| CN116881122A (en) | Test case generation method, device, equipment, storage medium and program product | |
| CN116881543A (en) | Financial resource object recommendation method, device, equipment, storage medium and product | |
| CN117436484A (en) | Image recognition model construction, image recognition model construction device, computer equipment and storage medium | |
| CN113010442A (en) | User interface test result classification method and device based on deep belief network | |
| CN117194976A (en) | Touch chain construction method and device based on semi-supervised learning, electronic equipment and storage medium | |
| CN118171953A (en) | Target enterprise screening method, device, computer equipment and storage medium | |
| CN117975473A (en) | Bill text detection model training and detection method, device, equipment and medium | |
| CN117954016A (en) | Battery material determining method, apparatus, computer device, and storage medium | |
| CN116881116A (en) | Interface test method, apparatus, computer device, storage medium, and program product | |
| CN118295664A (en) | Code generation method, code generation device, computer equipment, storage medium and product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |