CN115269674A - Data analysis method and device, electronic equipment and storage medium - Google Patents
Data analysis method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115269674A CN115269674A CN202210667972.0A CN202210667972A CN115269674A CN 115269674 A CN115269674 A CN 115269674A CN 202210667972 A CN202210667972 A CN 202210667972A CN 115269674 A CN115269674 A CN 115269674A
- Authority
- CN
- China
- Prior art keywords
- data
- behavior data
- sample
- information
- abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2465—Query processing support for facilitating data mining operations in structured databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Fuzzy Systems (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
The present disclosure relates to a data analysis method, apparatus, electronic device and storage medium, including: acquiring operation behavior data of a target account in real time; inputting the role information, the authority information and the operation behavior data of the target account into an early warning model obtained through pre-training, and performing behavior analysis on the role information, the authority information and the operation behavior data to obtain a prediction result, wherein the prediction result is used for indicating abnormal operation data in the operation behavior data; and triggering early warning operation aiming at the operation behavior data under the condition that the operation behavior data is abnormal operation data. Therefore, the operation behavior data acquired in real time can be analyzed through the pre-trained early warning model, abnormal operation behaviors are identified from the operation behavior data, and early warning is given out.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a data analysis method and apparatus, an electronic device, and a storage medium.
Background
With the popularization of the internet, more and more data are maintained in a data center, the operation on the data is more and more complicated, and data security is an important part which cannot be lost in the data center. The range of data security concerns is wide, with the problems involved with human operation being the most important uncontrollable factor and the central focus on maintaining data security.
In the prior art, the operation of personnel can not be effectively monitored, so that the abnormal operation of the personnel on data is difficult to be prevented in time, the data is likely to be damaged, tampered and leaked, and the safety of the data is further reduced.
Disclosure of Invention
The present disclosure provides a to-be-processed data analysis method, apparatus, electronic device, and storage medium, to at least solve the problem that operations of personnel cannot be effectively monitored in the related art, and therefore, it is difficult to timely stop abnormal operations of the personnel on the data, which may cause damage, falsification, and leakage of the data, and further reduce the security of the data. The technical scheme of the disclosure is as follows:
according to a first aspect of the embodiments of the present disclosure, there is provided a data analysis method, including:
acquiring operation behavior data of a target account in real time;
inputting the role information, the authority information and the operation behavior data of the target account into a pre-trained early warning model, and performing behavior analysis on the role information, the authority information and the operation behavior data to obtain a prediction result, wherein the prediction result is used for indicating abnormal operation data in the operation behavior data;
and triggering early warning operation aiming at the operation behavior data under the condition that the operation behavior data is abnormal operation data.
Optionally, the obtaining operation behavior data of the target account in real time includes:
acquiring an operation log of a target account in real time;
and performing data cleaning on the operation log, and extracting operation behavior data of the target account.
Optionally, the abnormal operation data includes multiple abnormal types, and the triggering of the early warning operation for the operation behavior data includes:
acquiring an information template of a target abnormal type corresponding to the operation behavior data;
generating early warning information corresponding to the operation behavior data according to the information template;
and sending the early warning information to a client corresponding to the target abnormal type.
Optionally, the method further includes:
acquiring sample behavior data, sample role information and sample authority information of an account corresponding to the sample behavior data, wherein the sample behavior data comprises pre-marked real abnormal data;
processing the sample behavior data, the sample role information and the sample authority information by using a preset random forest model, and outputting abnormal prediction data in the sample behavior data;
and calculating a loss value of the preset random forest model according to the real abnormal data and the predicted abnormal data, adjusting model parameters of the random forest model under the condition that the loss value does not meet a preset threshold value until the loss value meets the preset threshold value, and taking the obtained random forest model as an early warning model.
Optionally, the using the obtained random forest model as an early warning model includes:
taking the obtained random forest model as a reference model, and performing characteristic importance analysis on the sample behavior data, the sample role information and the sample authority information;
screening out information with characteristic importance not being 0 from the sample behavior data, the sample role information and the sample authority information, as target characteristic information;
processing the target characteristic information by using the reference model, and outputting updated abnormal data in the sample behavior data;
and calculating a loss value of the reference model according to the real abnormal data and the updated abnormal data, adjusting model parameters of the reference model under the condition that the loss value does not meet a preset threshold value until the loss value meets the preset threshold value, and taking the obtained reference model as an early warning model.
Optionally, the method further includes:
under the condition that the operation behavior data are abnormal operation data, feedback information aiming at the operation behavior data is obtained;
and carrying out iterative adjustment on the early warning model according to the feedback information.
According to a second aspect of the embodiments of the present disclosure, there is provided a data analysis apparatus including:
the acquisition unit is configured to execute real-time acquisition of operation behavior data of the target account;
the analysis unit is configured to input the role information, the authority information and the operation behavior data of the target account into a pre-trained early warning model, and perform behavior analysis on the role information, the authority information and the operation behavior data to obtain a prediction result, wherein the prediction result is used for indicating abnormal operation data in the operation behavior data;
the early warning unit is configured to trigger early warning operation aiming at the operation behavior data when the operation behavior data is abnormal operation data.
Optionally, the obtaining unit is configured to perform:
acquiring an operation log of a target account in real time;
and performing data cleaning on the operation log, and extracting operation behavior data of the target account.
Optionally, the abnormal operation data includes a plurality of abnormal types, and the early warning unit is further configured to perform:
acquiring an information template of a target abnormal type corresponding to the operation behavior data;
generating early warning information corresponding to the operation behavior data according to the information template;
and sending the early warning information to a client corresponding to the target abnormal type.
Optionally, the apparatus further comprises a training unit configured to perform:
acquiring sample behavior data, sample role information and sample authority information of an account corresponding to the sample behavior data, wherein the sample behavior data comprises pre-marked real abnormal data;
processing the sample behavior data, the sample role information and the sample authority information by using a preset random forest model, and outputting predicted abnormal data in the sample behavior data;
and calculating a loss value of the preset random forest model according to the real abnormal data and the predicted abnormal data, adjusting model parameters of the random forest model under the condition that the loss value does not meet a preset threshold value until the loss value meets the preset threshold value, and taking the obtained random forest model as an early warning model.
Optionally, the training unit is further configured to perform:
taking the obtained random forest model as a reference model, and performing characteristic importance analysis on the sample behavior data, the sample role information and the sample authority information;
screening out information with characteristic importance not being 0 from the sample behavior data, the sample role information and the sample authority information as target characteristic information;
processing the target characteristic information by using the reference model, and outputting updated abnormal data in the sample behavior data;
and calculating a loss value of the reference model according to the real abnormal data and the updated abnormal data, adjusting model parameters of the reference model under the condition that the loss value does not meet a preset threshold value until the loss value meets the preset threshold value, and taking the obtained reference model as an early warning model.
Optionally, the apparatus further includes an optimization unit configured to perform:
under the condition that the operation behavior data are abnormal operation data, feedback information aiming at the operation behavior data is obtained;
and carrying out iterative adjustment on the early warning model according to the feedback information.
According to a third aspect of embodiments of the present disclosure, there is provided a data analysis electronic device including:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement any of the data analysis methods described above.
According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium having instructions which, when executed by a processor of a data analysis electronic device, enable the data analysis electronic device to perform any one of the data analysis methods described above.
According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product comprising computer programs/instructions which, when executed by a processor, implement the data analysis method of any one of the above.
The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:
acquiring operation behavior data of a target account in real time; inputting the role information, the authority information and the operation behavior data of the target account into an early warning model obtained by pre-training, and performing behavior analysis on the role information, the authority information and the operation behavior data to obtain a prediction result, wherein the prediction result is used for indicating abnormal operation data in the operation behavior data; and triggering early warning operation aiming at the operation behavior data under the condition that the operation behavior data is abnormal operation data.
Therefore, the operation behavior data acquired in real time can be analyzed through the pre-trained early warning model, abnormal operation behaviors are identified from the operation behavior data, and early warning is given out.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.
FIG. 1 is a flow chart illustrating a method of data analysis in accordance with an exemplary embodiment.
FIG. 2 is a logical schematic diagram illustrating a method of data analysis in accordance with an exemplary embodiment.
FIG. 3 is a block diagram illustrating a data analysis device according to an exemplary embodiment.
FIG. 4 is a block diagram illustrating an electronic device for data analysis in accordance with an exemplary embodiment.
FIG. 5 is a block diagram illustrating an apparatus for data analysis according to an example embodiment.
Detailed Description
In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in other sequences than those illustrated or described herein. The implementations described in the exemplary embodiments below do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
Fig. 1 is a flow chart illustrating a data analysis method according to an exemplary embodiment, which includes the following steps, as shown in fig. 1.
In step S11, the operation behavior data of the target account is acquired in real time.
In this step, the obtaining operation behavior data of the target account in real time may include: acquiring an operation log of a target account in real time; and performing data cleaning on the operation log, and extracting operation behavior data of the target account. That is to say, the operation behavior data of the target account can be extracted from the operation log of the target account, and effective data can be extracted by cleaning the data, so that the subsequent data processing efficiency is improved.
The operation behavior data may include, but is not limited to: the contents of the library name, the table name, the field, the operation type, etc. are not limited specifically. Specifically, the operation log of the target account may be obtained by deploying a Java program in a log server and collecting the operation log in real time by using the Java program.
In one implementation, the data cleaning is performed on the operation log, and the operations include mapping chinese and english to numerical values, processing missing values, extracting the number of tags, determining whether the data is abnormal, and performing numerical value conversion on the tag values, and the like, which are not limited specifically.
In step S12, the role information, the authority information, and the operation behavior data of the target account are input to the pre-trained early warning model, and the role information, the authority information, and the operation behavior data are subjected to behavior analysis to obtain a prediction result, where the prediction result is used to indicate abnormal operation data in the operation behavior data.
The role information includes, but is not limited to: the field importance level, the database importance level, the data table importance level, the user role data and the like, and the authority information includes but is not limited to operation authority data and the like.
The role information and the authority information of the target account may be acquired from a preset database according to the identification information of the target account after the operation behavior data is acquired, or may be acquired from a preset database in advance, and after the operation behavior data is acquired, the role information and the authority information of a plurality of accounts are queried according to the identification information of the target account, which is not limited specifically. The preset database can be a multi-source database such as MySQL, oracle, hbase and the like.
In the present disclosure, the method for training the early warning model may include the following steps:
firstly, sample behavior data, sample role information of an account corresponding to the sample behavior data and sample authority information are obtained, wherein the sample behavior data comprises pre-marked real abnormal data. The sample behavior data, the sample role information of the account corresponding to the sample behavior data and the sample authority information can be obtained by cleaning an operation log obtained from a preset database. In one implementation, the sample behavior data can be divided into a training set and a test set according to the proportion of 7:3, and the training set and the test set are respectively used for training and testing the early warning model.
And then, processing the sample behavior data, the sample role information and the sample authority information by using a preset random forest model, and outputting the abnormal prediction data in the sample behavior data. The optimal parameters of the model can be obtained by using GridSearchCV (grid search method model parameter adjusting and interest benefiting), so that the early warning model is constructed by using the optimal parameters. In addition, in the embodiment of the invention, besides using the random forest model, the early warning model can be obtained by utilizing model training of XGboost, lightGBM and the like, and the method is not limited specifically.
And then calculating a loss value of the preset random forest model according to the real abnormal data and the predicted abnormal data, adjusting model parameters of the random forest model under the condition that the loss value does not meet a preset threshold value until the loss value meets the preset threshold value, and taking the obtained random forest model as an early warning model.
In one implementation, the method for using the obtained random forest model as an early warning model includes:
taking the obtained random forest model as a reference model, and performing characteristic importance analysis on the sample behavior data, the sample role information and the sample authority information; screening out information with the characteristic importance not being 0 from the sample behavior data, the sample role information and the sample authority information as target characteristic information; processing the target characteristic information by using a reference model, and outputting updated abnormal data in the sample behavior data; and calculating a loss value of the reference model according to the real abnormal data and the updated abnormal data, adjusting model parameters of the reference model under the condition that the loss value does not meet a preset threshold value until the loss value meets the preset threshold value, and taking the obtained reference model as an early warning model.
That is, after obtaining the reference Model, performing importance analysis on the features used for training the reference Model, outputting the importance degree of each feature, then removing the features with the importance degree of 0, and then using the other features after removing the features to retrain the random forest Model, wherein the trained early warning Model can be saved into a pmml (Predictive Model Markup Language, standard Language for describing and storing data mining models by using XML) file by using a skearn 2pmml library.
It can be understood that the feature with the importance degree of 0 makes a small contribution to the prediction result of the random forest model, so that the input features of the early warning model can be simplified by screening the features on the premise of not influencing the output result of the model, and the calculation efficiency of the early warning model is further improved.
In the embodiment of the invention, the Flink streaming media can be used for loading the early warning model, inputting the role information, the authority information and the operation behavior data of the target account into the early warning model obtained by pre-training, and performing behavior analysis on the role information, the authority information and the operation behavior data to obtain the prediction result.
That is, the operation behavior data of the target account may be sent to a Kafka queue, and then a pmml model file of the early warning model is loaded into a Flink Streaming media using a jpmml tool, and then the Flink Streaming media is used to perform subsequent processing on the operation behavior data in the Kafka. It can be understood that the Flink supports the storage of the stream data to the external system in a file form, and therefore, the processing of the operation behavior data of the target account in real time can be realized.
In step S13, if the operation behavior data is abnormal operation data, an early warning operation for the operation behavior data is triggered.
In this step, when the operation behavior data is abnormal operation data, an early warning operation for the operation behavior data is triggered, and on the other hand, when the operation behavior data is not abnormal operation data, the operation behavior data may not be processed next, or a confirmation operation for the operation behavior data may also be triggered, so as to prompt a user that the current operation behavior data is not abnormal data, and the specific limitation is not made.
Wherein, unusual operation data include multiple unusual type, trigger the early warning operation to operation behavior data, include: acquiring an information template of a target exception type corresponding to the operation behavior data; generating early warning information corresponding to the operation behavior data according to the information template; and sending the early warning information to a client corresponding to the target abnormal type.
That is to say, different information templates can be preset for different types of abnormal operations, so that when the operation behavior data are abnormal operation data, a short message notification corresponding to the abnormal type can be generated according to the information templates, relevant personnel can be informed of the short message notification to respond in time, occurrence of data safety accidents can be found and blocked in time, and an immeasurable effect is brought to information and data safety of enterprises. The information template may be loaded in advance when the FlinkStreaming is initialized, or may be acquired from a preset database after the current operation behavior data is confirmed to be abnormal, which is not limited specifically.
In one implementation, in the case that the operation behavior data is abnormal operation data, feedback information for the operation behavior data may be acquired; and carrying out iterative adjustment on the early warning model according to the feedback information. Therefore, the modeling input data is adjusted according to the abnormal recognition feedback effect, and the iterative optimization is carried out on the prediction model. Therefore, the prediction accuracy of the early warning model can be continuously improved, and the data safety is further improved.
Fig. 2 is a logic diagram of the data analysis method according to the present embodiment. Firstly, the log, the role and the authority data, that is, the sample behavior data, the sample role information and the sample authority information of the account corresponding to the sample behavior data, can be extracted, the data are fused, the obtained data are called sample data, and an abnormal identifier needs to be marked on abnormal data in the sample data. Then, a random forest model is constructed, the trained model is stored into a pmml file, and the pmml file is loaded into FlinkStreaming. And performing real-time prediction by FlinkStreaming, and determining whether the operation behavior data is abnormal operation data in real time. Under the condition that the operation behavior data is abnormal operation data, the operation behavior data can be notified in real time through FlinkStreaming, namely, early warning operation aiming at the operation behavior data is triggered. When the operation behavior data is abnormal operation data, the operation behavior data is discarded and no processing is performed. Meanwhile, feedback and optimization of the abnormal recognition effect can be carried out, namely feedback information is obtained, and iterative adjustment is carried out on the early warning model according to the feedback information.
As can be seen from the above, according to the technical scheme provided by the embodiment of the disclosure, the operation behavior data obtained in real time can be analyzed through the pre-trained early warning model, so that the abnormal operation behavior is identified, and an early warning is sent out.
FIG. 3 is a block diagram illustrating a data analysis apparatus according to an exemplary embodiment, the apparatus comprising:
an acquisition unit 201 configured to perform real-time acquisition of operation behavior data of a target account;
the analysis unit 202 is configured to perform input of role information, authority information and the operation behavior data of the target account into a pre-trained early warning model, perform behavior analysis on the role information, the authority information and the operation behavior data, and obtain a prediction result, where the prediction result is used for indicating abnormal operation data in the operation behavior data;
an early warning unit 203 configured to trigger an early warning operation for the operation behavior data if the operation behavior data is abnormal operation data.
In one implementation, the obtaining unit 201 is configured to perform:
acquiring an operation log of a target account in real time;
and performing data cleaning on the operation log, and extracting operation behavior data of the target account.
In one implementation, the abnormal operation data includes a plurality of abnormal types, and the early warning unit 203 is further configured to perform:
acquiring an information template of a target abnormal type corresponding to the operation behavior data;
generating early warning information corresponding to the operation behavior data according to the information template;
and sending the early warning information to a client corresponding to the target abnormal type.
In one implementation, the apparatus further includes a training unit configured to perform:
acquiring sample behavior data, sample role information and sample authority information of an account corresponding to the sample behavior data, wherein the sample behavior data comprises pre-marked real abnormal data;
processing the sample behavior data, the sample role information and the sample authority information by using a preset random forest model, and outputting predicted abnormal data in the sample behavior data;
and calculating a loss value of the preset random forest model according to the real abnormal data and the predicted abnormal data, adjusting model parameters of the random forest model under the condition that the loss value does not meet a preset threshold value until the loss value meets the preset threshold value, and taking the obtained random forest model as an early warning model.
In one implementation, the training unit is further configured to perform:
taking the obtained random forest model as a reference model, and performing characteristic importance analysis on the sample behavior data, the sample role information and the sample authority information;
screening out information with characteristic importance not being 0 from the sample behavior data, the sample role information and the sample authority information as target characteristic information;
processing the target characteristic information by using the reference model, and outputting updated abnormal data in the sample behavior data;
and calculating a loss value of the reference model according to the real abnormal data and the updated abnormal data, adjusting model parameters of the reference model under the condition that the loss value does not meet a preset threshold value until the loss value meets the preset threshold value, and taking the obtained reference model as an early warning model.
In one implementation, the apparatus further includes an optimization unit configured to perform:
under the condition that the operation behavior data are abnormal operation data, feedback information aiming at the operation behavior data is obtained;
and carrying out iterative adjustment on the early warning model according to the feedback information.
As can be seen from the above, according to the technical scheme provided by the embodiment of the disclosure, the operation behavior data obtained in real time can be analyzed through the pre-trained early warning model, so that the abnormal operation behavior can be identified, and an early warning can be sent.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
FIG. 4 is a block diagram illustrating an electronic device for data analysis including a processor and a memory for storing a computer program in accordance with an exemplary embodiment; the processor is used for executing the program stored in the memory.
The Memory may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
In an exemplary embodiment, a computer-readable storage medium comprising instructions, such as a memory comprising instructions, executable by a processor of an electronic device to perform the above-described method is also provided. Alternatively, the computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
In an exemplary embodiment, a computer program product is also provided, which, when run on a computer, causes the computer to implement the above-described method of analysis of data to be processed.
As can be seen from the above, according to the technical scheme provided by the embodiment of the disclosure, the operation behavior data obtained in real time can be analyzed through the pre-trained early warning model, so that the abnormal operation behavior can be identified, and an early warning can be sent.
Fig. 5 is a block diagram illustrating an apparatus 800 for data analysis according to an example embodiment.
For example, the apparatus 800 may be a mobile phone, a computer, a digital broadcast electronic device, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.
Referring to fig. 5, the apparatus 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814, and a communication component 816.
The processing component 802 generally controls overall operation of the device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.
The memory 804 is configured to store various types of data to support operation at the device 800. Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
Power supply components 807 provide power to the various components of device 800. The power components 807 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the apparatus 800.
The multimedia component 808 includes a screen that provides an output interface between the device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front-facing camera and/or the rear-facing camera may receive external multimedia data to be processed when the device 800 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the apparatus 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.
The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the device 800. For example, the sensor assembly 814 may detect the open/closed state of the device 800, the relative positioning of the components, such as a display and keypad of the apparatus 800, the sensor assembly 814 may also detect a change in position of the apparatus 800 or a component of the apparatus 800, the presence or absence of user contact with the apparatus 800, orientation or acceleration/deceleration of the apparatus 800, and a change in temperature of the apparatus 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 816 is configured to facilitate communications between the apparatus 800 and other devices in a wired or wireless manner. The apparatus 800 may access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G, or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the apparatus 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the methods of the first and second aspects.
In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 804 comprising instructions, executable by the processor 820 of the device 800 to perform the above-described method is also provided. Alternatively, for example, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
In an exemplary embodiment, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the data analysis method of any of the above embodiments.
As can be seen from the above, according to the technical scheme provided by the embodiment of the disclosure, the operation behavior data obtained in real time can be analyzed through the pre-trained early warning model, so that the abnormal operation behavior can be identified, and an early warning can be sent.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (14)
1. A method of data analysis, comprising:
acquiring operation behavior data of a target account in real time;
inputting the role information, the authority information and the operation behavior data of the target account into a pre-trained early warning model, and performing behavior analysis on the role information, the authority information and the operation behavior data to obtain a prediction result, wherein the prediction result is used for indicating abnormal operation data in the operation behavior data;
and triggering early warning operation aiming at the operation behavior data under the condition that the operation behavior data is abnormal operation data.
2. The data analysis method of claim 1, wherein the obtaining operation behavior data of the target account in real time comprises:
acquiring an operation log of a target account in real time;
and performing data cleaning on the operation log, and extracting operation behavior data of the target account.
3. The data analysis method of claim 1, wherein the abnormal operation data comprises a plurality of abnormal types, and the triggering of the early warning operation for the operation behavior data comprises:
acquiring an information template of a target abnormal type corresponding to the operation behavior data;
generating early warning information corresponding to the operation behavior data according to the information template;
and sending the early warning information to a client corresponding to the target abnormal type.
4. The data analysis method of claim 1, further comprising:
acquiring sample behavior data, sample role information and sample authority information of an account corresponding to the sample behavior data, wherein the sample behavior data comprises pre-marked real abnormal data;
processing the sample behavior data, the sample role information and the sample authority information by using a preset random forest model, and outputting predicted abnormal data in the sample behavior data;
and calculating a loss value of the preset random forest model according to the real abnormal data and the predicted abnormal data, adjusting model parameters of the random forest model under the condition that the loss value does not meet a preset threshold value until the loss value meets the preset threshold value, and taking the obtained random forest model as an early warning model.
5. The data analysis method of claim 4, wherein the using the obtained random forest model as an early warning model comprises:
taking the obtained random forest model as a reference model, and performing characteristic importance analysis on the sample behavior data, the sample role information and the sample authority information;
screening out information with characteristic importance not being 0 from the sample behavior data, the sample role information and the sample authority information as target characteristic information;
processing the target characteristic information by using the reference model, and outputting updated abnormal data in the sample behavior data;
and calculating a loss value of the reference model according to the real abnormal data and the updated abnormal data, adjusting model parameters of the reference model under the condition that the loss value does not meet a preset threshold value until the loss value meets the preset threshold value, and taking the obtained reference model as an early warning model.
6. The data analysis method of claim 1, further comprising:
under the condition that the operation behavior data are abnormal operation data, feedback information aiming at the operation behavior data is obtained;
and carrying out iterative adjustment on the early warning model according to the feedback information.
7. A data analysis apparatus, comprising:
the acquisition unit is configured to execute real-time acquisition of operation behavior data of the target account;
the analysis unit is configured to input the role information, the authority information and the operation behavior data of the target account into a pre-trained early warning model, and perform behavior analysis on the role information, the authority information and the operation behavior data to obtain a prediction result, wherein the prediction result is used for indicating abnormal operation data in the operation behavior data;
the early warning unit is configured to trigger early warning operation aiming at the operation behavior data when the operation behavior data is abnormal operation data.
8. The data analysis apparatus according to claim 7, wherein the acquisition unit is configured to perform:
acquiring an operation log of a target account in real time;
and performing data cleaning on the operation log, and extracting operation behavior data of the target account.
9. The data analysis device of claim 7, wherein the abnormal operation data comprises a plurality of abnormal types, and the early warning unit is further configured to perform:
acquiring an information template of a target abnormal type corresponding to the operation behavior data;
generating early warning information corresponding to the operation behavior data according to the information template;
and sending the early warning information to a client corresponding to the target abnormal type.
10. The data analysis apparatus of claim 7, wherein the apparatus further comprises a training unit configured to perform:
acquiring sample behavior data, sample role information and sample authority information of an account corresponding to the sample behavior data, wherein the sample behavior data comprises pre-marked real abnormal data;
processing the sample behavior data, the sample role information and the sample authority information by using a preset random forest model, and outputting predicted abnormal data in the sample behavior data;
and calculating a loss value of the preset random forest model according to the real abnormal data and the predicted abnormal data, adjusting model parameters of the random forest model under the condition that the loss value does not meet a preset threshold value until the loss value meets the preset threshold value, and taking the obtained random forest model as an early warning model.
11. The data analysis device of claim 10, wherein the training unit is further configured to perform:
taking the obtained random forest model as a reference model, and performing characteristic importance analysis on the sample behavior data, the sample role information and the sample authority information;
screening out information with characteristic importance not being 0 from the sample behavior data, the sample role information and the sample authority information as target characteristic information;
processing the target characteristic information by using the reference model, and outputting updated abnormal data in the sample behavior data;
and calculating a loss value of the reference model according to the real abnormal data and the updated abnormal data, adjusting model parameters of the reference model under the condition that the loss value does not meet a preset threshold value until the loss value meets the preset threshold value, and taking the obtained reference model as an early warning model.
12. The data analysis apparatus of claim 7, wherein the apparatus further comprises an optimization unit configured to perform:
under the condition that the operation behavior data are abnormal operation data, feedback information aiming at the operation behavior data is obtained;
and carrying out iterative adjustment on the early warning model according to the feedback information.
13. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the data analysis method of any one of claims 1 to 6.
14. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of a data analysis electronic device, enable the data analysis electronic device to perform the data analysis method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210667972.0A CN115269674A (en) | 2022-06-14 | 2022-06-14 | Data analysis method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210667972.0A CN115269674A (en) | 2022-06-14 | 2022-06-14 | Data analysis method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115269674A true CN115269674A (en) | 2022-11-01 |
Family
ID=83760352
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210667972.0A Pending CN115269674A (en) | 2022-06-14 | 2022-06-14 | Data analysis method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115269674A (en) |
-
2022
- 2022-06-14 CN CN202210667972.0A patent/CN115269674A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9691256B2 (en) | Method and device for presenting prompt information that recommends removing contents from garbage container | |
| CN109446994B (en) | Gesture key point detection method and device, electronic equipment and storage medium | |
| CN111274426A (en) | Category labeling method and device, electronic equipment and storage medium | |
| CN109255128B (en) | Multi-level label generation method, device and storage medium | |
| CN106202150A (en) | Method for information display and device | |
| CN109842612B (en) | Log security analysis method and device based on graph library model and storage medium | |
| CN111813932B (en) | Text data processing method, text data classifying device and readable storage medium | |
| CN110738267B (en) | Image classification method, device, electronic equipment and storage medium | |
| CN110941727B (en) | Resource recommendation method and device, electronic equipment and storage medium | |
| CN115310093A (en) | Vulnerability detection method and device based on code slicing and storage medium | |
| CN112256563A (en) | Android application stability testing method and device, electronic equipment and storage medium | |
| CN111046927A (en) | Method and device for processing labeled data, electronic equipment and storage medium | |
| CN116069612A (en) | Abnormality positioning method and device and electronic equipment | |
| CN111428806B (en) | Image tag determining method and device, electronic equipment and storage medium | |
| CN115827398B (en) | Method and device for calculating component value of alarm information, electronic equipment and storage medium | |
| CN111382061B (en) | Test method, test device, test medium and electronic equipment | |
| CN110213062A (en) | Handle the method and device of message | |
| CN116032782A (en) | Fault detection method, device and storage medium | |
| CN115269674A (en) | Data analysis method and device, electronic equipment and storage medium | |
| CN112333233B (en) | Event information reporting method and device, electronic equipment and storage medium | |
| CN114896165A (en) | Testing method and device of conversation robot system, electronic equipment and storage medium | |
| CN110929055B (en) | Multimedia quality detection method and device, electronic equipment and storage medium | |
| CN114338587B (en) | Multimedia data processing method and device, electronic equipment and storage medium | |
| CN113760946A (en) | Pre-verification processing method, device, equipment and medium applied to data source migration | |
| CN115225702B (en) | Information pushing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |