CN110287241A - A kind of method and device generating alarm data report - Google Patents
A kind of method and device generating alarm data report Download PDFInfo
- Publication number
- CN110287241A CN110287241A CN201910569785.7A CN201910569785A CN110287241A CN 110287241 A CN110287241 A CN 110287241A CN 201910569785 A CN201910569785 A CN 201910569785A CN 110287241 A CN110287241 A CN 110287241A
- Authority
- CN
- China
- Prior art keywords
- data
- block
- analysis task
- alarm
- alert analysis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Alarm Systems (AREA)
- Maintenance And Management Of Digital Transmission (AREA)
Abstract
The embodiment of the invention provides a kind of method and devices for generating alarm data report, it is related to financial technology technical field, this method comprises: constructing data frame using alarm data, for each alert analysis task, the corresponding local data's block of alert analysis task is extracted from data frame, then for statistical analysis to local data block, determine the result data block of alert analysis task.The result data block of each alert analysis task is carried out to splice determining alarm data report, therefore all alarm datas to be analyzed only generate an alarm data report, the alarm data report includes the content of all alert analysis tasks, is convenient for report management and user query report.Secondly, handling alarm data as a data frame, the corresponding categorical data of report is generated, the efficiency for generating alarm data report can be improved.By carrying out the operation of row and column to big data frame, the update to alarm data report is realized, so that the management service of alarm data report is more convenient.
Description
Technical field
The present embodiments relate to financial technology (Fintech) technical field more particularly to a kind of generation alarm data reports
The method and device of table.
Background technique
With the development of computer technology, more and more technical applications are in financial field, and traditional financial industry is gradually
Change to financial technology (Fintech), but due to the safety of financial industry, requirement of real-time, also technology is proposed higher
Requirement.Currently, the performance indicator of each machine and the health status of service software carry out alarm point in financial system
When analysis, the initial data of acquisition is stored in big data platform fairground after being handled.Due to the data set in big data platform
In city, the Data source table being related to can be very much, therefore use sql (Structured Query Language, structured query language)
When sentence extracts data from Data Mart different zones, the sql sentence of configuration report different zones access is especially complex, is not easy to
Data query.
Summary of the invention
There are many Data source table as involved in big data fairground, and query statement needs to extract data from different zones, lead
The problem of causing query statement complicated, being not easy to data query, the embodiment of the invention provides a kind of generation alarm data reports
Method and device.
On the one hand, the embodiment of the invention provides a kind of methods for generating alarm data report, comprising:
It obtains alarm data to be analyzed and data frame is constructed using the alarm data；
For each alert analysis task, the corresponding local data of the alert analysis task is extracted from the data frame
Block；
It is for statistical analysis to the corresponding local data's block of the alert analysis task, determine the alert analysis task
Result data block；
The corresponding result data block of each alert analysis task is spliced, determines alarm data report.
Optionally, described to splice the corresponding result data block of each alert analysis task, determine alarm data report
Table, comprising:
The corresponding result data block of each alert analysis task is added in corresponding data frame group；
All result data blocks in each data frame group are spliced, determine the corresponding knot of each alert analysis task
Fruit data frame；
The corresponding result data frame of each alert analysis task is spliced, determines alarm data report.
Optionally, described for statistical analysis to the corresponding local data's block of the alert analysis task, determine the announcement
The result data block of alert analysis task, comprising:
The corresponding local data's block of the alert analysis task is grouped and is counted, multiple statistical result blocks are obtained；
The multiple statistical result block is analyzed and sorted, and according to the analysis result generation for coming top N
The result data block of alert analysis task, N are default integer.
Optionally, the data frame is the two dimensional data structure for including row and column.
Optionally, described to be directed to each alert analysis task, from the alarm data corresponding data frame described in extraction
Before the corresponding local data's block of alert analysis task, further includes:
Data characteristics string is extracted from the mixed characters string of the data frame using regular expression set, and described in use
Data characteristics string replaces the mixed characters string of the data frame；
Type conversion is carried out to the data frame and to the missing values assignment of the data frame.
On the one hand, the embodiment of the invention provides a kind of devices for generating alarm data report, comprising:
Module is obtained, for obtaining alarm data to be analyzed and constructing data frame using the alarm data；
Abstraction module extracts the alert analysis task for being directed to each alert analysis task from the data frame
Corresponding local data's block；
Analysis module, for for statistical analysis to the corresponding local data's block of the alert analysis task, described in determination
The result data block of alert analysis task；
Splicing module determines alarm data for splicing the corresponding result data block of each alert analysis task
Report.
Optionally, the splicing module is specifically used for:
The corresponding result data block of each alert analysis task is added in corresponding data frame group；
All result data blocks in each data frame group are spliced, determine the corresponding knot of each alert analysis task
Fruit data frame；
The corresponding result data frame of each alert analysis task is spliced, determines alarm data report.
Optionally, the analysis module is specifically used for:
The corresponding local data's block of the alert analysis task is grouped and is counted, multiple statistical result blocks are obtained；
The multiple statistical result block is analyzed and sorted, and according to the analysis result generation for coming top N
The result data block of alert analysis task, N are default integer.
Optionally, the data frame is the two dimensional data structure for including row and column.
It optionally, further include preprocessing module；
The preprocessing module is specifically used for:
For each alert analysis task, the alert analysis task is extracted from the corresponding data frame of the alarm data
Before corresponding local data's block, data characteristics is extracted from the mixed characters string of the data frame using regular expression set
It goes here and there, and replaces the mixed characters string of the data frame using the data characteristics string；
Type conversion is carried out to the data frame and to the missing values assignment of the data frame.
On the one hand, the embodiment of the invention provides a kind of computer equipment, including memory, processor and it is stored in storage
On device and the computer program that can run on a processor, the processor is realized when executing described program generates alarm data report
The step of method of table.
On the one hand, the embodiment of the invention provides a kind of computer readable storage medium, being stored with can be set by computer
The standby computer program executed, when described program is run on a computing device, so that the computer equipment executes generation
The step of method of alarm data report.
In the embodiment of the present invention, obtains alarm data to be analyzed and data frame is constructed using alarm data.For each
Alert analysis task extracts the corresponding local data's block of alert analysis task, then to alert analysis task pair from data frame
The local data's block answered is for statistical analysis, determines the result data block of alert analysis task, later again by each alert analysis
The corresponding result data block of task is spliced, and determines alarm data report.Therefore all alarm datas to be analyzed only generate one
Alarm data report is opened, which includes the content of all alert analysis tasks, is convenient for report management and user
Inquire report.Secondly, handling alarm data as a data frame, the corresponding categorical data of report is generated, generation can be improved and accused
The efficiency of alert data sheet.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly introduced, it should be apparent that, drawings in the following description are only some embodiments of the invention, for this
For the those of ordinary skill in field, without any creative labor, it can also be obtained according to these attached drawings
His attached drawing.
Fig. 1 is a kind of application scenarios schematic diagram provided in an embodiment of the present invention；
Fig. 2 is a kind of flow diagram of method for generating alarm data report provided in an embodiment of the present invention；
Fig. 3 is a kind of flow diagram of method for generating alarm data report provided in an embodiment of the present invention；
Fig. 4 is a kind of structural schematic diagram of device for generating alarm data report provided in an embodiment of the present invention；
Fig. 5 is a kind of structural schematic diagram of computer equipment provided in an embodiment of the present invention.
Specific embodiment
In order to which the purpose of the present invention, technical solution and beneficial effect is more clearly understood, below in conjunction with attached drawing and implementation
Example, the present invention will be described in further detail.It should be appreciated that specific embodiment described herein is only used to explain this hair
It is bright, it is not intended to limit the present invention.
In order to facilitate understanding, noun involved in the embodiment of the present invention is explained below.
Pandas: one python data analysis bag provides tool needed for efficiently operating large data collection.
Mysql: one Relational DBMS.
CMDB:Configuration Management Database, configuration management database.
The method of generation alarm data report in the embodiment of the present invention can be applied to application scenarios as shown in Figure 1,
It include big data synchronization system 101, source database 102, report generating system 103, report database in the application scenarios
104.Big data synchronization system 101 acquires alarm data from data source and source database 102 is written in alarm data, and data source can
To be the operation system of bank and other financial mechanism.Report generating system 103 generates alarm based on the alarm data in source database
Data sheet.Specifically, report generating system 103 loads alarm data to be analyzed from source database and using alarm data
Construct data frame.For each alert analysis task, the corresponding local data's block of alert analysis task is extracted from data frame, so
It is for statistical analysis to the corresponding local data's block of alert analysis task afterwards, determine the result data block of alert analysis task, it will
The corresponding result data block of each alert analysis task is spliced, and determines alarm data report.Later again by alarm data report
Table is stored in report database 104.
Based on application scenario diagram shown in FIG. 1, the embodiment of the invention provides a kind of methods for generating alarm data report
Process, the process of this method can execute by the device of generation alarm data report, and the device for generating alarm data report can
To be report generating system 103 shown in FIG. 1, as shown in Figure 2, comprising the following steps:
Step S201 obtains alarm data to be analyzed and constructs data frame using alarm data.
Specifically, big data synchronization system downloads alarm data from data source and alarm data is stored in source database
In.For the ease of analyzing alarm data, the alarm data in source database can be updated according to analytical cycle, for example,
When analytical cycle is one month, it can set and regularly update source database every month, when analytical cycle is one week, can set every
Week regularly updates source database.When analytical cycle is one, it can set and regularly update source database daily.
Report generating system loads alarm data to be analyzed from source database, and data frame is the two dimension for including row and column
Data structure can carry out operation to the row and column of data frame, can also be with the row and column of growth data frame.In specific implementation, make
It is loaded into memory with the alarm data that pandas interface is analysed to from source database, then by alarm data with the table of row and column
Mode is arranged in two dimensional data structure, to construct pandas data frame.Since load data volume is big, therefore can screen first
Feature field is only loaded into data and analyzes required field, reduces load data scale；Secondly, can be by multi-process/multithreading simultaneously
Row loading technique accelerates the speed of load alarm data；Finally, being loaded into the alarm data compressed storage of memory, connect with pandas
Alarm data is configured to data frame by mouth, and it is time-consuming to reduce load data.
Step S202 extracts the corresponding local number of alert analysis task for each alert analysis task from data frame
According to block.
Specifically, alert analysis task can be the analysis task for different field, such as analysis loan field, analysis
Deposit field, analysis clearing field etc..Alert analysis task is also possible to the analysis task for different levels, for example analysis is accused
Warn the time etc. that content, the machine that analysis warning content takes place frequently, analysis warning content take place frequently.Part is extracted by column from data frame
Data block, local data's block of extraction correspond to the same alert analysis task.In specific implementation, it can be matched in advance by parametrization
The local data's block for configuring each alert analysis task is set, local data's block of each alert analysis task can be not identical
's.
Step S203, it is for statistical analysis to the corresponding local data's block of alert analysis task, determine alert analysis task
Result data block.
In a kind of possible embodiment, the corresponding local data's block of alert analysis task is grouped and is counted,
Obtain multiple statistical result blocks；Multiple statistical result blocks are analyzed and sorted, are generated according to the analysis result for coming top N
The result data block of the alert analysis task, N are default integer.
In specific implementation, statistics can be grouped to local data block using pivot table, be calculated and be grouped using crosstab
Frequency, pivot table and crosstab are that two kinds of modes of operation of data block were not only related but also were had any different, connection is that crosstab is one
The special pivot table of kind, can be replaced with pivot table.Difference is that crosstab is exclusively used in calculating grouped frequency, and pivot table is a kind of
It is grouped the function of statistics, passes through parameter designated statistics type.
After analyzing statistical result block, analysis result block can be generated, wherein the corresponding new column of statistical result block,
The corresponding new column of result block are analyzed, used column and new column splice when by necessary grouping, obtain result data block.Row
It can also sort from low to high according to sorting from high to low when sequence.
Illustratively, local data block is grouped according to the time, local data's block in same analysis period is divided into
One group, obtain multiple packet data blocks.It is then based on alarm header field and counts identical warning content in each packet data block
Frequency of occurrence, then newly add save each analysis period in each warning content frequency of occurrence.By different analytical cycles
Corresponding new column are horizontally-spliced, count the increment of each warning content by spliced column, increase ratio.Then it is corresponding to increase increment newly
New column increase than corresponding new column and horizontally-spliced, and incrementally sequence from high to low is ranked up the incremental value in new column,
Take the top N incremental value in new column.The increasing ratio in new column is ranked up than sequence from high to low according to increasing, is taken in new column
Top N increase ratio, later again based on new column-generation result data block and the type of annotation results data block.
It should be noted that when needing the analysis result for continuing to top N to continue statistical analysis, for example, obtaining
In the case where the increasing ratio for knowing the quantity of warning content, the increment of warning content, warning content, need to position the machine that alarm takes place frequently
Device, alarm take place frequently date, alarm take place frequently object when, can the analysis result based on top N extracted from data frame again
Then new local data's block is grouped, counts, analyzes and sorts to new local data's block, take new point of M before coming
Analysis is as a result, generate new result data block, the scale of new result data block is N*M, wherein N can be equal to M, can also not
Equal to M.
The corresponding result data block of each alert analysis task is spliced, determines alarm data report by step S204.
The object types such as result data block and text, table, figure in data sheet are corresponding, each result data block pair
Answer unique identification.After the corresponding result data block of each alert analysis task is spliced, a big data frame can be obtained.
CMDB is accessed by http (s) interface and configures system, and the data item in big data frame is updated.It will using pandas interface
Big data frame is converted into alarm data report, and alarm data report is written in report database.By to big data frame into
The operation of row row and column can update big data frame, when newly-increased alert analysis task, increase newly in big data frame different types of
Result data block, and then field is newly added in big data frame, so that the storage organization of big data frame be made to change.
In the embodiment of the present invention, using alarm data construct data frame, then extract data frame in local data's block into
Row alert analysis obtains the result data block of alert analysis task, later again by the corresponding number of results of each alert analysis task
Spliced according to block, determines alarm data report, therefore all alarm datas to be analyzed only generate an alarm data report, it should
Alarm data report includes the content of all alert analysis tasks, is convenient for report management and user query report.Secondly, with number
According to the formal layout alarm data of frame, the corresponding categorical data of report is generated, the efficiency for generating alarm data report can be improved.It is logical
The operation for carrying out row and column to big data frame is crossed, the update to alarm data report is realized, so that the management of alarm data report
Maintenance is more convenient.
Optionally, in above-mentioned steps S204, when generating alarm data report, the embodiment of the present invention at least provides following
Two kinds of embodiments:
In a kind of possible embodiment, after the corresponding result data block marking types of each alert analysis task,
It is added in global data frame group.Then all result data blocks in global data frame group are spliced, obtains big data
Frame converts alarm data report for big data frame again later.
In specific implementation, all result data blocks in global data frame group are subjected to longitudinal spliced, each result data
Block uses type column field identification type, automatic extending transversely when type column field difference, while filling out to missing values
It fills, forms big data frame.The line number amount of the big data frame of acquisition is the sum of the line number of all result data blocks, and number of columns is all
The union of the column of result data block, that is, the identical column of result data block share when splicing.
In a kind of possible embodiment, each alert analysis task can correspond to a data frame group, the data frame
Group is generating the corresponding result data block of each alert analysis task for saving the result data block of alert analysis task
Afterwards, the corresponding result data block of each alert analysis task can be added in corresponding data frame group；Then by every number
Spliced according to all result data blocks in frame group, determines the corresponding result data frame of each alert analysis task；It will be each
The corresponding result data frame of alert analysis task is spliced, and determines alarm data report.
In specific implementation, all result data blocks in data frame group are carried out to longitudinal spliced, acquisition result data frame, it is vertical
Process to splicing is identical as the process in aforementioned embodiments, and details are not described herein again.Then all result data frames are carried out
It is longitudinal spliced, big data frame is obtained, converts alarm data report for big data frame again later.By the way that different data frames is arranged
Group saves the result data block of different alert analysis tasks respectively, and the result data block first spliced in data frame group obtains number of results
According to frame, result data frame is then subjected to splicing and obtains alarm data report, to improve the efficiency for generating alarm data report.
Optionally, before above-mentioned steps S202, i.e., alert analysis task is extracted from the corresponding data frame of alarm data
It before corresponding local data's block, needs to pre-process data frame, pretreated mode includes: using regular expression collection
It closes and extracts data characteristics string from the mixed characters string of data frame, and using the mixed characters of data characteristics string replacement data frame
String carries out type conversion to data frame and to the missing values assignment of data frame.
In specific implementation, data frame pretreatment is executed by the way of the calculating of pandas matrix.When according to a keyword
When corresponding one regular expression of execution extracts data characteristics string, since data volume is big, time-consuming is 0.8-1s or so.If being arranged
Keyword is more, and sequential processes time-consuming will linearly increase, for example, 5 keyword time-consumings are in 4-5s or so.If by this 5
It is time-consuming similar with one regular expression of execution when the corresponding regular expression of keyword is merged into a regular expression.
Therefore, regular expression similar in keyword difference but extraction feature string pattern is merged in the embodiment of the present invention, constructs canonical
Expression formula set.Then data characteristics string is extracted from the mixed characters string of data frame using regular expression set, and used
The mixed characters string of data characteristics string replacement data frame, to improve the efficiency that feature string is extracted and replaced.It is arranged by data frame
Particular column time execution date type or integer type are converted in type conversion, are according to condition done by pandas interface to data frame
Missing values processing and to data frame line according to condition assignment.By being pre-processed to data frame, so that data frame meets sequence
With the requirement of statistics, alert analysis is carried out based on data frame convenient for subsequent.
Embodiment in order to preferably explain the present invention describes the embodiment of the present invention below with reference to specific implement scene and provides
A kind of generation alarm data report method, this method is by big data synchronization system, source database, report generating system, report
The interaction of table database executes, as shown in figure 3, method includes the following steps:
Big data synchronization system acquires alarm data from data source and alarm data is written source database, and data source can be with
It is the operation system of bank and other financial mechanism.Alarm data in source database regularly updates.Report generating system includes big number
According to loading module, preprocessing module, report data generation module, data frame update module, report memory module.Big data load
Module is loaded into memory using the alarm data that pandas interface is analysed to from source database, is then constructed using alarm data
Pandas data frame.Preprocessing module executes data frame pretreatment by the way of the calculating of pandas matrix, and pretreatment includes number
According to feature string extraction, data type conversion, missing values processing.Report data generation module extracts alert analysis from data frame and appoints
It is engaged in corresponding local data's block, the corresponding local data's block of alert analysis task is grouped and is counted, multiple statistics are obtained
Result block.The progress of multiple result data blocks is horizontally-spliced, and multiple statistical result blocks carry out analysis and obtain analysis result block, it is right
Analysis result in each analysis result block is ranked up, and takes the analysis for coming top N in each analysis result block as a result, being based on
The analysis result block that value operation is carried out after alert analysis task, statistical result block, sequence generates the result of alert analysis task
Data block, N are default integer.It will be added in global data frame group after result data block marking types.Judge whether to continue to take out
Local data's block is taken, if so, executing above-mentioned extraction data block, grouping, statistics, analysis, sequence and the step for generating result data block
Suddenly, otherwise, the result data block progress in global data frame group is longitudinal spliced, each result data block uses type column field
Identity type, it is automatic extending transversely when type column field difference, while missing values are filled, form big data frame.Number
CMDB is accessed by http (s) interface according to frame update module and configures system, the data item in big data frame is updated.Report
Memory module writes data-interface using pandas and converts alarm data report for big data frame, and alarm data report is written
In report database.
In the embodiment of the present invention, using alarm data construct data frame, then extract data frame in local data's block into
Row alert analysis obtains the result data block of alert analysis task, later again by the result data block of same alert analysis task
It carries out splicing and obtains result data frame, the result data frame of all alert analysis tasks is subjected to splicing and obtains alarm data report
Table, therefore all alarm datas to be analyzed only generate an alarm data report, which includes all alarms point
The content of analysis task is convenient for report management and user query report.Secondly, alarm data is handled as a data frame, it is raw
At the corresponding categorical data of report, the efficiency for generating alarm data report can be improved.By carrying out row and column to big data frame
The update to alarm data report is realized in operation, so that the management service of alarm data report is more convenient.
Based on the same technical idea, the embodiment of the invention provides a kind of structures of device for generating alarm data report
Schematic diagram, as shown in figure 4, the device 400 includes:
Module 401 is obtained, for obtaining alarm data to be analyzed and constructing data frame using the alarm data；
Abstraction module 402 extracts the alert analysis from the data frame and appoints for being directed to each alert analysis task
It is engaged in corresponding local data's block；
Analysis module 403 determines institute for for statistical analysis to the corresponding local data's block of the alert analysis task
State the result data block of alert analysis task；
Splicing module 404 determines alarm number for splicing the corresponding result data block of each alert analysis task
According to report.
Optionally, the splicing module 404 is specifically used for:
The corresponding result data block of each alert analysis task is added in corresponding data frame group；
All result data blocks in each data frame group are spliced, determine the corresponding knot of each alert analysis task
Fruit data frame；
The corresponding result data frame of each alert analysis task is spliced, determines alarm data report.
Optionally, the analysis module 403 is specifically used for:
The corresponding local data's block of the alert analysis task is grouped and is counted, multiple statistical result blocks are obtained；
The multiple statistical result block is analyzed and sorted, the announcement is generated according to the analysis result for coming top N
The result data block of alert analysis task, N are default integer.
Optionally, the data frame is the two dimensional data structure for including row and column.
It optionally, further include preprocessing module 405；
The preprocessing module 405 is specifically used for:
For each alert analysis task, the alert analysis task is extracted from the corresponding data frame of the alarm data
Before corresponding local data's block, data characteristics is extracted from the mixed characters string of the data frame using regular expression set
It goes here and there, and replaces the mixed characters string of the data frame using the data characteristics string；
Type conversion is carried out to the data frame and to the missing values assignment of the data frame.
Based on the same technical idea, the embodiment of the invention provides a kind of computer equipments, as shown in figure 5, including extremely
Lack a processor 501, and the memory 502 connecting at least one processor, does not limit processing in the embodiment of the present invention
Specific connection medium between device 501 and memory 502 passes through bus between processor 501 and memory 502 in Fig. 5 and connects
For.Bus can be divided into address bus, data/address bus, control bus etc..
In embodiments of the present invention, memory 502 is stored with the instruction that can be executed by least one processor 501, at least
The instruction that one processor 501 is stored by executing memory 502 can execute the method above-mentioned for generating alarm data report
In included step.
Wherein, processor 501 is the control centre of computer equipment, can use various interfaces and connection computer
The various pieces of equipment are stored in memory 502 by running or executing the instruction being stored in memory 502 and calling
Data, to generate alarm data report.Optionally, processor 501 may include one or more processing units, processor
501 can integrate application processor and modem processor, wherein the main processing operation system of application processor, user interface
With application program etc., modem processor mainly handles wireless communication.It is understood that above-mentioned modem processor
It can not be integrated into processor 501.In some embodiments, processor 501 and memory 502 can be real on the same chip
Existing, in some embodiments, they can also be realized respectively on independent chip.
Processor 501 can be general processor, such as central processing unit (CPU), digital signal processor, dedicated integrated
Circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array or other can
Perhaps transistor logic, discrete hardware components may be implemented or execute present invention implementation for programmed logic device, discrete gate
Each method, step and logic diagram disclosed in example.General processor can be microprocessor or any conventional processor
Deng.The step of method in conjunction with disclosed in the embodiment of the present invention, can be embodied directly in hardware processor and execute completion, Huo Zheyong
Hardware and software module combination in processor execute completion.
Memory 502 is used as a kind of non-volatile computer readable storage medium storing program for executing, can be used for storing non-volatile software journey
Sequence, non-volatile computer executable program and module.Memory 502 may include the storage medium of at least one type,
It such as may include flash memory, hard disk, multimedia card, card-type memory, random access storage device (Random Access
Memory, RAM), static random-access memory (Static Random Access Memory, SRAM), may be programmed read-only deposit
Reservoir (Programmable Read Only Memory, PROM), read-only memory (Read Only Memory, ROM), band
Electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory,
EEPROM), magnetic storage, disk, CD etc..Memory 502 can be used for carrying or storing have instruction or data
The desired program code of structure type and can by any other medium of computer access, but not limited to this.The present invention is real
Applying the memory 502 in example can also be circuit or other devices that arbitrarily can be realized store function, for storing program
Instruction and/or data.
Based on the same technical idea, it the embodiment of the invention provides a kind of computer readable storage medium, is stored with
The computer program that can be executed by computer equipment, when described program is run on a computing device, so that the computer
Equipment executes the step of method for generating alarm data report.
It should be understood by those skilled in the art that, the embodiment of the present invention can provide as method or computer program product.
Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in the present invention
Form.It is deposited moreover, the present invention can be used to can be used in the computer that one or more wherein includes computer usable program code
The shape for the computer program product implemented on storage media (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Although preferred embodiments of the present invention have been described, it is created once a person skilled in the art knows basic
Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as
It selects embodiment and falls into all change and modification of the scope of the invention.
Obviously, various changes and modifications can be made to the invention without departing from essence of the invention by those skilled in the art
Mind and range.In this way, if these modifications and changes of the present invention belongs to the range of the claims in the present invention and its equivalent technologies
Within, then the present invention is also intended to include these modifications and variations.
Claims (10)
1. a kind of method for generating alarm data report characterized by comprising
It obtains alarm data to be analyzed and data frame is constructed using the alarm data；
For each alert analysis task, the corresponding local data's block of the alert analysis task is extracted from the data frame；
It is for statistical analysis to the corresponding local data's block of the alert analysis task, determine the result of the alert analysis task
Data block；
The corresponding result data block of each alert analysis task is spliced, determines alarm data report.
2. the method as described in claim 1, which is characterized in that described by the corresponding result data block of each alert analysis task
Spliced, determine alarm data report, comprising:
The corresponding result data block of each alert analysis task is added in corresponding data frame group；
All result data blocks in each data frame group are spliced, determine the corresponding number of results of each alert analysis task
According to frame；
The corresponding result data frame of each alert analysis task is spliced, determines alarm data report.
3. the method as described in claim 1, which is characterized in that described to the corresponding local data's block of the alert analysis task
It is for statistical analysis, determine the result data block of the alert analysis task, comprising:
The corresponding local data's block of the alert analysis task is grouped and is counted, multiple statistical result blocks are obtained；
The multiple statistical result block is analyzed and sorted, the alarm point is generated according to the analysis result for coming top N
The result data block of analysis task, N are default integer.
4. the method as described in claim 1, which is characterized in that the data frame is the two dimensional data structure for including row and column.
5. the method as described in Claims 1-4 is any, which is characterized in that it is described to be directed to each alert analysis task, from described
Before extracting the corresponding local data's block of the alert analysis task in the corresponding data frame of alarm data, further includes:
Data characteristics string is extracted from the mixed characters string of the data frame using regular expression set, and uses the data
Feature string replaces the mixed characters string of the data frame；
Type conversion is carried out to the data frame and to the missing values assignment of the data frame.
6. a kind of device for generating alarm data report characterized by comprising
Module is obtained, for obtaining alarm data to be analyzed and constructing data frame using the alarm data；
It is corresponding to extract the alert analysis task for being directed to each alert analysis task from the data frame for abstraction module
Local data's block；
Analysis module determines the alarm for for statistical analysis to the corresponding local data's block of the alert analysis task
The result data block of analysis task；
Splicing module determines alarm data report for splicing the corresponding result data block of each alert analysis task.
7. device as claimed in claim 6, which is characterized in that the splicing module is specifically used for:
The corresponding result data block of each alert analysis task is added in corresponding data frame group；
All result data blocks in each data frame group are spliced, determine the corresponding number of results of each alert analysis task
According to frame；
The corresponding result data frame of each alert analysis task is spliced, determines alarm data report.
8. device as claimed in claim 6, which is characterized in that the analysis module is specifically used for:
The corresponding local data's block of the alert analysis task is grouped and is counted, multiple statistical result blocks are obtained；
The multiple statistical result block is analyzed and sorted, and the alarm is generated according to the analysis result for coming top N
The result data block of analysis task, N are default integer.
9. a kind of computer equipment including memory, processor and stores the meter that can be run on a memory and on a processor
Calculation machine program, which is characterized in that the processor is realized described in Claims 1 to 5 any claim when executing described program
The step of method.
10. a kind of computer readable storage medium, which is characterized in that it is stored with the computer journey that can be executed by computer equipment
Sequence, when described program is run on a computing device, so that computer equipment perform claim requirement 1~5 is any described
The step of method.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910569785.7A CN110287241B (en) | 2019-06-27 | 2019-06-27 | Method and device for generating alarm data report |
| PCT/CN2020/091932 WO2020259155A1 (en) | 2019-06-27 | 2020-05-22 | Method and apparatus for generating alarm data report |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910569785.7A CN110287241B (en) | 2019-06-27 | 2019-06-27 | Method and device for generating alarm data report |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110287241A true CN110287241A (en) | 2019-09-27 |
| CN110287241B CN110287241B (en) | 2023-09-08 |
Family
ID=68019964
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910569785.7A Active CN110287241B (en) | 2019-06-27 | 2019-06-27 | Method and device for generating alarm data report |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110287241B (en) |
| WO (1) | WO2020259155A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020259155A1 (en) * | 2019-06-27 | 2020-12-30 | 深圳前海微众银行股份有限公司 | Method and apparatus for generating alarm data report |
| CN113204416A (en) * | 2021-04-07 | 2021-08-03 | 上海多维度网络科技股份有限公司 | Data report task execution method, device, equipment and storage medium |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114372189A (en) * | 2021-12-31 | 2022-04-19 | 上海金仕达软件科技有限公司 | Processing method and device of user-defined report, storage medium and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170187737A1 (en) * | 2015-12-28 | 2017-06-29 | Le Holdings (Beijing) Co., Ltd. | Method and electronic device for processing user behavior data |
| CN108763038A (en) * | 2018-08-08 | 2018-11-06 | 平安科技（深圳）有限公司 | Management method, device, computer equipment and the storage medium of alarm data |
| CN109933771A (en) * | 2019-03-22 | 2019-06-25 | 广州市玄武无线科技股份有限公司 | A kind of automatic merging method of report, device, equipment and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101170729A (en) * | 2007-11-30 | 2008-04-30 | 中国移动通信集团重庆有限公司 | A GSM network alarming analysis system and network alarm analysis method |
| US9043269B2 (en) * | 2008-05-27 | 2015-05-26 | Appfolio, Inc. | Systems and methods for automatically identifying data dependencies for reports |
| CN102841943B (en) * | 2012-08-24 | 2015-07-08 | 上海泰宇信息技术有限公司 | Data safety supervision early warning and backup strategy system and method |
| CN110287241B (en) * | 2019-06-27 | 2023-09-08 | 深圳前海微众银行股份有限公司 | Method and device for generating alarm data report |
-
2019
- 2019-06-27 CN CN201910569785.7A patent/CN110287241B/en active Active
-
2020
- 2020-05-22 WO PCT/CN2020/091932 patent/WO2020259155A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170187737A1 (en) * | 2015-12-28 | 2017-06-29 | Le Holdings (Beijing) Co., Ltd. | Method and electronic device for processing user behavior data |
| CN108763038A (en) * | 2018-08-08 | 2018-11-06 | 平安科技（深圳）有限公司 | Management method, device, computer equipment and the storage medium of alarm data |
| CN109933771A (en) * | 2019-03-22 | 2019-06-25 | 广州市玄武无线科技股份有限公司 | A kind of automatic merging method of report, device, equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020259155A1 (en) * | 2019-06-27 | 2020-12-30 | 深圳前海微众银行股份有限公司 | Method and apparatus for generating alarm data report |
| CN113204416A (en) * | 2021-04-07 | 2021-08-03 | 上海多维度网络科技股份有限公司 | Data report task execution method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110287241B (en) | 2023-09-08 |
| WO2020259155A1 (en) | 2020-12-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8065326B2 (en) | System and method for building decision trees in a database | |
| CN111538731B (en) | Automatic report generation system for industrial data | |
| US20190121810A1 (en) | System and method for querying a graph model | |
| CN110287241A (en) | A kind of method and device generating alarm data report | |
| CN109241159B (en) | Partition query method and system for data cube and terminal equipment | |
| US20150032708A1 (en) | Database analysis apparatus and method | |
| CN106293891B (en) | Multidimensional investment index monitoring method | |
| US9153051B2 (en) | Visualization of parallel co-ordinates | |
| CN110134845A (en) | Project public sentiment monitoring method, device, computer equipment and storage medium | |
| CN114153839B (en) | Integration method, device, equipment and storage medium of multi-source heterogeneous data | |
| CN103077192B (en) | A kind of data processing method and system thereof | |
| US20150379073A1 (en) | Virtual split dictionary for search optimization | |
| CN109471718A (en) | Computing resource configuration method, device, equipment and medium based on recognition of face | |
| CN104050193B (en) | Generate the method for message and realize the data handling system of this method | |
| CN105843788A (en) | Report generation method and device | |
| WO2021012861A1 (en) | Method and apparatus for evaluating data query time consumption, and computer device and storage medium | |
| CN113535758B (en) | Big data system and method for converting traditional database scripts into cloud in batch | |
| CN113806429A (en) | Canvas type log analysis method based on large data stream processing framework | |
| CN113947468B (en) | Data management method and platform | |
| CN105138656A (en) | Method and device for processing data | |
| CN111159213A (en) | Data query method, device, system and storage medium | |
| CN110019195A (en) | A kind of storage method and device of data | |
| CN116010417A (en) | Automatic driving data mining method, system, terminal and medium | |
| CN113515528B (en) | Asset screening system and method based on big data and ORACLE mass data | |
| CN114116773A (en) | Structured Query Language (SQL) text auditing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |