CN110347653A - Data processing method and device, electronic equipment and readable storage medium storing program for executing - Google Patents
Data processing method and device, electronic equipment and readable storage medium storing program for executing Download PDFInfo
- Publication number
- CN110347653A CN110347653A CN201910622196.0A CN201910622196A CN110347653A CN 110347653 A CN110347653 A CN 110347653A CN 201910622196 A CN201910622196 A CN 201910622196A CN 110347653 A CN110347653 A CN 110347653A
- Authority
- CN
- China
- Prior art keywords
- log
- online transaction
- analysis model
- concentrated
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
Present disclose provides a kind of data processing methods executed by electronic equipment, comprising: multiple online transaction logs of the acquisition from enterprise operation system；Classified according to default element of taxonomy to multiple online transaction logs, obtain multiple log collection, wherein each log collection that multiple logs are concentrated has corresponding classification information；The online transaction log input that each log is concentrated Data Analysis Model corresponding with the classification information of log collection, the online transaction log concentrated to the log being entered is handled to obtain processing result, wherein, each log collection that multiple logs are concentrated is respectively provided with corresponding Data Analysis Model；And output processing result.The disclosure additionally provides a kind of a kind of a kind of data processing equipment, data processing system, a kind of electronic equipment and computer readable storage medium executed by electronic equipment.
Description
Technical field
This disclosure relates to field of computer technology, more particularly, to a kind of data processing side executed by electronic equipment
Method, a kind of a kind of a kind of data processing equipment, data processing system, a kind of electronic equipment and computer executed by electronic equipment
Readable storage medium storing program for executing.
Background technique
Currently, more and more large enterprises have set up data center, to realize centralized processing core business, to meet
The high timeliness of enterprise's Transaction Processing, High Availabitity etc. require, online transaction refer to client in diverse geographic location with
The affairs that interaction generates between server, feature are that affairs amount is big, and affairs content is fairly simple and repetitive rate is high.However in city
When field quickly changes, customer demand increases, enterprise product is abundant, the concurrency of online transaction, complexity all substantially increase
It is long, to bring bigger performance risk and O&M pressure to the enterprise operation system of data center.
Since the type in large-scale data center, online transaction might have thousands of, the quantity of daily online transaction has
Hundred million may be crossed.In face of so big data, there are the following problems and difficult for meeting when carrying out online transaction analysis: how
Anomalous variation situation in discovery online transaction in time, thus the reason of assessing anomalous variation, and carry out optimization counter-measure；Such as
The growth trend situation of what Accurate Prediction online transaction to carry out the capacity requirement assessment of operation system, and carries out system expansion
Hold the prediction sex work such as tuning.
Summary of the invention
In view of this, present disclose provides a kind of data processing method executed by electronic equipment, one kind by electronic equipment
The data processing equipment of execution, a kind of data processing system, a kind of electronic equipment and a kind of computer readable storage medium.
An aspect of this disclosure provides a kind of data processing method executed by electronic equipment, comprising: acquisition comes from
Multiple online transaction logs of enterprise operation system；Above-mentioned multiple online transaction logs are divided according to default element of taxonomy
Class obtains multiple log collection, wherein each log collection that above-mentioned multiple logs are concentrated has corresponding classification information；It will be above-mentioned
The online transaction log input Data Analysis Model corresponding with the classification information of log collection that each log is concentrated, to what is be entered
The online transaction log that log is concentrated is handled to obtain processing result, wherein each log that above-mentioned multiple logs are concentrated
Collection is respectively provided with corresponding Data Analysis Model；And the above-mentioned processing result of output.
In accordance with an embodiment of the present disclosure, classification packet is carried out to above-mentioned multiple online transaction logs according to default element of taxonomy
It includes: each online transaction log in above-mentioned multiple online transaction logs is parsed, from above-mentioned each online transaction log
The one or more keywords of middle extraction；And by by above-mentioned each online transaction log one or more keywords with it is upper
It states default element of taxonomy to be compared, classify to above-mentioned multiple online transaction logs.
In accordance with an embodiment of the present disclosure, exporting above-mentioned processing result includes: according to above-mentioned processing result automatically generated data
Report；And visualize above-mentioned data sheet.
In accordance with an embodiment of the present disclosure, above-mentioned multiple log collection include at least the first log collection and category for belonging to first category
In the second log collection of second category, above-mentioned Data Analysis Model includes at least the first analysis model and analyzes mould with above-mentioned first
The second different analysis model of type, the classification information pair of online transaction log input and log collection that above-mentioned each log is concentrated
The Data Analysis Model answered includes: that above-mentioned first log collection is inputted above-mentioned first analysis model；And by above-mentioned second log
Collection inputs above-mentioned second analysis model.
In accordance with an embodiment of the present disclosure, above-mentioned first analysis model and above-mentioned second analysis model are with one of drag:
Transaction abnormality detection model, trend prediction model.
Another aspect of the disclosure provides a kind of data processing equipment executed by electronic equipment, comprising: acquisition mould
Block, for acquiring multiple online transaction logs in enterprise operation system；Categorization module, for according to preset element of taxonomy to
It states multiple online transaction logs to classify, obtains multiple log collection, wherein each log collection tool that above-mentioned multiple logs are concentrated
There is corresponding classification information；Input module, online transaction log input and log collection for concentrating above-mentioned each log
The corresponding Data Analysis Model of classification information, the online transaction log concentrated to the log being entered are handled to be handled
As a result, wherein each log collection that above-mentioned multiple logs are concentrated is respectively provided with corresponding Data Analysis Model；And output mould
Block, for exporting above-mentioned processing result.
Another aspect of the disclosure provides a kind of data processing system, comprising: enterprise operation system is stored with multiple
Online transaction log；And data statistic analysis server, it is used for: to multiple in the above-mentioned enterprise operation system got
Machine transaction journal is classified according to default element of taxonomy, obtains multiple log collection, wherein each of above-mentioned multiple logs concentrations
Log collection has corresponding classification information；The classification of online transaction log input and log collection that above-mentioned each log is concentrated is believed
Corresponding Data Analysis Model is ceased, the online transaction log concentrated to the log being entered is handled to obtain processing result,
Wherein, each log collection that above-mentioned multiple logs are concentrated is respectively provided with corresponding Data Analysis Model；And the above-mentioned processing of output
As a result.
In accordance with an embodiment of the present disclosure, data processing system further include: management server, for managing above-mentioned data statistics
The Data Analysis Model stored in Analysis server.
In accordance with an embodiment of the present disclosure, data processing system further include: information alert server, for receiving above-mentioned data
The above-mentioned processing result of statistic analysis server output, and determine whether that external equipment sends prompt according to above-mentioned processing result
Information.
Another aspect of the disclosure provides a kind of electronic equipment, comprising: one or more processors；Memory is used
In the one or more instructions of storage, wherein when said one or multiple instruction are executed by said one or multiple processors, make
It obtains said one or multiple processors realizes method as described above.
Another aspect of the disclosure provides a kind of computer readable storage medium, is stored thereon with executable instruction,
The instruction makes processor realize method as described above when being executed by processor.
Another aspect of the present disclosure provides a kind of computer program, and the computer program, which includes that computer is executable, to be referred to
It enables, described instruction is when executed for realizing method as described above.
In accordance with an embodiment of the present disclosure, by the way that different classes of log collection is inputted model corresponding with classification information respectively
The technological means automatically processed, so that the log capacity of enterprise operation system can satisfy current and future online transaction
Process demand is analyzed by the accurate and effective of online transaction log, the measures such as adjustment can be optimized to enterprise operation system,
Ensure that enterprise operation system is stablized, the processing of online transaction, which meets, calculates service request, so at least partially overcoming correlation
The concurrency of online transaction and complexity are high in technology, cause the stability of enterprise operation system low, the big technology of O&M pressure
Problem, and then reached the technical effect for improving the stability of enterprise operation system.
Detailed description of the invention
By referring to the drawings to the description of the embodiment of the present disclosure, the above-mentioned and other purposes of the disclosure, feature and
Advantage will be apparent from, in the accompanying drawings:
Fig. 1 is diagrammatically illustrated can be using the exemplary system architecture of the data processing method and device of the disclosure；
Fig. 2 diagrammatically illustrates the flow chart of the data processing method according to the embodiment of the present disclosure；
Fig. 3 diagrammatically illustrate according to the embodiment of the present disclosure basis preset element of taxonomy to multiple online transaction logs into
The flow chart of row classification；
Fig. 4 diagrammatically illustrates the block diagram of the data processing system according to the embodiment of the present disclosure；
Fig. 5 diagrammatically illustrates the flow chart of the data processing method according to the embodiment of the present disclosure；
Fig. 6 diagrammatically illustrates the block diagram of the data processing equipment according to the embodiment of the present disclosure；And
Fig. 7 diagrammatically illustrates the frame of the electronic equipment for being adapted for carrying out method as described above according to the embodiment of the present disclosure
Figure.
Specific embodiment
Hereinafter, will be described with reference to the accompanying drawings embodiment of the disclosure.However, it should be understood that these descriptions are only exemplary
, and it is not intended to limit the scope of the present disclosure.In the following detailed description, to elaborate many specific thin convenient for explaining
Section is to provide the comprehensive understanding to the embodiment of the present disclosure.It may be evident, however, that one or more embodiments are not having these specific thin
It can also be carried out in the case where section.In addition, in the following description, descriptions of well-known structures and technologies are omitted, to avoid
Unnecessarily obscure the concept of the disclosure.
Term as used herein is not intended to limit the disclosure just for the sake of description specific embodiment.It uses herein
The terms "include", "comprise" etc. show the presence of the feature, step, operation and/or component, but it is not excluded that in the presence of
Or add other one or more features, step, operation or component.
There are all terms (including technical and scientific term) as used herein those skilled in the art to be generally understood
Meaning, unless otherwise defined.It should be noted that term used herein should be interpreted that with consistent with the context of this specification
Meaning, without that should be explained with idealization or excessively mechanical mode.
It, in general should be according to this using statement as " at least one in A, B and C etc. " is similar to
Field technical staff is generally understood the meaning of the statement to make an explanation (for example, " system at least one in A, B and C "
Should include but is not limited to individually with A, individually with B, individually with C, with A and B, with A and C, have B and C, and/or
System etc. with A, B, C).Using statement as " at least one in A, B or C etc. " is similar to, generally come
Saying be generally understood the meaning of the statement according to those skilled in the art to make an explanation (for example, " having in A, B or C at least
One system " should include but is not limited to individually with A, individually with B, individually with C, with A and B, have A and C, have
B and C, and/or the system with A, B, C etc.).
Embodiment of the disclosure provides a kind of data processing method executed by electronic equipment, comprising: acquisition is from enterprise
Multiple online transaction logs of industry operation system；Classified according to default element of taxonomy to above-mentioned multiple online transaction logs,
Obtain multiple log collection, wherein each log collection that above-mentioned multiple logs are concentrated has corresponding classification information；It will be above-mentioned each
The online transaction log input Data Analysis Model corresponding with the classification information of log collection that log is concentrated, the log to being entered
The online transaction log of concentration is handled to obtain processing result, wherein each log collection point that above-mentioned multiple logs are concentrated
It Ju You not corresponding Data Analysis Model；And the above-mentioned processing result of output.
Fig. 1 diagrammatically illustrate according to the embodiment of the present disclosure can be with the exemplary system of application data processing method and device
System framework 100.It should be noted that being only the example that can apply the system architecture of the embodiment of the present disclosure shown in Fig. 1, to help
Those skilled in the art understand that the technology contents of the disclosure, but it is not meant to that the embodiment of the present disclosure may not be usable for other and set
Standby, system, environment or scene.
As shown in Figure 1, system architecture 100 may include server 101, server 102, server according to this embodiment
103 and network 104.Network 104 between server 101, server 102 and server 103 to provide Jie of communication link
Matter.Network 104 may include various connection types, such as wired and or wireless communications link etc..
User can be used server 101 and pass through network 104 and server 102, the interaction of server 103, to receive or send out
Send message etc..
Server 101 can be interacted with terminal device, for example, processing result can be sent to end by server 101
End equipment is shown.Terminal device includes but is not limited to smart phone, tablet computer, pocket computer on knee and desk-top meter
Calculation machine etc..
It can be the service of multiple online transaction logs of storage enterprise operation system in server 102 and server 103
Device.Server 102 can be the log for belonging to different enterprise operation systems, example from the online transaction log that server 103 stores
Such as, server 102 stores the online transaction log of enterprise operation system 1, and server 103 stores the online of enterprise operation system 2
Transaction journal.Server 102 and server 103 can carry out the processing such as analyzing to data such as the user's requests received, will use
The data etc. of family request feed back to other equipment.
It should be noted that data processing method provided by the embodiment of the present disclosure can generally be executed by server 101.
Correspondingly, data processing equipment provided by the embodiment of the present disclosure generally can be set in server 101.The embodiment of the present disclosure
Provided data processing method can also be by being different from server 101 and the server or clothes that can communicate with server 101
Business device cluster executes.Correspondingly, data processing equipment provided by the embodiment of the present disclosure also can be set in different from server
101 and the server or server cluster that can be communicated with server 101 in.
It should be understood that the number of server and network in Fig. 1 is only schematical.According to needs are realized, can have
There are any number of server and network.
Fig. 2 diagrammatically illustrates the flow chart of the data processing method according to the embodiment of the present disclosure.
As shown in Fig. 2, including operation S201~S204 by the data processing method that electronic equipment executes.
In operation S201, multiple online transaction logs from enterprise operation system are acquired.
In accordance with an embodiment of the present disclosure, it can use log collection tool and acquire online thing in different enterprise operation systems
Business log.
In operation S202, is classified according to default element of taxonomy to multiple online transaction logs, obtains multiple log collection,
Wherein, each log collection that multiple logs are concentrated has corresponding classification information.
In accordance with an embodiment of the present disclosure, default element of taxonomy may include at least one of: time, area, channel, industry
Business type, type of transaction etc..It should be noted that default element of taxonomy is without being limited thereto.
In accordance with an embodiment of the present disclosure, online transaction journal can be carried out according to certain rule according to default element of taxonomy
Classification, and collect statistics are carried out, basis of formation data.
In accordance with an embodiment of the present disclosure, each log collection includes one or more online transaction logs.Different log collection tools
There is corresponding classification information.For example, the online transaction log in log collection 1 belongs to classification 1, the online transaction day in log collection 2
Will belongs to classification 2, and classification 1 is different from classification 2.
In accordance with an embodiment of the present disclosure, multiple log collection include at least the first log collection for belonging to first category and belong to the
Second log collection of two classifications, Data Analysis Model include at least the first analysis model and different from the first analysis model second
Analysis model, the online transaction log input that each log is concentrated Data Analysis Model corresponding with the classification information of log collection
Including the first log collection is inputted the first analysis model, the second log collection is inputted into the second analysis model.
In accordance with an embodiment of the present disclosure, the first analysis model and the second analysis model are with one of drag: transaction is abnormal
Detection model, trend prediction model.
For example, the first analysis model is transaction abnormality detection model, the second analysis model is trend prediction model.Or the
One analysis model is trend prediction model, and the second analysis model is transaction abnormality detection model.
Transaction abnormality detection model can detecte abnormal conditions, such as transaction data bust is more than normal fluctuation range, is handed over
Easy data and system performance are unsatisfactory for regression model etc..
Trend prediction model can estimate the business of following a period of time according to the online transaction log that current log is concentrated
Trade development trend assesses following a period of time online transaction and enterprise operation system with the presence or absence of availability according to conclusion is estimated
With the risk of aspect of performance.
In accordance with an embodiment of the present disclosure, the first analysis model and the second analysis model are not limited to this, for example, it is also possible to wrap
Include O&M state portrait model etc..
In operation S203, the online transaction log input that each log is concentrated number corresponding with the classification information of log collection
According to analysis model, the online transaction log concentrated to the log being entered is handled to obtain processing result, wherein Duo Ge
Each log collection that will is concentrated is respectively provided with corresponding Data Analysis Model.
In accordance with an embodiment of the present disclosure, the corresponding relationship of classification information and analysis model can be preset, it can also root
According to the selection operation of user, real-time determination analysis model corresponding with the classification information of log collection.It can achieve flexibly using not
Same analysis model, effectively analyzes the effect of different classes of log collection feature.
In operation S204, processing result is exported.
In accordance with an embodiment of the present disclosure, when exporting processing result, can according to processing result automatically generated data report,
And carry out visual presentation data sheet.
In accordance with an embodiment of the present disclosure, can divide according to the log that existing Data Analysis Model concentrates log
Analysis forms analysis conclusion, and analysis conclusion is shown by all kinds of reports.In accordance with an embodiment of the present disclosure, can also will divide
The abnormal conditions or estimation results found when analysing data are alerted or are prompted by forms such as mail or short messages.
In accordance with an embodiment of the present disclosure, by the way that different classes of log collection is inputted model corresponding with classification information respectively
The technological means automatically processed, so that the log capacity of enterprise operation system can satisfy current and future online transaction
Process demand is analyzed by the accurate and effective of online transaction log, the measures such as adjustment can be optimized to enterprise operation system,
Ensure that enterprise operation system is stablized, the processing of online transaction, which meets, calculates service request, so at least partially overcoming correlation
The concurrency of online transaction and complexity are high in technology, cause the stability of enterprise operation system low, the big technology of O&M pressure
Problem, and then reached the technical effect for improving the stability of enterprise operation system.
Below with reference to Fig. 3~Fig. 5, method shown in Fig. 2 is described further in conjunction with specific embodiments.
Fig. 3 diagrammatically illustrate according to the embodiment of the present disclosure basis preset element of taxonomy to multiple online transaction logs into
The flow chart of row classification.
As shown in figure 3, according to default element of taxonomy to multiple online transaction logs carry out classification include operation S301~
S302。
In operation S301, each online transaction log in multiple online transaction logs is parsed, from each online
One or more keywords are extracted in transaction journal.
In accordance with an embodiment of the present disclosure, after collecting the online transaction log in enterprise operation system, to online transaction
Log is parsed, and extracts keyword from online transaction log using Keyword Extraction Technique.
For example, default element of taxonomy includes area and the two kinds of element of type of transaction, using Keyword Extraction Technique
Keyword " Nanchang " and " transferring accounts " are extracted from a certain online transaction log.
Operation S302, by by each online transaction log one or more keywords and default element of taxonomy into
Row compares, and classifies to multiple online transaction logs.
For example, by the keyword " Nanchang " extracted from a certain online transaction log and default element of taxonomy " Jiangxi " into
Row compares, and determines whether " Nanchang " belongs to " Jiangxi ", the keyword " transferring accounts " that will be extracted from a certain online transaction log
It is compared with default element of taxonomy " transferred account service ", determines whether " transferring accounts " belongs to " transferred account service ".It should be noted that
The above-mentioned process for being compared keyword with default element of taxonomy is not limited to above-mentioned two comparison procedure, for example, further include by
Keyword " Nanchang " is compared with default element of taxonomy " transferred account service ", by keyword " transferring accounts " and default element of taxonomy " river
West " is compared.
In accordance with an embodiment of the present disclosure, it when determining that keyword belongs to the classification that default element of taxonomy is characterized, can incite somebody to action
The corresponding online transaction log of the keyword is determined as the classification that corresponding default element of taxonomy is characterized.For example, by different connection
Machine transaction journal is divided into the log of different regions, alternatively, different online transaction logs to be divided into the log of different type of transaction
Etc..
Fig. 4 diagrammatically illustrates the block diagram of the data processing system according to the embodiment of the present disclosure.
As shown in figure 4, data processing system 400 includes enterprise operation system 410 and data statistics Analysis server 420.
Enterprise operation system 410 is stored with multiple online transaction logs.
In accordance with an embodiment of the present disclosure, enterprise operation system 410 includes one or more enterprise operation systems, each enterprise
Operation system is all stored with multiple online transaction logs.
Data statistic analysis server 420 be used for multiple online transaction logs in the enterprise operation system got by
Classify according to default element of taxonomy, obtain multiple log collection, wherein each log collection that multiple logs are concentrated has corresponding
Classification information；The online transaction log input that each log is concentrated data corresponding with the classification information of log collection analyze mould
Type, the online transaction log concentrated to the log being entered are handled to obtain processing result, wherein what multiple logs were concentrated
Each log collection is respectively provided with corresponding Data Analysis Model；And output processing result.
In accordance with an embodiment of the present disclosure, data statistic analysis server 420 may include log collection tool 421, log
Analytical tool 422, data statistics device 423, data analysis set-up 424 and report generation device 425.
In accordance with an embodiment of the present disclosure, log collection tool 421 can be used for collecting online in enterprise operation system 401
Transaction journal.Log analytical tool 422 can be used for parsing log, and (element may include element needed for extraction and analysis
Time, area, channel, type of business, type of transaction etc.).Data statistics device 423 can be used for online transaction journal according to
Certain rule carries out collect statistics, basis of formation data.Data analysis set-up 424 can be used for according to existing data model pair
Basic data is analyzed, formed analysis conclusion, existing model for example can be abnormality detection model, trend prediction model,
O&M state portrait model etc..Report generation device 425 can be used for data analyzing conclusion is opened up by all kinds of reports
Show.
In accordance with an embodiment of the present disclosure, log collection tool 421 and log analytical tool 422 can also be provided separately within separately
In one server, it is connect with data statistic analysis server 420.
In accordance with an embodiment of the present disclosure, data processing system 400 further includes management server 430.
Management server 430 is for managing the Data Analysis Model stored in data statistic analysis server.
In accordance with an embodiment of the present disclosure, management server 430 can create Data Analysis Model, and to Data Analysis Model
Parameter be adjusted.
In accordance with an embodiment of the present disclosure, data processing system 400 further includes information alert server 440.
The processing result that statistic analysis server exports for receiving data of information alert server 440, and according to processing
As a result determine whether that external equipment sends prompt information.
In accordance with an embodiment of the present disclosure, for example, the abnormal feelings that information alert server 440 can will be found in processing result
Condition, estimation results are alerted or are prompted by forms such as mail, short messages.
Fig. 5 diagrammatically illustrates the flow chart of the data processing method according to another embodiment of the disclosure.
As shown in figure 5, the data processing method includes operation S501~S511.
Log collect statistics rule is worked out according to the needs that data are analyzed in operation S501.
In operation S502, collect statistics are carried out by data of the statistical rules to online transaction journal.
In operation S503, need to optimize collect statistics rule according to data analysis can be met.
In operation S504, demand is analyzed according to abnormity diagnosis, trend prediction etc., according to historical data, combined data excavates,
The Data Analysis Model of machine learning scheduling algorithm model foundation online transaction.
In operation S505, the data after being summarized using Data Analysis Model to statistics are analyzed.
In operation S506, result is analyzed by all kinds of personalized report query online transaction datas.
Operation S507, for discovery abnormal conditions (such as transaction data bust be more than normal fluctuation range, number of deals
Regression model is unsatisfactory for according to system performance), whether analysis and assessment are really abnormal.
It establishes questionnaire for true abnormal conditions in operation S508 and optimizes.
In operation S509, sent out according to the business transaction that trend prediction model and current data situation estimate following a period of time
Exhibition trend.
In operation S510, following a period of time online transaction and application system are assessed with the presence or absence of available according to conclusion is estimated
The risk of property and aspect of performance.
Data Analysis Model is optimized according to the accuracy of abnormal conditions discovery and Trend Prediction in operation S511.
Fig. 6 diagrammatically illustrates the block diagram of the data processing equipment according to the embodiment of the present disclosure.
As shown in fig. 6, by electronic equipment execute data processing equipment 600 include acquisition module 601, categorization module 602,
Input module 603 and output module 604.
Acquisition module 601 is used to acquire multiple online transaction logs in enterprise operation system.
Categorization module 602 is used to classify to multiple online transaction logs according to default element of taxonomy, obtains multiple days
Will collection, wherein each log collection that multiple logs are concentrated has corresponding classification information.
The online transaction log input that input module 603 is used to concentrate each log is corresponding with the classification information of log collection
Data Analysis Model, the online transaction log concentrated to the log that is entered handled to obtain processing result, wherein more
Each log collection that a log is concentrated is respectively provided with corresponding Data Analysis Model.
Output module 604 is for exporting processing result.
In accordance with an embodiment of the present disclosure, output module 604 is also used to according to the processing result automatically generated data report；
And visualize the data sheet.
In accordance with an embodiment of the present disclosure, categorization module includes: resolution unit, for every in multiple online transaction logs
A online transaction log is parsed, and one or more keywords are extracted from each online transaction log；And comparing unit,
For by being compared one or more keywords in each online transaction log with default element of taxonomy, to multiple
Machine transaction journal is classified.
In accordance with an embodiment of the present disclosure, comparing unit is also used to determine one or more passes in each online transaction log
The similarity of key word and default element of taxonomy；And it according to one or more keywords in each online transaction log and presets
The similarity of element of taxonomy classifies to multiple online transaction logs.
In accordance with an embodiment of the present disclosure, multiple log collection include at least the first log collection for belonging to first category and belong to the
Second log collection of two classifications, Data Analysis Model include at least the first analysis model and different from the first analysis model second
Analysis model, the online transaction log input that each log is concentrated Data Analysis Model corresponding with the classification information of log collection
It include: that the first log collection is inputted into the first analysis model；And the second log collection is inputted into the second analysis model.
In accordance with an embodiment of the present disclosure, the first analysis model and the second analysis model are with one of drag: transaction is abnormal
Detection model, trend prediction model.
It is module according to an embodiment of the present disclosure, submodule, unit, any number of or in which any more in subelement
A at least partly function can be realized in a module.It is single according to the module of the embodiment of the present disclosure, submodule, unit, son
Any one or more in member can be split into multiple modules to realize.According to the module of the embodiment of the present disclosure, submodule,
Any one or more in unit, subelement can at least be implemented partly as hardware circuit, such as field programmable gate
Array (FPGA), programmable logic array (PLA), system on chip, the system on substrate, the system in encapsulation, dedicated integrated electricity
Road (ASIC), or can be by the hardware or firmware for any other rational method for integrate or encapsulate to circuit come real
Show, or with any one in three kinds of software, hardware and firmware implementations or with wherein any several appropriately combined next reality
It is existing.Alternatively, can be at least by part according to one or more of the module of the embodiment of the present disclosure, submodule, unit, subelement
Ground is embodied as computer program module, when the computer program module is run, can execute corresponding function.
For example, any number of in acquisition module 601, categorization module 602, input module 603 and output module 604 can be with
Merging is realized in a module/unit/subelement or any one module/unit/subelement therein can be split
At multiple module/unit/subelements.Alternatively, one or more module/units/son in these module/unit/subelements is single
Member at least partly function can be combined with other modules/unit/subelement at least partly function, and a module/
It is realized in unit/subelement.In accordance with an embodiment of the present disclosure, acquisition module 601, categorization module 602, input module 603 and defeated
At least one of module 604 can at least be implemented partly as hardware circuit, such as field programmable gate array out
(FPGA), programmable logic array (PLA), system on chip, the system on substrate, the system in encapsulation, specific integrated circuit
(ASIC), it or can be realized by carrying out the hardware such as any other rational method that is integrated or encapsulating or firmware to circuit,
Or it several appropriately combined is realized with any one in three kinds of software, hardware and firmware implementations or with wherein any.
Alternatively, at least one of acquisition module 601, categorization module 602, input module 603 and output module 604 can be at least by portions
Divide ground to be embodied as computer program module, when the computer program module is run, corresponding function can be executed.
It should be noted that in embodiment of the disclosure in data processing equipment part and embodiment of the disclosure at data
Reason method part be it is corresponding, the description of data processing equipment part is with specific reference to data processing method part, herein no longer
It repeats.
Fig. 7 diagrammatically illustrates the frame of the electronic equipment for being adapted for carrying out method as described above according to the embodiment of the present disclosure
Figure.Electronic equipment shown in Fig. 7 is only an example, should not function to the embodiment of the present disclosure and use scope bring it is any
Limitation.
As shown in fig. 7, include processor 701 according to the electronic equipment 700 of the embodiment of the present disclosure, it can be according to being stored in
Program in read-only memory (ROM) 702 is loaded into the journey in random access storage device (RAM) 703 from storage section 708
Sequence and execute various movements appropriate and processing.Processor 701 for example may include general purpose microprocessor (such as CPU), instruction
Set processor and/or related chip group and/or special microprocessor (for example, specific integrated circuit (ASIC)), etc..Processor
701 can also include the onboard storage device for caching purposes.Processor 701 may include being implemented for executing according to the disclosure
Single treatment unit either multiple processing units of the different movements of the method flow of example.
In RAM 703, it is stored with system 700 and operates required various programs and data.Processor 701, ROM 702 with
And RAM 703 is connected with each other by bus 704.Processor 701 is held by executing the program in ROM 702 and/or RAM 703
The various operations gone according to the method flow of the embodiment of the present disclosure.It is noted that described program also can store except ROM 702
In one or more memories other than RAM 703.Processor 701 can also be stored in one or more of by execution
Program in memory executes the various operations of the method flow according to the embodiment of the present disclosure.
In accordance with an embodiment of the present disclosure, system 700 can also include input/output (I/O) interface 705, input/output
(I/O) interface 705 is also connected to bus 704.System 700 can also include be connected to I/O interface 705 with one in lower component
Item is multinomial: the importation 706 including keyboard, mouse etc.；Including such as cathode-ray tube (CRT), liquid crystal display (LCD)
Deng and loudspeaker etc. output par, c 707；Storage section 708 including hard disk etc.；And including such as LAN card, modulatedemodulate
Adjust the communications portion 709 of the network interface card of device etc..Communications portion 709 executes communication process via the network of such as internet.
Driver 710 is also connected to I/O interface 705 as needed.Detachable media 711, such as disk, CD, magneto-optic disk, semiconductor
Memory etc. is mounted on as needed on driver 710, in order to be pacified as needed from the computer program read thereon
It is packed into storage section 708.
In accordance with an embodiment of the present disclosure, computer software journey may be implemented as according to the method flow of the embodiment of the present disclosure
Sequence.For example, embodiment of the disclosure includes a kind of computer program product comprising be carried on computer readable storage medium
Computer program, which includes the program code for method shown in execution flow chart.In such implementation
In example, which can be downloaded and installed from network by communications portion 709, and/or from detachable media 711
It is mounted.When the computer program is executed by processor 701, the above-mentioned function limited in the system of the embodiment of the present disclosure is executed
Energy.In accordance with an embodiment of the present disclosure, system as described above, unit, module, unit etc. can pass through computer program
Module is realized.
The disclosure additionally provides a kind of computer readable storage medium, which can be above-mentioned reality
It applies included in equipment/device/system described in example；Be also possible to individualism, and without be incorporated the equipment/device/
In system.Above-mentioned computer readable storage medium carries one or more program, when said one or multiple program quilts
When execution, the method according to the embodiment of the present disclosure is realized.
In accordance with an embodiment of the present disclosure, computer readable storage medium can be non-volatile computer-readable storage medium
Matter.Such as it can include but is not limited to: portable computer diskette, hard disk, random access storage device (RAM), read-only memory
(ROM), erasable programmable read only memory (EPROM or flash memory), portable compact disc read-only memory (CD-ROM), light
Memory device, magnetic memory device or above-mentioned any appropriate combination.In the disclosure, computer readable storage medium can
With to be any include or the tangible medium of storage program, the program can be commanded execution system, device or device use or
Person is in connection.
For example, in accordance with an embodiment of the present disclosure, computer readable storage medium may include above-described ROM 702
And/or one or more memories other than RAM 703 and/or ROM 702 and RAM 703.
Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the disclosure, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of above-mentioned module, program segment or code include one or more
Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box
The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical
On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants
It is noted that the combination of each box in block diagram or flow chart and the box in block diagram or flow chart, can use and execute rule
The dedicated hardware based systems of fixed functions or operations is realized, or can use the group of specialized hardware and computer instruction
It closes to realize.
Embodiment of the disclosure is described above.But the purpose that these embodiments are merely to illustrate that, and
It is not intended to limit the scope of the present disclosure.Although respectively describing each embodiment above, but it is not intended that each reality
Use cannot be advantageously combined by applying the measure in example.The scope of the present disclosure is defined by the appended claims and the equivalents thereof.It does not take off
From the scope of the present disclosure, those skilled in the art can make a variety of alternatives and modifications, these alternatives and modifications should all fall in this
Within scope of disclosure.
Claims (12)
1. a kind of data processing method executed by electronic equipment, comprising:
Acquire multiple online transaction logs from enterprise operation system；
Classified according to default element of taxonomy to the multiple online transaction log, obtains multiple log collection, wherein described more
Each log collection that a log is concentrated has corresponding classification information；
The online transaction log input Data Analysis Model corresponding with the classification information of log collection that each log is concentrated,
The online transaction log concentrated to the log being entered is handled to obtain processing result, wherein the multiple log is concentrated
Each log collection be respectively provided with corresponding Data Analysis Model；And
Export the processing result.
2. according to the method described in claim 1, wherein, being carried out according to default element of taxonomy to the multiple online transaction log
Classification includes:
Each online transaction log in the multiple online transaction log is parsed, from each online transaction log
The one or more keywords of middle extraction；And
By by each online transaction log one or more keywords with it is described it is pre- do not have element of taxonomy to be compared,
Classify to the multiple online transaction log.
3. according to the method described in claim 1, exporting the processing result and including:
According to the processing result automatically generated data report；And
Visualize the data sheet.
4. according to the method described in claim 1, wherein, the multiple log collection, which includes at least, to be belonged to first day of first category
Will collection and the second log collection for belonging to second category, the Data Analysis Model include at least the first analysis model and with described the
The second different analysis model of one analysis model, the class of online transaction log input and log collection that each log is concentrated
The corresponding Data Analysis Model of other information includes:
The first log collection is inputted into first analysis model；And
The second log collection is inputted into second analysis model.
5. according to the method described in claim 4, wherein, first analysis model and second analysis model are with lower die
One of type: transaction abnormality detection model, trend prediction model.
6. a kind of data processing equipment executed by electronic equipment, comprising:
Acquisition module, for acquiring multiple online transaction logs in enterprise operation system；
Categorization module obtains multiple logs for classifying according to default element of taxonomy to the multiple online transaction log
Collection, wherein each log collection that the multiple log is concentrated has corresponding classification information；
Input module, the online transaction log input for concentrating each log are corresponding with the classification information of log collection
Data Analysis Model, the online transaction log concentrated to the log being entered are handled to obtain processing result, wherein described
Each log collection that multiple logs are concentrated is respectively provided with corresponding Data Analysis Model；And
Output module, for exporting the processing result.
7. device according to claim 6, wherein the categorization module includes:
Resolution unit, for being parsed to each online transaction log in the multiple online transaction log, from described every
One or more keywords are extracted in a online transaction log；And
Comparing unit, for by by each online transaction log one or more keywords and the default classification
Element is compared, and is classified to the multiple online transaction log.
8. a kind of data processing system, comprising:
Enterprise operation system is stored with multiple online transaction logs；And
Data statistic analysis server, is used for:
Classify to multiple online transaction logs in the enterprise operation system got according to default element of taxonomy, obtains
To multiple log collection, wherein each log collection that the multiple log is concentrated has corresponding classification information；
The online transaction log input Data Analysis Model corresponding with the classification information of log collection that each log is concentrated,
The online transaction log concentrated to the log being entered is handled to obtain processing result, wherein the multiple log is concentrated
Each log collection be respectively provided with corresponding Data Analysis Model；And
Export the processing result.
9. data processing system according to claim 8, further includes:
Management server, for managing the Data Analysis Model stored in the data statistic analysis server.
10. data processing system according to claim 8, further includes:
Information alert server, for receiving the processing result of the data statistic analysis server output, and according to institute
It states processing result and determines whether that external equipment sends prompt information.
11. a kind of electronic equipment, comprising:
One or more processors；
Memory, for storing one or more instructions,
Wherein, when one or more of instructions are executed by one or more of processors, so that one or more of
Processor realizes method described in any one of claims 1 to 5.
12. a kind of computer readable storage medium, is stored thereon with executable instruction, which makes to handle when being executed by processor
Device realizes method described in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910622196.0A CN110347653A (en) | 2019-07-10 | 2019-07-10 | Data processing method and device, electronic equipment and readable storage medium storing program for executing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910622196.0A CN110347653A (en) | 2019-07-10 | 2019-07-10 | Data processing method and device, electronic equipment and readable storage medium storing program for executing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110347653A true CN110347653A (en) | 2019-10-18 |
Family
ID=68174804
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910622196.0A Pending CN110347653A (en) | 2019-07-10 | 2019-07-10 | Data processing method and device, electronic equipment and readable storage medium storing program for executing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110347653A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112363904A (en) * | 2020-11-30 | 2021-02-12 | 歌尔科技有限公司 | Log data analysis positioning method and device and computer readable storage medium |
| CN113326243A (en) * | 2021-05-27 | 2021-08-31 | 北京百度网讯科技有限公司 | Method and device for analyzing log data |
| CN113496031A (en) * | 2020-03-20 | 2021-10-12 | 拓尔思天行网安信息技术有限责任公司 | Enhanced analysis method, device, equipment and storage medium based on log security audit |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106940679A (en) * | 2017-02-23 | 2017-07-11 | 中科创达软件股份有限公司 | Data processing method and device |
| CN107256219A (en) * | 2017-04-24 | 2017-10-17 | 卡斯柯信号有限公司 | Big data convergence analysis method applied to automatic train control system massive logs |
| CN109063050A (en) * | 2018-07-19 | 2018-12-21 | 郑州云海信息技术有限公司 | A kind of database journal analysis and early warning method and apparatus |
| CN109558384A (en) * | 2018-09-29 | 2019-04-02 | 中国平安人寿保险股份有限公司 | Log classification method, device, electronic equipment and storage medium |
| CN109634818A (en) * | 2018-10-24 | 2019-04-16 | 中国平安人寿保险股份有限公司 | Log analysis method, system, terminal and computer readable storage medium |
| CN109783459A (en) * | 2019-01-04 | 2019-05-21 | 平安科技（深圳）有限公司 | The method, apparatus and computer readable storage medium of data are extracted from log |
-
2019
- 2019-07-10 CN CN201910622196.0A patent/CN110347653A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106940679A (en) * | 2017-02-23 | 2017-07-11 | 中科创达软件股份有限公司 | Data processing method and device |
| CN107256219A (en) * | 2017-04-24 | 2017-10-17 | 卡斯柯信号有限公司 | Big data convergence analysis method applied to automatic train control system massive logs |
| CN109063050A (en) * | 2018-07-19 | 2018-12-21 | 郑州云海信息技术有限公司 | A kind of database journal analysis and early warning method and apparatus |
| CN109558384A (en) * | 2018-09-29 | 2019-04-02 | 中国平安人寿保险股份有限公司 | Log classification method, device, electronic equipment and storage medium |
| CN109634818A (en) * | 2018-10-24 | 2019-04-16 | 中国平安人寿保险股份有限公司 | Log analysis method, system, terminal and computer readable storage medium |
| CN109783459A (en) * | 2019-01-04 | 2019-05-21 | 平安科技（深圳）有限公司 | The method, apparatus and computer readable storage medium of data are extracted from log |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113496031A (en) * | 2020-03-20 | 2021-10-12 | 拓尔思天行网安信息技术有限责任公司 | Enhanced analysis method, device, equipment and storage medium based on log security audit |
| CN113496031B (en) * | 2020-03-20 | 2023-09-22 | 拓尔思天行网安信息技术有限责任公司 | Log security audit-based enhanced analysis method, device, equipment and storage medium |
| CN112363904A (en) * | 2020-11-30 | 2021-02-12 | 歌尔科技有限公司 | Log data analysis positioning method and device and computer readable storage medium |
| CN112363904B (en) * | 2020-11-30 | 2022-11-22 | 歌尔科技有限公司 | Log data analysis positioning method and device and computer readable storage medium |
| CN113326243A (en) * | 2021-05-27 | 2021-08-31 | 北京百度网讯科技有限公司 | Method and device for analyzing log data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10311044B2 (en) | Distributed data variable analysis and hierarchical grouping system | |
| CN107577805B (en) | Business service system for log big data analysis | |
| US11631014B2 (en) | Computer-based systems configured for detecting, classifying, and visualizing events in large-scale, multivariate and multidimensional datasets and methods of use thereof | |
| CN111984499B (en) | Fault detection method and device for big data cluster | |
| US10181982B2 (en) | Distributed multi-data source performance management | |
| CN109344170B (en) | Stream data processing method, system, electronic device and readable storage medium | |
| CN110347653A (en) | Data processing method and device, electronic equipment and readable storage medium storing program for executing | |
| KR20160121806A (en) | Determining a temporary transaction limit | |
| US10031901B2 (en) | Narrative generation using pattern recognition | |
| CN111078980A (en) | Management method, device, equipment and storage medium based on credit investigation big data | |
| US11048677B1 (en) | Streamlined transaction and dimension data collection | |
| CN110677271B (en) | Big data alarm method, device, equipment and storage medium based on ELK | |
| CN110059234A (en) | Water utilities anomalous event method for detecting and device, computer installation and storage medium | |
| CN114995791A (en) | API (application program interface) polymerization method and device, electronic equipment and storage medium | |
| CN114625763A (en) | Information analysis method and device for database, electronic equipment and readable medium | |
| CN112035366A (en) | Test case generation method, device and equipment | |
| CN111612302A (en) | Group-level data management method and equipment | |
| CN112825165A (en) | Project quality management method and device | |
| CN112734352A (en) | Document auditing method and device based on data dimensionality | |
| US10152556B1 (en) | Semantic modeling platform | |
| CN117093609A (en) | Query statement processing method, device, equipment, medium and program product | |
| CN114048056A (en) | Root cause positioning method, apparatus, device, medium, and program product | |
| CN118154245A (en) | Client group circle selection method, device, electronic equipment and medium | |
| CN117271486A (en) | Data reporting method, device, equipment and storage medium based on artificial intelligence | |
| CN116205461A (en) | Service system peak guarantee method, device, equipment, medium and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |