CN109063150A - Big data extracting method, device, storage medium and server - Google Patents
Big data extracting method, device, storage medium and server Download PDFInfo
- Publication number
- CN109063150A CN109063150A CN201810895300.9A CN201810895300A CN109063150A CN 109063150 A CN109063150 A CN 109063150A CN 201810895300 A CN201810895300 A CN 201810895300A CN 109063150 A CN109063150 A CN 109063150A
- Authority
- CN
- China
- Prior art keywords
- data
- copy
- copied
- server
- total
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the present invention provides a kind of big data extracting method, for in emulating server, it the described method comprises the following steps: the total data of production server is copied, wherein the total data includes the configuration data of system itself and the status data that system operation generates；Simulation system is established in emulating server according to the total data of copy and carries out dry run；Obtain the data acquisition request with conditional information；Target data is extracted out from the configuration data of simulation system and status data according to the conditional information.The present invention can reduce the pressure to production server, improve the efficiency of production server.
Description
Technical field
The present invention relates to big data processing field, in particular to a kind of big data extracting method, device, storage medium and clothes
Business device.
Background technique
Server will not only receive the various data informations of external equipment in the prior art, and be sent out according to these data informations
Out indication signal give each external equipment, server can also to the operation result data met certain condition in operational process with
And initial data such as extracts, analyzes and backs up at the processing.
But server proposes the operation result data and initial data that meet certain condition in operational process
It takes, analyze and back up etc. and comprehend to the very big CPU usage of server is occupied, influence its normal control to external equipment
System.
Therefore, the prior art is defective, needs to improve.
Summary of the invention
The embodiment of the present invention provides a kind of big data extracting method, device, storage medium and server, can reduce production
The pressure of server improves the operational efficiency of production server.
The embodiment of the present invention provides a kind of big data extracting method, in emulating server, the method includes following
Step:
The total data of production server is copied, wherein the total data include system itself configuration data and
The status data that system operation generates；
Simulation system is established in emulating server according to the total data of copy and carries out dry run；
Obtain the data acquisition request with conditional information；
Target data is extracted out from the configuration data of simulation system and status data according to the conditional information.
In big data extracting method described in the embodiment of the present invention, the total data to production server is copied
The step of include:
Every preset time period copied totally with the instead preceding total data copied to the total data of production server.
In big data extracting method described in the embodiment of the present invention, the total data to production server is copied
The step of include:
When getting copy trigger signal, the type of identification copy trigger signal；
When the copy trigger signal is the first kind, to the system self configuration data and fortune in the total data
Row data are copied；
When the copy trigger signal is the second class, the operation data in the total data is copied.
In big data extracting method described in the embodiment of the present invention, the conditional information includes the conditional information packet
Include: one of dry run result feature, data type feature, temporal characteristics and Site characterization are a variety of.
In big data extracting method described in the embodiment of the present invention, it is described according to the conditional information from simulation system
The step of extracting target data out:
Extract the of corresponding data type out from the simulation system according to the data type feature in the conditional information
Two data；
Third data are filtered out from second data according to the temporal characteristics in the conditional information；
Target data is extracted out from the third data according to the dry run result feature of the simulation system.
A kind of big data extraction element, comprising:
Module is copied, is copied for the total data to production server, wherein the total data includes system itself
The status data that configuration data and system operation generate；
Dry run module establishes simulation system in emulating server for the total data according to copy and carries out simulation fortune
Row；
Module is obtained, for obtaining the data acquisition request with conditional information；
Extraction module, for extracting mesh out from the configuration data of simulation system and status data according to the conditional information
Mark data.
In big data extraction element described in the embodiment of the present invention, the copy module is used for every preset time period pair
The total data of production server copied totally with the instead preceding total data copied.
The copy module is used for copy module described in every preset time period
Recognition unit, for when getting copy trigger signal, identification to copy the type of trigger signal；
First copy cell, for when the copy trigger signal is the first kind, to the system in the total data
Self configuration data and operation data are copied；
Second copy cell, for when the copy trigger signal is the second class, to the operation in the total data
Data are copied.
From the foregoing, it will be observed that the present invention is by simulating the initial data obtained from production server, and generate simulation
Result data；Obtain the conditional information for the target data that will be extracted；According to the conditional information from the initial data and
Target data is obtained in the result data；The target data is analyzed and processed；To realize in production server
Data extraction and analysis processing, the pressure to production server can be reduced, improve the efficiency of production server.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly described.It should be evident that drawings in the following description are only some embodiments of the invention, for
For those skilled in the art, without creative efforts, it can also be obtained according to these attached drawings other attached
Figure.
Fig. 1 is the flow diagram of big data extracting method provided in an embodiment of the present invention.
Fig. 2 is the structural schematic diagram of big data extraction element provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description.Obviously, described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, those skilled in the art's every other implementation obtained under that premise of not paying creative labor
Example, belongs to protection scope of the present invention.
Description and claims of this specification and term " first " in above-mentioned attached drawing, " second ", " third " etc.
(if present) is to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should be appreciated that this
The object of sample description is interchangeable under appropriate circumstances.In addition, term " includes " and " having " and their any deformation, meaning
Figure, which is to cover, non-exclusive includes.For example, containing the process, method of series of steps or containing a series of modules or list
The device of member, server, system those of are not necessarily limited to be clearly listed step or module or unit, can also include without clear
The step of listing to Chu or module or unit also may include intrinsic for these process, methods, device, server or system
Other steps or module or unit.
It is the flow chart of the big data extracting method in the embodiment of the present invention with reference to Fig. 1, Fig. 1.Method can be applied to mould
In quasi- server.In the present embodiment, method includes the following steps:
S101, the total data of production server is copied, wherein the total data include system self configuration data with
And operation data.
Wherein, the total data may include the system self configuration data of the operating system of this composition production server with
And the operation data run in the system.The system self configuration data was updated once every the disconnected time, just as common
Operating system it is the same.
In some embodiments, step S101 includes: that every preset time period carries out the total data of production server
Copy.For example, can carry out once again at interval of 12 hours, the content once copied afterwards once copies interior cover comprehensively before
Hold.
In some embodiments, step S101 includes:
S1011, when get copy trigger signal when, identification copy trigger signal type.
Wherein, the type of the copy trigger signal is generated according to default.For example, default is to service on every Wendesdays production
The system of device is upgraded, and therefore, when Wednesday, the simulation system of emulating server also needs to be updated, therefore it is given birth to
At copy trigger signal be the first kind, other times generate copy trigger signal be then the second class.
S1012, when the copy trigger signal is the first kind, to the system self configuration data in the total data
And operation data is copied.
When the copy trigger signal is the second class, the operation data in the total data is copied.
S102, simulation system is established in emulating server according to the total data of copy and carries out dry run.
Virtual operation platform is provided in the emulating server, from production server obtain initial data, and according to
The identical logic of production server, simulates the initial data.Wherein, the operation data in the total data may include with
Under a variety of data: image data, audio data, image data, numerical data, lteral data, finger print data etc..
In some embodiments, step S102 includes:
S1021, the data characteristic information that the target data block is required to meet is obtained；
S1022, the result type that the target data block is required to meet is obtained.
Wherein, which includes at least one of following characteristics: data type feature, temporal characteristics,
Point feature, key characteristics.The result type can be the result that initial data is identified, the result of processing or other
The result of processing.
S103, the data acquisition request with conditional information is obtained.
Preferably, it includes: dry run result feature, data type spy that the conditional information, which includes the conditional information,
One of sign, temporal characteristics and Site characterization are a variety of.
S104, target data is extracted out from simulation system according to the conditional information.
Wherein, which can be the operation data in the total data most started, be also possible to operation data process
The operation result data generated after dry run.
Wherein it is possible to be backed up, classified, be associated with, artificial intelligence to the target data extracted according to existing technology
The processing such as study.
For example, the data characteristic information of multiple target data blocks can be identical or not identical, multiple target data blocks
Mode it is different, for example, it may be possible to which having some target data blocks is image data, some target data blocks are lteral datas.And
It is that identification identifies image data to image data, and is identification keyword therein to the simulation of lteral data.
In some embodiments, step S104 includes:
S1041, corresponding data class is extracted out from the simulation system according to the data type feature in the conditional information
Second data of type；
S1042, third data are filtered out from second data according to the temporal characteristics in the conditional information；
S1043, target data is extracted out from the third data according to the dry run result feature of the simulation system.
For example, including image data, lteral data and video data in the initial data.Data characteristic information includes time spy
Sign, data type feature, Site characterization.Herein, the screening conditions of the data characteristic information be XX YY days ZZ when, picture number
Accordingly and place is located at the Tian'anmen Square.And in the present embodiment, the type of simulation is picture recognition, the item of operation result type
Part are as follows: identify the black race in picture to be screened.
According to data type feature, temporal characteristics and the Site characterization of second data block and the third data
Block is grouped storage to the multiple target data block.
From the foregoing, it will be observed that the present invention is by simulating the initial data obtained from production server, and generate simulation
Result data；Obtain the conditional information for the target data that will be extracted；According to the conditional information from the initial data and
Target data is obtained in the result data；The target data is analyzed and processed；To realize in production server
Data extraction and analysis processing, the pressure to production server can be reduced, improve the efficiency of production server.
Referring to figure 2., Fig. 2 is the structure chart of one of embodiment of the present invention big data extraction element.The device includes:
Module 201 is copied, is copied for the total data to production server, wherein the total data includes system itself
Configuration data and operation data；
Dry run module 202 establishes simulation system in emulating server for the total data according to copy and carries out mould
Quasi- operation；
Module 203 is obtained, for obtaining the data acquisition request with conditional information；
Extraction module 204, for extracting target data out from simulation system according to the conditional information.
Preferably, the copy module 201 copies the total data of production server for every preset time period.
Preferably, copy module 201 includes: recognition unit, the first copy cell and the second copy cell.
Wherein, recognition unit is used for when getting copy trigger signal, the type of identification copy trigger signal.
Wherein, the first copy cell is used for when the copy trigger signal is the first kind, in the total data
System self configuration data and operation data are copied.
Wherein, the second copy cell is used for when the copy trigger signal is the second class, in the total data
Operation data is copied.Wherein, the type of the copy trigger signal is generated according to default.For example, default is on every Wendesdays
The system of production server is upgraded, therefore, when Wednesday, the simulation system of emulating server also needs to carry out more
Newly, therefore its copy trigger signal generated is the first kind, and the copy trigger signal that other times generate then is the second class.
The embodiment of the present invention also provides a kind of storage medium, computer program is stored in the storage medium, when described
When computer program is run on computers, so that the computer executes the big data extracting method in above-described embodiment.
The embodiment of the present invention also provides a kind of server, including processor and memory, is stored with meter in the memory
Calculation machine program, the processor is by calling the computer program stored in the memory, for executing above-mentioned implementation
Big data extracting method in example.
It should be noted that those of ordinary skill in the art will appreciate that whole in the various methods of above-described embodiment or
Part steps are relevant hardware can be instructed to complete by program, which can store in computer-readable storage medium
In matter, which be can include but is not limited to: read-only memory (ROM, Read Only Memory), random access memory
Device (RAM, Random Access Memory), disk or CD etc..
Big data extracting method, device, storage medium and server is provided for the embodiments of the invention above to carry out
It is discussed in detail, used herein a specific example illustrates the principle and implementation of the invention, above embodiments
Illustrate to be merely used to help understand method and its core concept of the invention；Meanwhile for those skilled in the art, according to this
The thought of invention, there will be changes in the specific implementation manner and application range, in conclusion the content of the present specification is not answered
It is interpreted as limitation of the present invention.
Claims (10)
1. a kind of big data extracting method, in emulating server, which is characterized in that the described method comprises the following steps:
The total data of production server is copied, wherein the total data includes the configuration data and system of system itself
Run the status data generated；
Simulation system is established in emulating server according to the total data of copy and carries out dry run；
Obtain the data acquisition request with conditional information；
Target data is extracted out from the configuration data of simulation system and status data according to the conditional information.
2. big data extracting method according to claim 1, which is characterized in that the total data to production server into
Row copy the step of include:
Every preset time period copied totally with the instead preceding total data copied to the total data of production server.
3. big data extracting method according to claim 2, which is characterized in that the total data to production server into
Row copy the step of include:
When getting copy trigger signal, the type of identification copy trigger signal；
When the copy trigger signal is the first kind, to the system self configuration data and operation number in the total data
According to being copied；
When the copy trigger signal is the second class, the operation data in the total data is copied.
4. big data extracting method according to claim 1, which is characterized in that the conditional information includes the condition letter
Breath includes: one of dry run result feature, data type feature, temporal characteristics and Site characterization or a variety of.
5. big data extracting method according to claim 4, which is characterized in that it is described according to the conditional information from simulation
The step of extracting target data in system out:
Extract the second number of corresponding data type out from the simulation system according to the data type feature in the conditional information
According to；
Third data are filtered out from second data according to the temporal characteristics in the conditional information；
Target data is extracted out from the third data according to the dry run result feature of the simulation system.
6. a kind of big data extraction element characterized by comprising
Module is copied, is copied for the total data to production server, wherein the total data includes the configuration of system itself
The status data that data and system operation generate；
Dry run module establishes simulation system in emulating server for the total data according to copy and carries out dry run；
Module is obtained, for obtaining the data acquisition request with conditional information；
Extraction module, for extracting number of targets out from the configuration data of simulation system and status data according to the conditional information
According to.
7. big data extraction element according to claim 6, which is characterized in that the copy module is used for when default
Between section copy totally is carried out to copy before instead to the total data of production server total data.
8. big data extraction element according to claim 7, which is characterized in that the copy module includes:
Recognition unit, for when getting copy trigger signal, identification to copy the type of trigger signal；
First copy cell, for when the copy trigger signal is the first kind, to the system in the total data itself
Configuration data and operation data are copied；
Second copy cell, for when the copy trigger signal is the second class, to the operation data in the total data
It is copied.
9. a kind of storage medium, which is characterized in that computer program is stored in the storage medium, when the computer program
When running on computers, so that the computer perform claim requires 1 to 5 described in any item methods.
10. a kind of server, which is characterized in that including processor and memory, it is stored with computer program in the memory,
The processor requires any one of 1 to 5 by calling the computer program stored in the memory, for perform claim
The method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810895300.9A CN109063150A (en) | 2018-08-08 | 2018-08-08 | Big data extracting method, device, storage medium and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810895300.9A CN109063150A (en) | 2018-08-08 | 2018-08-08 | Big data extracting method, device, storage medium and server |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109063150A true CN109063150A (en) | 2018-12-21 |
Family
ID=64678503
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810895300.9A Pending CN109063150A (en) | 2018-08-08 | 2018-08-08 | Big data extracting method, device, storage medium and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109063150A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105335253A (en) * | 2015-10-28 | 2016-02-17 | 北京百度网讯科技有限公司 | Method and device for creating virtual machine system disk snapshots |
| CN105354113A (en) * | 2015-11-27 | 2016-02-24 | 上海爱数信息技术股份有限公司 | Server, and system and method for managing server |
| CN105389335A (en) * | 2015-10-15 | 2016-03-09 | 佛山市云端容灾信息技术有限公司 | Data mining analysis method and device based on automatic exercise |
| CN106776998A (en) * | 2016-12-06 | 2017-05-31 | 华为技术有限公司 | A kind of database service provides method and server |
| US20170300550A1 (en) * | 2015-11-02 | 2017-10-19 | StoreReduce | Data Cloning System and Process |
| US20180191753A1 (en) * | 2014-02-03 | 2018-07-05 | Intuit Inc. | System and method for providing a self-monitoring, self-reporting, and self-repairing virtual asset configured for extrusion and intrusion detection and threat scoring in a cloud computing environment |
| CN108388493A (en) * | 2018-02-11 | 2018-08-10 | 佛山市云端容灾信息技术有限公司 | Big data extraction and analytical method, device, storage medium and server |
-
2018
- 2018-08-08 CN CN201810895300.9A patent/CN109063150A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180191753A1 (en) * | 2014-02-03 | 2018-07-05 | Intuit Inc. | System and method for providing a self-monitoring, self-reporting, and self-repairing virtual asset configured for extrusion and intrusion detection and threat scoring in a cloud computing environment |
| CN105389335A (en) * | 2015-10-15 | 2016-03-09 | 佛山市云端容灾信息技术有限公司 | Data mining analysis method and device based on automatic exercise |
| CN105335253A (en) * | 2015-10-28 | 2016-02-17 | 北京百度网讯科技有限公司 | Method and device for creating virtual machine system disk snapshots |
| US20170300550A1 (en) * | 2015-11-02 | 2017-10-19 | StoreReduce | Data Cloning System and Process |
| CN105354113A (en) * | 2015-11-27 | 2016-02-24 | 上海爱数信息技术股份有限公司 | Server, and system and method for managing server |
| CN106776998A (en) * | 2016-12-06 | 2017-05-31 | 华为技术有限公司 | A kind of database service provides method and server |
| CN108388493A (en) * | 2018-02-11 | 2018-08-10 | 佛山市云端容灾信息技术有限公司 | Big data extraction and analytical method, device, storage medium and server |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102413013B (en) | Method and device for detecting abnormal network behavior | |
| CN105740707B (en) | The recognition methods of malicious file and device | |
| US11221904B2 (en) | Log analysis system, log analysis method, and log analysis program | |
| CN109241780B (en) | Method, device and equipment for detecting hidden information of image | |
| CN103608765A (en) | Virtual machine snapshotting and analysis | |
| CN108521588B (en) | Anchor ranking method and system based on time slicing, server and storage medium | |
| CN110278201B (en) | Security policy evaluation method and device, computer readable medium and electronic device | |
| CN110471945B (en) | Active data processing method, system, computer equipment and storage medium | |
| CN110414375B (en) | Low-altitude target identification method and device, storage medium and electronic equipment | |
| CN108304322B (en) | Pressure testing method and terminal equipment | |
| CN115033876A (en) | Log processing method, log processing device, computer device and storage medium | |
| CN115269359A (en) | Terminal interface testing method and device | |
| CN108388493A (en) | Big data extraction and analytical method, device, storage medium and server | |
| CN109063150A (en) | Big data extracting method, device, storage medium and server | |
| CN112365269A (en) | Risk detection method, apparatus, device and storage medium | |
| WO2009097241A1 (en) | A method and system for assessing deployment and un-deployment of software installations | |
| CN116707859A (en) | Feature rule extraction method and device, and network intrusion detection method and device | |
| CN111240923A (en) | Automatic test method and device for recurring problems of vehicle navigation system and storage medium | |
| CN112182441A (en) | Method and device for detecting violation data | |
| CN110544166A (en) | Sample generation method, device and storage medium | |
| CN114090650A (en) | Sample data identification method and device, electronic equipment and storage medium | |
| CN109542722A (en) | Anomaly analysis processing method, device and storage medium | |
| US20190294523A1 (en) | Anomaly identification system, method, and storage medium | |
| CN115526859A (en) | Method for identifying production defects, distributed processing platform, equipment and storage medium | |
| CN114003784A (en) | Request recording method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20181221 |
|
| WD01 | Invention patent application deemed withdrawn after publication |