CN112291121B - Data processing method and related equipment - Google Patents
Data processing method and related equipment Download PDFInfo
- Publication number
- CN112291121B CN112291121B CN202011600584.8A CN202011600584A CN112291121B CN 112291121 B CN112291121 B CN 112291121B CN 202011600584 A CN202011600584 A CN 202011600584A CN 112291121 B CN112291121 B CN 112291121B
- Authority
- CN
- China
- Prior art keywords
- consumer
- network
- network data
- target
- message queue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 34
- 239000000523 sample Substances 0.000 claims abstract description 141
- 238000012545 processing Methods 0.000 claims abstract description 24
- 238000000034 method Methods 0.000 claims abstract description 19
- 238000012544 monitoring process Methods 0.000 claims abstract description 12
- 238000003860 storage Methods 0.000 claims description 16
- 230000000694 effects Effects 0.000 abstract description 4
- 238000004590 computer program Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000011156 evaluation Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000009826 distribution Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013481 data capture Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/02—Capturing of monitoring data
- H04L43/028—Capturing of monitoring data by filtering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L61/00—Network arrangements, protocols or services for addressing or naming
- H04L61/50—Address allocation
- H04L61/5007—Internet protocol [IP] addresses
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer And Data Communications (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the application provides a data processing method and related equipment, which belong to the technical field of networks and specifically comprise the following steps: the method comprises the steps that a producer obtains network probe interface parameters and stores the network probe interface parameters into a message queue; monitoring whether the network data corresponding to the target network probe interface parameters of the message queue is the target network data needing to be consumed or not by a consumer; and under the condition that the network data corresponding to the network probe interface parameters of the message queue is the target network data needing to be consumed, consuming the target network data through the consumer. Through the processing scheme, the consumer can sequentially consume the network data acquired by the network probe, the network data of the network probe is prevented from being lost, the integrity of network data processing is effectively improved, and a network data processing effect is provided.
Description
Technical Field
The present application relates to the field of network technologies, and in particular, to a data processing method and a related device.
Background
As network technologies continue to develop, various services may be provided through the network. There are a variety of software programs that require the production of large amounts of data, and a variety of software programs that require data produced based on other software programs to operate. The probe is used as a device for acquiring network data, can also be called as a network probe, can intercept network data packets, and can realize capture, filtration and analysis of the data packets on the internet probe. The probe captures network data from the network according to a certain interval time, the system also acquires the network data from the probe according to a certain interval time, and the network data are lost due to network abnormality and other conditions.
Disclosure of Invention
In view of the above, embodiments of the present disclosure provide a data processing method and a related device, which at least partially solve the problems in the prior art.
In a first aspect, an embodiment of the present disclosure provides a data processing method, including:
acquiring network probe interface parameters by a producer, and storing the network probe interface parameters into a message queue;
monitoring whether the network data corresponding to the target network probe interface parameters of the message queue is the target network data needing to be consumed or not by a consumer;
and under the condition that the network data corresponding to the network probe interface parameters of the message queue is the target network data needing to be consumed, the target network data is consumed through the consumer.
In a second aspect, an embodiment of the present disclosure provides a data processing apparatus, including:
the acquisition module is used for acquiring network probe interface parameters through a producer and storing the network probe interface parameters into a message queue;
the monitoring module is used for monitoring whether the network data corresponding to the target network probe interface parameters of the message queue is the target network data needing to be consumed or not through a consumer;
and the processing module is used for consuming the target network data by the consumer under the condition that the network data corresponding to the network probe interface parameters of the message queue is the target network data required to be consumed.
In a third aspect, an embodiment of the present disclosure further provides an electronic device, where the electronic device includes:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the data processing method of the first aspect or any implementation manner of the first aspect.
In a fourth aspect, the disclosed embodiments also provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the data processing method of the first aspect or any implementation manner of the first aspect.
In a fifth aspect, the disclosed embodiments also provide a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to perform the data processing method of the first aspect or any of the implementations of the first aspect.
The data processing method in the embodiment of the disclosure includes the steps that a producer obtains network probe interface parameters, and the network probe interface parameters are stored in a message queue; monitoring whether the network data corresponding to the target network probe interface parameters of the message queue is the target network data needing to be consumed or not by a consumer; and under the condition that the network data corresponding to the network probe interface parameters of the message queue is the target network data needing to be consumed, the target network data is consumed through the consumer. According to the scheme, the producer stores the network probe interface parameters into the message queue, when the consumer monitors that the network data corresponding to the network probe interface parameters in the message queue are the network data needing to be consumed, the consumer consumes the network data, the network data obtained by the network probe can be consumed in order, the network data of the network probe is prevented from being lost, the integrity of network data processing is effectively improved, and the network data processing effect is provided.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flow chart of a data processing method according to an embodiment of the present invention;
fig. 2 is a flowchart of step S102 in the data processing method according to the embodiment of the present invention;
FIG. 3 is a block diagram of a data processing apparatus according to an embodiment of the present invention;
fig. 4 is a structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The embodiments of the present application will be described in detail below with reference to the accompanying drawings.
The following description of the embodiments of the present application is provided by way of specific examples, and other advantages and effects of the present application will be readily apparent to those skilled in the art from the disclosure herein. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. The present application is capable of other and different embodiments and its several details are capable of modifications and/or changes in various respects, all without departing from the spirit of the present application. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It is noted that various aspects of the embodiments are described below within the scope of the appended claims. It should be apparent that the aspects described herein may be embodied in a wide variety of forms and that any specific structure and/or function described herein is merely illustrative. Based on the present application, one skilled in the art should appreciate that one aspect described herein may be implemented independently of any other aspects and that two or more of these aspects may be combined in various ways. For example, an apparatus may be implemented and/or a method practiced using any number of the aspects set forth herein. Additionally, such an apparatus may be implemented and/or such a method may be practiced using other structure and/or functionality in addition to one or more of the aspects set forth herein.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present application, and the drawings only show the components related to the present application rather than the number, shape and size of the components in actual implementation, and the type, amount and ratio of the components in actual implementation may be changed arbitrarily, and the layout of the components may be more complicated.
In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects may be practiced without these specific details.
The embodiment of the application provides a data processing method.
Referring to fig. 1, a data processing method provided in an embodiment of the present disclosure includes:
and step S101, obtaining the network probe interface parameters through a producer, and storing the network probe interface parameters into a message queue.
In this embodiment, the producer may be a software program and the producer may produce data. The network probe can monitor the network data packet and can realize the functions of capturing, filtering, analyzing and the like on the network data packet.
Preferably, the network probe interface parameters include a start time, an end time, probe parameters and network indexes, wherein the probe parameters include parameters such as a probe address and a probe port, and the network indexes include indexes such as a session number and a flow.
It should be noted that the probe address is similar to the IP address, and one probe address includes a plurality of probe ports, for example, the probe address may be 10.0.12.1, and the probe ports may be 1, 2, 3, 4, 5, and so on. In this embodiment, the message queue may be stored in a local database, or may be stored in a server.
Preferably, the message queue further includes producer information of the producer, and the producer information is stored in correspondence with the network probe interface parameter. The message queue also comprises consumer information of the consumers and contents generated after the consumers consume, and the consumer information and the contents generated after the consumers consume are correspondingly stored. The producer information of the message queue includes a producer key and the consumer information of the message queue includes a consumer key.
In this embodiment, the parameters produced by different producers are different, for example, the number of sessions, the flow rate, etc. produced by different producers are different. The producer is uniquely identified by annotating the corresponding producer key to the producer, for example, by setting the producer key to a.b.c, numbering with letters and point numbers, and uniquely identifying. The consumers are uniquely identified by annotating corresponding consumer keywords to the consumers, for example, the consumer keywords are set to a.b #, and are numbered by letters, dot numbers and well numbers.
Therefore, the message queue correspondingly stores the producer information and the network probe interface parameters, the consumer information and the content generated after the consumer consumes, the information can be backed up and stored, and the subsequent data processing process is facilitated.
And step S102, monitoring whether the network data corresponding to the target network probe interface parameters of the message queue is the target network data needing to be consumed or not by the consumer.
In this embodiment, the consumer may be a software program, and the consumer may consume the related data. The message queue comprises one or more network probe interface parameters, one network probe interface parameter can be read in sequence according to the storage sequence, and the read network probe interface parameter is used as the target network probe interface parameter. The target network probe interface parameter of the message queue corresponds to a target network probe, and the target network probe comprises corresponding network data. When the consumer judges that the network data is the target network data which needs to be consumed, the consumer can consume the target network data.
Preferably, referring to fig. 2, step S102 includes the following processes:
step S1021, whether producer information of the message queue comprises the target keyword is judged through the consumer.
In this embodiment, a producer key corresponding to a producer is determined according to a consumer key of a consumer, the determined producer information is used as a target key, and if the producer information of the message queue includes the target key, it indicates that data produced by the corresponding producer is data that the consumer needs to consume.
Step S1022, when the producer information of the message queue includes the target keyword, determining that the producer information of the message queue is the target producer information, determining a target network probe interface parameter corresponding to the target producer information according to the message queue, and determining network data corresponding to the target network probe interface parameter as target network data to be consumed.
Therefore, the target network data which needs to be consumed by the consumer can be quickly determined, and the data processing speed is improved.
Preferably, the determining the network data corresponding to the target network probe interface parameter in step S1022 as the target network data to be consumed includes:
and determining a target probe address and a target probe port of the target network probe interface parameters by a consumer, acquiring corresponding network data from the target probe address and the target probe port, and consuming the acquired network data.
For example, the target probe address is 10.0.12.1, and the target probe port is probe port 5, the customer obtains the corresponding network data from probe port 5 of probe address 10.0.12.1. In this embodiment, each of 5 minutes and 3 minutes can be preset, the network probe performs network data capture once, the preset time is not limited, and the network probe interface parameters including the start time and the end time can be obtained according to the preset time. The method comprises the steps that a consumer obtains network probe interface parameters, and obtains network data from a network probe in a mode of POST or GET in a Hypertext transfer Protocol (HTTP), namely, the network data are obtained from a target probe address and a target probe port.
Step S1023, in a case that the producer information of the message queue does not include the target keyword, determining that the network data corresponding to the network probe interface parameter of the message queue is network data that does not need to be consumed.
Therefore, network data which do not need to be consumed can be screened out, and data processing time is saved.
And step S103, consuming the target network data by the consumer under the condition that the network data corresponding to the network probe interface parameters of the message queue is the target network data required to be consumed.
Therefore, the producer stores the network probe interface parameters into the message queue, and when the consumer monitors that the network data corresponding to the network probe interface parameters in the message queue are the network data needing to be consumed, the consumer consumes the network data, can sequentially consume the network data acquired by the network probe, avoids the network data loss of the network probe, effectively improves the integrity of network data processing, and provides a network data processing effect.
Preferably, after step S103, the data processing method further includes:
and under the condition that the consumer successfully consumes the target network data, displaying the data generated after the consumer consumes through the front-end equipment.
In the embodiment, the data generated by the consumer is displayed through the front end, so that the consumption result of the consumer can be visually displayed, the richness of the displayed data is improved, the user can conveniently know the working progress of the consumer, and the accuracy of front end display is ensured.
Preferably, after step S103, the data processing method further includes:
and under the condition that the consumer successfully consumes the target network data, comparing the data generated after the consumer consumes with a preset alarm number threshold value, and under the condition that the alarm threshold value is reached, carrying out alarm operation.
Therefore, when the data generated by the consumer reaches the preset alarm data, the alarm operation is carried out, the alarm accuracy is improved, and the alarm error is reduced.
Preferably, after step S103, the data processing method further includes:
and under the condition that the consumer fails to consume the target network data, sending the target network probe interface parameters corresponding to the target network data to the producer.
Therefore, after receiving the network probe interface parameters sent by the consumer, the producer can determine the network data required to be consumed by the consumer according to the received network probe interface parameters, re-determine the network probe interface parameters corresponding to the network data required to be consumed from the network probe, write the re-confirmed network probe interface parameters into the message queue, obtain the re-confirmed network probe interface parameters from the message queue by the consumer, write the re-confirmed network probe interface parameters into the message queue, and continue to complete data consumption.
Corresponding to the above method embodiment, referring to fig. 3, the present disclosure also provides a data processing apparatus 300, including:
an obtaining module 301, configured to obtain network probe interface parameters through a producer, and store the network probe interface parameters in a message queue;
a monitoring module 302, configured to monitor, by a consumer, whether network data corresponding to a target network probe interface parameter of the message queue is target network data that needs to be consumed;
the processing module 303 is configured to consume, by the consumer, the target network data when the network data corresponding to the network probe interface parameter of the message queue is the target network data that needs to be consumed.
Preferably, the message queue further includes producer information of the producer, and the producer information is stored in correspondence with the network probe interface parameters acquired by the producer.
Preferably, the message queue further comprises consumer information of the consumer and content generated after the consumer consumes, and the consumer information and the content generated after the consumer consumes are stored correspondingly.
Preferably, the network probe interface parameters include a start time, an end time, probe parameters and a network index, and the probe parameters include a probe address and a probe port.
Preferably, the producer information of the message queue includes a producer key, and the consumer information of the message queue includes a consumer key.
Preferably, the monitoring module 302 includes:
a judging submodule, configured to judge, by the consumer, whether the producer information of the message queue includes a target keyword;
a first determining sub-module, configured to determine that the producer information of the message queue is target producer information when the producer information of the message queue includes a target keyword, determine a target network probe interface parameter corresponding to the target producer information according to the message queue, and determine network data corresponding to the target network probe interface parameter as target network data that needs to be consumed;
and the second determining submodule is used for determining that the network data corresponding to the network probe interface parameters of the message queue is network data which does not need to be consumed under the condition that the producer information of the message queue does not comprise the target keyword.
Preferably, the first determining sub-module is further configured to determine, by the consumer, a target probe address and a target probe port of the target network probe interface parameter, obtain corresponding network data from the target probe address and the target probe port, and determine the obtained network data as the target network data.
Preferably, the data processing apparatus 300 further includes:
and the display module is used for displaying the data generated after the consumer consumes the target network data through front-end equipment under the condition that the consumer successfully consumes the target network data.
Preferably, the data processing apparatus 300 further includes:
and the alarm module is used for comparing the data generated after the consumer consumes the target network data with a preset alarm number threshold value under the condition that the consumer successfully consumes the target network data, and carrying out alarm operation under the condition that the alarm threshold value is reached.
Preferably, the data processing apparatus 300 further includes:
and the sending module is used for sending the target network probe interface parameters corresponding to the target network data to the producer under the condition that the consumer fails to consume the target network data.
The apparatus shown in fig. 3 may correspondingly execute the content in the above method embodiment, and details of the part not described in detail in this embodiment refer to the content described in the above method embodiment, which is not described again here.
Referring to fig. 4, an embodiment of the present disclosure also provides an electronic device 40, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the data processing method of the foregoing method embodiments.
The disclosed embodiments also provide a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the data processing method in the foregoing method embodiments.
The disclosed embodiments also provide a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, cause the computer to perform the data processing method in the aforementioned method embodiments.
Referring now to FIG. 4, a block diagram of an electronic device 40 suitable for use in implementing embodiments of the present disclosure is shown. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., car navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 4 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 4, the electronic device 40 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 401 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM) 402 or a program loaded from a storage means 408 into a Random Access Memory (RAM) 403. In the RAM 403, various programs and data necessary for the operation of the electronic apparatus 40 are also stored. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.
Generally, the following devices may be connected to the I/O interface 405: input devices 406 including, for example, a touch screen, touch pad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; an output device 407 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 408 including, for example, tape, hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 40 to communicate wirelessly or by wire with other devices to exchange data. While the figures illustrate an electronic device 40 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication device 409, or from the storage device 408, or from the ROM 402. The computer program performs the above-described functions defined in the methods of the embodiments of the present disclosure when executed by the processing device 401.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring at least two internet protocol addresses; sending a node evaluation request comprising the at least two internet protocol addresses to node evaluation equipment, wherein the node evaluation equipment selects the internet protocol addresses from the at least two internet protocol addresses and returns the internet protocol addresses; receiving an internet protocol address returned by the node evaluation equipment; wherein the obtained internet protocol address indicates an edge node in the content distribution network.
Alternatively, the computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: receiving a node evaluation request comprising at least two internet protocol addresses; selecting an internet protocol address from the at least two internet protocol addresses; returning the selected internet protocol address; wherein the received internet protocol address indicates an edge node in the content distribution network.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a unit does not in some cases constitute a limitation of the unit itself, for example, the first retrieving unit may also be described as a "unit for retrieving at least two internet protocol addresses".
It should be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof.
The above description is only for the specific embodiments of the present disclosure, but the scope of the present disclosure is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present disclosure should be covered within the scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (13)
1. A data processing method, comprising:
acquiring network probe interface parameters by a producer, and storing the network probe interface parameters into a message queue;
monitoring whether the network data corresponding to the target network probe interface parameters of the message queue is the target network data needing to be consumed or not by a consumer;
consuming the target network data by the consumer under the condition that the network data corresponding to the network probe interface parameters of the message queue is the target network data required to be consumed,
the method comprises the steps that under the condition that a consumer fails to consume target network data, target network probe interface parameters corresponding to the target network data are sent to a producer, the producer determines the network data which the consumer needs to consume according to the received network probe interface parameters, the network probe interface parameters corresponding to the network data which the consumer needs to consume are determined from a network probe again, the network probe interface parameters which are confirmed again are written into a message queue, and the consumer obtains the network probe interface parameters which are confirmed again from the message queue and writes the network probe interface parameters into the message queue.
2. The data processing method according to claim 1, wherein the message queue further includes producer information of the producer, and the producer information is stored in correspondence with the network probe interface parameters acquired by the producer.
3. The data processing method of claim 2, wherein the message queue further comprises consumer information of the consumer and content generated after consumption by the consumer, and the consumer information is stored in correspondence with the content generated after consumption by the consumer.
4. The data processing method of claim 3, wherein the network probe interface parameters comprise a start time, an end time, probe parameters and a network index, and the probe parameters comprise a probe address and a probe port.
5. The data processing method of claim 4, wherein the producer information of the message queue comprises a producer key and the consumer information of the message queue comprises a consumer key.
6. The data processing method of claim 5, wherein the monitoring, by the consumer, whether the network data corresponding to the target network probe interface parameter of the message queue is the target network data that needs to be consumed comprises:
determining, by the consumer, whether the producer information of the message queue includes a target keyword;
determining that the producer information of the message queue is target producer information under the condition that the producer information of the message queue comprises a target keyword, determining a target network probe interface parameter corresponding to the target producer information according to the message queue, and determining network data corresponding to the target network probe interface parameter as target network data needing to be consumed;
and under the condition that the producer information of the message queue does not comprise a target keyword, determining that the network data corresponding to the network probe interface parameter of the message queue is network data which does not need to be consumed.
7. The data processing method according to claim 6, wherein the determining the network data corresponding to the target network probe interface parameter as the target network data to be consumed comprises:
and determining a target probe address and a target probe port of the target network probe interface parameters by the consumer, acquiring corresponding network data from the target probe address and the target probe port, and determining the acquired network data as the target network data.
8. The data processing method of claim 7, wherein after the consumption of the target network data by the consumer, the data processing method further comprises:
and under the condition that the consumer successfully consumes the target network data, displaying the data generated after the consumer consumes through front-end equipment.
9. The data processing method of claim 7, wherein after the target network data is consumed by the consumer, the data processing method further comprises:
and under the condition that the consumer successfully consumes the target network data, comparing the data generated after the consumer consumes with a preset alarm number threshold value, and under the condition that the alarm threshold value is reached, carrying out alarm operation.
10. The data processing method of claim 7, wherein the consuming the target network data by the consumer comprises:
and under the condition that the consumer fails to consume the target network data, sending the target network probe interface parameters corresponding to the target network data to the producer.
11. A data processing apparatus, comprising:
the acquisition module is used for acquiring network probe interface parameters through a producer and storing the network probe interface parameters into a message queue;
the monitoring module is used for monitoring whether the network data corresponding to the target network probe interface parameters of the message queue is the target network data needing to be consumed or not through a consumer;
a processing module, configured to consume, by the consumer, the target network data when the network data corresponding to the network probe interface parameter of the message queue is the target network data that needs to be consumed,
the method comprises the steps that under the condition that a consumer fails to consume target network data, target network probe interface parameters corresponding to the target network data are sent to a producer, the producer determines the network data which the consumer needs to consume according to the received network probe interface parameters, the network probe interface parameters corresponding to the network data which the consumer needs to consume are determined from a network probe again, the network probe interface parameters which are confirmed again are written into a message queue, and the consumer obtains the network probe interface parameters which are confirmed again from the message queue and writes the network probe interface parameters into the message queue.
12. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the data processing method of any one of the preceding claims 1-10.
13. A non-transitory computer readable storage medium storing a program
Storing computer instructions for causing the computer to perform the data processing method of any preceding claim 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011600584.8A CN112291121B (en) | 2020-12-30 | 2020-12-30 | Data processing method and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011600584.8A CN112291121B (en) | 2020-12-30 | 2020-12-30 | Data processing method and related equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112291121A CN112291121A (en) | 2021-01-29 |
| CN112291121B true CN112291121B (en) | 2021-08-03 |
Family
ID=74426359
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011600584.8A Active CN112291121B (en) | 2020-12-30 | 2020-12-30 | Data processing method and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112291121B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112882846B (en) * | 2021-02-19 | 2024-02-09 | 深圳市云网万店科技有限公司 | Data processing method and device for message queue, computer equipment and storage medium |
| CN113055480B (en) * | 2021-03-17 | 2023-05-23 | 网宿科技股份有限公司 | Scheduling method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102419833A (en) * | 2011-09-23 | 2012-04-18 | 方欣科技有限公司 | Workflow engine system and workflow processing method |
| CN104182467A (en) * | 2014-07-21 | 2014-12-03 | 安徽华贞信息科技有限公司 | Network data source detection method |
| CN111221664A (en) * | 2019-12-24 | 2020-06-02 | 视联动力信息技术股份有限公司 | Operation log processing system |
| US10771583B2 (en) * | 2014-12-29 | 2020-09-08 | Akamai Technologies, Inc. | Managing mobile device user subscription and service preferences to predictively pre-fetch content |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108021358A (en) * | 2017-12-15 | 2018-05-11 | 无线生活（杭州）信息科技有限公司 | A kind of data processing method and device |
-
2020
- 2020-12-30 CN CN202011600584.8A patent/CN112291121B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102419833A (en) * | 2011-09-23 | 2012-04-18 | 方欣科技有限公司 | Workflow engine system and workflow processing method |
| CN104182467A (en) * | 2014-07-21 | 2014-12-03 | 安徽华贞信息科技有限公司 | Network data source detection method |
| US10771583B2 (en) * | 2014-12-29 | 2020-09-08 | Akamai Technologies, Inc. | Managing mobile device user subscription and service preferences to predictively pre-fetch content |
| CN111221664A (en) * | 2019-12-24 | 2020-06-02 | 视联动力信息技术股份有限公司 | Operation log processing system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112291121A (en) | 2021-01-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111625473B (en) | Interface test case generation method and device, storage medium and electronic equipment | |
| CN111198859B (en) | Data processing method, device, electronic equipment and computer readable storage medium | |
| CN112291370B (en) | Method for processing service access relation and related equipment | |
| CN110516159B (en) | Information recommendation method and device, electronic equipment and storage medium | |
| CN112291121B (en) | Data processing method and related equipment | |
| CN112596738B (en) | Method and device for determining front-end page to be tested, storage medium and electronic equipment | |
| CN112395188B (en) | Test-oriented software project development method and device, electronic equipment and storage medium | |
| CN111708680A (en) | Error reporting information analysis method and device, electronic equipment and storage medium | |
| CN110059064B (en) | Log file processing method and device and computer readable storage medium | |
| CN111738316A (en) | Image classification method and device for zero sample learning and electronic equipment | |
| CN109902726B (en) | Resume information processing method and device | |
| CN112954056B (en) | Method and device for processing monitoring data, electronic equipment and storage medium | |
| CN112261176B (en) | Method for acquiring actual network access relationship and related equipment | |
| CN112884376A (en) | Work order processing method and device, electronic equipment and computer readable storage medium | |
| CN112882948A (en) | Stability testing method, device and system for application and storage medium | |
| CN111198853B (en) | Data processing method, device, electronic equipment and computer readable storage medium | |
| CN111274104B (en) | Data processing method, device, electronic equipment and computer readable storage medium | |
| CN109542743B (en) | Log checking method and device, electronic equipment and computer readable storage medium | |
| CN109408387B (en) | Page testing method and device | |
| CN116319932A (en) | Training method, device, equipment and storage medium of content push model | |
| CN112260880B (en) | Network access relation display method and related equipment | |
| CN111832354A (en) | Target object age identification method and device and electronic equipment | |
| CN112379967B (en) | Simulator detection method, device, equipment and medium | |
| CN109614137B (en) | Software version control method, device, equipment and medium | |
| CN114116480A (en) | Method, device, medium and equipment for determining application program test coverage rate |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |