CN103166845A - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN103166845A CN103166845A CN2013100666234A CN201310066623A CN103166845A CN 103166845 A CN103166845 A CN 103166845A CN 2013100666234 A CN2013100666234 A CN 2013100666234A CN 201310066623 A CN201310066623 A CN 201310066623A CN 103166845 A CN103166845 A CN 103166845A
- Authority
- CN
- China
- Prior art keywords
- clear text
- data flow
- buffer area
- state
- described clear
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Transfer Between Computers (AREA)
Abstract
The invention provides a data processing method and a device. Before service processing is conducted on a to-be-processed message, whether a message which is in the course of processing exists in a data flow or not is detected, wherein the data flow is where the to-be-processed message is located. If the message which is in the course of processing does not exist in the data flow where the to-be-processed message is located, the to-be-processed message is transmitted to a processing thread to be service processed. Due to the fact that the message which is in the course of processing does not exist in the data flow where the to-be-processed message is located, the to-be-processed message can be transmitted to any processing thread to be service processed, the problem that messages of the same data flow are processed in a concurrency mode does not occur, task loads of the processing threads can be balanced, and therefore, a resource utilization rate of a processor of network equipment can be improved.
Description
Technical field
The embodiment of the present invention relates to computer networking technology, relates in particular to a kind of data processing method and device.
Background technology
Along with the development of computer networking technology, Multi-core concurrent processing technology also obtain develop rapidly.
In the environment of Multi-core concurrent processing, each thread is before processing message, the intermediateness data of needs read-write message place traffic data stream, if the different messages of same traffic data stream are processed simultaneously by a plurality of threads, the situation that a plurality of threads read while write the intermediateness data of same traffic data stream can appear, and because the intermediateness data synchronization of a traffic data stream can only be by same thread read-write, therefore, have the problem of resource contention, different process synchronizations are processed can exist conflict with stream packet.For fear of conflict, in prior art, calculate cryptographic Hash according to the five-tuple information of message, by the Business Processing Thread Count, cryptographic Hash is asked modular arithmetic, operation result counterlogic thread identification (Identity, hereinafter to be referred as: ID), thereby produced the mapping relations that flow to thread, at last according to mapping relations, the message of a stream has been distributed to fixing thread process, data for a stream are bonding thread process like this, thereby avoid concurrency conflict.
Yet, due to the inhomogeneities of message distribution, adopt the scheme of prior art, can appear in a period of time, some thread is busier, and the more idle situation of some thread, thereby causes the resource utilization of processor of the performance of the concurrent execution of multi-service and the network equipment all not high.
Summary of the invention
The embodiment of the present invention provides a kind of data processing method and device, to increase the performance of the concurrent execution of multi-service, improves the resource utilization of the processor of the network equipment.
Embodiment of the present invention first aspect provides a kind of data processing method, comprising:
Before clear text is carried out Business Processing, detect in the data flow of described clear text place and whether exist just at processed message;
If just do not exist in the data flow of described clear text place at processed message, described clear text sent to the Business Processing thread, to carry out Business Processing.
In conjunction with first aspect, in the possible implementation of the first, just whether exist in the data flow of the described clear text of described detection place at processed message, comprising:
Detect the treatment state of described clear text place data flow, described treatment state comprises just at processed state and not processed state, wherein, just the one or more messages in processed state is used to indicate data flow are being by the Business Processing thread process, and not processed state is used to indicate in data flow without any message by the Business Processing thread process;
If just do not exist in the data flow of described clear text place at processed message, described clear text is sent to the Business Processing thread, to carry out Business Processing, comprising:
If the treatment state of described clear text place data flow being detected is not processed state, the treatment state of described clear text place data flow is set to just at processed state, described clear text is sent to the Business Processing thread, to carry out Business Processing, and after described Business Processing thread is disposed to described clear text, the treatment state of described clear text place data flow is set to not processed state.
In conjunction with first aspect or the possible implementation of the first, in the possible implementation of the second, described method also comprises:
If the treatment state that described clear text place data flow detected, is buffered in described clear text in the first buffer area at processed state for just.
In conjunction with the possible implementation of the second, in the third possible implementation, before the treatment state of the described clear text of described detection place data flow, also comprise:
Judge whether described the first buffer area is empty;
If, obtain a clear text from the second buffer area, wherein, described the second buffer area is used for buffer memory from the network interface message;
If not, obtain a clear text from described the first buffer area;
Correspondingly, the treatment state of the described clear text of described detection place data flow comprises: the treatment state that detects the clear text place data flow of obtaining from described the first buffer area or the second buffer area.
In conjunction with the third possible implementation, in the 4th kind of possible implementation, described the first buffer area comprises one or more formations, wherein, each formation is associated with a data flow, be used for the clear text of this data flow of buffer memory, the number of described formation is more than or equal to the number of all Business Processing threads;
Described described clear text is buffered in the first buffer area, comprises:
Judge and whether have the formation that is associated with described clear text place data flow in described the first buffer area;
If described clear text is buffered in the formation that is associated with described clear text place data flow;
If not, described clear text is buffered in the formation of the sky in described the first buffer area, and set up the formation of described sky and the incidence relation of described clear text place data flow, so that the formation of described sky is associated with the data flow at described clear text place.
In conjunction with the 4th kind of possible implementation, in the 5th kind of possible implementation, described treatment state with described clear text place data flow just is set at processed state, comprising:
Treatment state with described clear text place data flow in the stream table is revised as just at processed state; Wherein, described stream table stores the traffic identifier of described clear text and the corresponding relation for the treatment of state;
Described treatment state with described clear text place data flow is set to not processed state, comprising:
Treatment state with described clear text place data flow in the stream table is revised as not processed state;
The treatment state of the clear text place data flow that described detection is obtained from described the first buffer area or the second buffer area comprises:
If the clear text of described clear text for obtaining from described the second buffer area, according to the five-tuple information of described clear text, determine the traffic identifier of described clear text, inquire about the treatment state that obtains described clear text place data flow according to described traffic identifier from described stream table;
If described clear text is the clear text that obtains from the formation of described the first buffer area, the traffic identifier corresponding according to described formation inquired about the treatment state that obtains described clear text place data flow from described stream table.
In conjunction with the 4th kind of possible implementation or the 5th kind of possible implementation, in the 6th kind of possible implementation, described obtain a clear text from described the first buffer area after, also comprise:
If the formation of the described clear text of buffer memory is empty, delete the incidence relation between described formation and described clear text place data flow.
In conjunction with the third possible implementation, in the 7th kind of possible implementation, described the first buffer area comprises one or more chained lists, wherein, each chained list is associated with a data flow, be used for the clear text of this data flow of buffer memory, the number of described chained list is more than or equal to the number of all Business Processing threads;
Described described clear text is buffered in the first buffer area, comprises:
Judge and whether have the chained list that is associated with described clear text place data flow in described the first buffer area;
If described clear text is buffered in the chained list that is associated with described clear text place data flow;
If not, described clear text is buffered in the chained list of the sky in described the first buffer area, and set up the chained list of described sky and the incidence relation of described clear text place data flow, so that the chained list of described sky is associated with the data flow at described clear text place.
Embodiment of the present invention second aspect provides a kind of data processing equipment, comprising:
Whether detection module was used for before clear text is carried out Business Processing, detect in the data flow of described clear text place to exist just at processed message;
Processing module does not just exist at processed message if be used for described clear text place data flow, described clear text is sent to the Business Processing thread, to carry out Business Processing.
In conjunction with second aspect, in the possible implementation of the first, described detection module, specifically for detection of the treatment state of described clear text place data flow, described treatment state comprises just at processed state and not processed state, wherein, just the one or more messages in processed state is used to indicate data flow are being by the Business Processing thread process, and not processed state is used to indicate in data flow without any message by the Business Processing thread process;
Described processing module, if concrete to be used for the treatment state that detection module detects described clear text place data flow be not processed state, the treatment state of described clear text place data flow is set to just at processed state, described clear text is sent to the Business Processing thread, to carry out Business Processing, and after described Business Processing thread is disposed to described clear text, the treatment state of described clear text place data flow is set to not processed state.
In conjunction with second aspect or the possible implementation of the first, in the possible implementation of the second, described processing module, if also be used for treatment state that detection module detects described clear text place data flow for just at processed state, described clear text is buffered in the first buffer area.
In conjunction with the possible implementation of the second, in the third possible implementation, described processing module also before the treatment state for detection of described clear text place data flow, judges whether described the first buffer area is empty; If, obtain a clear text from the second buffer area, wherein, described the second buffer area is used for buffer memory from the network interface message; If not, obtain a clear text from described the first buffer area;
Correspondingly, described detection module is specifically for detection of the treatment state of the clear text place data flow of obtaining from described the first buffer area or the second buffer area.
In conjunction with the third possible implementation, in the 4th kind of possible implementation, described the first buffer area comprises one or more formations, wherein, each formation is associated with a data flow, be used for the clear text of this data flow of buffer memory, the number of described formation is more than or equal to the number of all Business Processing threads;
Described processing module concrete is used for judging whether described the first buffer area exists the formation that is associated with described clear text place data flow; If described clear text is buffered in the formation that is associated with described clear text place data flow; If not, described clear text is buffered in the formation of the sky in described the first buffer area, and set up the formation of described sky and the incidence relation of described clear text place data flow, so that the formation of described sky is associated with the data flow at described clear text place.
In conjunction with the 4th kind of possible implementation, in the 5th kind of possible implementation, described processing module, concrete being used for when the treatment state of clear text place data flow is not processed state, the treatment state with described clear text place data flow in the stream table is revised as just at processed state; Wherein, described stream table stores the traffic identifier of described clear text and the corresponding relation for the treatment of state; And after described Business Processing thread was disposed to described clear text, the treatment state with described clear text place data flow in the stream table was revised as not processed state;
Described detection module, if the concrete clear text of described clear text for obtaining from described the second buffer area that be used for, according to the five-tuple information of described clear text, determine the traffic identifier of described clear text, inquire about the treatment state that obtains described clear text place data flow according to described traffic identifier from described stream table; If described clear text is the clear text that obtains from the formation of described the first buffer area, the traffic identifier corresponding according to described formation inquired about the treatment state that obtains described clear text place data flow from described stream table.
In conjunction with the 4th kind of possible implementation or the 5th kind of possible implementation, in the 6th kind of possible implementation, described processing module, concrete for described after described the first buffer area obtains a clear text, if the formation of the described clear text of buffer memory is empty, delete the incidence relation between described formation and described clear text place data flow.
In conjunction with the third possible implementation, in the 7th kind of possible implementation, described the first buffer area comprises one or more chained lists, wherein, each chained list is associated with a data flow, be used for the clear text of this data flow of buffer memory, the number of described chained list is more than or equal to the number of all Business Processing threads;
Described processing module concrete is used for judging whether described the first buffer area exists the chained list that is associated with described clear text place data flow; If described clear text is buffered in the chained list that is associated with described clear text place data flow; If not, described clear text is buffered in the chained list of the sky in described the first buffer area, and set up the chained list of described sky and the incidence relation of described clear text place data flow, so that the chained list of described sky is associated with the data flow at described clear text place.the data processing method that the embodiment of the present invention provides and device, by before clear text is carried out Business Processing, whether exist just at processed message in the data flow of detection clear text place, if do not exist in the data flow of clear text place just at processed message, clear text is sent to the Business Processing thread, to carry out Business Processing, owing to not existing just at processed message in the data flow of clear text place, therefore, clear text can be sent in any one Business Processing thread and carry out Business Processing, the problem of the message concurrent processing of same data flow can not occur, the task amount of balanced each Business Processing thread, increase the performance of the concurrent execution of multi-service, improve the resource utilization of the processor of the network equipment.
Description of drawings
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, the below will do to introduce simply to the accompanying drawing of required use in embodiment or description of the Prior Art, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain according to these accompanying drawings other accompanying drawing.
Fig. 1 is the schematic flow sheet of data processing method embodiment one of the present invention;
Fig. 2 is the schematic flow sheet of data processing method embodiment two of the present invention;
Fig. 3 is the structural representation of data processing equipment embodiment one of the present invention;
Fig. 4 is the structural representation of data processing equipment embodiment two of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is only the present invention's part embodiment, rather than whole embodiment.Based on the embodiment in the present invention, those of ordinary skills belong to the scope of protection of the invention not making the every other embodiment that obtains under the creative work prerequisite.
Fig. 1 is the schematic flow sheet of data processing method embodiment one of the present invention, and as shown in Figure 1, the method for the present embodiment comprises:
S101: before clear text is carried out Business Processing, detect and just whether to exist in the data flow of clear text place at processed message, if not, carry out S102, if carry out S103.
Particularly, can determine whether exist just at processed message in the data flow of clear text place by the message treatment state of inquiry clear text place data flow.Also can determine whether exist just at processed message in the data flow of clear text place by inquiring about data flow that each Business Processing thread processing, whether can detect by other means yet and just exist in the data flow of clear text place at processed message, the embodiment of the present invention is not restricted this.
S102: clear text is sent to the Business Processing thread, to carry out Business Processing.
if do not exist in the data flow of clear text place just at processed message, clear text is sent to and carry out Business Processing in the Business Processing thread, owing to not existing just at processed message in the data flow of clear text place, therefore, clear text can be sent in any one Business Processing thread and carry out Business Processing, the problem of the message concurrent processing of same data flow can not occur, be each Business Processing thread synchronization concurrent processing be the message of different data streams, when clear text being sent to the Business Processing thread and carries out Business Processing, can be according to scheduling strategy, clear text is sent to more idle Business Processing thread carry out Business Processing, the task amount of balanced each Business Processing thread, improve the resource utilization of processor.
S103: other processing.
As a kind of feasible implementation: if exist just at processed message in the data flow of clear text place, clear text is buffered in the first buffer area, in order to after the data flow state of each Business Processing thread process changes, obtain clear text from the first buffer area.
in the present embodiment, by before clear text is carried out Business Processing, whether exist just at processed message in the data flow of detection clear text place, if do not exist in the data flow of clear text place just at processed message, clear text is sent to the Business Processing thread, to carry out Business Processing, owing to not existing just at processed message in the data flow of clear text place, therefore, clear text can be sent in any one Business Processing thread and carry out Business Processing, the problem of the message concurrent processing of same data flow can not occur, the task amount of balanced each Business Processing thread, increase the performance of the concurrent execution of multi-service, improve the resource utilization of the processor of the network equipment.
Fig. 2 is the schematic flow sheet of data processing method embodiment two of the present invention, and as shown in Figure 2, the method for the present embodiment comprises:
S201: judge whether the first buffer area is empty, if, carry out S202, if not, carry out S203.
S202: obtain a clear text from the second buffer area.
Particularly, the second buffer area is used for buffer memory from the message of network interface.
S203: obtain a clear text from the first buffer area.
Particularly, exist just at processed message if detect in the data flow of clear text place, clear text is buffered in the first buffer area, alternatively, clear text in the first buffer area is buffered in one or more formations, and wherein, each formation is associated with a data flow, be used for the clear text of this data flow of buffer memory, the number of formation is more than or equal to the number of all Business Processing threads in the network equipment.
Step by S201 ~ S203 has guaranteed that message arrives first the principle of first processing, and namely preferably obtains the message in the first buffer area, in the situation that the first buffer area is empty, then obtains the message in the second buffer area.
S204: detect in the data flow of clear text place and whether exist just at processed message.If, carry out S205, if not, carry out S206.
Particularly, detect and just whether to exist in the data flow of clear text place at processed message and can realize by the treatment state that detects clear text place data flow, treatment state comprises just at processed state and not processed state, when the treatment state of clear text place data flow for just at processed state, determine to exist just at processed message in the data flow of clear text place; When the treatment state of clear text place data flow is not processed state, determine not exist just at processed message in the data flow of clear text place; Wherein, just the one or more messages in processed state is used to indicate data flow are being by the Business Processing thread process, and not processed state is used to indicate in data flow without any message by the Business Processing thread process; Be divided into following two kinds of situations:
The first situation: if the clear text of clear text for obtaining from the formation of the first buffer area, whether the traffic identifier corresponding according to clear text place formation inquired about the treatment state that obtains clear text place data flow from the stream table, exist just at processed message to determine clear text place data flow.
The second situation: if the clear text of clear text for obtaining from the second buffer area, according to the five-tuple information of clear text, determine the traffic identifier of clear text, inquire about the treatment state that obtains clear text place data flow according to traffic identifier, to determine whether exist just at processed message in the data flow of clear text place from the stream table.
Have identical five-tuple information between the different messages of same data flow, five-tuple information comprises: the source network agreement (Internet Protocol, hereinafter to be referred as: IP) address, purpose IP address, source port, destination interface, transport layer protocol.Therefore, if the clear text of clear text for obtaining from the second buffer area, can determine the traffic identifier of clear text according to the five-tuple information of clear text, and then detect in the data flow of clear text place according to traffic identifier and whether exist just at processed message.Specifically can determine whether clear text place data flow exists by message treatment state corresponding to traffic identifier in inquiry stream table and process message.
what deserves to be explained is, before carrying out S204, also comprise the step of setting up the stream table, can be specifically to set up one-to-one relationship between five-tuple information and traffic identifier according to the five-tuple information of stream, alternatively, Data Stream Processing state that can traffic identifier is corresponding is stored in the stream table, Data Stream Processing state that also can traffic identifier is corresponding is stored in independent memory space outside the stream table, in a word, can inquire Data Stream Processing state corresponding to traffic identifier by traffic identifier gets final product, concrete storage mode to Data Stream Processing state corresponding to traffic identifier, the present invention does not limit this.
Alternatively, except five-tuple information, can also set up the stream table by other information that can identify the different messages of same data flow in the above-described embodiments, hexa-atomic group of information for example, its specific implementation and five-tuple info class seemingly repeat no more herein.
S205: clear text is buffered in the first buffer area.
Particularly, judge whether there is the formation that is associated with clear text place data flow in the first buffer area; If exist, clear text is buffered in formation that clear text institute data flow is associated in, alternatively, can set up incidence relation between clear text place data flow and formation by traffic identifier.
If do not exist, clear text is buffered in the formation of the sky in the first buffer area, and set up the incidence relation of empty formation and clear text place data flow, so that empty formation is associated with the data flow at clear text place, alternatively, can be by the formation of sky is corresponding with the traffic identifier of clear text place data flow.The incidence relation of the traffic identifier by identifying to set up formation and clear text place data flow in the queue heads storage flow.That is to say, the message of a data flow is only preserved in a formation.
S206: the treatment state of clear text place data flow is set to just at processed state.
According to the traffic identifier of clear text place data flow, in the stream table, the treatment state of clear text place data flow is put and just be revised as at processed state, wherein, the stream table stores the traffic identifier of clear text and the corresponding relation for the treatment of state.
Alternatively, if the clear text of clear text for obtaining from the first buffer area, after clear text in obtaining the first buffer area, also comprise: whether the formation that judges the clear text buffer memory is empty, if, delete the incidence relation between this formation and clear text place data flow, make this formation become the formation of a sky, in order to store the clear text of new data flow.
S207: clear text is sent to the Business Processing thread, to carry out Business Processing.
Alternatively, can directly clear text be sent to the Business Processing thread of current free time and carry out Business Processing.Clear text can be carried out buffer memory, concrete cache way the present invention is not restricted this yet, then sends to the Business Processing thread according to scheduling strategy and carries out Business Processing.Owing to not existing just at processed message in the data flow of clear text place, therefore, clear text can be sent in any one Business Processing thread and carry out Business Processing, the problem of the message concurrent processing of same data flow can not occur, the scheduling mode of any task amount that can balanced each Business Processing thread can, thereby, the resource utilization of the processor of the raising network equipment.
S208: the treatment state of clear text place data flow is set to not processed state.
After the complete clear text of Business Processing thread process, the treatment state of clear text place data flow is set to not processed state, particularly, treatment state with clear text place data flow in the stream table is revised as not processed state, just do not exist in this data flow of not processed status indicator at processed message, carry out Business Processing so that new clear text is sent to the Business Processing thread.
in the present embodiment, whether exist just at processed message by detecting in the data flow of clear text place, if not, the treatment state of clear text place data flow is set to just at processed state, clear text is sent to the Business Processing thread carry out Business Processing, after the complete clear text of Business Processing thread process, the treatment state of clear text place data flow is set to not processed state, so that being sent to the Business Processing thread, new clear text carries out Business Processing, owing to not existing just at processed message in the data flow of clear text place, just clear text is sent to and carry out Business Processing in the Business Processing thread, therefore, clear text can be sent in any one Business Processing thread and carry out Business Processing, the problem of the message concurrent processing of same data flow can not occur, be each Business Processing thread synchronization concurrent processing be the message of different data streams, when clear text being sent to the Business Processing thread and carries out Business Processing, can be according to scheduling strategy, clear text is sent to more idle Business Processing thread carry out Business Processing, the task amount of balanced each Business Processing thread, to increase the performance of the concurrent execution of multi-service, improve the resource utilization of the processor of the network equipment.
In embodiment illustrated in fig. 2, alternatively, clear text in the first buffer area can be buffered in arbitrarily in data structure, for example, clear text in the first buffer area is buffered in one or more chained lists or Hash table, wherein, and the traffic identifier of the corresponding data flow of each chained list or Hash table, the number of chained list or Hash table is more than or equal to the number of Business Processing thread, and the present invention does not limit this.
Fig. 3 is the structural representation of data processing equipment embodiment one of the present invention, as shown in Figure 3, the device of the present embodiment comprises detection module 301 and processing module 302, wherein, whether detection module 301 was used for before clear text is carried out Business Processing, detect in the data flow of clear text place to exist just at processed message; Just do not exist at processed message if processing module 302 is used for clear text place data flow, clear text is sent to the Business Processing thread, to carry out Business Processing.
The device of the present embodiment is used for carrying out the technical scheme of embodiment of the method shown in Figure 1, its realize principle and technique effect similar, repeat no more herein.
In the above-described embodiments, detection module 301 is specifically for detection of the treatment state of clear text place data flow, treatment state comprises just at processed state and not processed state, wherein, just the one or more messages in processed state is used to indicate data flow are being by the Business Processing thread process, and not processed state is used to indicate in data flow without any message by the Business Processing thread process;
To be used for the treatment state that detection module detects clear text place data flow be not processed state if processing module 302 is concrete, the treatment state of clear text place data flow is set to just at processed state, clear text is sent to the Business Processing thread, to carry out Business Processing, and after the Business Processing thread is disposed to clear text, the treatment state of clear text place data flow is set to not processed state.
In the above-described embodiments, if processing module 302 also be used for treatment state that detection module detects clear text place data flow for just at processed state, clear text is buffered in the first buffer area.
In the above-described embodiments, processing module 302 also before the treatment state for detection of clear text place data flow, judges whether the first buffer area is empty; If, obtain a clear text from the second buffer area, wherein, the second buffer area is used for buffer memory from the network interface message; If not, obtain a clear text from the first buffer area;
Correspondingly, detection module 301 is specifically for detection of the treatment state of the clear text place data flow of obtaining from the first buffer area or the second buffer area.
In the above-described embodiments, the first buffer area comprises one or more formations, and wherein, each formation is associated with a data flow, is used for the clear text of this data flow of buffer memory, and the number of formation is more than or equal to the number of all Business Processing threads;
In the above-described embodiments, processing module 302 processing modules 302 are concrete to be used for when the treatment state of clear text place data flow is not processed state, and the treatment state with clear text place data flow in the stream table is revised as just at processed state; Wherein, the stream table stores the traffic identifier of clear text and the corresponding relation for the treatment of state; And after the Business Processing thread was disposed to clear text, the treatment state with clear text place data flow in the stream table was revised as not processed state; If the concrete clear text of clear text for obtaining from the second buffer area that be used for of detection module 301, according to the five-tuple information of clear text, determine the traffic identifier of clear text, inquire about the treatment state that obtains clear text place data flow according to traffic identifier from the stream table; If clear text is the clear text that obtains from the formation of the first buffer area, the traffic identifier corresponding according to formation inquired about the treatment state that obtains clear text place data flow from the stream table.
In the above-described embodiments, processing module 302 is concrete to be used for after the first buffer area obtains a clear text, if the formation of buffer memory clear text be sky, deletes the incidence relation between formation and clear text place data flow.
In the above-described embodiments, the first buffer area comprises one or more chained lists, and wherein, each chained list is associated with a data flow, is used for the clear text of this data flow of buffer memory, and the number of chained list is more than or equal to the number of all Business Processing threads;
The device of the present embodiment is used for carrying out the technical scheme of embodiment of the method shown in Figure 2, its realize principle and technique effect similar, repeat no more herein.
Fig. 4 is the structural representation of data processing equipment embodiment two of the present invention, as shown in Figure 4, the device of the present embodiment the present embodiment comprises communication interface 401, at least one processor 402 and memory 403, and processor 402, memory 403 are connected with communication interface and are connected and complete mutual communication by bus.Described bus can be industry standard architecture (Industry Standard Architecture, referred to as ISA) bus, peripheral component interconnect (Peripheral Component, referred to as PCI) bus or extended industry-standard architecture (Extended Industry Standard Architecture is referred to as EISA) bus etc.Described bus can be divided into address bus, data/address bus, control bus etc.For ease of expression, only represent with a thick line in Fig. 4, but do not represent only to have the bus of a bus or a type.Wherein, communication interface 401 can realize device or the unit of data transmit-receive function for network interface, USB interface, radio frequency unit, antenna, wi-fi communication module etc.; Memory 403 is used for the stores executable programs code, and this program code comprises computer-managed instruction.Memory 403 may comprise the high-speed RAM memory, also may also comprise nonvolatile memory (non-volatile memory), for example at least one magnetic disc store.In one embodiment, processor 402 moves the program corresponding with described executable program code by the executable program code of storage in read memory 403, to be used for: before clear text is carried out Business Processing, detect in the data flow of described clear text place and whether exist just at processed message; If just do not exist in the data flow of clear text place at processed message, described clear text sent to the Business Processing thread by communication interface 401, to carry out Business Processing.Processor 402 may be a central processing unit (Central Processing Unit, referred to as CPU), or specific integrated circuit (Application Specific Integrated Circuit, referred to as ASIC), or be configured to implement one or more integrated circuits of the embodiment of the present invention.
Above-mentioned processor 402 also can be used for carrying out other flow processs in said method embodiment except realizing above-mentioned functions, do not repeat them here.
Wherein, the detail of above-mentioned flow process can with reference to said method embodiment, repeat no more herein.
In the above-described embodiments, concrete program corresponding to executive program code that be used for of processor 402, to be used for: the treatment state that detects clear text place data flow, treatment state comprises just at processed state and not processed state, wherein, just the one or more messages in processed state is used to indicate data flow are being by the Business Processing thread process, and not processed state is used to indicate in data flow without any message by the Business Processing thread process;
If it is not processed state that detection module detects the treatment state of clear text place data flow, the treatment state of clear text place data flow is set to just at processed state, clear text is sent to the Business Processing thread, to carry out Business Processing, and after the Business Processing thread is disposed to clear text, the treatment state of clear text place data flow is set to not processed state.
In the above-described embodiments, processor 402 also is used for executive program, to be used for: if the treatment state that clear text place data flow detected for just at processed state, clear text is buffered in the first buffer area.
In the above-described embodiments, processor 402 also is used for executive program, to be used for: before detecting the treatment state of clear text place data flow, judge whether the first buffer area is sky; If, obtain a clear text from the second buffer area, wherein, the second buffer area is used for buffer memory from the network interface message; If not, obtain a clear text from the first buffer area;
Correspondingly, the concrete executive program that is used for of processor 402 is to be used for: the treatment state that detects the clear text place data flow of obtaining from the first buffer area or the second buffer area.
In the above-described embodiments, the first buffer area and the second buffer area can be two in memory 403 memory spaces independently, further, the first buffer area comprises one or more formations, wherein, each formation is associated with a data flow, is used for the clear text of this data flow of buffer memory, and the number of formation is more than or equal to the number of all Business Processing threads;
The concrete executive program that is used for of processor 402 is to be used for: judge whether the first buffer area exists the formation that is associated with clear text place data flow; If clear text is buffered in the formation that is associated with clear text place data flow; If not, clear text is buffered in the formation of the sky in the first buffer area, and sets up the incidence relation of empty formation and clear text place data flow, so that the formation of sky is associated with the data flow at clear text place.
In the above-described embodiments, the concrete executive program that is used for of processor 402, to be used for: at the treatment state of clear text place data flow during as not processed state, the treatment state with clear text place data flow in the stream table is revised as just at processed state; Wherein, the stream table stores the traffic identifier of clear text and the corresponding relation for the treatment of state; And after the Business Processing thread was disposed to clear text, the treatment state with clear text place data flow in the stream table was revised as not processed state;
The concrete executive program that is used for of processor 402, to be used for: if the clear text of clear text as obtaining from the second buffer area, according to the five-tuple information of clear text, determine the traffic identifier of clear text, inquire about the treatment state that obtains clear text place data flow according to traffic identifier from the stream table; If clear text is the clear text that obtains from the formation of the first buffer area, the traffic identifier corresponding according to formation inquired about the treatment state that obtains clear text place data flow from the stream table.
In the above-described embodiments, the concrete executive program that is used for of processor 402, to be used for: after the first buffer area obtains a clear text, if the formation of buffer memory clear text is empty, delete the incidence relation between formation and clear text place data flow.
In another embodiment, the first buffer area comprises one or more chained lists, and wherein, each chained list is associated with a data flow, is used for the clear text of this data flow of buffer memory, and the number of chained list is more than or equal to the number of all Business Processing threads;
The concrete executive program that is used for of processor 402 is to be used for: judge whether the first buffer area exists the chained list that is associated with clear text place data flow; If clear text is buffered in the chained list that is associated with clear text place data flow; If not, clear text is buffered in the chained list of the sky in the first buffer area, and sets up the incidence relation of empty chained list and clear text place data flow, so that the chained list of sky is associated with the data flow at clear text place.
The device of the present embodiment is used for carrying out the technical scheme of embodiment of the method shown in Figure 2, its realize principle and technique effect similar, repeat no more herein.
One of ordinary skill in the art will appreciate that: all or part of step that realizes above-mentioned each embodiment of the method can be completed by the hardware that program command is correlated with.Aforesaid program can be stored in a computer read/write memory medium.This program is carried out the step that comprises above-mentioned each embodiment of the method when carrying out; And aforesaid storage medium comprises: the various media that can be program code stored such as ROM, RAM, magnetic disc or CD.
It should be noted that at last: above each embodiment is not intended to limit only in order to technical scheme of the present invention to be described; Although with reference to aforementioned each embodiment, the present invention is had been described in detail, those of ordinary skill in the art is to be understood that: it still can be modified to the technical scheme that aforementioned each embodiment puts down in writing, and perhaps some or all of technical characterictic wherein is equal to replacement; And these modifications or replacement do not make the essence of appropriate technical solution break away from the scope of various embodiments of the present invention technical scheme.
Claims (16)
1. a data processing method, is characterized in that, comprising:
Before clear text is carried out Business Processing, detect in the data flow of described clear text place and whether exist just at processed message;
If just do not exist in the data flow of described clear text place at processed message, described clear text sent to the Business Processing thread, to carry out Business Processing.
2. whether method according to claim 1, is characterized in that, just exist in the data flow of the described clear text of described detection place at processed message, comprising:
Detect the treatment state of described clear text place data flow, described treatment state comprises just at processed state and not processed state, wherein, just the one or more messages in processed state is used to indicate data flow are being by the Business Processing thread process, and not processed state is used to indicate in data flow without any message by the Business Processing thread process;
If just do not exist in the data flow of described clear text place at processed message, described clear text is sent to the Business Processing thread, to carry out Business Processing, comprising:
If the treatment state of described clear text place data flow being detected is not processed state, the treatment state of described clear text place data flow is set to just at processed state, described clear text is sent to the Business Processing thread, to carry out Business Processing, and after described Business Processing thread is disposed to described clear text, the treatment state of described clear text place data flow is set to not processed state.
3. method according to claim 2, is characterized in that, also comprises:
If the treatment state that described clear text place data flow detected, is buffered in described clear text in the first buffer area at processed state for just.
4. method according to claim 3, is characterized in that, before the treatment state of the described clear text of described detection place data flow, also comprises:
Judge whether described the first buffer area is empty;
If, obtain a clear text from the second buffer area, wherein, described the second buffer area is used for buffer memory from the network interface message;
If not, obtain a clear text from described the first buffer area;
Correspondingly, the treatment state of the described clear text of described detection place data flow comprises: the treatment state that detects the clear text place data flow of obtaining from described the first buffer area or the second buffer area.
5. method according to claim 4, is characterized in that,
Described the first buffer area comprises one or more formations, and wherein, each formation is associated with a data flow, is used for the clear text of this data flow of buffer memory, and the number of described formation is more than or equal to the number of all Business Processing threads;
Described described clear text is buffered in the first buffer area, comprises:
Judge and whether have the formation that is associated with described clear text place data flow in described the first buffer area;
If described clear text is buffered in the formation that is associated with described clear text place data flow;
If not, described clear text is buffered in the formation of the sky in described the first buffer area, and set up the formation of described sky and the incidence relation of described clear text place data flow, so that the formation of described sky is associated with the data flow at described clear text place.
6. method according to claim 5, is characterized in that, described treatment state with described clear text place data flow just is set at processed state, comprising:
Treatment state with described clear text place data flow in the stream table is revised as just at processed state; Wherein, described stream table stores the traffic identifier of described clear text and the corresponding relation for the treatment of state;
Described treatment state with described clear text place data flow is set to not processed state, comprising:
Treatment state with described clear text place data flow in the stream table is revised as not processed state;
The treatment state of the clear text place data flow that described detection is obtained from described the first buffer area or the second buffer area comprises:
If the clear text of described clear text for obtaining from described the second buffer area, according to the five-tuple information of described clear text, determine the traffic identifier of described clear text, inquire about the treatment state that obtains described clear text place data flow according to described traffic identifier from described stream table;
If described clear text is the clear text that obtains from the formation of described the first buffer area, the traffic identifier corresponding according to described formation inquired about the treatment state that obtains described clear text place data flow from described stream table.
7. according to claim 5 or 6 described methods, is characterized in that, described obtain a clear text from described the first buffer area after, also comprise:
If the formation of the described clear text of buffer memory is empty, delete the incidence relation between described formation and described clear text place data flow.
8. method according to claim 4, it is characterized in that, described the first buffer area comprises one or more chained lists, wherein, each chained list is associated with a data flow, be used for the clear text of this data flow of buffer memory, the number of described chained list is more than or equal to the number of all Business Processing threads;
Described described clear text is buffered in the first buffer area, comprises:
Judge and whether have the chained list that is associated with described clear text place data flow in described the first buffer area;
If described clear text is buffered in the chained list that is associated with described clear text place data flow;
If not, described clear text is buffered in the chained list of the sky in described the first buffer area, and set up the chained list of described sky and the incidence relation of described clear text place data flow, so that the chained list of described sky is associated with the data flow at described clear text place.
9. a data processing equipment, is characterized in that, comprising:
Whether detection module was used for before clear text is carried out Business Processing, detect in the data flow of described clear text place to exist just at processed message;
Processing module does not just exist at processed message if be used for described clear text place data flow, described clear text is sent to the Business Processing thread, to carry out Business Processing.
10. device according to claim 9, it is characterized in that, described detection module, specifically be used for, detect the treatment state of described clear text place data flow, described treatment state comprises just at processed state and not processed state, wherein, just the one or more messages in processed state is used to indicate data flow are being by the Business Processing thread process, and not processed state is used to indicate in data flow without any message by the Business Processing thread process;
Described processing module, specifically be used for, if it is not processed state that described detection module detects the treatment state of described clear text place data flow, the treatment state of described clear text place data flow is set to just at processed state, described clear text is sent to the Business Processing thread, carrying out Business Processing, and after described Business Processing thread is disposed to described clear text, the treatment state of described clear text place data flow is set to not processed state.
11. device according to claim 10, it is characterized in that, described processing module, if also be used for treatment state that detection module detects described clear text place data flow for just at processed state, described clear text is buffered in the first buffer area.
12. device according to claim 11 is characterized in that, described processing module also is used for judge whether described the first buffer area is sky before detection module detects the treatment state of described clear text place data flow; If, obtain a clear text from the second buffer area, wherein, described the second buffer area is used for buffer memory from the network interface message; If not, obtain a clear text from described the first buffer area;
Correspondingly, described detection module is specifically for detection of the treatment state of the clear text place data flow of obtaining from described the first buffer area or the second buffer area.
13. device according to claim 12 is characterized in that,
Described the first buffer area comprises one or more formations, and wherein, each formation is associated with a data flow, is used for the clear text of this data flow of buffer memory, and the number of described formation is more than or equal to the number of all Business Processing threads;
Described processing module concrete is used for judging whether described the first buffer area exists the formation that is associated with described clear text place data flow; If described clear text is buffered in the formation that is associated with described clear text place data flow; If not, described clear text is buffered in the formation of the sky in described the first buffer area, and set up the formation of described sky and the incidence relation of described clear text place data flow, so that the formation of described sky is associated with the data flow at described clear text place.
14. device according to claim 13, it is characterized in that, described processing module, concrete being used for when the treatment state of clear text place data flow is not processed state, the treatment state with described clear text place data flow in the stream table is revised as just at processed state; Wherein, described stream table stores the traffic identifier of described clear text and the corresponding relation for the treatment of state; And after described Business Processing thread was disposed to described clear text, the treatment state with described clear text place data flow in the stream table was revised as not processed state;
Described detection module, if the concrete clear text of described clear text for obtaining from described the second buffer area that be used for, according to the five-tuple information of described clear text, determine the traffic identifier of described clear text, inquire about the treatment state that obtains described clear text place data flow according to described traffic identifier from described stream table; If described clear text is the clear text that obtains from the formation of described the first buffer area, the traffic identifier corresponding according to described formation inquired about the treatment state that obtains described clear text place data flow from described stream table.
15. according to claim 13 or 14 described devices, it is characterized in that, described processing module, concrete for described after described the first buffer area obtains a clear text, if the formation of the described clear text of buffer memory is empty, delete the incidence relation between described formation and described clear text place data flow.
16. device according to claim 12, it is characterized in that, described the first buffer area comprises one or more chained lists, wherein, each chained list is associated with a data flow, be used for the clear text of this data flow of buffer memory, the number of described chained list is more than or equal to the number of all Business Processing threads;
Described processing module concrete is used for judging whether described the first buffer area exists the chained list that is associated with described clear text place data flow; If described clear text is buffered in the chained list that is associated with described clear text place data flow; If not, described clear text is buffered in the chained list of the sky in described the first buffer area, and set up the chained list of described sky and the incidence relation of described clear text place data flow, so that the chained list of described sky is associated with the data flow at described clear text place.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013100666234A CN103166845A (en) | 2013-03-01 | 2013-03-01 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013100666234A CN103166845A (en) | 2013-03-01 | 2013-03-01 | Data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103166845A true CN103166845A (en) | 2013-06-19 |
Family
ID=48589596
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2013100666234A Pending CN103166845A (en) | 2013-03-01 | 2013-03-01 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103166845A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104735025A (en) * | 2013-12-18 | 2015-06-24 | 国家计算机网络与信息安全管理中心 | Data identification method and corresponding device thereof |
| CN105096400A (en) * | 2015-07-27 | 2015-11-25 | 努比亚技术有限公司 | Card punching machine and card punching method |
| CN107451012A (en) * | 2014-07-04 | 2017-12-08 | 华为技术有限公司 | A kind of data back up method and stream calculation system |
| CN107450971A (en) * | 2017-06-29 | 2017-12-08 | 北京五八信息技术有限公司 | Task processing method and device |
| CN109495404A (en) * | 2018-12-14 | 2019-03-19 | 北京金山云网络技术有限公司 | A kind of message processing method, device, switch device and storage medium |
| CN112749028A (en) * | 2021-01-11 | 2021-05-04 | 科大讯飞股份有限公司 | Network traffic processing method, related device and readable storage medium |
| CN113407334A (en) * | 2021-04-08 | 2021-09-17 | 贵州全安密灵科技有限公司 | Method for business processing of detonator based on LINUX system and detonator |
| CN112749028B (en) * | 2021-01-11 | 2024-06-07 | 科大讯飞股份有限公司 | Network traffic processing method, related equipment and readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101221539A (en) * | 2008-01-28 | 2008-07-16 | 杭州华三通信技术有限公司 | Caching system and method for improving caching performance |
| CN101282303A (en) * | 2008-05-19 | 2008-10-08 | 杭州华三通信技术有限公司 | Method and apparatus for processing service packet |
| CN101616083A (en) * | 2009-08-06 | 2009-12-30 | 杭州华三通信技术有限公司 | A kind of message forwarding method and device |
| CN102035719A (en) * | 2009-09-29 | 2011-04-27 | 华为技术有限公司 | Method and device for processing message |
| CN102130833A (en) * | 2011-03-11 | 2011-07-20 | 中兴通讯股份有限公司 | Memory management method and system of traffic management chip chain tables of high-speed router |
| CN102664815A (en) * | 2012-05-21 | 2012-09-12 | 华为技术有限公司 | Load sharing method, device and system for message traffic |
-
2013
- 2013-03-01 CN CN2013100666234A patent/CN103166845A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101221539A (en) * | 2008-01-28 | 2008-07-16 | 杭州华三通信技术有限公司 | Caching system and method for improving caching performance |
| CN101282303A (en) * | 2008-05-19 | 2008-10-08 | 杭州华三通信技术有限公司 | Method and apparatus for processing service packet |
| CN101616083A (en) * | 2009-08-06 | 2009-12-30 | 杭州华三通信技术有限公司 | A kind of message forwarding method and device |
| CN102035719A (en) * | 2009-09-29 | 2011-04-27 | 华为技术有限公司 | Method and device for processing message |
| CN102130833A (en) * | 2011-03-11 | 2011-07-20 | 中兴通讯股份有限公司 | Memory management method and system of traffic management chip chain tables of high-speed router |
| CN102664815A (en) * | 2012-05-21 | 2012-09-12 | 华为技术有限公司 | Load sharing method, device and system for message traffic |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104735025A (en) * | 2013-12-18 | 2015-06-24 | 国家计算机网络与信息安全管理中心 | Data identification method and corresponding device thereof |
| CN104735025B (en) * | 2013-12-18 | 2018-01-09 | 国家计算机网络与信息安全管理中心 | A kind of Data Identification method and its corresponding intrument |
| CN107451012A (en) * | 2014-07-04 | 2017-12-08 | 华为技术有限公司 | A kind of data back up method and stream calculation system |
| CN107451012B (en) * | 2014-07-04 | 2020-09-25 | 华为技术有限公司 | Data backup method and stream computing system |
| CN105096400A (en) * | 2015-07-27 | 2015-11-25 | 努比亚技术有限公司 | Card punching machine and card punching method |
| CN107450971A (en) * | 2017-06-29 | 2017-12-08 | 北京五八信息技术有限公司 | Task processing method and device |
| CN107450971B (en) * | 2017-06-29 | 2021-01-29 | 北京五八信息技术有限公司 | Task processing method and device |
| CN109495404A (en) * | 2018-12-14 | 2019-03-19 | 北京金山云网络技术有限公司 | A kind of message processing method, device, switch device and storage medium |
| CN109495404B (en) * | 2018-12-14 | 2021-03-26 | 北京金山云网络技术有限公司 | Message processing method and device, switch equipment and storage medium |
| CN112749028A (en) * | 2021-01-11 | 2021-05-04 | 科大讯飞股份有限公司 | Network traffic processing method, related device and readable storage medium |
| CN112749028B (en) * | 2021-01-11 | 2024-06-07 | 科大讯飞股份有限公司 | Network traffic processing method, related equipment and readable storage medium |
| CN113407334A (en) * | 2021-04-08 | 2021-09-17 | 贵州全安密灵科技有限公司 | Method for business processing of detonator based on LINUX system and detonator |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105511954B (en) | Message processing method and device | |
| CN101616083B (en) | Message forwarding method and device | |
| US10572290B2 (en) | Method and apparatus for allocating a physical resource to a virtual machine | |
| US8533334B2 (en) | Message binding processing technique | |
| CN103166845A (en) | Data processing method and device | |
| US8626955B2 (en) | Directing packets to a processor unit | |
| JP2008507201A5 (en) | ||
| CN106537367B (en) | Techniques for proxy-based multi-threaded message passing communication | |
| US20170364266A1 (en) | Method and device for managing input/output (i/o) of storage device | |
| CN106776395B (en) | A kind of method for scheduling task and device of shared cluster | |
| US20190007339A1 (en) | Method and device for managing stateful application on server | |
| CN112835695B (en) | Method for communication between Pod and distributed computing system | |
| CN102316043A (en) | Port virtualization method, switch and communication system | |
| US8832215B2 (en) | Load-balancing in replication engine of directory server | |
| CN103995751A (en) | Cross-platform communication system and method | |
| CN103986585A (en) | Message preprocessing method and device | |
| US9594584B2 (en) | Apparatus and method for mapping of tenant based dynamic processor | |
| CN109672575B (en) | Data processing method and electronic equipment | |
| US9753769B2 (en) | Apparatus and method for sharing function logic between functional units, and reconfigurable processor thereof | |
| CN105718304A (en) | Virtual machine management method and system | |
| CN203966126U (en) | Encryption device | |
| CN107896196B (en) | Method and device for distributing messages | |
| US9894012B2 (en) | Method and system to improve network connection locality on multicore systems | |
| CN106933646B (en) | Method and device for creating virtual machine | |
| CN103647729A (en) | Delaying request processing method and apparatus based on token bucket |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20130619 |
|
| RJ01 | Rejection of invention patent application after publication |