CN116996417A - Multi-source port data storage method and device - Google Patents
Multi-source port data storage method and device Download PDFInfo
- Publication number
- CN116996417A CN116996417A CN202311247018.7A CN202311247018A CN116996417A CN 116996417 A CN116996417 A CN 116996417A CN 202311247018 A CN202311247018 A CN 202311247018A CN 116996417 A CN116996417 A CN 116996417A
- Authority
- CN
- China
- Prior art keywords
- data
- buffer
- packet
- signal period
- buffer line
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000013500 data storage Methods 0.000 title claims abstract description 52
- 239000000872 buffer Substances 0.000 claims abstract description 354
- 230000003139 buffering effect Effects 0.000 claims abstract description 33
- 230000000737 periodic effect Effects 0.000 claims description 48
- 238000004590 computer program Methods 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 13
- 238000001514 detection method Methods 0.000 abstract description 9
- 238000004891 communication Methods 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 6
- 238000012805 post-processing Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0823—Errors, e.g. transmission errors
- H04L43/0829—Packet loss
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/12—Avoiding congestion; Recovering from congestion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
- H04L49/9063—Intermediate storage in different physical parts of a node or terminal
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Environmental & Geological Engineering (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention belongs to the technical field of communication, and provides a multi-source port data storage method and a device, wherein the multi-source port data storage method comprises the following steps: acquiring signal period data of a plurality of equipment network ports; marking the signal period data of the network ports of the plurality of devices respectively, and caching the signal period data to the target buffer line areas of the buffer pools corresponding to the network ports of the devices respectively; and under the condition that any data packet in the signal period data is received in the next buffer line area of the target buffer line areas of the buffer pools corresponding to the plurality of equipment network ports, determining that the signal period data buffering is finished. According to the method, the plurality of buffer pools are utilized to orderly buffer the input data of different network ports, so that the situation that the receiving end is congested with the data can be improved, the data packet loss information can be rapidly determined by utilizing the buffer result of the current buffer line area, the real-time receiving buffer and the real-time packet loss detection flow of the data oriented to the multi-source port are optimized, and the data storage efficiency of the multi-source port is improved.
Description
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a method and an apparatus for storing multi-source port data.
Background
With the development of modern technologies, the processing amount of signal data is more and more huge, and the number of ports passing through the switching device is more and more, for example, the port bandwidth reaches the rate of 10Gbps, 40Gbps and even 100 Gbps.
In the related art, a plurality of network device ports are generally configured for each server according to the configuration of the back-end signal processing system to transmit signal data with a regular period, but a large amount of ethernet data packets are easy to congestion and lose after passing through the network switching device, and when the data in the next period arrives at the receiving end earlier than the data in the previous period, the tasks of real-time receiving cache and real-time packet loss detection for the data of the multi-source ports become very complex under the high-speed data bandwidth.
Disclosure of Invention
The invention provides a multi-source port data storage method and device, which are used for solving the defects that when a plurality of network equipment ports are utilized to transmit signal data with a periodic rule in the prior art, congestion and loss are easy to occur at a receiving end, and the data in the next period arrives at the receiving end earlier than the data in the previous period, so that the real-time receiving buffer and the real-time packet loss detection task of the data facing the multi-source port are very complex, and the data buffer and packet loss detection efficiency of the multi-source port is improved.
The invention provides a multi-source port data storage method, which comprises the following steps:
acquiring signal period data of a plurality of equipment network ports, wherein the signal period data comprises a plurality of data packets, each equipment network port corresponds to a buffer pool, each buffer pool comprises a plurality of buffer line areas, and the size of each buffer line area is determined based on the number of the plurality of equipment network ports, the number of the plurality of data packets corresponding to each equipment network port and the size of each data packet;
marking the signal period data of the plurality of equipment network ports respectively, and caching the signal period data to target buffer line areas of the buffer pools corresponding to the equipment network ports respectively;
and under the condition that any data packet in the signal period data is received by the next buffer line area of the target buffer line areas of the buffer pools corresponding to the plurality of equipment network ports, determining that the signal period data is cached.
According to the multi-source port data storage method provided by the invention, the target buffer line area comprises a plurality of period buffer positions, and each period buffer position corresponds to a period number;
the marking the signal period data of the plurality of equipment network ports respectively, and buffering the signal period data to the target buffer line areas of the buffer pools corresponding to the equipment network ports respectively, including:
under the condition that the signal period data of the plurality of equipment network ports belong to the same period, obtaining a period number of each signal period data and a packet number of a plurality of data packets corresponding to each signal period data based on a first identifier, wherein the period number is associated with a buffer pool corresponding to each signal period data;
and sequentially caching each data packet of the marked signal periodic data to the plurality of periodic caching positions according to the caching sequence according to the periodic numbers and the packet numbers.
According to the method for storing multi-source port data provided by the invention, each data packet of the marked signal periodic data is sequentially cached to the plurality of periodic caching positions according to the period number and the packet number, and the method further comprises the following steps:
determining a position index of each data packet based on the cycle number, the packet number and the plurality of cycle cache positions;
and caching each data packet to the plurality of periodic caching positions based on the position index.
According to the multi-source port data storage method provided by the invention, N buffer line areas correspond to N line numbers;
after each data packet of the marked signal periodic data is sequentially cached to the plurality of periodic caching positions according to the period number and the packet number, the method further comprises:
setting a writing buffer area state as the ith buffer line area under the condition that the packet number of a data packet cached in the ith buffer line area is the line number of the mth buffer line area, wherein i is more than or equal to 1 and less than or equal to N, i is more than or equal to m and less than or equal to N, and m, i and N are positive integers;
when the difference between the packet number of the data packet cached in the ith buffer line area and the line number of the ith buffer line area is an integer multiple of the number of the equipment network ports, determining the line number of the (i+1) th buffer line area as the line number of the ith buffer line area;
under the condition that the packet number of a data packet cached in an ith buffer line area is the line number of an n+1th buffer line area, determining the state of a writing buffer area as the n+1th buffer line area, wherein N is a positive integer, and i+1 is less than or equal to N is less than or equal to N;
and when the packet number of the data packet cached in the ith buffer line area is the line number of the n+1th buffer line area and the number of the data packets cached in the n+1th buffer line area is smaller than a number threshold value, determining that the writing buffer area state is the ith buffer line area.
According to the multi-source port data storage method provided by the invention, the target buffer line area comprises a plurality of period buffer positions, and each period buffer position corresponds to a period number;
and under the condition that any data packet in the signal period data is received by a next buffer line area of target buffer line areas of the buffer pools corresponding to the plurality of equipment network ports, determining that the signal period data is cached is completed, including:
determining that the signal period data buffering is completed under the condition that any data packet in the signal period data is received by a target period buffer memory position of a next buffer line area of the target buffer line area; wherein the target periodic buffer location belongs to one of the plurality of periodic buffer locations, and the target periodic buffer location is determined based on the period number of the signal periodic data and the plurality of periodic buffer locations.
According to the multi-source port data storage method provided by the invention, the target buffer line area comprises a plurality of periodic buffer positions;
after the determining that the signal period data buffering is completed, the method further comprises:
acquiring the number of cached data packets in the target buffer line area;
and determining the packet loss rate of the target buffer line area according to the number of the buffered data packets and the plurality of periodic buffer positions.
The invention also provides a multi-source port data storage device, comprising:
the data acquisition module is used for acquiring signal cycle data of a plurality of equipment network ports, wherein the signal cycle data comprises a plurality of data packets, each equipment network port corresponds to a buffer pool, each buffer pool comprises a plurality of buffer line areas, and the size of each buffer line area is determined based on the number of the plurality of equipment network ports, the number of the plurality of data packets corresponding to each equipment network port and the size of each data packet;
the first processing module is used for marking the signal period data of the plurality of equipment network ports respectively and caching the signal period data to the target buffer line areas of the buffer pools corresponding to the equipment network ports respectively;
and the second processing module is used for determining that the signal period data buffering is completed under the condition that any data packet in the signal period data is received by the next buffer line area of the target buffer line areas of the buffer pools corresponding to the plurality of equipment network ports.
According to the present invention, there is provided a multi-source port data storage device, wherein a target buffer line area includes a plurality of periodic buffer locations, the device further including:
a third processing module, configured to obtain, after the determining that the signal period data buffering is completed, the number of buffered data packets in the target buffer line area;
and determining the packet loss rate of the target buffer line area according to the number of the buffered data packets and the plurality of periodic buffer positions.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing any of the multi-source port data storage methods described above when executing the program.
The invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a multi-source port data storage method as described in any of the above.
The invention also provides a computer program product comprising a computer program which when executed by a processor implements a multi-source port data storage method as described in any one of the above.
According to the multi-source port data storage method and device, the signal period data of the network ports of the devices are marked respectively and are cached to the target buffer line areas of the buffer pools corresponding to the network ports of the devices respectively, when any data packet in the signal period data is received in the next buffer line area of the target buffer line areas, the completion of the signal period data caching is determined, the input data of different network ports are cached orderly by utilizing the buffer pools, the condition that the receiving end is congested with the data can be improved, the data packet loss information can be determined quickly by utilizing the caching result of the current buffer line areas, the real-time receiving caching and the real-time packet loss detection flow of the data facing the multi-source ports are optimized, and the data storage efficiency of the multi-source ports is improved.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a multi-source port data storage method according to the present invention;
FIG. 2 is a schematic diagram of an interface for a device portal to transmit data packets to a buffer pool via threads;
FIG. 3 is a second flow chart of a multi-source data storage method according to the present invention;
FIG. 4 is a schematic diagram of a multi-source data storage device according to the present invention;
fig. 5 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The multi-source port data storage method and apparatus of the present invention are described below in conjunction with fig. 1-4.
Fig. 1 is a flow chart of a multi-source port data storage method according to the present invention, as shown in fig. 1, including the following steps:
step 110, acquiring signal cycle data of a plurality of equipment network ports, wherein the signal cycle data comprises a plurality of data packets, each equipment network port corresponds to a buffer pool, each buffer pool comprises a plurality of buffer line areas, and the size of each buffer line area is determined based on the number of the plurality of equipment network ports, the number of the plurality of data packets corresponding to each equipment network port and the size of each data packet.
In this step, the data received by the signal post-processing system is usually periodic data, that is, signal periodic data, and the data in the same period can be scattered to multiple device networks for simultaneous reception due to the large data volume.
In this embodiment, signal data of adjacent periods or different periods may be received simultaneously through a plurality of device networks.
In this embodiment, the cyclic buffer pool buffering technology is used to receive and buffer the signal cycle data input by different network ports, where the number of buffer pools is consistent with the number of network ports, so as to ensure that the signal cycle data input from each device network has a corresponding buffer pool.
In this embodiment, the size of each buffer line area is determined by the number of device ports that need to receive data, the number of packets that are scattered to each port per cycle, and the size of each packet.
For example, the P buffer pools correspond to E device interfaces, each buffer pool includes a plurality of buffer line areas, and the size of each buffer line area is B, where:
P=E；B=E*N*S；
wherein E is the number of network ports of the equipment, N is the number of data packets in data of each signal period, and S is the size of each data packet; e=2, n=12, s=8192.
And 120, marking the signal period data of the network ports of the devices respectively, and caching the signal period data in the target buffer line areas of the buffer pools corresponding to the network ports of the devices respectively.
In this step, the signal period data may be marked by custom setting the relevant number of each data packet, for example, the packet header of the data packet includes a period number and a data packet number in the same period.
In this embodiment, the signal post-processing system receives signal period data of multiple device interfaces through multiple threads at the same time, after determining a data packet format of an input interface, a general network transmission protocol, such as UDP (User Datagram Protocol ), is adopted, and after adding a label with a relevant number to the signal period data, each data packet in the labeled signal period data is buffered in a corresponding buffer pool, for example: the data of the network port 1 received by the thread 1 is cached in the buffer pool 1, and the data of the network port 2 received by the thread 2 is cached in the buffer pool 2.
In this embodiment, the corresponding position of each data packet received and buffered by the buffer is determined according to the period number I and the packet number PID in the marked signal period data.
In this embodiment, after the input signal period data is respectively transmitted to each buffer pool for buffering, the buffered data in the buffer line area at the same position of each buffer pool is combined to be the complete signal period data, for example, the nth data of the buffer pool 1 and the nth data of the buffer pools 2, 3, … and P in the buffer design are combined to be the complete E signal period data.
Step 130, determining that the signal period data buffering is completed when any one data packet in the signal period data is received in a next buffer line area of the target buffer line areas of the buffer pools corresponding to the plurality of equipment network ports.
In this step, when the next buffer line area of the target buffer line area can receive the data packet, the data of the target buffer line area is buffered.
For example, the buffer area performs packet buffer and packet loss detection on the data, and the packet loss judging method comprises the following steps: when the data at the m-th period position in the n+1th line buffer area receives k (0 < k < PID), the E period of the n line buffer area is judged to be received completely, and the data packet which is not received by the line buffer area is judged to be a lost data packet.
In this embodiment, the value of m is calculated from the period number of the signal period data and the number of the device network ports.
According to the multi-source port data storage method provided by the embodiment of the invention, the signal period data of the network ports of the plurality of devices are respectively marked and respectively cached to the target buffer line areas of the buffer pools corresponding to the network ports of the devices, the completion of the signal period data caching is determined under the condition that any data packet in the signal period data is received in the next buffer line area of the target buffer line area, the input data of different network ports are orderly cached by utilizing the plurality of buffer pools, the condition that the receiving end is congested with data can be improved, the data packet loss information can be rapidly determined by utilizing the caching result of the current buffer line area, the real-time receiving caching and the real-time packet loss detection flow of the data facing the multi-source port are optimized, and the data storage efficiency of the multi-source port is improved.
In some embodiments, the target buffer line region includes a plurality of cycle buffer locations, each cycle buffer location corresponding to a cycle number; marking the signal period data of the network ports of the devices respectively, and caching the signal period data to the target buffer line areas of the buffer pools corresponding to the network ports of the devices respectively, wherein the method comprises the following steps: under the condition that signal period data of a plurality of equipment network ports belong to the same period, obtaining a period number of each signal period data and a packet number of a plurality of data packets corresponding to each signal period data based on a first identifier, wherein the period number is associated with a buffer pool corresponding to each signal period data; and sequentially caching each data packet of the marked signal periodic data to a plurality of periodic caching positions according to the caching sequence according to the period number and the packet number.
In this embodiment, the first identifier may be identifier information added by the user in each data packet of the signal period data, for example, the first identifier is a period number added in each signal period data, and a packet number corresponding to a different data packet included in each signal period data.
For example, since the data received in the signal post-processing system is usually periodic data, and since the received data volume is large, the data in the same period is dispersed to a plurality of equipment network ports, and before the data is input into the corresponding buffer pool, the period number I of the packet header of the data packet and the UDP data packet number PID in the same period are customized through the first identifier; wherein I is counted from 1, the number of data packets in each period is 24, and PID is counted from 1.
In this embodiment, the signal post-processing system receives data of E network ports simultaneously through multiple threads, and the network port data packet received by each thread is cached in a corresponding buffer pool.
FIG. 2 is a schematic diagram of an interface of a device portal transmitting data packets to a buffer pool through a thread, in the embodiment shown in FIG. 2, data of a portal 1 received by a thread 1 is buffered to the buffer pool 1, data of a portal 2 received by a thread 2 is buffered to the buffer pool 2, and so on; each buffer pool comprises 2 period buffer positions (corresponding to the 1 st period and the 2 nd period), and a plurality of data packets of signal period data input by each network port are buffered to the corresponding period buffer positions one by one according to the period numbers and the packet numbers.
In this embodiment, the buffer position of each data packet in the current buffer pool may be determined by the packet number of each data packet and the association relationship between the period number corresponding to the data packet and the multiple period buffer positions of the buffer pool, so as to facilitate the subsequent statistics of packet loss information.
According to the multi-source port data storage method provided by the embodiment of the invention, under the condition that the signal period data of the network ports of the plurality of devices belong to the same period, the period number of each signal period data and the packet number of the plurality of data packets corresponding to each signal period data are obtained based on the first identifier, and each data packet of the marked signal period data is sequentially cached to the plurality of period cache positions according to the period number and the packet number, so that the data packets distributed to different network ports can be orderly stored to the corresponding period cache positions for caching, and the signal period data caching efficiency is improved.
In some embodiments, buffering each packet of the marked signal period data to a plurality of period buffering positions according to the buffering sequence in sequence according to the period number and the packet number, and further including: determining a position index of each data packet based on the period number, the packet number and a plurality of period buffer positions; each data packet is buffered to a plurality of periodic buffer locations based on the location index.
In this embodiment, each line buffer is set with a partial data size of E periods, and the position index of each received data packet stored in each line buffer can be calculated by the period number I and the packet number PID by the following formula:
Index=(I%E)×PID；
where Index is a location Index indicating the location number of the buffer where the received packet was stored.
In this embodiment, the data packets buffered at the same location in each buffer pool in the buffer design of the signal post-processing system are combined to be complete signal period data, for example, the nth data of the buffer pool 1 and the nth data of the buffer pool 2 are combined to be complete 2 period data.
According to the multi-source port data storage method provided by the embodiment of the invention, the position index of each data packet is calculated through the corresponding period number and packet number of each data packet and the plurality of period cache positions in the buffer pool, and each data packet is cached to the plurality of period cache positions according to the position index, so that the caching accuracy of each data packet of signal period data is improved.
In some embodiments, the N buffer line regions correspond to N line numbers; after each data packet of the marked signal periodic data is sequentially cached to a plurality of periodic caching positions according to the caching sequence according to the period number and the packet number, the method further comprises the steps of: setting the state of a writing buffer area as the ith buffer line area under the condition that the packet number of the data packet cached in the ith buffer line area is the line number of the mth buffer line area, i is more than or equal to 1 and less than or equal to N, i is more than or equal to m and less than or equal to N, and m, i and N are positive integers; when the difference between the packet number of the data packet cached in the ith buffer line area and the line number of the ith buffer line area is an integer multiple of the number of the equipment network ports, determining the line number of the (i+1) th buffer line area as the line number of the ith buffer line area; under the condition that the packet number of the data packet cached in the ith buffer line area is the line number of the (n+1) th buffer line area, determining the state of the writing buffer area as the (n+1) th buffer line area, wherein N is a positive integer, and i+1 is less than or equal to N; and when the packet number of the data packet cached in the ith buffer line area is the line number of the (n+1) th buffer line area and the number of the data packets cached in the (n+1) th buffer line area is smaller than the number threshold value, determining that the writing buffer area state is the ith buffer line area.
In this embodiment, the simultaneous reception of multi-port packets may occur in several cases:
(1) The data of the previous period of the period critical position arrives later than the data of the next period;
(2) A small amount of packets are lost in a certain period;
(3) And a large amount of packets are lost in a certain small period of time.
In this embodiment, several variables need to be set within each receiving thread before buffering the signal cycle data into the buffer pool: the cycle number of the current data packet, the cycle number of the current buffer line, the cycle number of the next line of the buffer, and the write buffer line state.
Fig. 3 is a second flow chart of the multi-source port data storage method provided by the present invention, in the embodiment shown in fig. 3, after a buffer receives a data packet and determines a period number and a packet number corresponding to each data packet by analyzing the data packet header, if the period number of the current data packet is equal to the period number of the current buffer line area (or the period number+1 of the current buffer line), setting the write buffer area state as the current buffer line area; when the current data packet period number is greater than the current buffer line period number +1, the following situations are: (1) The first data packet cycle number-the current buffer line cycle number is a multiple of 2, and the next line cycle number = current packet cycle number is set; (2) The current data packet cycle number is equal to the cycle number of the next line of the buffer (or the cycle number of the next line+1), and the state of the write buffer is set as the next line; (3) The cycle number of the current data packet exceeds the cycle number +1 of the next line of the buffer zone, the number of the data packets received by the next line is smaller than 6 (corresponding number threshold), the state of the next line is reset, and the write buffer zone is set to be the current state; if the number of the data packets received by the next line is greater than or equal to 6, changing the states of the current line and the next line, and recording the packet loss information of the buffer line area, wherein the packet loss information comprises the packet loss number and the packet loss position.
The multi-source port data storage method provided by the embodiment of the invention is characterized in that when the packet number of a data packet buffered in an ith buffer line area is the line number of an mth buffer line area, a write buffer area state is set as the ith buffer line area, and when the difference between the packet number of the data packet buffered in the ith buffer line area and the line number of the ith buffer line area is an integer multiple of the number of a device network port, the line number of the (i+1) th buffer line area is determined as the line number of the ith buffer line area; under the condition that the packet number of the data packet buffered in the ith buffer line area is the line number of the (n+1) th buffer line area, determining that the writing buffer area state is the (n+1) th buffer line area, and when the packet number of the data packet buffered in the ith buffer line area is the line number of the (n+1) th buffer line area and the number of the data packet buffered in the (n+1) th buffer line area is smaller than the number threshold, determining that the writing buffer area state is the (i) th buffer line area, the problem that the data of the next period arrives at the receiving end earlier than the data of the previous period can be solved, and the packet loss statistical accuracy of the buffer line area is improved.
In some embodiments, the target buffer line region includes a plurality of cycle buffer locations, each cycle buffer location corresponding to a cycle number; under the condition that any data packet in the signal period data is received in a next buffer line area of target buffer line areas of the buffer pools corresponding to the plurality of equipment network ports, determining that the signal period data buffering is completed comprises the following steps: under the condition that any data packet in the signal period data is received in the next buffer line area of the target buffer line area, determining that the signal period data is cached; the target periodic buffer position belongs to one of a plurality of periodic buffer positions, and the target periodic buffer position is determined based on the period number of the signal periodic data and the plurality of periodic buffer positions.
In this embodiment, packet loss detection is performed on data while the buffer pool buffers the data packets, and when 6 data packets are received from the data at the mth cycle position in the n+1th line buffer, it is determined that 2 cycles of the n line buffer are received, and the data packets that should be received but not received by the line buffer are determined to be lost.
The value of the target period buffer position m can be determined by the following formula:
m=I%E；
wherein, the liquid crystal display device comprises a liquid crystal display device,the method is characterized in that the method is used for numbering the period of signal period data, E is the number of network ports of equipment, and% is modulo operation.
According to the multi-source port data storage method provided by the embodiment of the invention, under the condition that any data packet in the signal period data is received at the target period cache position of the next buffer line area of the target buffer line area, the completion of the data caching of the target buffer line area is determined, the condition that the receiving end receives data congestion occurs is improved, and the multi-source port data storage efficiency is improved.
In some embodiments, the target buffer line region includes a plurality of periodic buffer locations; after determining that the signal period data buffering is complete, the method further comprises: acquiring the number of cached data packets in a target buffer line area; and determining the packet loss rate of the target buffer line area according to the number of the buffered data packets and the buffer positions of a plurality of periods.
In this embodiment, under the condition that the mth position of the n+1th line of the buffer pool receives k (0 < k is less than or equal to PID) data packets, determining that the nth line of the buffer pool has completed buffering of the data packets, checking the buffering condition of each period buffering position of the nth line to determine the number of the buffered data packets, and further calculating the ratio of the number of the packets to the total number of the period buffering positions, namely, the packet loss rate of the nth buffer line area.
The multi-source port data storage method provided by the embodiment of the invention is characterized in that the number of the cached data packets in a target buffer line area is obtained; and determining the packet loss rate of the target buffer line area according to the number of the buffered data packets and a plurality of period buffering positions, and providing data support for the packet loss information of the signal period data buffered in the subsequent analysis buffer area.
The multi-source port data storage device provided by the invention is described below, and the multi-source port data storage device described below and the multi-source port data storage method described above can be referred to correspondingly.
Fig. 4 is a schematic structural diagram of a multi-source data storage device according to the present invention, and as shown in fig. 4, the multi-source data storage device includes a data acquisition module 410, a first processing module 420, and a second processing module 430.
A data obtaining module 410, configured to obtain signal cycle data of a plurality of device interfaces, where the signal cycle data includes a plurality of data packets, each device interface corresponds to a buffer pool, each buffer pool includes a plurality of buffer line areas, and a size of each buffer line area is determined based on a number of the plurality of device interfaces, a number of the plurality of data packets corresponding to each device interface, and a size of each data packet;
the first processing module 420 is configured to mark signal period data of multiple device network ports respectively, and buffer the signal period data to a target buffer line area of a buffer pool corresponding to each device network port respectively;
the second processing module 430 is configured to determine that buffering of the signal period data is completed when any one of the data packets in the signal period data is received in a next buffer line area of the target buffer line areas of the buffer pools corresponding to the plurality of device interfaces.
According to the multi-source port data storage device provided by the embodiment of the invention, the signal period data of the network ports of the plurality of devices are respectively marked and respectively cached to the target buffer line areas of the buffer pools corresponding to the network ports of the devices, the completion of the signal period data caching is determined under the condition that any data packet in the signal period data is received in the next buffer line area of the target buffer line area, the input data of different network ports are orderly cached by utilizing the plurality of buffer pools, the condition that the receiving end is congested with data can be improved, the data packet loss information can be rapidly determined by utilizing the caching result of the current buffer line area, the real-time receiving caching and the real-time packet loss detection flow of the data facing the multi-source port are optimized, and the data storage efficiency of the multi-source port is improved.
In some embodiments, the target buffer line region includes a plurality of periodic buffer locations, the apparatus further comprising: the third processing module is used for acquiring the number of the cached data packets in the target buffer line area after the completion of the data caching of the signal period is determined; and determining the packet loss rate of the target buffer line area according to the number of the buffered data packets and the buffer positions of a plurality of periods.
The multi-source port data storage device provided by the embodiment of the invention obtains the number of the cached data packets in the target buffer line area; and determining the packet loss rate of the target buffer line area according to the number of the buffered data packets and a plurality of period buffering positions, and providing data support for the packet loss information of the signal period data buffered in the subsequent analysis buffer area.
Fig. 5 is a schematic structural diagram of an electronic device according to the present invention, and as shown in fig. 5, the electronic device may include: processor 510, communication interface (Communications Interface) 520, memory 530, and communication bus 540, wherein processor 510, communication interface 520, memory 530 complete communication with each other through communication bus 540. Processor 510 may invoke logic instructions in memory 530 to perform a multi-source port data storage method comprising: acquiring signal period data of a plurality of equipment network ports, wherein the signal period data comprises a plurality of data packets, each equipment network port corresponds to a buffer pool, each buffer pool comprises a plurality of buffer line areas, and the size of each buffer line area is determined based on the number of the plurality of equipment network ports, the number of the plurality of data packets corresponding to each equipment network port and the size of each data packet; marking the signal period data of the network ports of the plurality of devices respectively, and caching the signal period data to the target buffer line areas of the buffer pools corresponding to the network ports of the devices respectively; and under the condition that any data packet in the signal period data is received in the next buffer line area of the target buffer line areas of the buffer pools corresponding to the plurality of equipment network ports, determining that the signal period data buffering is finished.
Further, the logic instructions in the memory 530 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of performing the multi-source port data storage method provided by the methods described above, the method comprising: acquiring signal period data of a plurality of equipment network ports, wherein the signal period data comprises a plurality of data packets, each equipment network port corresponds to a buffer pool, each buffer pool comprises a plurality of buffer line areas, and the size of each buffer line area is determined based on the number of the plurality of equipment network ports, the number of the plurality of data packets corresponding to each equipment network port and the size of each data packet; marking the signal period data of the network ports of the plurality of devices respectively, and caching the signal period data to the target buffer line areas of the buffer pools corresponding to the network ports of the devices respectively; and under the condition that any data packet in the signal period data is received in the next buffer line area of the target buffer line areas of the buffer pools corresponding to the plurality of equipment network ports, determining that the signal period data buffering is finished.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the multi-source port data storage method provided by the above methods, the method comprising: acquiring signal period data of a plurality of equipment network ports, wherein the signal period data comprises a plurality of data packets, each equipment network port corresponds to a buffer pool, each buffer pool comprises a plurality of buffer line areas, and the size of each buffer line area is determined based on the number of the plurality of equipment network ports, the number of the plurality of data packets corresponding to each equipment network port and the size of each data packet; marking the signal period data of the network ports of the plurality of devices respectively, and caching the signal period data to the target buffer line areas of the buffer pools corresponding to the network ports of the devices respectively; and under the condition that any data packet in the signal period data is received in the next buffer line area of the target buffer line areas of the buffer pools corresponding to the plurality of equipment network ports, determining that the signal period data buffering is finished.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A multi-source port data storage method, comprising:
acquiring signal period data of a plurality of equipment network ports, wherein the signal period data comprises a plurality of data packets, each equipment network port corresponds to a buffer pool, each buffer pool comprises a plurality of buffer line areas, and the size of each buffer line area is determined based on the number of the plurality of equipment network ports, the number of the plurality of data packets corresponding to each equipment network port and the size of each data packet;
marking the signal period data of the plurality of equipment network ports respectively, and caching the signal period data to target buffer line areas of the buffer pools corresponding to the equipment network ports respectively;
and under the condition that any data packet in the signal period data is received by the next buffer line area of the target buffer line areas of the buffer pools corresponding to the plurality of equipment network ports, determining that the signal period data is cached.
2. The multi-source port data storage method of claim 1, wherein the target buffer line region comprises a plurality of cycle buffer locations, each cycle buffer location corresponding to a cycle number;
the marking the signal period data of the plurality of equipment network ports respectively, and buffering the signal period data to the target buffer line areas of the buffer pools corresponding to the equipment network ports respectively, including:
under the condition that the signal period data of the plurality of equipment network ports belong to the same period, obtaining a period number of each signal period data and a packet number of a plurality of data packets corresponding to each signal period data based on a first identifier, wherein the period number is associated with a buffer pool corresponding to each signal period data;
and sequentially caching each data packet of the marked signal periodic data to the plurality of periodic caching positions according to the caching sequence according to the periodic numbers and the packet numbers.
3. The method for storing multi-source port data according to claim 2, wherein sequentially buffering each packet of the marked signal period data to the plurality of period buffering positions according to the period number and the packet number in a buffering order, further comprising:
determining a position index of each data packet based on the cycle number, the packet number and the plurality of cycle cache positions;
and caching each data packet to the plurality of periodic caching positions based on the position index.
4. The multi-source port data storage method of claim 2, wherein the N buffer line areas correspond to N line numbers;
after each data packet of the marked signal periodic data is sequentially cached to the plurality of periodic caching positions according to the period number and the packet number, the method further comprises:
setting a writing buffer area state as the ith buffer line area under the condition that the packet number of a data packet cached in the ith buffer line area is the line number of the mth buffer line area, wherein i is more than or equal to 1 and less than or equal to N, i is more than or equal to m and less than or equal to N, and m, i and N are positive integers;
when the difference between the packet number of the data packet cached in the ith buffer line area and the line number of the ith buffer line area is an integer multiple of the number of the equipment network ports, determining the line number of the (i+1) th buffer line area as the line number of the ith buffer line area;
under the condition that the packet number of a data packet cached in an ith buffer line area is the line number of an n+1th buffer line area, determining the state of a writing buffer area as the n+1th buffer line area, wherein N is a positive integer, and i+1 is less than or equal to N is less than or equal to N;
and when the packet number of the data packet cached in the ith buffer line area is the line number of the n+1th buffer line area and the number of the data packets cached in the n+1th buffer line area is smaller than a number threshold value, determining that the writing buffer area state is the ith buffer line area.
5. The multi-source port data storage method of claim 1, wherein the target buffer line region comprises a plurality of cycle buffer locations, each cycle buffer location corresponding to a cycle number;
and under the condition that any data packet in the signal period data is received by a next buffer line area of target buffer line areas of the buffer pools corresponding to the plurality of equipment network ports, determining that the signal period data is cached is completed, including:
determining that the signal period data buffering is completed under the condition that any data packet in the signal period data is received by a target period buffer memory position of a next buffer line area of the target buffer line area; wherein the target periodic buffer location belongs to one of the plurality of periodic buffer locations, and the target periodic buffer location is determined based on the period number of the signal periodic data and the plurality of periodic buffer locations.
6. The multi-source port data storage method of claim 1 or 5 wherein the target buffer line region comprises a plurality of periodic buffer locations;
after the determining that the signal period data buffering is completed, the method further comprises:
acquiring the number of cached data packets in the target buffer line area;
and determining the packet loss rate of the target buffer line area according to the number of the buffered data packets and the plurality of periodic buffer positions.
7. A multi-source port data storage device, comprising:
the data acquisition module is used for acquiring signal cycle data of a plurality of equipment network ports, wherein the signal cycle data comprises a plurality of data packets, each equipment network port corresponds to a buffer pool, each buffer pool comprises a plurality of buffer line areas, and the size of each buffer line area is determined based on the number of the plurality of equipment network ports, the number of the plurality of data packets corresponding to each equipment network port and the size of each data packet;
the first processing module is used for marking the signal period data of the plurality of equipment network ports respectively and caching the signal period data to the target buffer line areas of the buffer pools corresponding to the equipment network ports respectively;
and the second processing module is used for determining that the signal period data buffering is completed under the condition that any data packet in the signal period data is received by the next buffer line area of the target buffer line areas of the buffer pools corresponding to the plurality of equipment network ports.
8. The multi-source port data storage device of claim 7, wherein the target buffer line region comprises a plurality of periodic buffer locations, the device further comprising:
a third processing module, configured to obtain, after the determining that the signal period data buffering is completed, the number of buffered data packets in the target buffer line area;
and determining the packet loss rate of the target buffer line area according to the number of the buffered data packets and the plurality of periodic buffer positions.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the multi-source port data storage method of any one of claims 1 to 6 when the program is executed by the processor.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the multi-source port data storage method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311247018.7A CN116996417B (en) | 2023-09-26 | 2023-09-26 | Multi-source port data storage method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311247018.7A CN116996417B (en) | 2023-09-26 | 2023-09-26 | Multi-source port data storage method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116996417A true CN116996417A (en) | 2023-11-03 |
| CN116996417B CN116996417B (en) | 2024-01-16 |
Family
ID=88521700
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311247018.7A Active CN116996417B (en) | 2023-09-26 | 2023-09-26 | Multi-source port data storage method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116996417B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016183155A1 (en) * | 2015-05-13 | 2016-11-17 | Cisco Technology, Inc. | Dynamic protection of shared memory used by output queues in a network device |
| CN109189339A (en) * | 2018-08-28 | 2019-01-11 | 郑州云海信息技术有限公司 | Caching accelerated method is automatically configured under a kind of storage system |
| CN110943941A (en) * | 2019-12-06 | 2020-03-31 | 北京天融信网络安全技术有限公司 | Message receiving method, message sending method, network card and electronic equipment |
| US20210297354A1 (en) * | 2020-03-18 | 2021-09-23 | Marvell Israel (M.I.S.L) Ltd. | Packet Buffer Spill-Over in Network Devices |
-
2023
- 2023-09-26 CN CN202311247018.7A patent/CN116996417B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016183155A1 (en) * | 2015-05-13 | 2016-11-17 | Cisco Technology, Inc. | Dynamic protection of shared memory used by output queues in a network device |
| CN109189339A (en) * | 2018-08-28 | 2019-01-11 | 郑州云海信息技术有限公司 | Caching accelerated method is automatically configured under a kind of storage system |
| CN110943941A (en) * | 2019-12-06 | 2020-03-31 | 北京天融信网络安全技术有限公司 | Message receiving method, message sending method, network card and electronic equipment |
| US20210297354A1 (en) * | 2020-03-18 | 2021-09-23 | Marvell Israel (M.I.S.L) Ltd. | Packet Buffer Spill-Over in Network Devices |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116996417B (en) | 2024-01-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10097464B1 (en) | Sampling based on large flow detection for network visibility monitoring | |
| US9979624B1 (en) | Large flow detection for network visibility monitoring | |
| US10536360B1 (en) | Counters for large flow detection | |
| US10003515B1 (en) | Network visibility monitoring | |
| CN112688837B (en) | Network measurement method and device based on time sliding window | |
| CN107800644A (en) | Dynamically configurable pipelined token bucket speed limiting method and device | |
| EP3200391B1 (en) | Counter and counting method | |
| US8533355B2 (en) | Techniques for improved clock offset measuring | |
| CN108173780B (en) | Data processing method, data processing device, computer and storage medium | |
| CN102882809B (en) | Network speed-limiting method and device based on message buffering | |
| CN116996417B (en) | Multi-source port data storage method and device | |
| CN112995809B (en) | Control method and device of FC switch, FC switch and storage medium | |
| CN113572655A (en) | Congestion detection method and system for loss-free network | |
| CN109412939A (en) | Record the communication gate and working method of industrial network communication cycle process data | |
| US8254399B1 (en) | Method and apparatus for adaptive buffer management for traffic optimization on switches | |
| US11489904B2 (en) | Poor-QoE assessment method and related device | |
| CN109308180B (en) | Processing method and processing device for cache congestion | |
| US9667546B2 (en) | Programmable partitionable counter | |
| CN115208845A (en) | Method, apparatus, device, storage medium, and program product for transmitting data packet | |
| CN117527641B (en) | Packet loss observation method, device and equipment for data message and storage medium | |
| CN112073318A (en) | Method, device, equipment and storage medium for transmitting and receiving aggregation link message | |
| US9306854B2 (en) | Method and apparatus for diagnosing interface oversubscription and microbursts | |
| CN117527937B (en) | Ordering window-based data message ordering method, device and equipment | |
| CN114124754B (en) | Method for processing media data packets in a multimedia network and related products | |
| US9112766B2 (en) | Method and apparatus having improved line rate IP packet communication |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |