CN111371704B - Data caching method and device, terminal equipment and storage medium - Google Patents
Data caching method and device, terminal equipment and storage medium Download PDFInfo
- Publication number
- CN111371704B CN111371704B CN202010081796.3A CN202010081796A CN111371704B CN 111371704 B CN111371704 B CN 111371704B CN 202010081796 A CN202010081796 A CN 202010081796A CN 111371704 B CN111371704 B CN 111371704B
- Authority
- CN
- China
- Prior art keywords
- data packet
- buffer area
- size
- buffer
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 239000000872 buffer Substances 0.000 claims abstract description 297
- 238000004590 computer program Methods 0.000 claims description 17
- 230000005540 biological transmission Effects 0.000 claims description 10
- 238000012544 monitoring process Methods 0.000 claims description 9
- 238000004891 communication Methods 0.000 abstract description 18
- 230000008569 process Effects 0.000 abstract description 14
- 238000010586 diagram Methods 0.000 description 13
- 238000012545 processing Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
- H04L49/9005—Buffering arrangements using dynamic buffer space allocation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the invention provides a data caching method, a device, terminal equipment and a storage medium, comprising the following steps: acquiring a data packet sent by a target terminal through a pre-established IOCP model; determining the size of a data packet and the size of an available buffer area in a virtual container corresponding to a target terminal; if the size of the data packet is larger than the size of the available buffer area, calling the dynamic buffer area to expand the available buffer area to obtain an expanded buffer area; by storing the data packet into the extended buffer area, the buffer area in the virtual container corresponding to each terminal can be dynamically extended in real time under the condition of adopting the IOCP model to process large concurrent data, and the communication capacity of the IOCP network is improved to the greatest extent.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a data caching method, a data caching device, a terminal device, and a storage medium.
Background
With the rapid development of network technology, two-way communication such as video conference and video teaching is widely popularized in the aspects of life, work, study and the like of users.
The IOCP frame mode supports large concurrent data, each path of network communication connection needs to consume a certain memory space to bear network data acquired from an I/O layer, single path resources are relatively small, and when the concurrency quantity reaches a certain value, the required memory space is huge to a server and cannot bear, so that the large concurrency quantity and service equipment performance of the whole IOCP frame are limited.
In general, when performing service processing, each established network communication connection does not perform service processing in real time, only a small portion of memory resources are occupied, and most of memory resources applied for by the network communication connection are in a dormant and stagnant state and are not really utilized.
Disclosure of Invention
In view of the foregoing, embodiments of the present invention are provided to provide a data caching method, apparatus, terminal device, and storage medium that overcome or at least partially solve the foregoing problems.
In a first aspect, an embodiment of the present invention provides a data caching method, applied to a server, where the method includes:
acquiring a data packet sent by a target terminal through a pre-established IOCP model;
determining the size of the data packet and the size of an available buffer area in a virtual container corresponding to the target terminal;
if the size of the data packet is larger than the size of the available buffer area, calling a dynamic buffer area to expand the available buffer area to obtain an expanded buffer area;
and storing the data packet into the expansion buffer area.
Optionally, before the data packet sent by the target terminal is acquired through the pre-established IOCP model, the method further includes:
creating a plurality of virtual containers in advance;
when the monitoring thread in the IOCP model detects that the target terminal sends a data packet, the virtual container is associated with the target terminal to form a network channel of the target terminal and the virtual container, so that the target terminal and a server perform data transmission.
Optionally, if the size of the data packet is greater than the size of the available buffer, calling a dynamic buffer to expand the available buffer to obtain an expanded buffer, including:
determining the size of a buffer area needing to be expanded according to the difference between the size of the data packet and the size of the available buffer area;
and calling the dynamic buffer to expand the available buffer according to the size of the buffer to be expanded to obtain the expanded buffer, wherein the dynamic buffer comprises buffers except for a preset buffer which is applied for the virtual container corresponding to each terminal in advance and/or preset buffers of virtual containers corresponding to other terminals.
Optionally, the available buffer area is determined according to a buffer area occupied by a data packet written into the extended buffer area and a released buffer area after the data packet has been read from the extended buffer area.
Optionally, the storing the data packet in the extended buffer area includes:
after analyzing the data packet sent by the terminal, writing the analyzed data into the expansion buffer area, wherein the analyzed data at least comprises the byte length of the data packet, and the byte length of the data packet occupies the expansion buffer area with the preset byte length.
Optionally, after storing the data packet in the extended buffer, the method further includes:
reading a data packet from the expansion buffer area;
releasing the expansion buffer area occupied by the read data packet;
specifically, the reading the data packet from the extended buffer area includes:
and if the length of the remained bytes in the expansion buffer is greater than the preset length of the bytes, reading the data packet from the expansion buffer.
In a second aspect, an embodiment of the present invention provides a data caching apparatus, applied to a server, where the apparatus includes:
the acquisition module is used for acquiring a data packet sent by the target terminal through a pre-established IOCP model;
the determining module is used for determining the size of the data packet and the size of an available buffer area in the virtual container corresponding to the target terminal;
the capacity expansion module is used for calling the dynamic buffer area to expand the available buffer area if the size of the data packet is larger than the size of the available buffer area, so as to obtain an expanded buffer area;
and the writing module is used for storing the data packet into the expansion buffer area.
Optionally, before the obtaining module, the apparatus further includes an associating module, where the associating module is configured to:
creating a plurality of virtual containers in advance;
when the monitoring thread in the IOCP model detects that the target terminal sends a data packet, the virtual container is associated with the target terminal to form a network channel of the target terminal and the virtual container, so that the target terminal and a server perform data transmission.
Optionally, the capacity expansion module is specifically configured to:
determining the size of a buffer area needing to be expanded according to the difference between the size of the data packet and the size of the available buffer area;
and calling the dynamic buffer to expand the available buffer according to the size of the buffer to be expanded to obtain the expanded buffer, wherein the dynamic buffer comprises buffers except for a preset buffer which is applied for the virtual container corresponding to each terminal in advance and/or preset buffers of virtual containers corresponding to other terminals.
Optionally, the available buffer area is determined according to a buffer area occupied by a data packet written into the extended buffer area and a released buffer area after the data packet has been read from the extended buffer area.
Optionally, the writing module is specifically configured to:
after analyzing the data packet sent by the terminal, writing the analyzed data into the expansion buffer area, wherein the analyzed data at least comprises the byte length of the data packet, and the byte length of the data packet occupies the expansion buffer area with the preset byte length.
Optionally, after the writing module, the apparatus further includes a reading module for:
reading a data packet from the expansion buffer area;
releasing the expansion buffer area occupied by the read data packet;
the reading module is specifically configured to:
and if the length of the remained bytes in the expansion buffer is greater than the preset length of the bytes, reading the data packet from the expansion buffer.
In a third aspect, an embodiment of the present invention provides a terminal device, including: at least one processor and memory;
the memory stores a computer program; the at least one processor executes the computer program stored by the memory to implement the data caching method provided in the first aspect.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having stored therein a computer program that when executed implements the data caching method provided in the first aspect.
The technical scheme provided by the embodiment of the invention is applied to a server and comprises the following steps: acquiring a data packet sent by a target terminal through a pre-established IOCP model; determining the size of the data packet and the size of an available buffer area in a virtual container corresponding to the target terminal; if the size of the data packet is larger than the size of the available buffer area, calling the dynamic buffer area to expand the available buffer area to obtain an expanded buffer area; by storing the data packet into the extended buffer area, the buffer area in the virtual container corresponding to each terminal can be dynamically extended in real time under the condition of adopting the IOCP model to process large concurrent data, and the communication capacity of the IOCP network is improved to the greatest extent.
Drawings
FIG. 1 is a flow chart of steps of an embodiment of a data caching method of the present invention;
FIG. 2 is a flow chart of steps of another embodiment of a data caching method of the present invention;
FIG. 3 is a flow chart of steps of yet another embodiment of a data caching method of the present invention;
FIG. 4 is a flow chart of steps of yet another embodiment of a data caching method of the present invention;
FIG. 5 is a flow chart of steps of yet another embodiment of a data caching method of the present invention;
FIG. 6 is a flowchart illustrating steps of another embodiment of a data caching method of the present invention;
FIG. 7 is a block diagram illustrating an embodiment of a data caching apparatus of the present invention;
fig. 8 is a schematic structural view of a terminal device of the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
The nouns are explained as follows:
IOCP: I/O Completion Port, commonly referred to as an I/O completion port. The IOCP model belongs to a communication model, and is suitable for a technology of a high-load server capable of controlling concurrent execution. In a popular sense, it is a model for efficiently handling data exchange by a large number of clients. Or, as it were, a model that can operate asynchronously in I/O; "done" in the IOCP (i.e., done port) means that the system is notified after the network I/O operation "completes" that data from the network has been received. This may enable the application to do other things during the I/O transfer. The IOCP network communication mode is the network communication underlying framework supporting large concurrency and high load which is most widely used at present.
Referring to fig. 1, a flowchart illustrating steps of an embodiment of a data caching method of the present invention is shown, where the method may be applied to a server, and specifically may include the following steps:
s101, acquiring a data packet sent by a target terminal through a pre-established IOCP model;
specifically, on the server side, an IOCP model is pre-established, that is, the data packets sent by each of the plurality of internet terminals can be received through a plurality of asynchronous I/os at the same time, that is, the server receives the I/O layer network data from each channel through the IOCP model.
When the monitoring thread of the established IOCP model monitors a data packet sent by a terminal, the terminal is associated with the pre-established virtual container to form a network channel for data transmission between the terminal and the server, wherein the virtual container is used for storing the data packet sent by the terminal, and the data packet comprises terminal point information of the terminal, content of the data packet and the like.
S102, determining the size of the data packet and the size of an available buffer area in a virtual container corresponding to the target terminal;
specifically, after receiving a data packet sent by a target terminal, the server calculates the size of the data packet, and calculates the size of a practically available buffer area in a virtual container corresponding to the target terminal at the moment.
S103, if the size of the data packet is larger than the size of the available buffer area, calling a dynamic buffer area to expand the available buffer area to obtain an expanded buffer area;
the available buffer area is determined according to the buffer area occupied by the stored data packet and the released buffer area after the data packet is read from the preset buffer area.
Specifically, if the data packet is written into the virtual container for the first time, the available buffer area in the virtual container is the preset buffer area;
when the data packet is written into the virtual container subsequently, the available buffer area is determined according to the buffer area occupied by the data packet written into the expansion buffer area and the released buffer area after the data packet is read from the expansion buffer area.
The server judges the size of the data packet to be cached and the size of an actually available cache area, and if the size of the data packet to be cached is smaller than the size of the actually available cache area, the data packet is directly stored into the actually available cache area in a virtual container corresponding to the target terminal;
if the size of the data packet to be cached is larger than the size of the actually available cache area, calling a dynamic cache area to expand the actually available cache area to obtain an expanded cache area, wherein the dynamic cache area can be a memory area except for a cache area applied in advance for each virtual container or can be a preset cache area corresponding to other unoccupied virtual containers.
S104, storing the data packet into the expansion buffer area.
Illustratively, the server pre-establishes 3 network channels, namely pre-establishes 3 virtual containers, receives data packets sent by the terminal a, the terminal B and the terminal C through a pre-established IOCP model, associates the terminal a with the virtual container a, the terminal B with the virtual container B, and the terminal C with the virtual container C;
the server receives a data packet M and a data packet N sent by a terminal A, the data packet M is stored in a virtual container A, the size of the data packet M and the size of an available buffer area in the virtual container at the moment are calculated (the available buffer area is the preset buffer area at the moment because the data is not stored in the preset buffer area), and if the size of the data packet M is smaller than the size of the available buffer area, the data packet M is directly stored in an available storage area corresponding to the virtual container A;
the method comprises the steps that a data packet N is continuously stored in a virtual container A, the size of the data packet N and the size of an actually available buffer area are calculated, at this time, the actually available buffer area is a preset buffer area minus the space occupied by the data packet M, and if the size of the data packet N is larger than the size of the available buffer area, a dynamic buffer area is called to expand the actually available buffer area, wherein the dynamic buffer area can be a pre-applied buffer area corresponding to a virtual container corresponding to a terminal or a buffer space except for the pre-applied buffer area of the virtual container corresponding to the terminal;
specifically, in the process of data communication between the terminal a and the server, only the preset buffer area corresponding to the virtual container a is occupied, at this time, the preset buffer areas applied for the terminal B and the terminal C are in a dormant or stagnation state in advance, when the data packet N of the terminal a is stored, the size of the data packet N and the size of the available buffer area are calculated, if the size of the data packet N is greater than the size of the available buffer area, the dynamic buffer area is mobilized to expand the available buffer area, specifically, the preset buffer areas applied for other terminals are called to expand the available buffer area, or the memory space except the preset buffer areas applied for the terminal is called to expand the available buffer area, so as to obtain an expanded buffer area; the data packet N is stored in an extended buffer in the virtual container a.
According to the data caching method provided by the embodiment of the invention, the data packet sent by the target terminal is obtained through the pre-established IOCP model; determining the size of the data packet and the size of an available buffer area in a virtual container corresponding to the target terminal; if the size of the data packet is larger than the size of the available buffer area, calling the dynamic buffer area to expand the available buffer area to obtain an expanded buffer area; the data packet is stored in the expansion buffer area in the virtual container corresponding to the target terminal, and the preset memory in the virtual container corresponding to each terminal can be dynamically expanded in real time under the condition of adopting the IOCP model to process large concurrent data by the embodiment of the invention, so that the communication capacity of the IOCP network is improved to the greatest extent.
A further embodiment of the present invention further provides a method according to the above embodiment.
As shown in fig. 2, a step flow diagram of another embodiment of a data caching method of the present invention is shown, the method further comprising:
s201, creating a plurality of virtual containers in advance;
specifically, the server establishes a plurality of virtual containers based on the IOCP model for storing information of each terminal and transmitted data packets.
S202, when a monitoring thread in an IOCP model detects that a target terminal sends a data packet, associating the virtual container with the target terminal to form a network channel of the target terminal and the virtual container so as to enable the target terminal and a server to carry out data transmission;
specifically, the server and the terminal establish socket (socket) connection, and the terminal sends a data packet to the server through the socket, wherein the socket can be included in the internet communication protocol, and can be understood as an interface for interaction between an application layer protocol and a transport layer protocol, and can be used for distinguishing network communication and connection between different application program processes.
When a monitoring thread of an IOCP model pre-established on a server monitors a data packet sent by a terminal, a virtual container is distributed to the terminal, a network channel for data transmission is established, and the terminal performs data transmission with the server through the IOCP model and the virtual container corresponding to the terminal.
As shown in fig. 3, a step flow chart of still another embodiment of the data caching method of the present invention is shown, and step S103 specifically includes:
s1031, determining the size of a buffer area to be expanded according to the difference between the size of the data packet and the size of the available buffer area;
s1032, calling the dynamic buffer area to expand the available buffer area according to the size of the buffer area needing to be expanded to obtain the expanded buffer area, wherein the dynamic buffer area comprises buffer areas except for a preset buffer area which is applied for a virtual container corresponding to each terminal in advance and/or preset buffer areas of virtual containers corresponding to other terminals.
Specifically, the server calculates the size of a buffer area to be expanded according to the difference between the size of a data packet to be buffered and the size of an available buffer area;
as an implementation manner, according to the size of a buffer zone to be expanded, calling a buffer zone outside a preset buffer zone applied for a virtual container of each terminal in advance to expand the available buffer zone corresponding to the target terminal, so as to obtain an expanded buffer zone;
as another implementation manner, according to the size of the buffer area to be expanded, calling a preset buffer area corresponding to other terminals to expand the available buffer area corresponding to the target terminal, so as to obtain an expanded buffer area;
as another implementation manner, according to the size of the buffer area to be expanded, calling the preset buffer areas corresponding to other terminals and the available buffer areas corresponding to the target terminals except the preset buffer areas which are applied for the virtual container of the terminals in advance to expand the capacity, so as to obtain an expanded buffer area;
as shown in fig. 4, a step flow diagram of yet another embodiment of a data caching method of the present invention is shown, the data caching method comprising:
s401, acquiring a data packet sent by a target terminal through a pre-established IOCP model.
S402, determining the size of the data packet and the size of an available buffer area in a virtual container corresponding to the target terminal;
s403, if the size of the data packet is larger than the size of the available buffer area, calling a dynamic buffer area to expand the available buffer area to obtain an expanded buffer area;
since steps S401 to S403 are the same as steps S101 to S103 in the embodiment shown in fig. 1, steps S101 to S103 have been described in detail in fig. 1, and thus, the descriptions of steps S401 to S403 are not repeated here.
S404, after analyzing the data packet sent by the terminal, writing the analyzed data into the expansion buffer, wherein the analyzed data at least comprises the byte length of the data packet, and the byte length of the data packet occupies the expansion buffer with the preset byte length;
specifically, the server stores the data packet obtained from the IOCP model into an extended buffer area in the virtual container corresponding to the target terminal, parses the data packet, and obtains parsed data, where the parsed data is written into the extended buffer area by means of byte length of the data packet and data packet content, specifically, the byte length of the data packet occupies an extended buffer area with a preset byte length, for example, the byte length of the data packet occupies an extended buffer area with a byte length of 4 bytes, and the data packet content may be placed after the byte length of the data packet according to the actual byte length.
As shown in fig. 5, a step flow diagram of still another embodiment of a data caching method of the present invention is shown, the data caching method comprising:
s501, acquiring a data packet sent by a target terminal through a pre-established IOCP model.
S502, determining the size of the data packet and the size of an available buffer area in a virtual container corresponding to the target terminal;
s503, if the size of the data packet is larger than the size of the available buffer area, calling a dynamic buffer area to expand the available buffer area to obtain an expanded buffer area;
s504, after analyzing the data packet sent by the terminal, writing the analyzed data into the expansion buffer, wherein the analyzed data at least comprises the byte length of the data packet, and the byte length of the data packet occupies the expansion buffer with the preset byte length;
since steps S501 to S504 are the same as steps S401 to S404 in the embodiment shown in fig. 4, steps S401 to S404 have been described in detail in fig. 4, and thus, the descriptions of steps S501 to S504 are not repeated here.
S505, reading the data packet from the expansion buffer area;
and releasing the expansion buffer area occupied by the read data packet.
Specifically, as an executable manner, if the remaining byte length in the extended buffer is greater than a preset byte length, the data packet is read from the extended buffer;
as another executable manner, if the remaining byte length in the extended buffer is less than the preset byte length, a new data packet is read from the pre-established IOCP model.
As shown in fig. 6, a step flow diagram of another embodiment of a data caching method of the present invention is shown, the data caching method comprising:
s601, generating a certain required number of network channels, namely, creating a plurality of virtual containers in advance on a server, wherein the network channels are used for data transmission between a terminal and the virtual containers;
when detecting that a terminal sends a data packet to a server through a monitoring thread in an IOCP model, associating a pre-established virtual container with the terminal, and directly adopting a pre-established network channel, so that the time required for reproducing the network channel when the data packet sent by the terminal is received can be avoided, and meanwhile, a relatively small memory space buffer is pre-applied in each virtual container;
s602, selecting an idle network channel A, and receiving a data packet sent by a terminal by a server through an IOCP model;
s603, comparing the size of the data packet to be buffered with the size of an effective buffer area, namely tbuffer, wherein the size of the effective buffer area is determined according to the occupied buffer area and the buffer area read from the buffer area;
if the size Buffer of the received data packet is smaller than the effective Buffer size tbuffer, the Buffer is sufficient, and the received data packet is stored in the Buffer.
S604, if the size of the received data packet Buffer is larger than the effective Buffer size tbuffer, calculating a memory space Buffer-tbuffer which is actually required to be expanded, applying for a new memory space on the basis of the existing available Buffer to obtain an expansion Buffer, and writing the received data packet into the expansion Buffer;
s605, assigning and updating the effective data length in the buffer of the expansion buffer area;
s606, reading the data packet written in the buffer of the expansion buffer area;
s607, specifically, after analyzing the received data packet, writing the data packet into an expansion buffer area in a mode of data packet length and data packet content, wherein the data packet length occupies 4 bytes, and writing the data packet content according to the actual length;
s608, if the length of the remaining effective bytes is greater than 4 bytes when reading data in the available buffer zone tbuffer in the extended buffer zone, the data packet can be continuously read;
s609, if the remaining effective byte length is smaller than 4 bytes, a new data packet needs to be acquired from the IOCP model, and step S603 is executed again.
According to the data caching method provided by the embodiment of the invention, the data packet sent by the target terminal is obtained through the pre-established IOCP model; determining the size of the data packet and the size of an available buffer area in a preset buffer area; if the size of the data packet is larger than the size of the available buffer area, calling the dynamic buffer area to expand the available buffer area to obtain an expanded buffer area; the data packet is stored in the expansion buffer area in the virtual container corresponding to the terminal, and the preset memory in the virtual container corresponding to each terminal can be dynamically expanded in real time under the condition of adopting the IOCP model to process large concurrent data by the embodiment of the invention, so that the communication capacity of the IOCP network is improved to the greatest extent.
Another embodiment of the present invention provides a data caching apparatus, configured to execute the data caching method provided in the foregoing embodiment.
Referring to fig. 7, there is shown a block diagram of an embodiment of a data caching apparatus of the present invention, applied to a server, the apparatus comprising: an acquisition module 701, a determination module 702, a capacity expansion module 703 and a writing module 704, wherein:
the acquiring module 701 is configured to acquire a data packet sent by a target terminal through a pre-established IOCP model;
the determining module 702 is configured to determine a size of the data packet and an available buffer size in a virtual container corresponding to the target terminal;
the capacity expansion module 703 is configured to call a dynamic buffer area to expand the available buffer area if the size of the data packet is larger than the size of the available buffer area, so as to obtain an expanded buffer area;
the writing module 704 is configured to store the data packet in the extended buffer.
The data caching device provided by the embodiment of the invention acquires the data packet sent by the target terminal through the pre-established IOCP model; determining the size of the data packet and the size of an available buffer area in a preset buffer area; if the size of the data packet is larger than the size of the available buffer area, calling the dynamic buffer area to expand the available buffer area to obtain an expanded buffer area; the data packet is stored in the expansion buffer area in the virtual container corresponding to the terminal, and the preset memory in the virtual container corresponding to each terminal can be dynamically expanded in real time under the condition of adopting the IOCP model to process large concurrent data by the embodiment of the invention, so that the communication capacity of the IOCP network is improved to the greatest extent.
A further embodiment of the present invention provides a further supplementary explanation of the apparatus provided in the above embodiment.
Optionally, before the obtaining module, the apparatus further includes an associating module, where the associating module is configured to:
creating a plurality of virtual containers in advance;
when the monitoring thread in the IOCP model detects that the target terminal sends a data packet, the virtual container is associated with the target terminal to form a network channel of the target terminal and the virtual container, so that the target terminal and a server perform data transmission.
Optionally, the capacity expansion module 703 is specifically configured to:
determining the size of a buffer area needing to be expanded according to the difference between the size of the data packet and the size of the available buffer area;
and calling the dynamic buffer to expand the available buffer according to the size of the buffer to be expanded to obtain the expanded buffer, wherein the dynamic buffer comprises buffers except for a preset buffer which is applied for the virtual container corresponding to each terminal in advance and/or preset buffers of virtual containers corresponding to other terminals.
Optionally, the available buffer area is determined according to a buffer area occupied by a data packet written into the extended buffer area and a released buffer area after the data packet has been read from the extended buffer area.
Optionally, the writing module 704 is specifically configured to:
after analyzing the data packet sent by the terminal, writing the analyzed data into the expansion buffer area, wherein the analyzed data at least comprises the byte length of the data packet, and the byte length of the data packet occupies the expansion buffer area with the preset byte length.
Optionally, after the writing module, the apparatus further includes a reading module for:
reading a data packet from the expansion buffer area;
releasing the expansion buffer area occupied by the read data packet;
the reading module is specifically configured to:
and if the length of the remained bytes in the expansion buffer is greater than the preset length of the bytes, reading the data packet from the expansion buffer.
It should be noted that, in this embodiment, each of the possible embodiments may be implemented separately, or may be implemented in any combination without conflict, which is not limited to the implementation of the present application.
For the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points.
The data caching device provided by the embodiment of the invention acquires the data packet sent by the target terminal through a pre-established IOCP model; determining the size of the data packet and the size of an available buffer area in a preset buffer area; if the size of the data packet is larger than the size of the available buffer area, calling the dynamic buffer area to expand the available buffer area to obtain an expanded buffer area; the data packet is stored in the expansion buffer area in the virtual container corresponding to the terminal, and the preset memory in the virtual container corresponding to each terminal can be dynamically expanded in real time under the condition of adopting the IOCP model to process large concurrent data by the embodiment of the invention, so that the communication capacity of the IOCP network is improved to the greatest extent.
An embodiment of the present invention provides a terminal device, configured to execute the data caching method provided in the foregoing embodiment.
Fig. 8 is a schematic structural diagram of a terminal device of the present invention, and as shown in fig. 8, the electronic device includes: at least one processor 801 and memory 802;
the memory stores a computer program; the at least one processor executes the computer program stored in the memory to implement the data caching method provided in the foregoing embodiment.
The terminal equipment provided by the embodiment obtains the data packet sent by the target terminal through a pre-established IOCP model; determining the size of the data packet and the size of an available buffer area in a virtual container corresponding to the target terminal; if the size of the data packet is larger than the size of the available buffer area, calling the dynamic buffer area to expand the available buffer area to obtain an expanded buffer area; the data packet is stored in the expansion buffer area in the virtual container corresponding to the terminal, and the preset memory in the virtual container corresponding to each terminal can be dynamically expanded in real time under the condition of adopting the IOCP model to process large concurrent data by the embodiment of the invention, so that the communication capacity of the IOCP network is improved to the greatest extent.
Yet another embodiment of the present application provides a computer readable storage medium having a computer program stored therein, which when executed implements the data caching method provided in any one of the above embodiments.
According to the computer readable storage medium of the present embodiment, a data packet sent by a target terminal is acquired through a pre-established IOCP model; determining the size of the data packet and the size of an available buffer area in a virtual container corresponding to the target terminal; if the size of the data packet is larger than the size of the available buffer area, calling the dynamic buffer area to expand the available buffer area to obtain an expanded buffer area; the data packet is stored in the expansion buffer area in the virtual container corresponding to the terminal, and the preset memory in the virtual container corresponding to each terminal can be dynamically expanded in real time under the condition of adopting the IOCP model to process large concurrent data by the embodiment of the invention, so that the communication capacity of the IOCP network is improved to the greatest extent.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal device comprising the element.
The foregoing has described in detail a data caching method and a data caching apparatus according to the present invention, and specific examples have been applied to illustrate the principles and embodiments of the present invention, where the foregoing examples are only for aiding in understanding the method and core idea of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.
Claims (10)
1. A data caching method, applied to a server, the method comprising:
acquiring a data packet sent by a target terminal through a pre-established IOCP model;
determining the size of the data packet and the size of an available buffer area in a virtual container corresponding to the target terminal;
if the size of the data packet is larger than the size of the available buffer area, calling a dynamic buffer area to expand the available buffer area to obtain an expanded buffer area;
and storing the data packet into the expansion buffer area.
2. The method of claim 1, wherein prior to the acquiring, by the pre-established IOCP model, the data packet sent by the target terminal, the method further comprises:
creating a plurality of virtual containers in advance;
when the monitoring thread in the IOCP model detects that the target terminal sends a data packet, the virtual container is associated with the target terminal to form a network channel between the target terminal and the virtual container, so that the target terminal and a server perform data transmission.
3. The method of claim 1, wherein if the size of the data packet is larger than the size of the available buffer, calling a dynamic buffer to expand the available buffer to obtain an extended buffer, comprising:
determining the size of a buffer area needing to be expanded according to the difference between the size of the data packet and the size of the available buffer area;
and calling the dynamic buffer to expand the available buffer according to the size of the buffer to be expanded to obtain the expanded buffer, wherein the dynamic buffer comprises buffers except for a preset buffer which is applied for the virtual container corresponding to each terminal in advance and/or preset buffers of virtual containers corresponding to other terminals.
4. The method of claim 1, wherein the available buffers are determined based on buffers occupied by packets written to the extended buffer and buffers released after packets have been read from the extended buffer.
5. The method of claim 1, wherein storing the data packet in the extended buffer comprises:
after analyzing the data packet sent by the terminal, writing the analyzed data into the expansion buffer area, wherein the analyzed data at least comprises the byte length of the data packet, and the byte length of the data packet occupies the expansion buffer area with the preset byte length.
6. The method of claim 5, wherein after storing the data packet in the extended buffer, the method further comprises:
reading a data packet from the expansion buffer area;
releasing the expansion buffer area occupied by the read data packet;
specifically, the reading the data packet from the extended buffer area specifically includes:
and if the length of the remained bytes in the expansion buffer is greater than the preset length of the bytes, reading the data packet from the expansion buffer.
7. A data caching apparatus, applied to a server, the apparatus comprising:
the acquisition module is used for acquiring a data packet sent by the target terminal through a pre-established IOCP model;
the determining module is used for determining the size of the data packet and the size of an available buffer area in the virtual container corresponding to the target terminal;
the capacity expansion module is used for calling the dynamic buffer memory area to expand the available buffer memory area if the size of the data packet is larger than the size of the available buffer memory area, so as to obtain an expanded buffer memory area;
and the writing module is used for storing the data packet into the expansion buffer area.
8. The apparatus of claim 7, wherein prior to the obtaining module, the apparatus further comprises an associating module to:
creating a plurality of virtual containers in advance;
when the monitoring thread in the IOCP model detects that the target terminal sends a data packet, the virtual container is associated with the target terminal to form a network channel of the target terminal and the virtual container, so that the target terminal and a server perform data transmission.
9. A terminal device, comprising: at least one processor and memory;
the memory stores a computer program; the at least one processor executing the computer program stored by the memory to implement the data caching method of any one of claims 1-6.
10. A computer readable storage medium, characterized in that a computer program is stored in the computer readable storage medium, said computer program when executed implementing the data caching method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010081796.3A CN111371704B (en) | 2020-02-06 | 2020-02-06 | Data caching method and device, terminal equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010081796.3A CN111371704B (en) | 2020-02-06 | 2020-02-06 | Data caching method and device, terminal equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111371704A CN111371704A (en) | 2020-07-03 |
| CN111371704B true CN111371704B (en) | 2024-03-15 |
Family
ID=71209998
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010081796.3A Active CN111371704B (en) | 2020-02-06 | 2020-02-06 | Data caching method and device, terminal equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111371704B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112737977B (en) * | 2020-12-28 | 2022-05-20 | 苏州浪潮智能科技有限公司 | Data packet processing method and device |
| CN112887319B (en) * | 2021-02-01 | 2022-07-01 | 上海帆一尚行科技有限公司 | Network state monitoring method and device based on downlink traffic and electronic equipment |
| CN112905697B (en) * | 2021-02-10 | 2024-04-16 | 北京锐服信科技有限公司 | High-speed network data packet capturing method and system |
| CN113778326A (en) * | 2021-02-23 | 2021-12-10 | 北京沃东天骏信息技术有限公司 | Data processing method, apparatus, device, medium, and program product |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5920732A (en) * | 1996-07-01 | 1999-07-06 | Apple Computer, Inc. | System for preallocating additional larger buffer sizes in accordance with packet sizes of discarded packets that can't be stored in existing preallocated buffer sizes |
| KR20040058032A (en) * | 2002-12-26 | 2004-07-03 | 한국과학기술정보연구원 | File transfer management system and method, and storage media having program thereof |
| CN1677946A (en) * | 2004-04-02 | 2005-10-05 | 华为技术有限公司 | Buffer distribution method and apparatus |
| CN107689923A (en) * | 2016-08-04 | 2018-02-13 | 华为技术有限公司 | Message processing method and router |
| CN108834086A (en) * | 2018-06-25 | 2018-11-16 | 平安科技（深圳）有限公司 | Method, apparatus, computer equipment and the storage medium that short message is sent |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9961023B2 (en) * | 2014-05-13 | 2018-05-01 | Red Hat Israel, Ltd. | Adjusting buffer size for network interface controller |
-
2020
- 2020-02-06 CN CN202010081796.3A patent/CN111371704B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5920732A (en) * | 1996-07-01 | 1999-07-06 | Apple Computer, Inc. | System for preallocating additional larger buffer sizes in accordance with packet sizes of discarded packets that can't be stored in existing preallocated buffer sizes |
| KR20040058032A (en) * | 2002-12-26 | 2004-07-03 | 한국과학기술정보연구원 | File transfer management system and method, and storage media having program thereof |
| CN1677946A (en) * | 2004-04-02 | 2005-10-05 | 华为技术有限公司 | Buffer distribution method and apparatus |
| CN107689923A (en) * | 2016-08-04 | 2018-02-13 | 华为技术有限公司 | Message processing method and router |
| CN108834086A (en) * | 2018-06-25 | 2018-11-16 | 平安科技（深圳）有限公司 | Method, apparatus, computer equipment and the storage medium that short message is sent |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111371704A (en) | 2020-07-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111371704B (en) | Data caching method and device, terminal equipment and storage medium | |
| US11146502B2 (en) | Method and apparatus for allocating resource | |
| US10831612B2 (en) | Primary node-standby node data transmission method, control node, and database system | |
| CN105631035B (en) | Date storage method and device | |
| CN104202375A (en) | Method and system for synchronous data | |
| KR20220038827A (en) | Partitioning and rebalancing data storage | |
| CN104756080A (en) | Augmenting capabilities of a host device | |
| CN110020305B (en) | Webpage loading method, webpage loading device, computer equipment and storage medium | |
| CN110708377A (en) | Data transmission method, device and storage medium | |
| CN113806300B (en) | Data storage method, system, device, equipment and storage medium | |
| CN114564457B (en) | Storage space optimization method and system for database files | |
| WO2014194704A1 (en) | A grouping processing method and system | |
| CN104657435A (en) | Storage management method for application data and network management system | |
| CN111796935B (en) | Consumption instance distribution method and system for calling log information | |
| EP4057160A1 (en) | Data reading and writing method and device for database | |
| CN109564502A (en) | Treating method and apparatus applied to the access request in storage equipment | |
| CN107544298A (en) | A kind of camera call method and device | |
| CN111225003B (en) | NFS node configuration method and device | |
| CN112363980B (en) | Data processing method and device of distributed system | |
| CN112069152B (en) | Database cluster upgrading method, device, equipment and storage medium | |
| CN110891033B (en) | Network resource processing method, device, gateway, controller and storage medium | |
| CN104079368B (en) | A kind of the test data transmission method and server of application software | |
| CN110740138A (en) | Data transmission method and device | |
| CN106570068A (en) | Information recommendation method and device | |
| CN105554134A (en) | Information synchronization and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |