CN108874560B - Method and communication device for communication - Google Patents
Method and communication device for communication Download PDFInfo
- Publication number
- CN108874560B CN108874560B CN201810576190.XA CN201810576190A CN108874560B CN 108874560 B CN108874560 B CN 108874560B CN 201810576190 A CN201810576190 A CN 201810576190A CN 108874560 B CN108874560 B CN 108874560B
- Authority
- CN
- China
- Prior art keywords
- message
- data block
- data
- index number
- occupied
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Abstract
The invention provides a communication method and communication equipment, and belongs to the technical field of computers. The method comprises the following steps: when a first message is written into a message queue, determining the number of data blocks in the message queue occupied by the first message according to the data volume of the first message; updating the write offset corresponding to the message queue to be the sum of the write offset stored corresponding to the message queue and the number, wherein the write offset is used for indicating the maximum index number of a data block into which data is written currently; determining that the first message occupies a data block in the message queue according to the updated write offset and the number; and writing the first message into the determined data block. By adopting the invention, the storage space can be saved.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for performing communication.
Background
With the development of computer technology, when inter-process or inter-thread communication is carried out, a message queue mode is often adopted for communication.
In the related art, during inter-process communication, a process that sends a message belongs to a write process, a process that receives the message belongs to a read process, a message queue can be created in a shared memory, and the data volume of data that can be stored in each data block in the message queue is set to be greater than or equal to the maximum data volume of the sent message. The message queue comprises a head, an index area and a data area, wherein the head area is used for storing the total length of the queue and the data volume of data which can be stored in each data block, the index area is used for storing the index number of the data block in the data area, the index number corresponds to the data block one by one, and the data area comprises a plurality of data blocks and is used for storing the data.
The writing process may write a message into the data block, and the reading process may periodically detect whether a new message is written into the data block, and if so, may read the message in the data block. In this way, interprocess communication may be achieved.
In the process of implementing the invention, the inventor finds that the related art has at least the following problems:
because the data volume of the data which can be stored in each data block in the message queue is greater than or equal to the maximum data volume of the sent message, only a small part of the storage space of one data block is occupied for the message with smaller data volume, and thus, under the condition that the data volumes of the messages are different in size, a large amount of storage space is wasted.
Disclosure of Invention
In order to solve the problems of the related art, embodiments of the present invention provide a method and a communication device for performing communication. The technical scheme is as follows:
in a first aspect, a method for performing communication is provided, where the method includes:
when a first message is written into a message queue, determining the number of data blocks in the message queue occupied by the first message according to the data volume of the first message;
updating the write offset corresponding to the message queue to be the sum of the write offset currently stored corresponding to the message queue and the number, wherein the write offset is used for indicating the maximum index number of a data block into which data is currently written;
determining that the first message occupies a data block in the message queue according to the updated write offset and the number;
and writing the first message into the determined data block.
Optionally, the writing the first message into the determined data block includes:
and in the message queue, sequentially writing the first messages into the determined data blocks according to the sequence of the index numbers of the determined data blocks from large to small.
Optionally, the method further includes:
and when the writing of the first data block in the determined data blocks is finished, updating the state identifier corresponding to the index number of the first data block as a finished identifier.
Optionally, the method further includes:
and when the writing of a second data block in the determined data blocks is finished, storing the sequence number corresponding to the index number of the second data block according to the index number of the determined data block.
Optionally, the method further includes:
and creating a message queue, wherein the message queue comprises a header area, a read-write monitoring area, an index area and a data area, the read-write monitoring area is used for storing the write offset and the read offset, the read offset is used for indicating the maximum index number of a data block of which data is read currently, and the data area comprises at least one data block.
In a second aspect, a method of communicating is provided, the method comprising:
acquiring a read offset and a write offset corresponding to a current message queue, wherein the write offset is used for indicating the maximum index number of a data block into which data is written currently, and the read offset is used for indicating the maximum index number of a data block from which data is read currently;
if the read offset is smaller than the write offset, determining that the unread second message occupies the data blocks in the message queue and the data volume of the second message according to the read offset, and updating the read offset to be the sum of the read offset and the determined number of the data blocks;
and reading the second message from the determined data block according to the data volume of the second message.
Optionally, the determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the read offset includes:
determining the index number of the next data block adjacent to the data block corresponding to the read offset according to the read offset;
and determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number.
Optionally, the determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number includes:
acquiring the state identifier corresponding to the index number,
if the state identification is a completion identification, determining that unread second messages occupy data blocks in the message queue and the data volume of the second messages according to the index numbers;
and if the state identifier is not the completion identifier and the state identifier is updated to be the completion identifier within a preset time, determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number.
Optionally, if the state identifier is a completion identifier, determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number includes:
if the state identifier is a completion identifier and the sequence number corresponding to the index number is a preset value, determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number;
if the state identifier is not a completion identifier and the state identifier is updated to a completion identifier within a preset time period, determining that the unread second message occupies the data blocks in the message queue and the data volume of the second message according to the index number, including:
and if the state identifier is not the completion identifier, the sequence number corresponding to the index number is a preset numerical value, and the state identifier is updated to be the completion identifier within a preset time period, determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number.
Optionally, the method further includes:
and if the state identifier is not the completion identifier, the sequence number is the preset numerical value, and the state identifier is not updated to be the completion identifier within the preset time length, the reading of the second message is abandoned.
Optionally, the determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number includes:
acquiring the data volume of the unread second message corresponding to the index number;
determining the number of data blocks in the message queue occupied by the second message according to the data volume of the second message;
and determining that the second message occupies the data block in the message queue according to the index number and the number.
In a third aspect, a communication device is provided, which includes:
the first determining module is used for determining the number of data blocks in a message queue occupied by a first message according to the data volume of the first message when the first message is written into the message queue;
the updating module is used for updating the write offset corresponding to the message queue to be the sum of the write offset currently stored corresponding to the message queue and the number, wherein the write offset is used for indicating the maximum index number of a data block in which data is currently written;
a second determining module, configured to determine, according to the updated write offset and the updated number, that the first message occupies a data block in the message queue;
and the writing module is used for writing the first message into the determined data block.
Optionally, the writing module is configured to:
and in the message queue, sequentially writing the first messages into the determined data blocks according to the sequence of the index numbers of the determined data blocks from large to small.
Optionally, the update module is further configured to:
and when the writing of the first data block in the determined data blocks is finished, updating the state identifier corresponding to the index number of the first data block as a finished identifier.
Optionally, the communication device further includes:
and the storage module is used for storing the sequence number corresponding to the index number of the second data block according to the index number of the determined data block when the writing of the second data block in the determined data block is finished.
Optionally, the communication device further includes:
the device comprises a creating module and a processing module, wherein the creating module is used for creating a message queue, the message queue comprises a head area, a read-write monitoring area, an index area and a data area, the read-write monitoring area is used for storing the write offset and the read offset, the read offset is used for indicating the maximum index number of a data block of which data is read currently, and the data area comprises at least one data block.
In a fourth aspect, a communication device is provided, which includes:
the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a read offset and a write offset corresponding to a current message queue, the write offset is used for indicating the maximum index number of a data block into which data is written currently, and the read offset is used for indicating the maximum index number of a data block from which data is read currently;
a determining module, configured to determine, according to the read offset, that a second message that is not read occupies a data block in the message queue and a data amount of the second message, and update the read offset to a sum of the read offset and a number of the determined data blocks if the read offset is smaller than the write offset;
and the reading module is used for reading the second message from the determined data block according to the data volume of the second message.
Optionally, the determining module is configured to:
determining the index number of the next data block adjacent to the data block corresponding to the read offset according to the read offset;
and determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number.
Optionally, the determining module is configured to:
acquiring the state identifier corresponding to the index number,
if the state identifier is a completion identifier, determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number;
and if the state identifier is not the completion identifier and the state identifier is updated to be the completion identifier within a preset time, determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number.
Optionally, the determining module is configured to:
if the state identification is a completion identification and the serial number corresponding to the index number is a preset numerical value, determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number;
the determining module is configured to:
and if the state identification is not the completion identification, the sequence number corresponding to the index number is a preset numerical value, and the state identification is updated to be the completion identification within a preset time period, determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number.
Optionally, the determining module is further configured to:
and if the state identifier is not the completion identifier, the sequence number is the preset numerical value, and the state identifier is not updated to be the completion identifier within the preset time length, the reading of the second message is abandoned.
Optionally, the determining module is configured to:
acquiring the data volume of the unread second message corresponding to the index number;
determining the number of data blocks in the message queue occupied by the second message according to the data volume of the second message;
and determining that the second message occupies the data block in the message queue according to the index number and the number.
In a fifth aspect, a communication device is provided, which includes a processor and a memory, where at least one instruction is stored, and the instruction is loaded and executed by the processor to implement the method for communicating according to the first aspect and the second aspect.
In a sixth aspect, a computer-readable storage medium is provided, in which at least one instruction is stored, the instruction being loaded and executed by a processor to implement the method for communicating according to the first and second aspects.
In the embodiment of the invention, because one message can occupy a plurality of data blocks when the message is written into the message queue, even if the data volume of the message is larger, the data volume of the data which can be stored in the data blocks does not need to be set larger, so that the storage space can be saved.
Drawings
Fig. 1 is a schematic diagram of a message queue according to an embodiment of the present invention;
fig. 2 is a flow chart illustrating a method for performing communication according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a write message according to an embodiment of the present invention;
fig. 4 is a flowchart illustrating a method for performing communication according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a device for performing communication according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a device for performing communication according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a device for performing communication according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of an apparatus for performing communication according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of a server according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
The embodiment of the invention provides a method for communication, wherein an execution main body of the method is communication equipment, the communication equipment can be a sender and a receiver in a server or a terminal, the terminal can be a mobile phone, a tablet, a computer and the like, the sender can be a writing process, the receiver can be a reading process, or the sender can be a writing thread, and the receiver can be a reading thread.
The terminal may be provided with a processor, a memory, a transceiver, and the like, the processor may be used for processing of the communication process, the memory may be used for data required and generated in the communication process, and the transceiver may be used for receiving and transmitting data.
The server may be provided with a processor, a memory, a transceiver, and the like, wherein the processor may be used for processing the communication process, the memory may be used for data required and generated in the communication process, and the transceiver may be used for receiving and transmitting data.
The embodiment of the present invention takes the execution main body as an example of the server, the sender takes the writing process, and the receiver takes the reading process as an example of the detailed description of the scheme, and other situations are similar to the above and are not described again.
The application scenario of the application is as follows: a plurality of writing processes write messages into the message queue, and a reading process reads messages from the message queue, or a plurality of writing threads write messages into the message queue, and a reading thread reads messages from the message queue. The multiple writing processes and the one reading process can belong to the same application program or different application programs, the multiple writing threads and the one reading thread can belong to the same application program or different application programs, the multiple writing processes belong to the same application program or different application programs, and the multiple writing threads belong to the same application program or different application programs.
First, the process of creating a message queue in the embodiment of the present invention is described:
as shown in fig. 1, before writing a message, a write process may request a host process to create a message queue, or create a message queue by itself, where the message queue includes a header area, a read/write monitoring area, an index area, and a data area, and the message queue is a lock-free message queue.
The header area includes the total length of the message queue (the total amount of data that can be stored), the version number of the message queue, the amount of data that can be stored per data block, and the like.
The read-write monitoring area is used for storing read offset and write offset, the read offset is used for indicating the maximum index number of the data block of which the data is read currently, and the write offset is used for indicating the maximum index number of the data block of which the data is written currently.
The index area is used for storing an index number of each data block and index information corresponding to each index number, the index information includes a state identifier, a total data amount of a stored message or a data amount of a stored message in the data block (for a certain data block, if the certain data block is a first data block occupied by the message, the total data amount of the message is the total data amount of the message, if the certain data block is not the first data block occupied by the message, the data amount of the message written in the data block), a sequence number and the like, and the sequence number is used for indicating a few data blocks (for example, the sequence number is 3, and represents a third data block occupied by the message) in the data block occupied by the message, that is, when the message is read, the few data blocks are read.
The data area includes at least one data block capable of storing the same amount of data.
As shown in fig. 2, the process flow of writing a message in the message queue for each of the plurality of write processes may be as follows:
The first message may be any message sent by the write process to the read process, for example, the first message may be a message used to notify the read process that the read process may perform the next processing, and the like.
In implementation, when a write process writes a first message into a message queue, the data size of the first message may be determined, then the data size of data that can be stored in data blocks in the message queue is obtained (the data size of data that can be stored in each data block in the message queue is equal), then the data size of the first message is divided by the data size of data that can be stored in the data blocks to obtain a numerical value, if the obtained numerical value is an integer, the number of data blocks in the message queue occupied by the first message is the numerical value, and if the obtained numerical value is a non-integer, the number of data blocks in the message queue occupied by the first message is an integer portion of the numerical value plus one. For example, the data amount of the data that can be stored in each data block is 2k, the data amount of the first message is 7k, the obtained value is 3.5, and the number of data blocks in the message queue occupied by the first message is 4.
It should be noted that the data size of the data that can be stored in the data blocks in the message queue can be adjusted according to the service requirement, for example, the data size of the message is relatively large, and the data size of the data that can be stored in each data block can also be larger, and if the data size of the message is relatively small, the data size of the data that can be stored in each data block can also be smaller. In general, the data amount of data which can be stored in each data block is a little bit smaller, and the waste of storage resources can be reduced.
In addition, when the writing process writes the message into the message queue, the version number of the message queue can be acquired, and if the version number of the message queue is not the latest version number, the main process can be requested to update the message queue, or the message queue can be updated by the main process.
Wherein the write offset is used to indicate a maximum index number of a data block to which data has currently been written.
In implementation, after determining the number of data blocks in the message queue occupied by the first message, the write process may obtain the write offset stored in the current corresponding message queue, that is, the maximum index number of the data block into which data has been currently written, and then update the write offset of the message queue to the sum of the write offset and the number stored in the current corresponding message queue using an atomic function.
Thus, the write offset is updated in time, and consecutive data blocks can be preempted preferentially. This is because there are multiple writing processes, all writing messages into the message queue, and if not preempting the data block used by itself, other writing processes may occupy the data block that they want to occupy. For example, a certain writing process needs to occupy three data blocks to write a first message, and as a result, when the first message is written in the first data block, an adjacent second data block is occupied by another process to write its own data, which may cause data discontinuity of the first message.
In implementation, the write process may determine the maximum index number based on the write offset, then subtract the number of the data blocks in the message queue occupied by the first message from the maximum index number, and then add one to obtain the minimum index number of the data blocks in the message queue occupied by the first message, so as to obtain the minimum index number and the maximum index number of the data blocks in the message queue occupied by the first message, and further determine the data blocks in the message queue occupied by the first message according to the minimum index number and the minimum index number.
In an implementation, after determining that the first message occupies a data block in the message queue, the writing process may write the first message into the determined data block.
In addition, after writing the first message into the determined data block, the write process may send a message write notification to the read process, where the message write notification is used to indicate that a new message is written into the data block, and the read process may read the message from the message queue.
Optionally, in order to reduce errors when the read process reads the message, the write process may write the message in the data block in the following manner, and the corresponding processing of step 204 may be as follows:
and in the message queue, sequentially writing the first message into the determined data blocks according to the sequence of the index numbers of the determined data blocks from large to small.
In implementation, after determining that the first message occupies the data block in the message queue, the writing process may write the message in the data block with the largest index number first, and then write the message in the data with the next index number according to the sequence from the largest index number of the determined data block to the smallest index number, and write the first message in the data block according to the sequence. For example, as shown in fig. 3, the first message occupies three data blocks, with index numbers 1, 2, and 3, (1) the message is first written in the data block with index number 3, (2) the message is then written in the data block with index number 2, and (3) the message is finally written in the data block with index number 1.
It should be noted that, if the first message cannot exactly write the whole number of data blocks full, the data block with the largest index number may not be full, and the written data amount is the data amount corresponding to the remainder obtained by dividing the data amount of the first message by the data amount of the data stored in the data block. For example, the data size of the first message is 8k, the data size of the data stored in each data block is 3k, 2 remaining 2k are obtained by dividing 8k by 3k, three data blocks are occupied, the first message of 2k is written in the data block with the largest index number, and the first message of 3k is written in the remaining two data blocks.
In addition, after the first message is written in the data block with the smallest index number, the index number of the data block with the smallest index number is stored in the data volume of the first message correspondingly, and the index number of each data block in other data blocks is stored in the data volume written in the first message in the data block (that is, how much data is stored in the data block), and the other data blocks are the data blocks except the data block with the smallest index number in the determined data blocks.
Optionally, after writing the first message into the determined data block, the status identifier may be further updated to indicate that writing in a certain data is completed, and the corresponding processing may be as follows:
and when the first data block in the determined data blocks is completely written, updating the state identifier corresponding to the index number of the first data block as a completion identifier.
The index information corresponding to the index number of the data block comprises a state identifier, the state identifier is used for indicating whether writing in the data block is completed or not, and the first data block is any data block in the determined data.
In implementation, in the process of writing the first message into the determined data block according to the sequence of the index numbers from large to small, when the first message is written in the first data block, the state identifier in the index information corresponding to the index number of the first data block may be the completion identifier to identify that the message has been written in the first data block. In this way, as long as the status identifier corresponding to the first data block occupied by the first message (that is, the data block with the smallest index number in the determined data blocks) is the completion identifier, the first message is written completely.
Optionally, in order to prevent the read process from reading incomplete messages, after the write process writes a message in a data block, the index number storage sequence number (the sequence number is used to indicate the number of the data block occupied by the message) of the corresponding data block is required to be correspondingly processed as follows:
and when the writing of the second data block in the determined data block is finished, storing the sequence number corresponding to the index number of the second data block according to the index number of the determined data block.
The second data block may be the same data block as the first data block.
In implementation, in the process of writing the first message into the determined data block according to the order of the index numbers from large to small, when the portion of data which should store the first message is written into the second data block, the sequence number corresponding to the index number of the second data block can be stored according to the determined index number of the data block.
Optionally, when the determined data block is one, the processing of storing the sequence number corresponding to the index number of the second data may be as follows:
in the process of writing the first message into the determined data blocks according to the sequence of the index numbers from large to small, when the part of data which should store the first message is written into the second data block, the sequence number in the index information corresponding to the index number of the second data block can be stored as a preset value, and the preset value is used for indicating the data block with the smallest index number in the data blocks occupied by the first message.
Optionally, when at least two determined data blocks are present, the processing of storing the sequence numbers corresponding to the index numbers of the second data may be as follows:
and if the index numbers of the second data blocks are all smaller than the index numbers of other data blocks, storing the sequence numbers corresponding to the index numbers of the second data blocks as preset numerical values. And if the index numbers of the second data blocks are not all smaller than the index numbers of other data blocks, storing the sequence numbers corresponding to the index numbers of the second data blocks according to the index numbers of other data blocks.
The second data block may be the same data block as the first data block.
In implementation, in the process of writing the first message into the determined data block according to the order of the index numbers from large to small, when the portion of data in the second data block, which should store the first message, is written, if the index number of the second data block is the smallest in the index numbers of the determined data blocks, the sequence number in the index information corresponding to the index number of the second data block may be stored as a preset value.
If the index number of the second data block is not the minimum in the determined index numbers of the data blocks, the sequence number corresponding to the index number of the second data block can be stored according to the index numbers of other data blocks except the second data block in the determined data blocks and then according to the arrangement positions of the index numbers of the second data block in the index numbers of other data blocks.
Optionally, in a case that the index numbers of the second data chunks are not all smaller than the index numbers of the other data chunks, the method for storing the sequence numbers corresponding to the index numbers of the second data chunks may be as follows:
the first message occupies three data blocks, if the index numbers of the second data block are not smaller than the index numbers of other data blocks, the sequence number corresponding to the index number of the second data block can be stored as a first preset value (preset by a technician), if the index number of the second data block is smaller than the index numbers of other data blocks, and if the index number of the second data block is larger than two index numbers of the index numbers of other data blocks, the sequence number corresponding to the index number of the second data block can be stored as a second preset value (preset by the technician), the first preset value is used for indicating the second data block occupied by the first message, and the second preset value is used for indicating the third data block occupied by the first message.
In the embodiment of the invention, because one message can occupy a plurality of data blocks when the message is written into the message queue, even if the data volume of the message is larger, the data volume of the data which can be stored in the data blocks does not need to be set larger, so that the storage space can be saved.
Another embodiment of the present invention provides a process for reading a message by a read process, and as shown in fig. 4, a corresponding processing flow may be as follows:
Wherein the write offset is used to indicate a maximum index number of a data block to which data has been currently written, and the read offset is used to indicate a maximum index number of a data block from which data has been currently read.
In implementation, the read process may obtain the read offset and the write offset corresponding to the current message queue from the read-write monitoring area of the message queue according to a preset period (the period is relatively small, for example, 15 milliseconds).
In addition, after receiving the message write notification, the read offset and the write offset corresponding to the current message queue may be acquired.
The second message is any message which is not read in the message queue, and since the reading process only knows that a message can be read and does not know that the message is to be read, a certain message is not specified here, and the second message can be the same as the first message.
In implementation, after the read process acquires the read offset and the write offset, the read offset and the write offset may be determined, and if the read offset is smaller than the write offset (if the read offset is smaller than the write offset, it indicates that the write process writes a message in the message queue), the data block in the message queue and the data amount of the second message occupied by the second message may be determined according to the read offset.
And the read process may update the read offset to be the sum of the read offset and the determined number of data blocks so that the message can be read next time according to the updated read offset.
Optionally, the data block in the message queue and the data amount of the second message occupied by the second message may be determined according to the index number, and the corresponding processing of step 402 may be as follows:
determining the index number of the next data block adjacent to the data block corresponding to the read offset according to the read offset; and determining the data block occupied by the unread second message in the message queue and the data volume of the second message according to the index number.
In implementation, the reading process may determine the index number of the data block corresponding to the read offset, use the index number, and add one to the index number, that is, the index number of the next adjacent data block, that is, the first data block occupied by the second message. And then determining the data block occupied by the second message in the message queue and the data volume of the second message according to the index number. For example, the read offset is the maximum index number of the data block for which data has been currently read, the read offset is 5, and the read offset is incremented by one to obtain an index number of 6.
Since the next data block is the first data block (the data block with the smallest index number) occupied by the second message, the data amount included therein is the data amount of the second message.
Optionally, before reading the message, it is further required to determine whether the message is written into the data block, and the corresponding processing may be as follows:
acquiring a state identifier corresponding to the index number, and if the state identifier is a completion identifier, determining that the unread second message occupies a data block in the message queue and the data volume of the second message according to the index number; and if the state identifier is not the completion identifier and the state identifier is updated to be the completion identifier within the preset time length, determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number.
The completion flag is used to indicate that all the second messages stored in the data block have been written in, and the preset duration may be preset by a technician and stored in the server.
In implementation, the read process may obtain a state identifier corresponding to the index number, if the state identifier is the completion identifier, obtain a data amount of the second message corresponding to the index number, divide the data amount of the second message by the data amount of the data that can be stored in each data block to obtain a value, if the obtained value is an integer, the number of data blocks in the message queue occupied by the second message is the value, and if the obtained value is a non-integer, the number of data blocks in the message queue occupied by the second message is an integer portion plus one of the value.
If the state identifier is not the completion identifier, waiting for a preset time length, updating the state identifier to be the completion identifier within the preset time length, obtaining a numerical value by the reading process by obtaining the data volume of the second message corresponding to the index number and dividing the data volume of the second message by the data volume of the data which can be stored in each data block, wherein if the obtained numerical value is an integer, the number of the data blocks in the message queue occupied by the second message is the numerical value, and if the obtained numerical value is a non-integer, the number of the data blocks in the message queue occupied by the second message is the integer part of the numerical value plus one.
After the number of the data blocks in the message queue occupied by the second message is obtained, the number can be increased and then subtracted from the index number of the next data block to obtain the maximum index number of the data blocks in the message queue occupied by the second message, and then the data blocks in the message queue occupied by the second message can be determined according to the index number of the next data block (namely the minimum index number of the data blocks occupied by the second message) and the maximum index number.
Optionally, when determining the data amount of the second message, it is further required to determine whether the determined next data block is the first data block occupied by the second message, and the corresponding processing may be as follows:
if the state identifier is a completion identifier and the serial number corresponding to the index number is a preset value, determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number; and if the state identification is not the completion identification, the sequence number corresponding to the index number is a preset numerical value, and the state identification is updated to be the completion identification within a preset time length, determining that the unread second message occupies the data blocks in the message queue and the data volume of the second message according to the index number.
The preset value is preset by a technician and is used for indicating a first data block occupied by a certain message, such as 1, 0 and the like.
In implementation, each data block in the message queue corresponds to an index number, each index number also corresponds to a sequence number, and for a certain data block, the sequence number corresponding to the index number of the data block is used to indicate that the data block is the second data block occupied by the second message, for example, the data block with sequence number 1 is the first data block occupied by the second message, and the data block with sequence number 2 is the second data block occupied by the second message.
The reading process can obtain index information corresponding to the index number, obtain a state identifier corresponding to the index number from the index information, obtain a sequence number in the index information if the state identifier is a completion identifier, obtain a data volume of a second message from the index information if the sequence number is a preset value, divide the data volume of the second message by the data volume of data which can be stored in each data block to obtain a value, if the obtained value is an integer, the number of data blocks occupied by the second message in the message queue is the value, and if the obtained value is a non-integer, the number of data blocks occupied by the second message in the message queue is the integer part plus one of the value.
If the status identifier is not a completion identifier, a preset time duration can be waited, if the status identifier is updated to be the completion identifier within the preset time duration, a serial number in the index information can be obtained, if the serial number is a preset numerical value, the reading process can read the data volume of the second message from the index message, then the data volume of the second message is divided by the data volume of the data which can be stored in each data block to obtain a numerical value, if the obtained numerical value is an integer, the number of the data blocks in the message queue occupied by the second message is the numerical value, and if the obtained numerical value is a non-integer, the number of the data blocks in the message queue occupied by the second message is the integer part of the numerical value plus one.
In this way, since the writing process writes the second message into the data blocks in the order of the index numbers from large to small, as long as the writing of the first data block (i.e. the data block with the smallest index number) occupied by the second message is completed, it can be said that the writing of the second message is completed completely, so that the following situations can be reduced: and writing according to the sequence of the index numbers from small to large, and when an error occurs in the writing process, only writing is carried out in the first data block, and the writing fails in other data blocks, so that the reading of the second message fails.
Optionally, the data amount of the second message may be first obtained, and then it is determined that the second message occupies the data block in the message queue, and the corresponding processing may be as follows:
acquiring the data volume of the unread second message corresponding to the index number; determining the number of data blocks in the message queue occupied by the second message according to the data volume of the second message; and determining that the second message occupies the data block in the message queue according to the index number and the number.
In implementation, first, the data amount of the second message in the index information corresponding to the index number may be obtained, and then the data amount of the second message is divided by the data amount of the data that can be stored in each data block to obtain a numerical value, if the obtained numerical value is an integer, the number of the data blocks in the message queue occupied by the second message is the numerical value, and if the obtained numerical value is a non-integer, the number of the data blocks in the message queue occupied by the second message is the integer portion plus one of the numerical value. And then adding the index number to the obtained number, subtracting the maximum index number of the data block occupied by the second message, and then using the index number of the next data block (namely the minimum index number of the data block occupied by the second message) and the maximum index number to determine that the second message occupies the data block in the message queue.
Optionally, if the state identifier is not updated to the completion identifier within the preset time length, the reading process may give up reading the second message, and the corresponding processing may be as follows:
and if the state identifier is not the completion identifier, the sequence number is a preset numerical value, and the state identifier is not updated to be the completion identifier within the preset time length, the reading of the second message is abandoned.
In implementation, the reading process may obtain the index information corresponding to the index number, obtain the state identifier corresponding to the index number from the index information, wait for a preset duration if the state identifier is not the completion identifier, and no longer read the second message if the state identifier is not updated to the completion identifier within the preset duration.
In addition, after the second message is abandoned to be read, the message stored in the next data block corresponding to the updated read offset (explained in step 403) may be read, and the method for reading the message stored in the next data block corresponding to the updated read offset is the same as the aforementioned method, and is not described herein again.
In implementation, after determining that the second message occupies the data block in the message queue and the data amount of the second message, the read process may read the second message from the determined data block according to the data amount of the second message. For example, the second message is 8k, the data amount of the data that can be stored in the data block is 3k, 3 data blocks are occupied, the second message of 3k is written in the first data block, the second message of 3k is written in the second data block, the second message of 2k is written in the third data block, the read process can read all the data from the first data block, all the data from the second data block, and the data of 2k can be read from the third data block.
In addition, the data volume corresponding to the index number of the first data block occupied by the second message is the data volume of the second message, except the first data block, the data volume corresponding to the index number of the other data blocks occupied by the second message is the data volume of the data actually stored in the data block by the second message, for example, the second message is 8k, the data volume of the data which can be stored in the data block is 3k, 3 data blocks are occupied, the second message of 3k is written in the first data block, the second message of 3k is written in the second data block, the second message of 2k is written in the third data block, the data volume corresponding to the index number of the first data block is 8k, the data volume corresponding to the index number of the second data block is 3k, and the data volume corresponding to the index number of the third data block is 2 k. Through the above description, when the second message is read in each data block occupied by the second message, the data amount of the data that can be stored in the data block can be read in the first data block, and in other data blocks except the first data block, the data amount corresponding to the index number of the other data block can be used to read the data of the corresponding data amount in the other data blocks respectively.
In addition, in the present application, after data is read from a data block each time, the data in the data block may be erased, so that the entire queue is circular, and when the index number of the data block with the largest offset is written, a writing process may write a message into the message queue, and the writing process may write a message from the data block with the smallest index number. In this way, the message queue can be reused.
In the embodiment of the invention, because one message can occupy a plurality of data blocks when the message is written into the message queue, even if the data volume of the message is larger, the data volume of the data which can be stored in the data blocks does not need to be set larger, so that the storage space can be saved.
Based on the same technical concept, an embodiment of the present invention further provides a communication device, as shown in fig. 5, the communication device includes:
a first determining module 510, configured to determine, when a first message is written into a message queue, a number of data blocks in the message queue occupied by the first message according to a data amount of the first message;
an updating module 520, configured to update the write offset corresponding to the message queue to a sum of the write offset currently stored corresponding to the message queue and the number, where the write offset is used to indicate a maximum index number of a data block to which data has been currently written;
a second determining module 530, configured to determine, according to the updated write offset and the number, that the first message occupies a data block in the message queue;
a writing module 540, configured to write the first message into the determined data block.
Optionally, the writing module 540 is configured to:
and in the message queue, sequentially writing the first messages into the determined data blocks according to the sequence of the index numbers of the determined data blocks from large to small.
Optionally, the updating module 520 is further configured to:
and when the writing of the first data block in the determined data blocks is completed, updating the state identifier corresponding to the index number of the first data block as a completion identifier.
Optionally, as shown in fig. 6, the communication device further includes:
and a storage module 550, configured to store, when writing of a second data block in the determined data block is completed, a sequence number corresponding to the index number of the second data block according to the index number of the determined data block.
Optionally, as shown in fig. 7, the communication device further includes:
the creating module 560 is configured to create a message queue, where the message queue includes a header area, a read-write monitoring area, an index area, and a data area, the read-write monitoring area is configured to store the write offset and the read offset, the read offset is configured to indicate a maximum index number of a data block from which data has been currently read, and the data area includes at least one data block.
In the embodiment of the invention, one message can occupy a plurality of data blocks when the message is written into the message queue, so that even if the data volume of the message is larger, the data volume of the data which can be stored in the data blocks does not need to be set to be larger, and the storage space can be saved.
Based on the same technical concept, an embodiment of the present invention further provides a communication device, as shown in fig. 8, the communication device includes:
an obtaining module 810, configured to obtain a read offset and a write offset corresponding to a current message queue, where the write offset is used to indicate a maximum index number of a data block into which data has been currently written, and the read offset is used to indicate a maximum index number of a data block from which data has been currently read;
a determining module 820, configured to determine, according to the read offset, that a second message that is not read occupies data blocks in the message queue and the data amount of the second message, and update the read offset to a sum of the read offset and the determined number of data blocks if the read offset is smaller than the write offset;
a reading module 830, configured to read the second message from the determined data block according to the data amount of the second message.
Optionally, the determining module 820 is configured to:
determining the index number of the next data block adjacent to the data block corresponding to the read offset according to the read offset;
and determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number.
Optionally, the determining module 820 is configured to:
acquiring the state identifier corresponding to the index number,
if the state identifier is a completion identifier, determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number;
and if the state identification is not the completion identification and is updated to be the completion identification within a preset time length, determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number.
Optionally, the determining module 820 is configured to:
if the state identification is a completion identification and the serial number corresponding to the index number is a preset numerical value, determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number;
the determining module is configured to:
and if the state identification is not the completion identification, the sequence number corresponding to the index number is a preset numerical value, and the state identification is updated to be the completion identification within a preset time period, determining that the unread second message occupies the data block in the message queue and the data volume of the second message according to the index number.
Optionally, the determining module 820 is further configured to:
and if the state identifier is not the completion identifier, the sequence number is the preset numerical value, and the state identifier is not updated to be the completion identifier within the preset time length, the reading of the second message is abandoned.
Optionally, the determining module 820 is configured to:
acquiring the data volume of the unread second message corresponding to the index number;
determining the number of data blocks in the message queue occupied by the second message according to the data volume of the second message;
and determining that the second message occupies the data block in the message queue according to the index number and the number.
In the embodiment of the invention, because one message can occupy a plurality of data blocks when the message is written into the message queue, even if the data volume of the message is larger, the data volume of the data which can be stored in the data blocks does not need to be set larger, so that the storage space can be saved.
It should be noted that: in the communication device provided in the above embodiment, only the division of the functional modules is illustrated when performing communication, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the communication device is divided into different functional modules to complete all or part of the above described functions. In addition, the communication device and the method for performing communication provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in detail in the method embodiments and are not described herein again.
Fig. 9 is a schematic structural diagram of a server according to an embodiment of the present invention, where the server 900 may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 901 and one or more memories 902, where the memory 902 stores at least one instruction, and the at least one instruction is loaded by the processor 901 and executes the processing of the method for performing communication.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and should not be taken as limiting the scope of the present invention, which is intended to cover any modifications, equivalents, improvements, etc. within the spirit and scope of the present invention.
Claims (6)
1. A method of communicating, the method comprising:
acquiring a read offset and a write offset corresponding to a current message queue, wherein the write offset is used for indicating the maximum index number of a data block into which data has been written currently, and the read offset is used for indicating the maximum index number of a data block from which data has been read currently;
if the read offset is smaller than the write offset, acquiring a state identifier corresponding to the index number of the next data block adjacent to the data block corresponding to the read offset; if the state identifier is a completion identifier, and the sequence number corresponding to the index number of the next data block is a preset value, or if the state identifier is not a completion identifier, and the sequence number corresponding to the index number of the next data block is a preset value, and the state identifier is updated to be a completion identifier within a preset time period, adding one to the read offset to obtain the minimum index number of the data block occupied by the second message to be read, determining the data volume of the second message according to the minimum index number, determining the number of the data blocks occupied by the second message according to the data volume of the second message, obtaining the maximum index number of the data block occupied by the second message according to the minimum index number and the number, and determining the data block occupied by the second message in the message queue according to the minimum index number and the maximum index number, updating the reading offset to be the sum of the reading offset and the determined number of the data blocks;
reading the second message from the determined data block according to the data volume of the second message;
wherein, the sequence number is used to indicate that the corresponding data block is the fourth data block in the data block occupied by the message, the preset value is used to indicate the first data block occupied by the second message, the sequence number stored correspondingly to the minimum index number of the data block occupied by the second message is the preset value, and the minimum index number also correspondingly stores the total data volume of the second message, for other index numbers except the minimum index number in the data block occupied by the second message, the corresponding sequence number is correspondingly stored according to the arrangement sequence of the data block corresponding to the other index numbers and the data block corresponding to the minimum index number, and each index number in the other index numbers also correspondingly stores the data volume written in the data block corresponding to each index number,
and the second message is written into each data block in sequence from large to small according to the index number of the data block required to be occupied in the process of writing into the message queue.
2. The method of claim 1, further comprising:
and if the state identifier is not the completion identifier, the sequence number is the preset numerical value, and the state identifier is not updated to be the completion identifier within the preset time length, the reading of the second message is abandoned.
3. A communication device, characterized in that the communication device comprises:
the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a read offset and a write offset corresponding to a current message queue, the write offset is used for indicating the maximum index number of a data block into which data is written currently, and the read offset is used for indicating the maximum index number of a data block from which data is read currently;
a determining module, configured to obtain a state identifier corresponding to an index number of a next data block adjacent to the data block corresponding to the read offset if the read offset is smaller than the write offset; if the state identification is a completion identification, and the sequence number corresponding to the index number of the next data block is a preset numerical value, or if the state identification is not the completion identification, and the sequence number corresponding to the index number of the next data block is a preset numerical value, and the state identification is updated to be the completion identification within a preset time period, adding one to the read offset to obtain the minimum index number of the data block occupied by the second message to be read, determining the data volume of the second message according to the minimum index number, determining the number of the data blocks occupied by the second message according to the data volume of the second message, obtaining the maximum index number of the data block occupied by the second message according to the minimum index number and the number, determining that the data block in the message queue is occupied by the second message according to the minimum index number and the maximum index number, updating the reading offset to be the sum of the reading offset and the determined number of the data blocks;
a reading module, configured to read the second message from the determined data block according to the data size of the second message;
wherein, the sequence number is used to indicate that the corresponding data block is the fourth data block in the data block occupied by the message, the preset value is used to indicate the first data block occupied by the second message, the sequence number stored correspondingly to the minimum index number of the data block occupied by the second message is the preset value, and the minimum index number also correspondingly stores the total data volume of the second message, for other index numbers except the minimum index number in the data block occupied by the second message, the corresponding sequence number is correspondingly stored according to the arrangement sequence of the data block corresponding to the other index numbers and the data block corresponding to the minimum index number, and each index number in the other index numbers also correspondingly stores the data volume written in the data block corresponding to each index number,
and the second message is written into each data block in sequence from large to small according to the index number of the data block required to be occupied in the process of writing into the message queue.
4. The communications device of claim 3, wherein the determining module is further configured to:
and if the state identifier is not the completion identifier, the sequence number is the preset numerical value, and the state identifier is not updated to be the completion identifier within the preset time length, the reading of the second message is abandoned.
5. A communication device comprising a processor and a memory, the memory having stored therein at least one instruction that is loaded and executed by the processor to implement a method of communicating as claimed in any one of claims 1 to 2.
6. A computer-readable storage medium having stored thereon at least one instruction which is loaded and executed by a processor to implement a method of communicating as claimed in any one of claims 1 to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810576190.XA CN108874560B (en) | 2018-06-05 | 2018-06-05 | Method and communication device for communication |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810576190.XA CN108874560B (en) | 2018-06-05 | 2018-06-05 | Method and communication device for communication |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108874560A CN108874560A (en) | 2018-11-23 |
| CN108874560B true CN108874560B (en) | 2022-09-09 |
Family
ID=64337233
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810576190.XA Active CN108874560B (en) | 2018-06-05 | 2018-06-05 | Method and communication device for communication |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108874560B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111552575B (en) * | 2019-12-31 | 2023-09-12 | 远景智能国际私人投资有限公司 | Message consumption method, device and equipment based on message queue |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6976260B1 (en) * | 1999-09-24 | 2005-12-13 | International Business Machines Corporation | Method and apparatus for serializing a message queue in a multiprocessing environment |
| CN101783740B (en) * | 2009-01-21 | 2012-02-15 | 大唐移动通信设备有限公司 | Method and device for managing message file |
| CN101561773B (en) * | 2009-06-03 | 2011-08-24 | 成都市华为赛门铁克科技有限公司 | Method for recovering disk data and device thereof |
| US9672274B1 (en) * | 2012-06-28 | 2017-06-06 | Amazon Technologies, Inc. | Scalable message aggregation |
| CN102890631B (en) * | 2012-09-13 | 2016-12-07 | 新浪网技术(中国)有限公司 | Method based on persistent message queue transmission message and massage transmission device |
| CN103645942B (en) * | 2013-12-12 | 2017-02-01 | 北京奇安信科技有限公司 | Message queue based write and read method and system of shared memory |
| CN103761194B (en) * | 2013-12-28 | 2017-06-06 | 华为技术有限公司 | A kind of EMS memory management process and device |
| CN104955075B (en) * | 2015-04-27 | 2018-10-26 | 哈尔滨工程大学 | A kind of delay-tolerant network cache management system and management method based on message fragment and node cooperation |
| US9319365B1 (en) * | 2015-10-09 | 2016-04-19 | Machine Zone, Inc. | Systems and methods for storing and transferring message data |
| CN106648933A (en) * | 2016-12-26 | 2017-05-10 | 北京奇虎科技有限公司 | Consuming method and device of message queue |
-
2018
- 2018-06-05 CN CN201810576190.XA patent/CN108874560B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108874560A (en) | 2018-11-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9128615B2 (en) | Storage systems that create snapshot queues | |
| CN104424030B (en) | Method and device for sharing memory by multi-process operation | |
| CN110018914B (en) | Shared memory based message acquisition method and device | |
| CN108845863B (en) | Communication method, device and system for virtual machine and host machine | |
| CN111309732B (en) | Data processing method, device, medium and computing equipment | |
| EP2565786A1 (en) | Information processing device and task switching method | |
| CN111124270A (en) | Method, apparatus and computer program product for cache management | |
| CN107277022B (en) | Process marking method and device | |
| CN115827506A (en) | Data writing method, data reading method, device, processing core and processor | |
| CN113010265A (en) | Pod scheduling method, scheduler, memory plug-in and system | |
| CN113867979A (en) | Data communication method, device, equipment and medium for heterogeneous multi-core processor | |
| CN113836184A (en) | Service persistence method and device | |
| CN115421787A (en) | Instruction execution method, apparatus, device, system, program product, and medium | |
| CN111385255B (en) | Asynchronous call implementation method and device, server and server cluster | |
| CN112395097A (en) | Message processing method, device, equipment and storage medium | |
| CN109117086B (en) | Storage device data position processing method, device, equipment and storage medium | |
| CN108874560B (en) | Method and communication device for communication | |
| CN113467719A (en) | Data writing method and device | |
| CN112433669A (en) | Method, system, equipment and medium for online migration of distributed storage volume | |
| CN112363980A (en) | Data processing method and device for distributed system | |
| CN109034668B (en) | ETL task scheduling method, ETL task scheduling device, computer equipment and storage medium | |
| CN107203339B (en) | Data storage method and device | |
| CN109656936A (en) | Method of data synchronization, device, computer equipment and storage medium | |
| CN115774724A (en) | Concurrent request processing method and device, electronic equipment and storage medium | |
| CN112231290A (en) | Method, device and equipment for processing local log and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230518 Address after: 518000 Room 201, building A, 1 front Bay Road, Shenzhen Qianhai cooperation zone, Shenzhen, Guangdong Patentee after: TENCENT MUSIC ENTERTAINMENT (SHENZHEN) Co.,Ltd. Address before: 518000 Room 201, building A, 1 front Bay Road, Shenzhen Qianhai cooperation zone, Shenzhen, Guangdong Patentee before: TENCENT MUSIC ENTERTAINMENT TECHNOLOGY (SHENZHEN) Co.,Ltd. |
|
| TR01 | Transfer of patent right |