Multithreading data synchronization method
Technical Field
The present application relates to the field of information processing, and in particular, to a method for multi-thread data synchronization.
Background
In an embedded system, if a large amount of data is communicated and the requirement on the real-time performance of the communication is high, when thread synchronization is carried out, the related technology provides a scheme of using a message queue, the scheme provides the message queue, and data needing synchronization is copied to the message queue to be transmitted to other threads.
If the synchronous data volume is large, a large amount of data copy is needed, the length of data which can be accommodated by the message queue is fixed, and a large amount of memory space is wasted for the data synchronization with the indefinite length.
Disclosure of Invention
In order to solve any one of the above technical problems, an embodiment of the present application provides a multithreading data synchronization method.
To achieve the purpose of the embodiments of the present application, an embodiment of the present application provides a method for multithreading data synchronization, including:
selecting a section of idle address space from a preset buffer area, wherein the buffer area is used for storing data of data synchronization operation of the thread and other threads;
storing data waiting for synchronization to the address space;
recording storage position information for storing the data in the address space;
and sending the storage position information to a thread requesting to synchronize the data.
A method of multi-threaded data synchronization, comprising:
acquiring storage position information of data waiting for synchronization in a preset cache region;
and synchronizing the data from the cache region according to the storage position information.
A storage medium having a computer program stored therein, wherein the computer program is arranged to perform the method as described above when executed.
An electronic device comprising a memory having a computer program stored therein and a processor arranged to execute the computer program to perform the method as described above.
One of the above technical solutions has the following advantages or beneficial effects:
the data synchronization is carried out through the storage position for transmitting the data waiting for synchronization, so that the problem of overlarge data volume caused by copying a large amount of data waiting for synchronization in the related technology is solved, and the processing performance is improved.
Additional features and advantages of the embodiments of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the embodiments of the application. The objectives and other advantages of the embodiments of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings are included to provide a further understanding of the embodiments of the present application and are incorporated in and constitute a part of this specification, illustrate embodiments of the present application and together with the examples of the embodiments of the present application do not constitute a limitation of the embodiments of the present application.
Fig. 1 is a flowchart of a multithreading data synchronization method according to an embodiment of the present application;
FIG. 2 is a flow chart of another method for multi-threaded data synchronization according to an embodiment of the present disclosure;
FIG. 3 is a diagram illustrating a method for multi-thread synchronization according to an embodiment of the present disclosure;
fig. 4 is a flowchart of a method for multi-thread synchronization on a data sending side according to an embodiment of the present application;
fig. 5 is a flowchart of a method for multi-thread synchronization on a data receiving side according to an embodiment of the present application;
FIG. 6(a) is a first diagram illustrating a method for multi-thread synchronization according to an embodiment of the present disclosure;
FIG. 6(b) is a second diagram illustrating a method for multi-thread synchronization according to an embodiment of the present application;
FIG. 6(c) is a third schematic diagram of a method for multi-thread synchronization according to an embodiment of the present application;
FIG. 7 is a diagram illustrating an address index table according to an embodiment of the present application;
fig. 8 is a schematic diagram of a cache area according to an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application more apparent, the embodiments of the present application will be described in detail below with reference to the accompanying drawings. It should be noted that, in the embodiments of the present application, features in the embodiments and the examples may be arbitrarily combined with each other without conflict.
Fig. 1 is a flowchart of a multithreading data synchronization method according to an embodiment of the present application. As shown in fig. 1, the method shown in fig. 1 includes:
step 101, selecting a segment of idle address space from a preset buffer area, wherein the buffer area is used for storing data of data synchronization operation of a thread and other threads;
step 102, storing data waiting for synchronization to the address space;
in an exemplary embodiment, when a thread has data waiting for synchronization, the data is stored in the cache region, wherein the address for storing the data is free, so as to avoid the content in the address from being covered.
For example, the addresses of the buffers add0 to add100, and the addresses with the contents not written in the addresses can be selected to perform the write operation of the data waiting for synchronization.
103, recording storage position information for storing the data in the address space;
in one exemplary embodiment, the data storage locations on thread A that are to be synchronized are add0 through add 10.
And 104, sending the storage position information to a thread requesting to synchronize the data.
In one exemplary embodiment, thread B and thread C need to synchronize thread A's data and then send memory location information to thread B and thread C.
Different from the way of sending data waiting for synchronization to the thread B and the thread C in the related art, the method provided by the embodiment of the application only needs to send the data address to other threads, does not need to copy a large amount of data, and improves the processing performance.
The method provided by the embodiment of the application selects a section of free address space from a preset buffer area, stores the data waiting for synchronization into the address space, records the storage position information of the data stored in the address space, sends the storage position information to the thread requesting for data synchronization, and performs data synchronization by sending the storage position of the data waiting for synchronization, so that the problem of overlarge data volume caused by mass copying of the data waiting for synchronization in the related technology is solved, and the processing performance is improved.
In an exemplary embodiment, the storing the data waiting for synchronization to the address space includes:
and selecting the first n addresses of the address space as mark bits of the address, setting the information of the mark bits to be in use, and performing storage operation by taking the (n + 1) th address as a starting address for storing the data, wherein n is an integer.
After the address in the cache region is used for storing the data waiting for synchronization, the address is marked to be in use, all the addresses in use are stored as the address index table, the problem that the content is covered when the subsequent new data is written into the address when the new data is written can be effectively avoided by searching the address index table, and the reasonable use of the address in the cache region is ensured.
In an exemplary embodiment, after the setting the information of the flag bit to be in use, the method further includes:
and when the synchronous operation of the data of the address space is finished, recycling the address space. .
After the synchronous operation of the data of the address space is completed, the address space is released by updating the address index, so that the storage of subsequent new data is facilitated.
In an exemplary embodiment, the detecting whether the synchronization operation of the data of the address space is completed by the following method includes:
acquiring a mark bit corresponding to the address space;
and determining whether the data of the address space completes the synchronous operation of the data of the address space or not according to the information of the marker bit.
If the content of the flag bit of the address space changes from being used to being free, it indicates that the data of the address space has completed the synchronization operation on the data of the address space. Whether the processing of the data synchronization operation is finished or not can be effectively determined through the change of the information of the marking bits, and the management is convenient.
In an exemplary embodiment, after recording the storage location information of the data stored in the buffer, the method includes:
storing the used address space into a preset address index table;
when new data waiting for synchronization is detected, determining used address information in the cache region according to the address index table;
determining an idle address space in the cache region according to the used address information;
and storing the new data by utilizing the free address space.
The used address space is recorded by using the address index table, and when new data waits for synchronization, the available free address space is determined according to the address index table to perform storage operation, so that the problem of data coverage can be avoided.
Fig. 2 is a flowchart of another multithreading data synchronization method according to an embodiment of the present application. As shown in fig. 2, the method shown in fig. 2 includes:
step 201, obtaining storage position information of data waiting for synchronization in a preset cache region;
in one exemplary embodiment, thread B receives memory address spaces addr0 through addr 11.
Step 202, synchronizing the data from the cache region according to the storage location information.
In an exemplary embodiment, thread B reads the content from addr0 through addr11 and completes processing of the content.
In an exemplary embodiment, after the synchronizing the data from the cache region according to the storage location information, the method further includes:
after the data of the storage position information are synchronized, the mark bit of the storage position information is obtained in the cache region;
and setting the information of the flag bit to be free.
After the processing is completed, the value of the flag bit of the addresses addr0 to addr11 is changed from 1 to 0, which indicates that the processing is completed, and thus the completion of the synchronization of the data in the address space is notified.
According to the method provided by the embodiment of the application, the storage position information of the data waiting for synchronization in the preset cache region is obtained, the data are synchronized from the cache region according to the storage position information, the purpose of carrying out data synchronization according to the storage position of the data waiting for synchronization is achieved, the problem that the data size is overlarge due to the fact that a large amount of data waiting for synchronization are copied in the related technology is solved, and the processing performance is improved.
The method provided by the embodiments of the present application is explained as follows:
global variables may be used for data synchronization in the related art. In the using process, two or more threads need to be prevented from accessing the same global variable at the same time so as not to damage the integrity of data. Therefore, functions with lock properties such as critical sections, mutexes, semaphores and the like need to be used to ensure that the same variable can only be accessed by one thread at one time.
The inventors have found that the use of lock functionality by multiple threads results in thread execution time uncertainty, reducing the real-time performance of the system. Therefore, the embodiment of the application provides a mode of not using a lock to ensure the access of a single thread.
The method provided by the embodiment of the application can be used for synchronizing the data among the threads on the basis of not using locks and copying a large amount of data, so that the real-time performance of the system is ensured, and the performance of the system is improved. Wherein the method comprises:
1) each thread initializes a buffer to send data.
2) And acquiring a free address space from the sending buffer, storing data required to be sent by the thread, setting the marking bit of the address as in use, and storing the address into a used address index table.
3) And sending the address to other threads through the message queue. After the other threads process the data, the address is marked as idle.
4) And when the next thread needs to send data, inquiring the mark of the sending buffer area, acquiring the idle address space and finishing the next sending.
Fig. 3 is a schematic diagram of a method for multi-thread synchronization according to an embodiment of the present application. As shown in FIG. 3, Task A performs a data synchronization process with Task B and Task C. The Task A sends the data address in the sending buffer to the tasks B and C. And the Task B and the Task C process the data, and after the data is processed, the address is marked to be free. And the Task A recovers the idle address and carries out the next data transmission.
The method provided by the embodiment of the application avoids the processing of the same group of data by a plurality of threads at the same time. Only when the sending thread has finished sending, the receiving thread starts processing. The sending thread can process and send the next group of data without waiting for the receiving thread to finish processing.
The method provided by the embodiments of the present application is explained as follows:
fig. 4 is a flowchart of a method for multithread synchronization on a data sending side according to an embodiment of the present application. As shown in fig. 4, the method includes:
step 401, initializing a sending buffer area, determining the size of the buffer area, and receiving the maximum length of data;
step 402, receiving data;
step 403, acquiring an idle data address from the sending buffer;
step 404, processing the data, and storing the data result in a sending buffer area;
step 405, sending the buffer address to the target task through the message queue;
step 406, judging whether the transmission is successful;
if successful, go to step 407; if not, step 408 is performed
Step 407, returning, starting the next processing, and executing step 402;
step 408, rolling back the update of the transmission buffer at this time.
Fig. 5 is a flowchart of a method for multithread synchronization on a data receiving side according to an embodiment of the present application. As shown in fig. 5, the method includes:
step 501, receiving data;
step 502, the data is processed and marked as used.
According to the method provided by the embodiment of the application, data integrity is guaranteed without using a lock, only one thread is used for processing the same group of data at the same time, thread blocking is avoided, and the real-time performance of the system is improved. In addition, only the data address needs to be sent to other threads through the message queue, and a large amount of data copying is not needed, so that the performance is improved.
The method provided by the embodiment of the application is explained by an application scene:
1. when the sending thread sends data, the free address addr is firstly obtained from the sending buffer area, and 4 bytes from the beginning of addr are used as a marking bit for judging whether the address is used or not and are marked as FLAG. Addr +4 is taken as the start address for storing actual data.
As shown in fig. 6 (a):
assume that at some point, thread a has three sets of data to synchronize:
data 0: addr0, addr1), addr0 through addr0+4 as address flag bits.
Data 1: addr1, addr2), addr1 through addr1+4 as address flag bits.
Data2 [ addr2, addr3), addr2 through addr2+4 as address flag bits.
The sending thread marks FLAG of the address in use as 1, updates the write pointer to the idle address addr3, and stores the address in use in the address index table. The current free address space is the write pointer to the end of the buffer.
The sending thread a sends the address to the other threads. For example, addr0 is sent to thread B, addr1 is sent to thread C, and addr2 is sent to thread B.
2. When receiving the data, the receiving thread processes the data first, and marks the FLAG corresponding to the address as 0 after the data is completed, which indicates that the use is finished. As shown in step two of fig. 6 (a). Thread B has finished processing Data0 and Data2 and has the address tag position 0, indicating that the Data has been processed. Thread C is processing Data 1.
Whether the data synchronization is completed can be determined by the flag bit of the address space.
The following explains the implementation method of the application scenario:
1. thread a opens up a segment of address space as a buffer for data synchronization.
2. At time t1, thread a has 3 sets of Data to be synchronized, Data0 (address segments addr0 to addr1) is Data that thread B needs to process, Data1 (address segments addr1 to addr2) is Data that thread C needs to process, Data2 (address segments addr2 to addr3) is Data that thread B needs to process, and the write pointer of the buffer points to addr 3. As in step one of fig. 6 (a).
3. At time t2, thread a sets the flag bits of addr0, addr1, and addr2 to 1, respectively, to indicate that the address is in use, stores the address information in the address index table, transmits addr0 and addr2 to thread B, and transmits address information addr1 to thread C. And after receiving the address information, the thread B and the thread C carry out processing according to the address information, and mark the position of the address information to be 0 after the processing is finished, so that the processing is finished. As in step two of fig. 6 (a).
4. At time t3, when thread A detects new Data3, it first searches the address index table, finds out the corresponding address mark bit according to the address in the index table, and updates the address index table according to the mark bit. Assuming thread B has already processed Data0 and Data2, the address index table deletes addr0 and addr2, leaving only addr 1. According to the address index table, the thread a can obtain the free space in the current buffer area and divide the space into two parts, wherein the first part is an address field: writing a pointer to the tail of the buffer; the second is the buffer header to addr 1. Whether the Data can be put into the remaining space is judged according to the Data length of the Data 3. If so, thread A deposits Data3 to address addr3, which ensures that the Data of Data3 does not overwrite addr 1. As in step three of fig. 6 (b).
The address in use is recorded through the address index table, and when new data is written, the current residual address space needs to be counted, so that the address coverage is avoided. Such as Data3 here, cannot cover the previous addr 1.
5. When the tail address space is too small, the write pointer needs to be pointed to the buffer head. As shown in fig. 6(c), when there is less remaining space from addr4 to the end of the buffer, only the address in use needs to be recorded, and by looking for the oldest used address, the write pointer will return to the head of the buffer, and the remaining address space can be found as the write pointer to addr 4. In this way, cyclic reading and writing of one address space are realized.
By the mode, the reasonable use of the storage space can be effectively ensured, and the data coverage is avoided. Storing an address index table, and finding the residual address space only by searching the earliest unread address space; at the same time, the same address space is read by only one thread, and the same group of data is controlled to be processed by only one thread at the same time.
The following describes a process of obtaining a free address from an address index table:
fig. 7 is a schematic diagram of an address index table according to an embodiment of the present application. As shown in fig. 7, the Next pointer points to the location that is Next filled with the address that has been used. The index table is filled from low order to high order. So it is only necessary to look up all the way from the position of the Next pointer to find a value whose address is not null, such as addr7 in fig. 7. Then according to the location of addr7, it can be determined that the current free address is the write pointer of the buffer to addr 7.
There are two cases that can be classified according to the location of addr7 in the buffer:
when the address of addr7 is greater than the write pointer address, as shown by a in FIG. 8, then the free address space is: write the pointer to addr 7. The newly added data need only not cover addr 7.
When the addr7 address is less than the write pointer address, as shown by b in FIG. 8, then the spare address space is write pointer to the end of the buffer and the buffer head to addr 7.
As can be seen from the above, the free address can be selected as a storage location of the data waiting for synchronization.
A storage medium having a computer program stored therein, wherein the computer program is arranged to perform the method of any of the above when executed.
An electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the method of any of the above.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.