CN113031908B - Ordered data processing method and computing device - Google Patents
Ordered data processing method and computing device Download PDFInfo
- Publication number
- CN113031908B CN113031908B CN202110320364.8A CN202110320364A CN113031908B CN 113031908 B CN113031908 B CN 113031908B CN 202110320364 A CN202110320364 A CN 202110320364A CN 113031908 B CN113031908 B CN 113031908B
- Authority
- CN
- China
- Prior art keywords
- data
- processing
- index
- unit
- processed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title description 5
- 238000000034 method Methods 0.000 claims abstract description 83
- 230000008569 process Effects 0.000 claims abstract description 32
- 238000004148 unit process Methods 0.000 claims description 3
- 238000004891 communication Methods 0.000 description 16
- 230000007246 mechanism Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 102100029469 WD repeat and HMG-box DNA-binding protein 1 Human genes 0.000 description 3
- 101710097421 WD repeat and HMG-box DNA-binding protein 1 Proteins 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 230000002093 peripheral effect Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000007723 transport mechanism Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
- G06F7/06—Arrangements for sorting, selecting, merging, or comparing data on individual record carriers
- G06F7/08—Sorting, i.e. grouping record carriers in numerical or other ordered sequence according to the classification of at least some of the information they carry
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for processing ordered data and a computing device, wherein an acquisition unit is utilized to read a plurality of data pointers of data to be processed from a cache unit at a time, then the quantity of the data processed by each processing unit is determined as a first numerical value, the acquired data pointers are divided into a plurality of data pointer segments by taking the first numerical value as a unit, then each processing unit is allocated with a data pointer segment, the data corresponding to the data pointer segments allocated to each processing unit are accessed and processed simultaneously by the plurality of processing units, a processing result is obtained, and finally the processing result is stored in the data pointers accessed by each processing unit. The method improves the processing efficiency of the ordered data, ensures the ordering of the processing result under the condition of no locking processing, and ensures that the processing process of the ordered data is simple and convenient.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method and a computing device for processing ordered data.
Background
Data processing is the collection, storage, retrieval, processing, transformation and transmission of data. The ordered data is data with a certain arrangement sequence, and it is important to ensure the order of output data (processing result) after the ordered data is processed in the process of processing the ordered data.
At present, the data processing method for ordered data is as follows: the ordered data is generated through a single producer thread, the ordered data is written into a cache queue, part of the ordered data is locked through a locking mechanism, and only one consumer thread holds the lock, so that the locked part of the ordered data can be processed through only one consumer thread holding the lock, and a processing result is obtained. The method can ensure the order of the processing result, but the efficiency is low because the ordered data can be processed by only one consumer thread at a time, and the locking mechanism is complex, so the process of processing the ordered data by the locking mechanism is complex and the efficiency is low.
For this reason, a new sequential data processing method is needed to solve the problems existing in the above technical solutions.
Disclosure of Invention
To this end, the present invention provides a method and computing device to solve or at least alleviate the above-identified problems.
According to an aspect of the present invention, there is provided a method of processing ordered data, adapted to be executed in a computing device, the computing device having previously created and loaded a cache unit for storing data to be processed, the data to be processed being ordered data, the ordered data being data having an arrangement order, the method comprising:
reading a plurality of data pointers of the data to be processed from the cache unit by using the acquisition unit at a time;
determining the quantity of data processed by each processing unit as a first numerical value;
dividing the acquired data pointer into a plurality of data pointer segments by taking the first numerical value as a unit;
allocating one data pointer segment to each processing unit, wherein the data pointer segments allocated to the processing units are not repeated;
accessing and processing data corresponding to the data pointer segments allocated to the processing units through a plurality of processing units at the same time to obtain a processing result;
and storing the processing result into the accessed data pointers of the processing units.
Optionally, the method further comprises the steps of:
storing the acquired data pointer into an array;
wherein the step of dividing the acquired data pointer into a plurality of data pointer segments in units of the first value comprises:
dividing the index of the array into a plurality of index segments by taking the first numerical value as a unit;
wherein said step of assigning each of said processing units with one of said data pointer segments comprises:
and allocating one index segment to each processing unit, wherein the index segments allocated to the processing units are not repeated.
Optionally, the step of determining the amount of data to be processed by each processing unit as the first value includes:
and determining the quantity of data to be processed of each processing unit according to the acquired data pointer quantity and the processing unit quantity in an equal division mode to serve as a first numerical value.
Optionally, the step of accessing, by a plurality of processing units, data corresponding to the data pointer segments allocated to each processing unit simultaneously, and processing includes:
accessing and processing data corresponding to the data pointers in the index segments allocated to the processing units through a plurality of processing units at the same time;
the step of accessing and processing the data corresponding to the data pointer in the allocated index segment by each processing unit and obtaining a processing result comprises the following steps:
determining a start index and an end index of the allocated index segment in the array;
accessing a data pointer corresponding to the start index to the end index;
and reading data from the accessed data pointer, and processing to obtain a data processing result.
Optionally, the method further comprises the steps of:
binding a data identifier for the data to be processed in the cache unit, wherein whether the data to be processed is processed or not is distinguished through the data identifier;
judging whether the processing unit processes the data corresponding to the data pointer in the allocated index section according to the data identifier of the data to be processed, if not, not changing the data identifier corresponding to the data pointer, and if so, judging whether the processing unit successfully processes the data corresponding to the data pointer in the allocated index section.
Optionally, the step of determining whether the processing unit successfully processes the data corresponding to the data pointer in the allocated index segment includes:
and judging whether the processing unit successfully processes the data corresponding to the data pointers of the allocated index segments, if so, setting the data identifiers of the data corresponding to the data pointers in the allocated index segments, and if not, setting the data identifiers of the data corresponding to the data pointers in the allocated index segments to be a preset value, wherein the preset value is different from the numerical value corresponding to the data identifier setting and the data identifier resetting.
Optionally, the method further comprises the steps of:
and judging whether all the processing units process the data corresponding to the data pointers in the assigned index section, if so, acquiring the data pointers of a plurality of data to be processed from the cache unit through the acquisition unit, and if not, continuously judging whether all the processing units process the data corresponding to the data pointers in the assigned index section.
Optionally, the processing unit is not destroyed after processing the data.
According to another aspect of the present invention, there is provided a computing device comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the method as described above.
According to yet another aspect of the present invention, there is provided a readable storage medium storing program instructions which, when read and executed by a computing device, cause the computing device to perform the method as described above.
According to the technical scheme of the invention, the method for processing the ordered data is provided, the method distributes the data pointer of the ordered data to be processed for each consumption thread, and a plurality of consumption threads can process the ordered data in the data pointer at the same time and store the data result to the corresponding data pointer. And the data is directly stored to the corresponding data pointer after being processed, which is equivalent to the change of the content of the data pointer but the data pointer is not changed, and the data pointer is ordered, so that the ordering of the processing result is ensured under the condition of no locking processing, and the processing process of the ordered data is simple and convenient.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.
Drawings
To the accomplishment of the foregoing and related ends, certain illustrative aspects are described herein in connection with the following description and the annexed drawings, which set forth the various ways in which the principles disclosed herein may be practiced, and all aspects and equivalents thereof are intended to fall within the scope of the claimed subject matter. The above, as well as additional objects, features, and advantages of the present disclosure will become more apparent from the following detailed description when read in conjunction with the accompanying drawings. Like reference numerals generally refer to like parts or elements throughout the present disclosure.
FIG. 1 shows a schematic diagram of a computing device 100 according to one embodiment of the invention;
FIG. 2 illustrates a flow chart of a method 200 of processing ordered data according to one embodiment of the invention; and
fig. 3 shows a schematic diagram of the connection of a producer unit, a cache unit, an acquisition unit and a processing unit according to an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Ordered data, i.e., data having a certain order of arrangement. In the prior art, the data processing method for ordered data comprises the following steps: generating ordered data through a single producer thread, writing the ordered data into a cache queue, locking part of the ordered data through a locking mechanism, and in the locking mechanism, only one consumer thread holding the lock can process the locked part of the ordered data, and obtain a processing result, then locking other parts of the ordered data, and processing the locked part of the ordered data through the one consumer thread holding the lock, so as to obtain a processing result, and repeating the process until all the ordered data of the cache queue are processed. The method can ensure the ordering of the processing result because the locked partial ordered data is processed by one consumer thread at a time, but has low efficiency because the ordered data can only be processed by one consumer thread at a time, and the locking mechanism is complex, so the process of processing the ordered data by the locking mechanism is complex and has low efficiency.
To this end, the present invention proposes a method of processing ordered data, the method being performed in a computing device 100, fig. 1 showing a block diagram of the computing device 100 according to one embodiment of the present invention. A block diagram of a computing device 100 is shown in fig. 1. In a basic configuration 102, the computing device 100 typically includes a system memory 106 and one or more processors 104. The memory bus 108 may be used for communication between the processor 104 and the system memory 106.
Depending on the desired configuration, the processor 104 may be any type of processing including, but not limited to: a microprocessor (μp), a microcontroller (μc), a digital information processor (DSP), or any combination thereof. The processor 104 may include one or more levels of caches, such as a first level cache 110 and a second level cache 112, a processor core 114, and registers 116. The example processor core 114 may include an Arithmetic Logic Unit (ALU), a Floating Point Unit (FPU), a digital signal processing core (DSP core), or any combination thereof. The example memory controller 118 may be used with the processor 104, or in some implementations, the memory controller 118 may be an internal part of the processor 104.
Depending on the desired configuration, system memory 106 may be any type of memory including, but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. The system memory 106 may include an operating system 120, one or more applications 122, and program data 124. In some implementations, the application 122 may be arranged to operate on an operating system with program data 124. Program data 124 includes instructions, in computing device 100 according to the present invention, program data 124 contains instructions for performing program debug method 200.
The network communication link may be one example of a communication medium. Communication media may typically be embodied by computer readable instructions, data structures, program modules, and may include any information delivery media in a modulated data signal, such as a carrier wave or other transport mechanism. A "modulated data signal" may be a signal that has one or more of its data set or changed in such a manner as to encode information in the signal. By way of non-limiting example, communication media may include wired media such as a wired network or special purpose network, and wireless media such as acoustic, radio Frequency (RF), microwave, infrared (IR) or other wireless media. The term computer readable media as used herein may include both storage media and communication media.
Fig. 2 shows a schematic flow chart of a method 200 of processing ordered data according to one embodiment of the invention. The method 200 is performed in the computing device 100. Before executing the method 200 for processing ordered data, the computing device 100 creates and loads a producer unit, a cache unit, an acquisition unit, and a processing unit in advance, as shown in fig. 3, fig. 3 is a schematic diagram of connection relationships among the producer unit, the cache unit, the acquisition unit, and the processing unit, and fig. 3 illustrates one producer unit, one cache unit, one acquisition unit, and a plurality of processing units, where the cache unit is respectively communicatively connected to the producer unit and the acquisition unit, and the acquisition unit is also communicatively connected to the plurality of processing units.
The computing device 100 generates ordered data through the producer unit and assigns a status identifier to the ordered data, and distinguishes whether the ordered data to be processed is processed and whether the ordered data is successfully processed according to the status identifier, the status identifier may be set according to actual situations, in one embodiment of the present invention, the status identifier includes 0, 1 and-1, the status identifier is 0 indicating that the data is not processed, the status identifier is 1 indicating that the data is processed, the data identifier is-1 indicating that the data processing is wrong, wherein when the status identifier is 1, it indicates that the data corresponding to the data identifier (1) does not need to be processed any more, when the data identifiers are 0 and-1, it indicates that the data corresponding to the data identifiers (0 and-1) need to be processed again, when the ordered data is generated by the producer unit, the computing device 100 assigns the data identifier to the ordered data as 0, and when the ordered data is processed, the data identifier is changed according to successful or failed state of the data processing.
The cache unit is used for storing the ordered data generated by the producer unit, the acquisition unit is used for acquiring the ordered data from the cache unit, and the plurality of processing units are used for processing the effective data acquired by the acquisition unit, wherein the sum of the number of the acquisition unit and the number of the processing units does not exceed the number of kernels of the computing equipment.
The producer unit is a producer, the producer may be a class, a function, a thread, a process, etc., the cache unit is a cache queue, the computing device 100 generates ordered data through the producer and writes the ordered data into the queue head of the cache queue, the one obtaining unit and the plurality of processing units are consumer threads, the computing device 100 creates the plurality of consumer threads in advance, sets the plurality of consumer threads to a dormant state, and activates one of the created plurality of consumer threads as the obtaining unit when the ordered data needs to be processed.
As shown in fig. 2, the method starts at step S210. In step S210, the data pointers of a plurality of data to be processed are read from the buffer unit at a time by the acquisition unit. Specifically, the acquiring unit polls the ordered data to be processed in the cache unit, and when the ordered data to be processed exists, acquires a plurality of data pointers of the ordered data to be processed from the cache unit at a time, wherein the data pointers are memory addresses of the data. The number of data pointers for reading the data to be processed from the buffer unit is set according to the actual scenario, and the present invention is not limited thereto, for example, the obtaining unit obtains 20 data pointers of ordered data from the buffer unit at a time. Since the data pointer is acquired in the present invention, not the data itself, each data in the buffer queue is lengthened.
And storing the acquired data pointers into the array, wherein each data pointer corresponds to one index of the array. And then in step S220, the amount of data processed by each processing unit is determined as a first value. Specifically, the number of processing units processing ordered data is determined based on the number of data pointers acquired and the number of processing units. In one embodiment of the present invention, the number of available data processed by each processing unit is obtained by dividing the number of acquired data pointers by the number of processing units (i.e., in equal divisions). For example, if the number of processing units is 4 and the number of pointers of acquired data is 20, the number of valid data to be processed by each processing unit is 20 divided by 4 and is equal to 5, i.e. the number of valid data to be processed by each processing unit is 5.
Next, in step S230, the acquired data pointer is divided into a plurality of data pointer segments by taking the first value as a unit. Because the data pointers are stored in the array, and each data pointer corresponds to one index of the array, dividing the acquired data pointers into a plurality of data pointer segments is equivalent to dividing the index of the array into a plurality of index segments. For example, the number of processing units is 4 (including processing unit 1, processing unit 2, processing unit 3 and processing unit 4 respectively), the data amount of the effective data to be processed by each unit to be processed is 5, the number of pointers of the acquired data is 20, and thus the index of the corresponding array is 0-19, and then the index of the array is divided into 4 index segments by taking the first value as a unit (namely by taking 5 as a unit), including: 0-4, 5-9, 10-14 and 15-19.
Then in step S240, a data pointer segment is allocated to each processing unit, that is, an index segment is allocated to each processing unit, which is essentially allocating a portion of the ordered data to be processed to each processing unit. Likewise, for example, the number of processing units is 4 (including the processing unit 1, the processing unit 2, the processing unit 3 and the processing unit 4 respectively), the data amount of the ordered data to be processed by each unit to be processed is 5, the number of acquired data pointers is 20, and the array index corresponding to the data pointers is 0-19, then the index of the array is divided into 4 index segments by taking the first numerical value as a unit (namely, taking 5 as a unit), and the method includes: 0-4, 5-9, 10-14, 15-19, then the index segment allocated for processing unit 1 is 0-4, the index segment allocated for processing unit 2 is 5-9, the index segment allocated for processing unit 3 is 10-14, and the index segment allocated for processing unit 4 is 15-19. Here, it should be noted that, the index segment allocated to each processing unit is not repeated, so that each processing unit is prevented from repeatedly processing the ordered data, thereby improving the processing efficiency of the ordered data.
In one embodiment of the present invention, whether the processing unit processes the ordered data corresponding to the data pointer in the allocated index segment is further determined according to the data identifier of the data to be processed, if not, the data identifier is kept to be 0 (the data is not processed) without changing the data identifier, if yes, in step S250, the plurality of processing units access and process the ordered data corresponding to the data pointer segment allocated to each processing unit at the same time, so as to obtain a processing result, i.e. the plurality of processing units access and process the ordered data corresponding to the data pointer in the index segment allocated to each processing unit at the same time, so as to obtain a processing result. As the plurality of processing units access and process the ordered data corresponding to the data pointers in the index segments allocated to the processing units at the same time, the multi-thread simultaneous processing of the ordered data is realized, and the processing efficiency of the ordered data is improved.
Specifically, the step of accessing and processing the ordered data corresponding to the data pointer in the allocated index segment by each processing unit to obtain a processing result includes: determining the starting index and the ending index of the allocated index segments in the array (for example, the starting indexes and the ending indexes corresponding to the index segments 0-4, 5-9, 10-14 and 15-19 are respectively 0 and 4, 5 and 9, 10 and 14, 15 and 19), accessing the data pointers corresponding to the starting index and the ending index, reading ordered data from the accessed data pointers, and processing to obtain a data processing result.
After the plurality of processing units process the ordered data corresponding to the data pointers in the allocated index segment at the same time, continuing to judge whether the processing units process the ordered data corresponding to the data pointers in the allocated index segment successfully, if so, setting the data identification of the data corresponding to the data pointers in the allocated index segment (namely, setting the data identification of the data corresponding to the data pointers to be 1 to indicate that the data is processed), and if not, setting the data identification of the data corresponding to the data pointers in the allocated index segment to be a preset value (for example, setting the data identification of the data corresponding to the data pointers to be-1 to indicate that the data processing is wrong).
In addition, the processing unit is not destroyed after processing the data. And continuously judging whether all processing units process the data corresponding to the data pointers in the allocated index segments, if so, selecting one consumer thread from the consumer threads as an acquisition unit, acquiring a plurality of ordered data to be processed from a cache unit through the acquisition unit, and processing the acquired ordered data based on the rest of the plurality of consumer threads as a plurality of processing units, thereby avoiding resource consumption caused by creating or destroying the consumer threads, saving time required for creating or destroying the consumer threads, and further improving the processing efficiency of the ordered data. If not, continuing to judge whether all the processing units process the data corresponding to the data pointers in the appointed index section.
Finally, in step S260, the data processing result is saved to the accessed data pointers of each processing unit, so that it is ensured that the processing result of the ordered data is still stored in the original data pointer, and because the original data pointers are stored according to a certain sequence, the processing result is also stored according to a certain sequence, thereby ensuring the order of the processing result of the ordered data, and the above process can realize the processing of the ordered data without a locking mechanism, so that the processing process of the ordered data is simple and convenient.
As can be seen from the above, the method for processing ordered data provided by the present invention allocates the data pointer of the ordered data to be processed for each consuming thread, and the multiple consuming threads can process the data in the data pointer at the same time and store the data result to the corresponding data pointer. And the processing result is directly stored to the corresponding data pointer, which is equivalent to the change of the content of the data pointer but the unchanged data pointer, and the data pointer is ordered, so that the ordering of the processing result is ensured under the condition of no locking processing, and the processing process of ordered data is simple and convenient.
The various techniques described herein may be implemented in connection with hardware or software or, alternatively, with a combination of both. Thus, the methods and apparatus of the present invention, or certain aspects or portions of the methods and apparatus of the present invention, may take the form of program code (i.e., instructions) embodied in tangible media, such as removable hard drives, U-drives, floppy diskettes, CD-ROMs, or any other machine-readable storage medium, wherein, when the program is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention.
In the case of program code execution on programmable computers, the mobile terminal will generally include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. Wherein the memory is configured to store program code; the processor is configured to execute the method of processing ordered data of the invention according to instructions in said program code stored in the memory.
By way of example, and not limitation, readable media comprise readable storage media and communication media. The readable storage medium stores information such as computer readable instructions, data structures, program modules, or other data. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. Combinations of any of the above are also included within the scope of readable media.
In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with examples of the invention. The required structure for a construction of such a system is apparent from the description above. In addition, the present invention is not directed to any particular programming language. It will be appreciated that the teachings of the present invention described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present invention.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in a device as described in this embodiment, or alternatively may be located in one or more devices different from the devices in this example. The modules in the foregoing examples may be combined into one module or may be further divided into a plurality of sub-modules.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.
Furthermore, some of the embodiments are described herein as methods or combinations of method elements that may be implemented by a processor of a computer system or by other means of performing the functions. Thus, a processor with the necessary instructions for implementing the described method or method element forms a means for implementing the method or method element. Furthermore, the elements of the apparatus embodiments described herein are examples of the following apparatus: the apparatus is for carrying out the functions performed by the elements for carrying out the objects of the invention.
As used herein, unless otherwise specified the use of the ordinal terms "first," "second," "third," etc., to describe a general object merely denote different instances of like objects, and are not intended to imply that the objects so described must have a given order, either temporally, spatially, in ranking, or in any other manner.
While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of the above description, will appreciate that other embodiments are contemplated within the scope of the invention as described herein. Furthermore, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is intended to be illustrative, but not limiting, of the scope of the invention, which is defined by the appended claims.
Claims (10)
1. A method of processing ordered data, adapted to be executed in a computing device, the computing device having previously created and loaded a cache unit, an acquisition unit, and a plurality of processing units, the cache unit being configured to store data to be processed, wherein the data to be processed is ordered data, the ordered data being data having an order of arrangement, the method comprising:
reading a plurality of data pointers of the data to be processed from the cache unit by using the acquisition unit at a time;
determining the quantity of data processed by each processing unit as a first numerical value;
dividing the acquired data pointer into a plurality of data pointer segments by taking the first numerical value as a unit;
allocating one data pointer segment to each processing unit, wherein the data pointer segments allocated to the processing units are not repeated;
accessing and processing data corresponding to the data pointer segments allocated to the processing units through a plurality of processing units at the same time to obtain a processing result;
and storing the processing result into the accessed data pointers of the processing units.
2. The method of claim 1, further comprising the step of:
storing the acquired data pointer into an array;
wherein the step of dividing the acquired data pointer into a plurality of data pointer segments in units of the first value comprises:
dividing the index of the array into a plurality of index segments by taking the first numerical value as a unit;
wherein said step of assigning each of said processing units with one of said data pointer segments comprises:
and allocating one index segment to each processing unit, wherein the index segments allocated to the processing units are not repeated.
3. The method of claim 1, wherein the step of determining the amount of data to be processed by each of the processing units as a first value comprises:
and determining the quantity of data to be processed of each processing unit according to the acquired data pointer quantity and the processing unit quantity in an equal division mode to serve as a first numerical value.
4. The method of claim 2, wherein the step of simultaneously accessing, by a plurality of the processing units, data corresponding to the data pointer segments allocated to each of the processing units, and processing includes:
accessing and processing data corresponding to the data pointers in the index segments allocated to the processing units through a plurality of processing units at the same time;
the step of accessing and processing the data corresponding to the data pointer in the allocated index segment by each processing unit and obtaining a processing result comprises the following steps:
determining a start index and an end index of the allocated index segment in the array;
accessing a data pointer corresponding to the start index to the end index;
and reading data from the accessed data pointer, and processing to obtain a data processing result.
5. The method of claim 4, further comprising the step of:
binding a data identifier for the data to be processed in the cache unit, wherein whether the data to be processed is processed or not is distinguished through the data identifier;
judging whether the processing unit processes the data corresponding to the data pointer in the allocated index section according to the data identifier of the data to be processed, if not, not changing the data identifier corresponding to the data pointer, and if so, judging whether the processing unit successfully processes the data corresponding to the data pointer in the allocated index section.
6. The method of claim 5, wherein the step of determining whether the processing unit successfully processes the data corresponding to the data pointer in the allocated index segment comprises:
and judging whether the processing unit successfully processes the data corresponding to the data pointers of the allocated index segments, if so, setting the data identifiers of the data corresponding to the data pointers in the allocated index segments, and if not, setting the data identifiers of the data corresponding to the data pointers in the allocated index segments to be a preset value, wherein the preset value is different from the numerical value corresponding to the data identifier setting and the data identifier resetting.
7. The method of any of claims 4 to 6, further comprising the step of:
and judging whether all the processing units process the data corresponding to the data pointers in the assigned index section, if so, acquiring the data pointers of a plurality of data to be processed from the cache unit through the acquisition unit, and if not, continuously judging whether all the processing units process the data corresponding to the data pointers in the assigned index section.
8. The method of any one of claims 1 to 6, wherein the processing unit is not destroyed after processing the data.
9. A computing device, comprising:
at least one processor; and
a memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the method of any of claims 1-8.
10. A readable storage medium storing program instructions which, when read and executed by a mobile terminal, cause the mobile terminal to perform the method of any of claims 1-8.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110320364.8A CN113031908B (en) | 2021-03-25 | 2021-03-25 | Ordered data processing method and computing device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110320364.8A CN113031908B (en) | 2021-03-25 | 2021-03-25 | Ordered data processing method and computing device |
Publications (2)
Publication Number | Publication Date |
---|---|
CN113031908A CN113031908A (en) | 2021-06-25 |
CN113031908B true CN113031908B (en) | 2023-06-06 |
Family
ID=76473722
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110320364.8A Active CN113031908B (en) | 2021-03-25 | 2021-03-25 | Ordered data processing method and computing device |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN113031908B (en) |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106227583A (en) * | 2016-07-18 | 2016-12-14 | 青岛科信安全技术有限公司 | A kind of multiple thread video Real-time Data Processing Method and device |
WO2018094509A1 (en) * | 2016-11-25 | 2018-05-31 | Genetec Inc. | System for inter-process communication |
WO2019046716A1 (en) * | 2017-08-31 | 2019-03-07 | MIPS Tech, LLC | Pointer-size controlled instruction processing |
CN110399396A (en) * | 2018-04-16 | 2019-11-01 | 计算系统有限公司 | Efficient data processing |
CN111221652A (en) * | 2020-01-03 | 2020-06-02 | 北京百度网讯科技有限公司 | Data processing method and device |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9031077B2 (en) * | 2013-03-14 | 2015-05-12 | Genband Us Llc | Systems, methods, and computer program products providing a data unit sequencing queue |
-
2021
- 2021-03-25 CN CN202110320364.8A patent/CN113031908B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106227583A (en) * | 2016-07-18 | 2016-12-14 | 青岛科信安全技术有限公司 | A kind of multiple thread video Real-time Data Processing Method and device |
WO2018094509A1 (en) * | 2016-11-25 | 2018-05-31 | Genetec Inc. | System for inter-process communication |
WO2019046716A1 (en) * | 2017-08-31 | 2019-03-07 | MIPS Tech, LLC | Pointer-size controlled instruction processing |
CN110399396A (en) * | 2018-04-16 | 2019-11-01 | 计算系统有限公司 | Efficient data processing |
CN111221652A (en) * | 2020-01-03 | 2020-06-02 | 北京百度网讯科技有限公司 | Data processing method and device |
Non-Patent Citations (1)
Title |
---|
面向高速实时数据处理的无锁内存分配算法;李文浩;方景龙;;杭州电子科技大学学报(自然科学版)(第04期);全文 * |
Also Published As
Publication number | Publication date |
---|---|
CN113031908A (en) | 2021-06-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN108052643B (en) | Data storage method and device based on LSM Tree structure and storage engine | |
CN111324427B (en) | Task scheduling method and device based on DSP | |
CN110928803B (en) | Memory management method and device | |
US20060225060A1 (en) | Code swapping in embedded DSP systems | |
CN114416252A (en) | Window display method and device, computing equipment and storage medium | |
CN114416310A (en) | Multiprocessor load balancing method, computing device and storage medium | |
CN114327917A (en) | Memory management method, computing device and readable storage medium | |
CN113031908B (en) | Ordered data processing method and computing device | |
CN115617255A (en) | Management method and management device for cache files | |
CN115658625B (en) | Data decompression system, graphic processing system, device, equipment and decompression method | |
CN111078384A (en) | Method and device for migrating core data, computer equipment and storage medium | |
US9405470B2 (en) | Data processing system and data processing method | |
CN114791854B (en) | Scheduling method, device, equipment and storage medium of user-state virtual machine task | |
CN110674086A (en) | Data merging method and device, electronic equipment and storage medium | |
CN113805971B (en) | Application program running method, computing device and storage medium | |
US20150212759A1 (en) | Storage device with multiple processing units and data processing method | |
CN113296788B (en) | Instruction scheduling method, device, equipment and storage medium | |
CN113076069A (en) | Double-cache method and device, electronic equipment and computer readable storage medium | |
CN113535338A (en) | Interaction method, system, storage medium and electronic device for data access | |
CN112711490B (en) | Message processing method, computing device and storage medium | |
CN117271440B (en) | File information storage method, reading method and related equipment based on freeRTOS | |
CN113342756B (en) | Temporary file occupying storage method, client and computing device | |
CN110968577A (en) | Method and system for writing and reading resources and time sequence storage system | |
CN112947677B (en) | Timer reading method, computing device and readable storage medium | |
CN111880743B (en) | Data storage method, device, equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |