CN113031908B - Ordered data processing method and computing device - Google Patents

Ordered data processing method and computing device Download PDF

Info

Publication number
CN113031908B
CN113031908B CN202110320364.8A CN202110320364A CN113031908B CN 113031908 B CN113031908 B CN 113031908B CN 202110320364 A CN202110320364 A CN 202110320364A CN 113031908 B CN113031908 B CN 113031908B
Authority
CN
China
Prior art keywords
data
processing
index
unit
processed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110320364.8A
Other languages
Chinese (zh)
Other versions
CN113031908A (en
Inventor
刘杨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Uniontech Software Technology Co Ltd
Original Assignee
Uniontech Software Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Uniontech Software Technology Co Ltd filed Critical Uniontech Software Technology Co Ltd
Priority to CN202110320364.8A priority Critical patent/CN113031908B/en
Publication of CN113031908A publication Critical patent/CN113031908A/en
Application granted granted Critical
Publication of CN113031908B publication Critical patent/CN113031908B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F7/00Methods or arrangements for processing data by operating upon the order or content of the data handled
    • G06F7/06Arrangements for sorting, selecting, merging, or comparing data on individual record carriers
    • G06F7/08Sorting, i.e. grouping record carriers in numerical or other ordered sequence according to the classification of at least some of the information they carry
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a method for processing ordered data and a computing device, wherein an acquisition unit is utilized to read a plurality of data pointers of data to be processed from a cache unit at a time, then the quantity of the data processed by each processing unit is determined as a first numerical value, the acquired data pointers are divided into a plurality of data pointer segments by taking the first numerical value as a unit, then each processing unit is allocated with a data pointer segment, the data corresponding to the data pointer segments allocated to each processing unit are accessed and processed simultaneously by the plurality of processing units, a processing result is obtained, and finally the processing result is stored in the data pointers accessed by each processing unit. The method improves the processing efficiency of the ordered data, ensures the ordering of the processing result under the condition of no locking processing, and ensures that the processing process of the ordered data is simple and convenient.

Description

Ordered data processing method and computing device
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method and a computing device for processing ordered data.
Background
Data processing is the collection, storage, retrieval, processing, transformation and transmission of data. The ordered data is data with a certain arrangement sequence, and it is important to ensure the order of output data (processing result) after the ordered data is processed in the process of processing the ordered data.
At present, the data processing method for ordered data is as follows: the ordered data is generated through a single producer thread, the ordered data is written into a cache queue, part of the ordered data is locked through a locking mechanism, and only one consumer thread holds the lock, so that the locked part of the ordered data can be processed through only one consumer thread holding the lock, and a processing result is obtained. The method can ensure the order of the processing result, but the efficiency is low because the ordered data can be processed by only one consumer thread at a time, and the locking mechanism is complex, so the process of processing the ordered data by the locking mechanism is complex and the efficiency is low.
For this reason, a new sequential data processing method is needed to solve the problems existing in the above technical solutions.
Disclosure of Invention
To this end, the present invention provides a method and computing device to solve or at least alleviate the above-identified problems.
According to an aspect of the present invention, there is provided a method of processing ordered data, adapted to be executed in a computing device, the computing device having previously created and loaded a cache unit for storing data to be processed, the data to be processed being ordered data, the ordered data being data having an arrangement order, the method comprising:
reading a plurality of data pointers of the data to be processed from the cache unit by using the acquisition unit at a time;
determining the quantity of data processed by each processing unit as a first numerical value;
dividing the acquired data pointer into a plurality of data pointer segments by taking the first numerical value as a unit;
allocating one data pointer segment to each processing unit, wherein the data pointer segments allocated to the processing units are not repeated;
accessing and processing data corresponding to the data pointer segments allocated to the processing units through a plurality of processing units at the same time to obtain a processing result;
and storing the processing result into the accessed data pointers of the processing units.
Optionally, the method further comprises the steps of:
storing the acquired data pointer into an array;
wherein the step of dividing the acquired data pointer into a plurality of data pointer segments in units of the first value comprises:
dividing the index of the array into a plurality of index segments by taking the first numerical value as a unit;
wherein said step of assigning each of said processing units with one of said data pointer segments comprises:
and allocating one index segment to each processing unit, wherein the index segments allocated to the processing units are not repeated.
Optionally, the step of determining the amount of data to be processed by each processing unit as the first value includes:
and determining the quantity of data to be processed of each processing unit according to the acquired data pointer quantity and the processing unit quantity in an equal division mode to serve as a first numerical value.
Optionally, the step of accessing, by a plurality of processing units, data corresponding to the data pointer segments allocated to each processing unit simultaneously, and processing includes:
accessing and processing data corresponding to the data pointers in the index segments allocated to the processing units through a plurality of processing units at the same time;
the step of accessing and processing the data corresponding to the data pointer in the allocated index segment by each processing unit and obtaining a processing result comprises the following steps:
determining a start index and an end index of the allocated index segment in the array;
accessing a data pointer corresponding to the start index to the end index;
and reading data from the accessed data pointer, and processing to obtain a data processing result.
Optionally, the method further comprises the steps of:
binding a data identifier for the data to be processed in the cache unit, wherein whether the data to be processed is processed or not is distinguished through the data identifier;
judging whether the processing unit processes the data corresponding to the data pointer in the allocated index section according to the data identifier of the data to be processed, if not, not changing the data identifier corresponding to the data pointer, and if so, judging whether the processing unit successfully processes the data corresponding to the data pointer in the allocated index section.
Optionally, the step of determining whether the processing unit successfully processes the data corresponding to the data pointer in the allocated index segment includes:
and judging whether the processing unit successfully processes the data corresponding to the data pointers of the allocated index segments, if so, setting the data identifiers of the data corresponding to the data pointers in the allocated index segments, and if not, setting the data identifiers of the data corresponding to the data pointers in the allocated index segments to be a preset value, wherein the preset value is different from the numerical value corresponding to the data identifier setting and the data identifier resetting.
Optionally, the method further comprises the steps of:
and judging whether all the processing units process the data corresponding to the data pointers in the assigned index section, if so, acquiring the data pointers of a plurality of data to be processed from the cache unit through the acquisition unit, and if not, continuously judging whether all the processing units process the data corresponding to the data pointers in the assigned index section.
Optionally, the processing unit is not destroyed after processing the data.
According to another aspect of the present invention, there is provided a computing device comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the method as described above.
According to yet another aspect of the present invention, there is provided a readable storage medium storing program instructions which, when read and executed by a computing device, cause the computing device to perform the method as described above.
According to the technical scheme of the invention, the method for processing the ordered data is provided, the method distributes the data pointer of the ordered data to be processed for each consumption thread, and a plurality of consumption threads can process the ordered data in the data pointer at the same time and store the data result to the corresponding data pointer. And the data is directly stored to the corresponding data pointer after being processed, which is equivalent to the change of the content of the data pointer but the data pointer is not changed, and the data pointer is ordered, so that the ordering of the processing result is ensured under the condition of no locking processing, and the processing process of the ordered data is simple and convenient.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.
Drawings
To the accomplishment of the foregoing and related ends, certain illustrative aspects are described herein in connection with the following description and the annexed drawings, which set forth the various ways in which the principles disclosed herein may be practiced, and all aspects and equivalents thereof are intended to fall within the scope of the claimed subject matter. The above, as well as additional objects, features, and advantages of the present disclosure will become more apparent from the following detailed description when read in conjunction with the accompanying drawings. Like reference numerals generally refer to like parts or elements throughout the present disclosure.
FIG. 1 shows a schematic diagram of a computing device 100 according to one embodiment of the invention;
FIG. 2 illustrates a flow chart of a method 200 of processing ordered data according to one embodiment of the invention; and
fig. 3 shows a schematic diagram of the connection of a producer unit, a cache unit, an acquisition unit and a processing unit according to an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Ordered data, i.e., data having a certain order of arrangement. In the prior art, the data processing method for ordered data comprises the following steps: generating ordered data through a single producer thread, writing the ordered data into a cache queue, locking part of the ordered data through a locking mechanism, and in the locking mechanism, only one consumer thread holding the lock can process the locked part of the ordered data, and obtain a processing result, then locking other parts of the ordered data, and processing the locked part of the ordered data through the one consumer thread holding the lock, so as to obtain a processing result, and repeating the process until all the ordered data of the cache queue are processed. The method can ensure the ordering of the processing result because the locked partial ordered data is processed by one consumer thread at a time, but has low efficiency because the ordered data can only be processed by one consumer thread at a time, and the locking mechanism is complex, so the process of processing the ordered data by the locking mechanism is complex and has low efficiency.
To this end, the present invention proposes a method of processing ordered data, the method being performed in a computing device 100, fig. 1 showing a block diagram of the computing device 100 according to one embodiment of the present invention. A block diagram of a computing device 100 is shown in fig. 1. In a basic configuration 102, the computing device 100 typically includes a system memory 106 and one or more processors 104. The memory bus 108 may be used for communication between the processor 104 and the system memory 106.
Depending on the desired configuration, the processor 104 may be any type of processing including, but not limited to: a microprocessor (μp), a microcontroller (μc), a digital information processor (DSP), or any combination thereof. The processor 104 may include one or more levels of caches, such as a first level cache 110 and a second level cache 112, a processor core 114, and registers 116. The example processor core 114 may include an Arithmetic Logic Unit (ALU), a Floating Point Unit (FPU), a digital signal processing core (DSP core), or any combination thereof. The example memory controller 118 may be used with the processor 104, or in some implementations, the memory controller 118 may be an internal part of the processor 104.
Depending on the desired configuration, system memory 106 may be any type of memory including, but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. The system memory 106 may include an operating system 120, one or more applications 122, and program data 124. In some implementations, the application 122 may be arranged to operate on an operating system with program data 124. Program data 124 includes instructions, in computing device 100 according to the present invention, program data 124 contains instructions for performing program debug method 200.
Computing device 100 also includes storage device 132, storage device 132 including removable storage 136 and non-removable storage 138, both removable storage 136 and non-removable storage 138 being connected to storage interface bus 134. In the present invention, the related data of each event occurring during the execution of the program and the time information indicating the occurrence of each event may be stored in the storage device 132, and the operating system 120 is adapted to manage the storage device 132. Wherein the storage device 132 may be a magnetic disk.
Computing device 100 may also include an interface bus 140 that facilitates communication from various interface devices (e.g., output devices 142, peripheral interfaces 144, and communication devices 146) to basic configuration 102 via bus/interface controller 130. The example output device 142 includes a graphics processing unit 148 and an audio processing unit 150. They may be configured to facilitate communication with various external devices such as a display or speakers via one or more a/V ports 152. Example peripheral interfaces 144 may include a serial interface controller 154 and a parallel interface controller 156, which may be configured to facilitate communication with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device) or other peripherals (e.g., printer, scanner, etc.) via one or more I/O ports 158. An example communication device 146 may include a network controller 160, which may be arranged to facilitate communication with one or more other computing devices 162 via one or more communication ports 164 over a network communication link.
The network communication link may be one example of a communication medium. Communication media may typically be embodied by computer readable instructions, data structures, program modules, and may include any information delivery media in a modulated data signal, such as a carrier wave or other transport mechanism. A "modulated data signal" may be a signal that has one or more of its data set or changed in such a manner as to encode information in the signal. By way of non-limiting example, communication media may include wired media such as a wired network or special purpose network, and wireless media such as acoustic, radio Frequency (RF), microwave, infrared (IR) or other wireless media. The term computer readable media as used herein may include both storage media and communication media.
Computing device 100 may be implemented as a server, such as a file server, a database server, an application server, a WEB server, etc., as part of a small-sized portable (or mobile) electronic device, such as a cellular telephone, a Personal Digital Assistant (PDA), a personal media player device, a wireless WEB-watch device, a personal headset device, an application-specific device, or a hybrid device that may include any of the above functions. Computing device 100 may also be implemented as a personal computer including desktop and notebook computer configurations. In some embodiments, the operating system 120 of the computing device 100 is configured to perform a method 200 of processing ordered data in accordance with the present invention.
Fig. 2 shows a schematic flow chart of a method 200 of processing ordered data according to one embodiment of the invention. The method 200 is performed in the computing device 100. Before executing the method 200 for processing ordered data, the computing device 100 creates and loads a producer unit, a cache unit, an acquisition unit, and a processing unit in advance, as shown in fig. 3, fig. 3 is a schematic diagram of connection relationships among the producer unit, the cache unit, the acquisition unit, and the processing unit, and fig. 3 illustrates one producer unit, one cache unit, one acquisition unit, and a plurality of processing units, where the cache unit is respectively communicatively connected to the producer unit and the acquisition unit, and the acquisition unit is also communicatively connected to the plurality of processing units.
The computing device 100 generates ordered data through the producer unit and assigns a status identifier to the ordered data, and distinguishes whether the ordered data to be processed is processed and whether the ordered data is successfully processed according to the status identifier, the status identifier may be set according to actual situations, in one embodiment of the present invention, the status identifier includes 0, 1 and-1, the status identifier is 0 indicating that the data is not processed, the status identifier is 1 indicating that the data is processed, the data identifier is-1 indicating that the data processing is wrong, wherein when the status identifier is 1, it indicates that the data corresponding to the data identifier (1) does not need to be processed any more, when the data identifiers are 0 and-1, it indicates that the data corresponding to the data identifiers (0 and-1) need to be processed again, when the ordered data is generated by the producer unit, the computing device 100 assigns the data identifier to the ordered data as 0, and when the ordered data is processed, the data identifier is changed according to successful or failed state of the data processing.
The cache unit is used for storing the ordered data generated by the producer unit, the acquisition unit is used for acquiring the ordered data from the cache unit, and the plurality of processing units are used for processing the effective data acquired by the acquisition unit, wherein the sum of the number of the acquisition unit and the number of the processing units does not exceed the number of kernels of the computing equipment.
The producer unit is a producer, the producer may be a class, a function, a thread, a process, etc., the cache unit is a cache queue, the computing device 100 generates ordered data through the producer and writes the ordered data into the queue head of the cache queue, the one obtaining unit and the plurality of processing units are consumer threads, the computing device 100 creates the plurality of consumer threads in advance, sets the plurality of consumer threads to a dormant state, and activates one of the created plurality of consumer threads as the obtaining unit when the ordered data needs to be processed.
As shown in fig. 2, the method starts at step S210. In step S210, the data pointers of a plurality of data to be processed are read from the buffer unit at a time by the acquisition unit. Specifically, the acquiring unit polls the ordered data to be processed in the cache unit, and when the ordered data to be processed exists, acquires a plurality of data pointers of the ordered data to be processed from the cache unit at a time, wherein the data pointers are memory addresses of the data. The number of data pointers for reading the data to be processed from the buffer unit is set according to the actual scenario, and the present invention is not limited thereto, for example, the obtaining unit obtains 20 data pointers of ordered data from the buffer unit at a time. Since the data pointer is acquired in the present invention, not the data itself, each data in the buffer queue is lengthened.
And storing the acquired data pointers into the array, wherein each data pointer corresponds to one index of the array. And then in step S220, the amount of data processed by each processing unit is determined as a first value. Specifically, the number of processing units processing ordered data is determined based on the number of data pointers acquired and the number of processing units. In one embodiment of the present invention, the number of available data processed by each processing unit is obtained by dividing the number of acquired data pointers by the number of processing units (i.e., in equal divisions). For example, if the number of processing units is 4 and the number of pointers of acquired data is 20, the number of valid data to be processed by each processing unit is 20 divided by 4 and is equal to 5, i.e. the number of valid data to be processed by each processing unit is 5.
Next, in step S230, the acquired data pointer is divided into a plurality of data pointer segments by taking the first value as a unit. Because the data pointers are stored in the array, and each data pointer corresponds to one index of the array, dividing the acquired data pointers into a plurality of data pointer segments is equivalent to dividing the index of the array into a plurality of index segments. For example, the number of processing units is 4 (including processing unit 1, processing unit 2, processing unit 3 and processing unit 4 respectively), the data amount of the effective data to be processed by each unit to be processed is 5, the number of pointers of the acquired data is 20, and thus the index of the corresponding array is 0-19, and then the index of the array is divided into 4 index segments by taking the first value as a unit (namely by taking 5 as a unit), including: 0-4, 5-9, 10-14 and 15-19.
Then in step S240, a data pointer segment is allocated to each processing unit, that is, an index segment is allocated to each processing unit, which is essentially allocating a portion of the ordered data to be processed to each processing unit. Likewise, for example, the number of processing units is 4 (including the processing unit 1, the processing unit 2, the processing unit 3 and the processing unit 4 respectively), the data amount of the ordered data to be processed by each unit to be processed is 5, the number of acquired data pointers is 20, and the array index corresponding to the data pointers is 0-19, then the index of the array is divided into 4 index segments by taking the first numerical value as a unit (namely, taking 5 as a unit), and the method includes: 0-4, 5-9, 10-14, 15-19, then the index segment allocated for processing unit 1 is 0-4, the index segment allocated for processing unit 2 is 5-9, the index segment allocated for processing unit 3 is 10-14, and the index segment allocated for processing unit 4 is 15-19. Here, it should be noted that, the index segment allocated to each processing unit is not repeated, so that each processing unit is prevented from repeatedly processing the ordered data, thereby improving the processing efficiency of the ordered data.
In one embodiment of the present invention, whether the processing unit processes the ordered data corresponding to the data pointer in the allocated index segment is further determined according to the data identifier of the data to be processed, if not, the data identifier is kept to be 0 (the data is not processed) without changing the data identifier, if yes, in step S250, the plurality of processing units access and process the ordered data corresponding to the data pointer segment allocated to each processing unit at the same time, so as to obtain a processing result, i.e. the plurality of processing units access and process the ordered data corresponding to the data pointer in the index segment allocated to each processing unit at the same time, so as to obtain a processing result. As the plurality of processing units access and process the ordered data corresponding to the data pointers in the index segments allocated to the processing units at the same time, the multi-thread simultaneous processing of the ordered data is realized, and the processing efficiency of the ordered data is improved.
Specifically, the step of accessing and processing the ordered data corresponding to the data pointer in the allocated index segment by each processing unit to obtain a processing result includes: determining the starting index and the ending index of the allocated index segments in the array (for example, the starting indexes and the ending indexes corresponding to the index segments 0-4, 5-9, 10-14 and 15-19 are respectively 0 and 4, 5 and 9, 10 and 14, 15 and 19), accessing the data pointers corresponding to the starting index and the ending index, reading ordered data from the accessed data pointers, and processing to obtain a data processing result.
After the plurality of processing units process the ordered data corresponding to the data pointers in the allocated index segment at the same time, continuing to judge whether the processing units process the ordered data corresponding to the data pointers in the allocated index segment successfully, if so, setting the data identification of the data corresponding to the data pointers in the allocated index segment (namely, setting the data identification of the data corresponding to the data pointers to be 1 to indicate that the data is processed), and if not, setting the data identification of the data corresponding to the data pointers in the allocated index segment to be a preset value (for example, setting the data identification of the data corresponding to the data pointers to be-1 to indicate that the data processing is wrong).
In addition, the processing unit is not destroyed after processing the data. And continuously judging whether all processing units process the data corresponding to the data pointers in the allocated index segments, if so, selecting one consumer thread from the consumer threads as an acquisition unit, acquiring a plurality of ordered data to be processed from a cache unit through the acquisition unit, and processing the acquired ordered data based on the rest of the plurality of consumer threads as a plurality of processing units, thereby avoiding resource consumption caused by creating or destroying the consumer threads, saving time required for creating or destroying the consumer threads, and further improving the processing efficiency of the ordered data. If not, continuing to judge whether all the processing units process the data corresponding to the data pointers in the appointed index section.
Finally, in step S260, the data processing result is saved to the accessed data pointers of each processing unit, so that it is ensured that the processing result of the ordered data is still stored in the original data pointer, and because the original data pointers are stored according to a certain sequence, the processing result is also stored according to a certain sequence, thereby ensuring the order of the processing result of the ordered data, and the above process can realize the processing of the ordered data without a locking mechanism, so that the processing process of the ordered data is simple and convenient.
As can be seen from the above, the method for processing ordered data provided by the present invention allocates the data pointer of the ordered data to be processed for each consuming thread, and the multiple consuming threads can process the data in the data pointer at the same time and store the data result to the corresponding data pointer. And the processing result is directly stored to the corresponding data pointer, which is equivalent to the change of the content of the data pointer but the unchanged data pointer, and the data pointer is ordered, so that the ordering of the processing result is ensured under the condition of no locking processing, and the processing process of ordered data is simple and convenient.
The various techniques described herein may be implemented in connection with hardware or software or, alternatively, with a combination of both. Thus, the methods and apparatus of the present invention, or certain aspects or portions of the methods and apparatus of the present invention, may take the form of program code (i.e., instructions) embodied in tangible media, such as removable hard drives, U-drives, floppy diskettes, CD-ROMs, or any other machine-readable storage medium, wherein, when the program is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention.
In the case of program code execution on programmable computers, the mobile terminal will generally include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. Wherein the memory is configured to store program code; the processor is configured to execute the method of processing ordered data of the invention according to instructions in said program code stored in the memory.
By way of example, and not limitation, readable media comprise readable storage media and communication media. The readable storage medium stores information such as computer readable instructions, data structures, program modules, or other data. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. Combinations of any of the above are also included within the scope of readable media.
In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with examples of the invention. The required structure for a construction of such a system is apparent from the description above. In addition, the present invention is not directed to any particular programming language. It will be appreciated that the teachings of the present invention described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present invention.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in a device as described in this embodiment, or alternatively may be located in one or more devices different from the devices in this example. The modules in the foregoing examples may be combined into one module or may be further divided into a plurality of sub-modules.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.
Furthermore, some of the embodiments are described herein as methods or combinations of method elements that may be implemented by a processor of a computer system or by other means of performing the functions. Thus, a processor with the necessary instructions for implementing the described method or method element forms a means for implementing the method or method element. Furthermore, the elements of the apparatus embodiments described herein are examples of the following apparatus: the apparatus is for carrying out the functions performed by the elements for carrying out the objects of the invention.
As used herein, unless otherwise specified the use of the ordinal terms "first," "second," "third," etc., to describe a general object merely denote different instances of like objects, and are not intended to imply that the objects so described must have a given order, either temporally, spatially, in ranking, or in any other manner.
While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of the above description, will appreciate that other embodiments are contemplated within the scope of the invention as described herein. Furthermore, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is intended to be illustrative, but not limiting, of the scope of the invention, which is defined by the appended claims.

Claims (10)

1. A method of processing ordered data, adapted to be executed in a computing device, the computing device having previously created and loaded a cache unit, an acquisition unit, and a plurality of processing units, the cache unit being configured to store data to be processed, wherein the data to be processed is ordered data, the ordered data being data having an order of arrangement, the method comprising:
reading a plurality of data pointers of the data to be processed from the cache unit by using the acquisition unit at a time;
determining the quantity of data processed by each processing unit as a first numerical value;
dividing the acquired data pointer into a plurality of data pointer segments by taking the first numerical value as a unit;
allocating one data pointer segment to each processing unit, wherein the data pointer segments allocated to the processing units are not repeated;
accessing and processing data corresponding to the data pointer segments allocated to the processing units through a plurality of processing units at the same time to obtain a processing result;
and storing the processing result into the accessed data pointers of the processing units.
2. The method of claim 1, further comprising the step of:
storing the acquired data pointer into an array;
wherein the step of dividing the acquired data pointer into a plurality of data pointer segments in units of the first value comprises:
dividing the index of the array into a plurality of index segments by taking the first numerical value as a unit;
wherein said step of assigning each of said processing units with one of said data pointer segments comprises:
and allocating one index segment to each processing unit, wherein the index segments allocated to the processing units are not repeated.
3. The method of claim 1, wherein the step of determining the amount of data to be processed by each of the processing units as a first value comprises:
and determining the quantity of data to be processed of each processing unit according to the acquired data pointer quantity and the processing unit quantity in an equal division mode to serve as a first numerical value.
4. The method of claim 2, wherein the step of simultaneously accessing, by a plurality of the processing units, data corresponding to the data pointer segments allocated to each of the processing units, and processing includes:
accessing and processing data corresponding to the data pointers in the index segments allocated to the processing units through a plurality of processing units at the same time;
the step of accessing and processing the data corresponding to the data pointer in the allocated index segment by each processing unit and obtaining a processing result comprises the following steps:
determining a start index and an end index of the allocated index segment in the array;
accessing a data pointer corresponding to the start index to the end index;
and reading data from the accessed data pointer, and processing to obtain a data processing result.
5. The method of claim 4, further comprising the step of:
binding a data identifier for the data to be processed in the cache unit, wherein whether the data to be processed is processed or not is distinguished through the data identifier;
judging whether the processing unit processes the data corresponding to the data pointer in the allocated index section according to the data identifier of the data to be processed, if not, not changing the data identifier corresponding to the data pointer, and if so, judging whether the processing unit successfully processes the data corresponding to the data pointer in the allocated index section.
6. The method of claim 5, wherein the step of determining whether the processing unit successfully processes the data corresponding to the data pointer in the allocated index segment comprises:
and judging whether the processing unit successfully processes the data corresponding to the data pointers of the allocated index segments, if so, setting the data identifiers of the data corresponding to the data pointers in the allocated index segments, and if not, setting the data identifiers of the data corresponding to the data pointers in the allocated index segments to be a preset value, wherein the preset value is different from the numerical value corresponding to the data identifier setting and the data identifier resetting.
7. The method of any of claims 4 to 6, further comprising the step of:
and judging whether all the processing units process the data corresponding to the data pointers in the assigned index section, if so, acquiring the data pointers of a plurality of data to be processed from the cache unit through the acquisition unit, and if not, continuously judging whether all the processing units process the data corresponding to the data pointers in the assigned index section.
8. The method of any one of claims 1 to 6, wherein the processing unit is not destroyed after processing the data.
9. A computing device, comprising:
at least one processor; and
a memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the method of any of claims 1-8.
10. A readable storage medium storing program instructions which, when read and executed by a mobile terminal, cause the mobile terminal to perform the method of any of claims 1-8.
CN202110320364.8A 2021-03-25 2021-03-25 Ordered data processing method and computing device Active CN113031908B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110320364.8A CN113031908B (en) 2021-03-25 2021-03-25 Ordered data processing method and computing device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110320364.8A CN113031908B (en) 2021-03-25 2021-03-25 Ordered data processing method and computing device

Publications (2)

Publication Number Publication Date
CN113031908A CN113031908A (en) 2021-06-25
CN113031908B true CN113031908B (en) 2023-06-06

Family

ID=76473722

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110320364.8A Active CN113031908B (en) 2021-03-25 2021-03-25 Ordered data processing method and computing device

Country Status (1)

Country Link
CN (1) CN113031908B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106227583A (en) * 2016-07-18 2016-12-14 青岛科信安全技术有限公司 A kind of multiple thread video Real-time Data Processing Method and device
WO2018094509A1 (en) * 2016-11-25 2018-05-31 Genetec Inc. System for inter-process communication
WO2019046716A1 (en) * 2017-08-31 2019-03-07 MIPS Tech, LLC Pointer-size controlled instruction processing
CN110399396A (en) * 2018-04-16 2019-11-01 计算系统有限公司 Efficient data processing
CN111221652A (en) * 2020-01-03 2020-06-02 北京百度网讯科技有限公司 Data processing method and device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9031077B2 (en) * 2013-03-14 2015-05-12 Genband Us Llc Systems, methods, and computer program products providing a data unit sequencing queue

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106227583A (en) * 2016-07-18 2016-12-14 青岛科信安全技术有限公司 A kind of multiple thread video Real-time Data Processing Method and device
WO2018094509A1 (en) * 2016-11-25 2018-05-31 Genetec Inc. System for inter-process communication
WO2019046716A1 (en) * 2017-08-31 2019-03-07 MIPS Tech, LLC Pointer-size controlled instruction processing
CN110399396A (en) * 2018-04-16 2019-11-01 计算系统有限公司 Efficient data processing
CN111221652A (en) * 2020-01-03 2020-06-02 北京百度网讯科技有限公司 Data processing method and device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
面向高速实时数据处理的无锁内存分配算法;李文浩;方景龙;;杭州电子科技大学学报(自然科学版)(第04期);全文 *

Also Published As

Publication number Publication date
CN113031908A (en) 2021-06-25

Similar Documents

Publication Publication Date Title
CN108052643B (en) Data storage method and device based on LSM Tree structure and storage engine
CN111324427B (en) Task scheduling method and device based on DSP
CN110928803B (en) Memory management method and device
US20060225060A1 (en) Code swapping in embedded DSP systems
CN114416252A (en) Window display method and device, computing equipment and storage medium
CN114416310A (en) Multiprocessor load balancing method, computing device and storage medium
CN114327917A (en) Memory management method, computing device and readable storage medium
CN113031908B (en) Ordered data processing method and computing device
CN115617255A (en) Management method and management device for cache files
CN115658625B (en) Data decompression system, graphic processing system, device, equipment and decompression method
CN111078384A (en) Method and device for migrating core data, computer equipment and storage medium
US9405470B2 (en) Data processing system and data processing method
CN114791854B (en) Scheduling method, device, equipment and storage medium of user-state virtual machine task
CN110674086A (en) Data merging method and device, electronic equipment and storage medium
CN113805971B (en) Application program running method, computing device and storage medium
US20150212759A1 (en) Storage device with multiple processing units and data processing method
CN113296788B (en) Instruction scheduling method, device, equipment and storage medium
CN113076069A (en) Double-cache method and device, electronic equipment and computer readable storage medium
CN113535338A (en) Interaction method, system, storage medium and electronic device for data access
CN112711490B (en) Message processing method, computing device and storage medium
CN117271440B (en) File information storage method, reading method and related equipment based on freeRTOS
CN113342756B (en) Temporary file occupying storage method, client and computing device
CN110968577A (en) Method and system for writing and reading resources and time sequence storage system
CN112947677B (en) Timer reading method, computing device and readable storage medium
CN111880743B (en) Data storage method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant