CN108491263A - Data processing method, data processing equipment, terminal and readable storage medium storing program for executing - Google Patents
Data processing method, data processing equipment, terminal and readable storage medium storing program for executing Download PDFInfo
- Publication number
- CN108491263A CN108491263A CN201810175811.3A CN201810175811A CN108491263A CN 108491263 A CN108491263 A CN 108491263A CN 201810175811 A CN201810175811 A CN 201810175811A CN 108491263 A CN108491263 A CN 108491263A
- Authority
- CN
- China
- Prior art keywords
- data processing
- task
- node tasks
- processor
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5066—Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Image Processing (AREA)
- Stored Programmes (AREA)
Abstract
A kind of data processing method of present invention offer, data processing equipment, terminal, readable storage medium storing program for executing.For：Receive data processing task；It determines the TU task unit in the data processing task, the TU task unit is split as at least one subtask；Determine at least one node tasks that at least one subtask includes；At least one node tasks are distributed to processor unit corresponding at least one node tasks.Through the above way, processing task of the present invention for intensive, it can be executed in each processor parallel in a manner of node tasks, task amount can more reasonably be distributed, the load balancing of realization system, execution efficiency is improved, shortens and executes the time, while system power dissipation can be effectively reduced.
Description
Technical field
The present invention relates to data processing field more particularly to a kind of data processing method, device, terminal and readable storage mediums
Matter.
Background technology
This part intends to provides background for the embodiments of the present invention stated in claims and specific implementation mode
Or context.Description herein recognizes it is the prior art not because not being included in this part.
With the fast development of mobile Internet, including portable notebook computer, the mobile terminal including mobile phone is in people
Life in play more and more important role, more and more functions are also integrated in the terminal to be carried to people
For more convenient, quickly service.But along with more multi-functional integrated, complication, at the data that mobile terminal is undertaken
Reason amount also just becomes increasing, this in terminal but also be equipped with more more processing chips, such as more first in industry
Into CPU, GPU of technique；More expert data processing chip:Neon、DSP(VP5、HVX、IPU)、NPU.Although being integrated with so
More data processors, but compared to the task amount of big data correlation computations required for fast development, one single chip computing capability
Promotion be still limited, especially in current manual's intelligence, machine learning increasingly fast-developing today, for more multiple
Miscellaneous, how more rapid more huge data processing task is under existing hardware condition, efficient, becomes in the industry
Urgent problem.
Invention content
Can be quick in consideration of it, it is necessary to provide one kind, efficient process complexity is high, the big data processing task of operand
Method.
The embodiment of the present application first aspect provides a kind of data processing method, a kind of data processing method, which is characterized in that
Receive data processing task；
Receive data processing task；
Determine at least one node tasks that the data processing task includes；
At least one node tasks are distributed to processor unit corresponding at least one node tasks.
In a kind of possible realization method, at least one node that the determination data processing task includes is appointed
Business, specifically includes：
Determine the TU task unit in the data processing task；
The TU task unit is split as at least one subtask；Determine that at least one subtask includes it is described extremely
Few node tasks.
In a kind of possible realization method, at least one node that determination at least one subtask includes
Task specifically includes：
Determine at least one subtask in requisition at least the one of execution according at least one subtask type
A node tasks.
In alternatively possible realization method, it is described by least one node tasks distribute to it is described at least one
Before the corresponding processor unit of node tasks, the method further includes：
According to the type and/or operating status of processor unit, the corresponding processing of at least one node tasks is determined
Device unit；
In alternatively possible realization method, the method further includes：By processor unit, treated described at least one
A node tasks combination.
In alternatively possible realization method, at least one node is being determined according to the operating status of processor unit
When the corresponding processor unit of task, the operating status of the processor unit includes the idle degrees of at least one processor；
It is described when determining the corresponding processor unit of at least one node tasks according to the type of processor unit
The type of processor unit includes the correspondence of at least one processor and at least one node tasks.
In alternatively possible realization method, dependence is not present between at least one subtask block.
The application second aspect provides a kind of data processing equipment, which is characterized in that including
Receiving module receives data processing task；
Determining module determines at least one node tasks that the data processing task includes；
Distribution module：Distribute at least one node tasks to processor corresponding at least one node tasks
Unit.
In alternatively possible realization method, determining module specifically includes：
First determination unit determines the TU task unit in the data processing task；
The TU task unit is split as at least one subtask by split cells；
Second determination unit determines at least one node tasks that at least one subtask includes.
In alternatively possible realization method, second determination unit is specifically used for：
Determine at least one subtask in requisition at least the one of execution according at least one subtask type
A node tasks.
In alternatively possible realization method, the distribution module is specifically used for, according to the type of processor unit and/
Or operating status, determine the corresponding processor unit of at least one node tasks；By at least one node tasks point
It is assigned to the corresponding processor unit of at least one node tasks.
In alternatively possible realization method, at least one node is being determined according to the operating status of processor unit
When the corresponding processor unit of task, the operating status of the processor unit includes the idle degrees of at least one processor；
It is described when determining the corresponding processor unit of at least one node tasks according to the type of processor unit
The type of processor unit includes the correspondence of at least one processor and at least one node tasks.
Alternatively possible realization method, above-mentioned data processing equipment further include：Composite module handles processor unit
At least one node tasks combination afterwards.
Dependence is not present between at least one subtask block for the mode of alternatively possible realization.
The application third aspect provides a kind of terminal, and the terminal includes processor, and the processor is for executing storage
The step of data processing method as described in any one of above-mentioned first aspect is realized when the computer program stored in device.
The application fourth aspect provides a kind of computer readable storage medium, is stored thereon with computer program, feature
It is, the step such as any one of above-mentioned first aspect data processing method is realized when the computer program is executed by processor
Suddenly.
Data processing method provided by the invention, data processing equipment, terminal, readable storage medium storing program for executing are each by analyzing
The type and/or current operating conditions of processor unit, and the attribute feature of task is combined to be divided into TU task unit, at least
One subtask, and further determine that at least one node tasks of each subtask, appointed with node in task handles queue
Business is that minimum unit is distributed to being more suitable for handling the processor unit of the node tasks, in this way holding for any one subtask
Row, can be executed in each processor parallel in a manner of node tasks, can more reasonably distribute task amount, realize system
The load balancing of system, improves execution efficiency, shortens and executes the time.Meanwhile for the king-sized data task of calculation amount, originally
Invention reduces calculation amount larger function or calculation by the dismantling of more rational task and the rational management of processor chips
The power problems that method is brought.
Description of the drawings
It is required in being described below to embodiment in order to illustrate more clearly of the technical solution of embodiment of the present invention
The attached drawing used is briefly described, it should be apparent that, the accompanying drawings in the following description is some embodiments of the present invention, for
For those of ordinary skill in the art, without creative efforts, other are can also be obtained according to these attached drawings
Attached drawing.
Figure 1A is the flow diagram of the data processing method of first embodiment of the invention.
Figure 1B is the step S102 flow diagrams of the data processing method of first embodiment of the invention.
Fig. 2 is corresponding Work flow model schematic diagram during data processing task executes.
Fig. 3 is the flow diagram of the data processing method of second embodiment of the invention.
Fig. 4 A are data processing methods of the present invention in JPEG decoding task flow diagrams.
Fig. 4 B are that data processing method of the present invention task in JPEG decoding tasks handles schematic diagram.
Fig. 5 is the structural schematic diagram for the data processing equipment that an embodiment of the present invention provides.
Fig. 6 is the schematic diagram for the terminal that an embodiment of the present invention provides.
Main element symbol description
Following specific implementation mode will be further illustrated the present invention in conjunction with above-mentioned attached drawing.
Specific implementation mode
To better understand the objects, features and advantages of the present invention, below in conjunction with the accompanying drawings and specific real
Applying mode, the present invention will be described in detail.It should be noted that in the absence of conflict, presently filed embodiment and reality
The feature applied in mode can be combined with each other.
Many details are elaborated in the following description to facilitate a thorough understanding of the present invention, described embodiment
Only some embodiments of the invention, rather than whole embodiments.Based on the embodiment in the present invention, this field
The every other embodiment that those of ordinary skill is obtained without making creative work, belongs to guarantor of the present invention
The range of shield.
Unless otherwise defined, all of technologies and scientific terms used here by the article and belong to the technical field of the present invention
The normally understood meaning of technical staff is identical.Used term is intended merely to description tool in the description of the invention herein
The purpose of the embodiment of body, it is not intended that in the limitation present invention.
Fig. 1 is the flow chart for the data processing method that an embodiment of the present invention provides, and the data processing method can be with
Applied to terminal.The terminal can have certain data processing such as smart mobile phone, tablet computer, personal digital assistant
The computer equipment of ability.It should be noted that the data processing method of embodiment of the present invention be not limited to it is shown in FIG. 1
Step in flow chart and sequence.The step in shown flow chart can increase, removes or change according to different requirements,
Sequentially.
In embodiment of the present invention one, data processing method according to figure 1 includes,
Step S101：Receive data processing task；
Start as terminal data processing flow, receives data processing task first, the user of terminal can be by defeated
Enter operation to come to terminal input data process instruction, so that terminal receives data processing task or terminal is mounted answers
With the pre-set mode of program data processing task is received come triggering terminal.Specifically, data processing task may include：Sound
Frequency decoding task can be mapped to the broadcasting of audio in terminal；The decoding of picture can be mapped to picture in terminal, video is checked；
Fingerprint, face verification task can be mapped in terminal and unlock, pay；And data parsing, deep learning, artificial intelligence calculate
Etc. tasks.Enumerating for those data tasks is merely exemplary to enumerate, and not exhaustive formula is enumerated, and those skilled in the art can know
Road, as long as the calculating completed by alignment processing device in terminal, processing task, are construed as at the data in the present invention
Reason task.
Step S102：Determine at least one node tasks that the data processing task includes.
Step S105：At least one node tasks are distributed to processing corresponding at least one node tasks
Device unit.
Wherein it is preferred to step S102：Determine at least one node tasks that the data processing task includes,
Specifically include S1021:Determine the TU task unit in the data processing task；
The data processing task that terminal receives is typically one has complete non-repetitive independence on task process step
Task or multiple independent tasks that there is batch to repeat on task process step, for example picture is checked in picture library, is exactly schemed
Piece decoding process, when picture library simultaneously check plurality of pictures when, the decoding of a pictures at this time as an independent task, and
The decoding of plurality of pictures corresponds to multiple independent tasks of repetition, when terminal only receives when checking instruction of a pictures,
That is processing task is only the decoding of a pictures, at this point, the data processing task received is also a TU task unit,
That is, itself is a TU task units for processing task, and the TU task unit in step S1021 is such independent task.Task
Unit is the separate unit repeated on process step in the processing task that terminal receives.
Step S1022：The TU task unit is split as at least one subtask；
In order to improve the degree of concurrence in processor unit processing speed and processing, TU task unit can be split as multiple
Subtask, in this way, the treating capacity of each subtask becomes a part for original TU task unit, compared with the prior art in by task
The mode that unit distributes to processor to handle is assigned to the treating capacity of each processor unit in present embodiment therefore becomes
Less, so as to improve processor unit it is per treatment in speed, in this case, for multiple processor units come
It says, therefore the degree of concurrence in processing is also improved.
It should be strongly noted that be split as multiple subtasks herein, only a kind of more preferred mode, at certain
In the case that process flow steps of being engaged in are relatively simple, using TU task unit as a complete task into next in the step
Step is equally included in present embodiment, can equally realize the effect of the invention to be realized.
In the data processing task of larger data amount, it is preferable that do not deposited between each at least one subtask block
In dependence.It can ensure in this manner, the energy when task handles each waiting task in queue in distribution
Higher degree of concurrence is enough kept, and then for the processing of big data quantity, in processing speed and efficiency, can keep more bright
Aobvious advantage.
Step S1023：Determine at least one node tasks that at least one subtask includes；
What needs to be explained here is that in the execution of task, completed by each step in execution, just
The execution for being task of saying is completed in the form of the workflow being made of step one by one, and the process of each task execution can be with
It is abstracted as a Work flow model, is illustrated in figure 2 a relatively simple Work flow model.It is appreciated that being held in a task
In row, need to execute 1., 2., 3., 4., 5. a step, and which step what each step executed below is, then according to specific
Condition judges, therefore, further includes 5 conditions redirected, that is, step and item in Work flow model other than including 5 steps
Part constitutes Work flow model.Each subtask, which can exist, corresponds to a Work flow model, and node tasks are workflows
Model step.
It, i.e., will be every after determining at least one node tasks that at least one subtask includes in present embodiment
A subtask is split as each step according to corresponding execution step in its Work flow model, then will be each after fractionation
A step is pushed into task processing queue, i.e., to be allocated in task processing queue is the distribution in this way in the form of node tasks
What it is to the processing of each processor unit is also node tasks, in such processing mode, for each subtask,
Each node tasks maintain parallel processing in the processing procedure of processor in included workflow, and the time per treatment and
Workload is simplified, and then can improve data-handling efficiency to the greatest extent, reduces the processing time of big data quantity,
And save power consumption.
Herein it should be noted that embodiment of the present invention one improves ten clearly for the efficiency of more complex data processing
It is aobvious.For extremely simple task workflow, there is also only there are one the possibilities of node tasks, for this kind of situation,
TU task unit is split as multiple subtasks only with step S1022, is equally applicable to side described in the embodiment of the present invention one
Method.
Compared to embodiment of the present invention one, traditional Data processing merely relate to by received task be split as to
A few subtask, each subtask is directly distributed to processor unit, at this point, each subtask is executed in processor unit
Process, be still entire Work flow model, in this way, each processor is still serial completion in entire Work flow model,
For in the processing of larger data amount, the above process of the prior art is extremely limited for the raising effect of efficiency.
It should be noted that step S1023 is determined at least one node tasks of at least one subtask, i.e., it will be each
Subtask is split as the corresponding node tasks of each step, then according to corresponding execution step in its Work flow model
Each node tasks after fractionation are pushed into task and handle queue, i.e., to be allocated in task processing queue is with node tasks
Form, distribute to the processing of each processor unit in this way is also node tasks.And the processor unit distributed is according to transmission
The initial data of the node tasks to come over and condition is executed, to carry out the execution of this node tasks and by the node after execution
The output data of task handles queue from the new task that is sent to.Wherein for the distribution of each node tasks, can be arranged individually
Intermediate processor complete, according to Work flow model, and the label and each processor of each node tasks type
And operating status, to distribute above-mentioned each node tasks, the labels of wherein node tasks be for identifying the node tasks type,
Atomic task attribute, the information of predecessor's business cell attribute.
The data processing method of embodiment one through the invention, by analyze the computing capability of each processor chips with
And calculation features, and it is gradually divided into TU task unit, at least one subtask in conjunction with the characteristics of task, and further determine that
At least one node tasks of each subtask are distributed by minimum unit of node tasks to being more suitable in task handles queue
The processor unit of the node tasks is handled, it, can be in a manner of node tasks in this way for the execution of any one subtask
It is executed in each processor parallel, can more reasonably distribute task amount, realize the load balancing of system, improve execution
Efficiency shortens and executes the time.
Fig. 3 shows the flow diagram of the data processing method of second embodiment of the invention.
In embodiment of the present invention two, data processing method according to Fig.3, includes step S201：Receive data
Processing task；Realization details in embodiment two involved by the step identical in embodiment one has had in embodiment one kind
Detailed description, to save space, is just no longer described in detail in the present embodiment；
After receiving data processing task, the data processing method further includes：
S202：Determine the TU task unit in the data processing task；
TU task unit is the separate unit repeated on process step in the processing task that terminal receives.
S203：The TU task unit is split as at least one subtask unit；
Specifically, in order to improve the degree of concurrence in processor unit processing speed and processing, TU task unit can be torn open
It is divided into multiple subtasks, in this way, the treating capacity of each subtask becomes a part for original TU task unit, compared with the prior art
The middle mode that TU task unit distributed to processor to handle is assigned to the treating capacity of each processor unit in present embodiment
Therefore become less, so as to improve processor unit it is per treatment in speed, in this case, for multiple processing
For device unit, therefore the degree of concurrence in processing is also improved.
S204：According at least one subtask type determine at least one subtask in requisition for execution extremely
Few node tasks；
S205：At least one node tasks for including according at least one subtask；
S206：According to the type and/or operating status of processor unit, determine that each node tasks after splitting are corresponding
Processor unit；
Specifically, the processor unit in present embodiment can be applied in existing mobile device to any have
General or proprietary data processing capacity processor unit, including but not limited to, CPU；GPU；More expert data processing chip:
Neon、DSP(VP5、HVX、IPU)；More intelligent AI chips：NPU.Since the type of different processor is different, can locate
The data and task type of reason also have difference, and processor type described herein includes processor and its times that can be handled
The correspondence of service type.
When distribution node task, other than considering processor type, in order to further increase data processing
Efficiency, it is also necessary in conjunction with each processor unit current operating conditions, that is, for some node tasks A, if had
Processor a/b/c can handle, it is still desirable to and in conjunction with the loading condition of current processor a/b/c, i.e., whether the free time judges,
It is preferred that node tasks to be distributed to processor unit that is available free, and being capable of the efficient process node tasks.
S207：Each at least one node tasks are distributed to corresponding with each at least one node tasks
Processor unit；
Type and/or operating status of the step S206 according to processor unit are completed, determines that each node after splitting is appointed
It is engaged in after corresponding processor unit, i.e., is split each subtask according to corresponding execution step in its Work flow model
For the corresponding node tasks of each step, each node tasks after fractionation are then pushed into task and handle queue, i.e. task
To be allocated in processing queue existed in the form of node tasks, and distribute to each processor unit processing in this way is also section
Point task.
S208：By processor unit treated each at least one node tasks combination.
It is the detailed description carried out to data processing method provided by the present invention above.Shown according to different requirements,
The execution sequence of square can change in flow chart, and certain squares can be omitted.Meanwhile it can be readily appreciated that above-mentioned first is real
Apply mode, second embodiment data processing method, can also both be combined together to form one and have both second embodiment, the
The function of one embodiment and the data processing method of effect.
The data processing method of embodiment two through the invention, by analyze the computing capability of each processor chips with
And calculation features, and it is gradually divided into TU task unit, at least one subtask in conjunction with the characteristics of task, and further determine that
At least one node tasks of each subtask are distributed by minimum unit of node tasks to being more suitable in task handles queue
The processor unit of the node tasks is handled, it, can be in a manner of node tasks in this way for the execution of any one subtask
It is executed in each processor parallel, can more reasonably distribute task amount, realize the load balancing of system, improve execution
Efficiency shortens and executes the time.
Illustrate the data of the present invention by taking specific processing mode involved in JPEG decoding task data handling procedures as an example below
Processing method.Fig. 4 is flow diagram of the data processing method of the present invention in JPEG decoding tasks, and wherein Fig. 4 A are the present invention
Data processing method is in JPEG decoding task flow diagrams；Fig. 4 B are data processing methods of the present invention in JPEG decoding tasks
Task handles schematic diagram.
For JPEG decoding tasks in mobile device, Hofmann decoding is generally included in Work flow model
Huffman decoding steps and idct＆color converting steps.Specifically：
For JPEG decodings, including step S301, terminal receive JPEG decoding tasks and jpeg data；Likewise,
The decoding task can input operation come to terminal input data process instruction, so that terminal receives at data by user
Reason task or the pre-set mode of the mounted application program of terminal carry out triggering terminal and receive data processing task；
Step S302 determines TU task unit according to JPEG decoding tasks；Selectively, when an only width JPEG picture,
This time the decoding task that receives of terminal constitutes a TU task unit, and when there is several JPEG pictures, every JPEG picture
Decoding task constitutes a TU task unit；
The TU task unit is split as four subtasks by step S303：Task0, task1, task2, task3, wherein
Fractionation mode may be selected, i.e., the data of an original JPEG picture are divided into four parts, it is easier to the mode of understanding, by one
JPEG picture is divided into four fritters, and the corresponding data of each fritter can individually be decoded task, and each fritter corresponds at this time
The decodings of data be a task.At this point, those skilled in the art are able to know that, the subtask that each TU task unit is split
Number, can be considered according to the combined factors such as the situation of actual processor unit and the calculation amount of TU task unit itself, this field
Technical staff may be selected preferably to split mode.
Step S304 determines the node tasks that four subtask task0/task1/task2/task3 include：①
Huffman decoding；②Idct&color coverting；
Terminal determines the node tasks that subtask includes according to JPEG decoding task Work flow models：①Huffman
decoding；②Idct&color coverting；It is later determined that the node tasks included by each subtask, and with subtask
Data correspond to each node tasks after determining, will include that each node tasks of subtask data are put into waiting task
Queue.Selectively, it is provided with intermediate processor, takes out the node tasks with data one by one from inactive queue of task,
And according to node tasks type and/or processor unit operating status, type, by least one node tasks distribute to
The corresponding processor unit of at least one node tasks.Wherein preferably, but intermediate processor its according to TU task unit and/
Or the Work flow model of subtask, and each label of node tasks and the type and operating status of each processor, to divide
With above-mentioned each node tasks, the label of wherein node tasks is for identifying the node tasks type, and atomic task attribute is former
The information of TU task unit attribute.
Later, handled after the node tasks should treated that node tasks (become work for each processor unit
Another node tasks in flow model) from newly it is put into inactive queue of task, wait for that intermediate processor continues allocation processing next time.Work as institute
After the completion of having node tasks to handle, according to the label information of node tasks, node tasks can be collectively referred to as each height and appointed by terminal
Business, and each subtask is collectively referred to as TU task unit, and shown by display unit.
It is above application of the data processing method of the present invention in JPEG picture decoding process, by present embodiment
Data processing method can improve data-handling efficiency during large batch of image data browses and shows, and then for
The data that picture shows and browses, which have, more significantly to be promoted, and can also be provided more to user during picture is checked
Good usage experience.
It should be noted that data processing method of the present invention, all for what is executed in terminal in addition to JPEG decoding tasks
As following calculating task is applicable：
1. scientific algorithm (such as linear algebra, data mining, finite element analysis, partial differential variance formula)；
2. spectrum analysis；
3. simulation particle dynamic system (simulated in such as astronomy the formation of galaxy, molecular dynamics, molecular simulation,
Body dynamics, quantum mechanics, weather forecasting, plasma physics)；
4. structured network (such as image Gaussian Blur, image sharpening)；
5. unstructured network (such as belief propagation)；
6.MapReduce (such as distributed search)；
7. combinational logic (as encrypted and decrypting, calculate check addition)；
8. figure tracking (such as collision detection)；
9. Dynamic Programming (such as alignment solves shortest path)；
10. network acceleration；
11. probability graph model (such as deep learning)；
12. finite state machine (such as video decodes, and parses, compression, and circulation pattern is searched in data mining)；
13. support vector machines train.
Specifically, such as LeNet network accelerations, Work flow model may include：Input-Convolutions-
Subsampling-Convolutions-Subsampling-Full connection-Full connection-Gaussian
coonections.Data processing determines TU task unit after having received LeNet network acceleration tasks, and by each task
Unit is split as at least one subtask unit, according to above-mentioned Work flow model, by each subtask unit with Work flow model
In node tasks be put into inactive queue of task, divided according to the type and state of processor unit by intermediate processor
Match, and handled one by one according to above-mentioned Work flow model and complete each node tasks, and finally forms complete LeNet network accelerations and appoint
Business output.
Equally, support vector machines (Support Vector Machine, SVM) is solving small sample, non-linear and higher-dimension
Many distinctive advantages are shown in pattern-recognition, and can promote the use of Function Fitting etc. in other machines problem concerning study.
In machine learning, support vector machines (SVM goes back support vector network) is supervised learning mould related with relevant learning algorithm
Type can analyze data, recognition mode, for classification and regression analysis.In SVM train applications, Work flow model can
To include：Input data-gram matrix (gram matrix)-solve QP problem (solving QP problems)-
Lagrange multipliers (lagrange's method of multipliers)-intercept (intercept)-model.In the supporting vector machine model
In foundation, after having received model foundation task and data, TU task unit is determined, and each TU task unit is split as at least one
Each subtask unit is put into the node tasks in Work flow model and is waited for according to above-mentioned Work flow model by subtask unit
Task queue is handled, is allocated according to the type and state of processor unit by intermediate processor, and according to above-mentioned work
Flow model is handled one by one completes each node tasks, and finally establishes complete supporting vector machine model and output model calculates
As a result.
The data processing method of the embodiment of the present application is illustrated above, below to the data of the embodiment of the present application at
Reason device illustrates, referring to Fig. 5, Fig. 5 is one embodiment figure of the data processing equipment of the application embodiment, this reality
Data processing equipment in mode is applied to may include：
Receiving module 11 receives data processing task；
First determining module 12, determines the TU task unit in the data processing task；
Module 13 is split, the TU task unit is split as at least one subtask；
Second determining module 14 determines at least one node tasks that at least one subtask includes；
Distribution module 15：At least one node tasks are distributed to place corresponding at least one node tasks
Manage device unit；
Wherein, the processor unit includes at least CPU, GPU and/or coprocessor；
Preferably mode, second determining module, can be specifically used for：According at least one subtask type
Determine at least one subtask at least one node tasks in requisition for execution；Determine at least one subtask packet
At least one node tasks included.
Preferably mode, the distribution module, is specifically used for, according to the type of processor unit and/or operation shape
State determines the corresponding processor unit of each node tasks after splitting；By each at least one node tasks distribute to
Each corresponding processor unit of at least one node tasks.
Preferably mode, data processing equipment further includes in present embodiment：Composite module 16, by processor unit
Each at least one node tasks combination that treated.
The operating status of preferably mode, at least one processor includes the idle journey of at least one processor
Degree；
The type of at least one processor includes pair of at least one processor and at least one node tasks
It should be related to.
Dependence is not present between at least one subtask block for preferably embodiment.
It should be noted that each embodiment of corresponding above-mentioned data processing method, data processing equipment may include Fig. 5
Shown in part or all in each function module, the function of each module will introduced in detail below.Data above processing
Specific explanation in each embodiment of mode in identical noun related terms and method is readily applicable to following
To the function introduction of each module.For the sake of saving space and avoiding repetition, details are not described herein again.
The embodiment of the present invention also provides a kind of terminal, including memory, processor and storage on a memory and can located
The computer program run on reason device, the processor are realized when executing described program at the data described in the above embodiment
The step of reason method.
Fig. 6 is the schematic diagram for the terminal that an embodiment of the present invention provides.As shown in fig. 6, terminal 1 includes at least：Processing
Device 20, memory 30, be stored in the memory 30 and can be run on the processor 20 computer program 40 (such as
The control program of data processing method).
Wherein, the terminal 1 can be the meter that smart mobile phone, tablet computer, personal digital assistant etc. have shooting function
Calculate machine equipment.It will be understood by those skilled in the art that the schematic diagram 3 is only the example of terminal 1, not structure paired terminal 1
Restriction, may include either combining certain components or different components, such as institute than illustrating more or fewer components
It can also includes input-output equipment, network access equipment, bus etc. to state terminal 1.
The processor 20 is realized when executing the computer program 40 in the control method embodiment of above-mentioned camera
The step of, such as step S1021-S1023 shown in step S101~S105 shown in FIG. 1, Fig. 2, step S201 shown in Fig. 3~
S207 or step S301~S305 shown in Fig. 4.The processor 20 is realized above-mentioned each when executing the computer program 40
Each module/unit in device embodiments, for example, module 11~13 function.
Illustratively, the computer program 40 can be divided into one or more module/units, it is one or
Multiple module/units are stored in the memory 30, and are executed by the processor 20, to complete the present invention.Described one
A or multiple module/units can be the series of computation machine program instruction section that can complete specific function, and described instruction section is used
In describing implementation procedure of the computer program 40 in the terminal 1.For example, the computer program 40 can be divided
At receiving module in Fig. 5 11；First determining module 12；Split module 13；Second determining module 14；Distribution module 15；Combination die
Block 16；The concrete function of each module 11~16 refers to the specific introduction of front, for the sake of saving space and avoiding repetition, herein
Just repeat no more.
Alleged processor 20 can be central processing unit (Central Processing Unit, CPU), can also be
Other general processors, digital signal processor (Digital Signal Processor, DSP), application-specific integrated circuit
(Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-
Programmable Gate Array, FPGA) either other programmable logic device, discrete gate or transistor logic,
Discrete hardware components etc..General processor can be microprocessor or the processor 20 can also be any conventional processing
Device etc., the processor 20 are the control centres of terminal 1 described in the control device 10/ of the camera, using various interfaces and
The various pieces of 10/ terminal 1 of control device of the entire camera of connection.
For the memory 30 for storing the computer program 40 and/or module/unit, the processor 20 passes through fortune
Row executes the computer program and/or module/unit being stored in the memory 30, and calls and be stored in the storage
Data in device 30 realize the various functions of the data processing equipment/terminal 1.The memory 30 can include mainly storage
Program area and storage data field, wherein storing program area can storage program area, the application program needed at least one function
(such as sound-playing function, image player function etc.) etc.；Storage data field can be stored uses created number according to terminal 1
According to (such as audio data, phone directory, the data etc. for being arranged, obtaining using the control method of above-mentioned camera) etc..In addition, institute
It may include high-speed random access memory to state memory 30, can also include nonvolatile memory, such as hard disk, memory,
Plug-in type hard disk, intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card dodge
Deposit card (Flash Card), at least one disk memory, flush memory device or other volatile solid-state parts.
The embodiment of the present invention also provides a kind of computer readable storage medium, is stored thereon with computer program, the meter
The step of data processing method described in the above embodiment is realized when calculation machine program is executed by processor.
If the integrated module/unit of 1/ computer installation of data processing equipment/terminal is with SFU software functional unit
Form is realized and when sold or used as an independent product, can be stored in a computer read/write memory medium.Base
In such understanding, the present invention realizes all or part of flow in the above embodiment method, can also pass through computer journey
Sequence is completed to instruct relevant hardware, and the computer program can be stored in a computer readable storage medium, described
Computer program is when being executed by processor, it can be achieved that the step of above-mentioned each method embodiment.Wherein, the computer journey
Sequence includes computer program code, and the computer program code can be source code form, object identification code form, executable text
Part or certain intermediate forms etc..The computer readable storage medium may include：The computer program code can be carried
Any entity or device, recording medium, USB flash disk, mobile hard disk, magnetic disc, CD, computer storage, read-only memory (ROM,
Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunications letter
Number and software distribution medium etc..It should be noted that the content that the computer-readable medium includes can be managed according to the administration of justice
Local legislation and the requirement of patent practice carry out increase and decrease appropriate, such as in certain jurisdictions, according to legislation and patent
Practice, computer-readable medium do not include electric carrier signal and telecommunication signal.
In several specific implementation modes provided by the present invention, it should be understood that disclosed terminal and method, it can be with
It realizes by another way.For example, termini embodiment described above is only schematical, for example, the module
Division, only a kind of division of logic function, formula that in actual implementation, there may be another division manner.
In addition, each function module in each embodiment of the present invention can be integrated in same treatment module, it can also
That modules physically exist alone, can also two or more modules be integrated in equal modules.Above-mentioned integrated mould
The form that hardware had both may be used in block is realized, can also be realized in the form of hardware adds software function module.
It is obvious to a person skilled in the art that the embodiment of the present invention is not limited to the details of above-mentioned exemplary embodiment,
And without departing substantially from the spirit or essential attributes of the embodiment of the present invention, this hair can be realized in other specific forms
Bright embodiment.Therefore, in all respects, the present embodiments are to be considered as illustrative and not restrictive, this
The range of inventive embodiments is indicated by the appended claims rather than the foregoing description, it is intended that being equal in claim will be fallen
All changes in the meaning and scope of important document are included in the embodiment of the present invention.It should not be by any attached drawing mark in claim
Note, which is considered as, to be limited the claims involved.Furthermore, it is to be understood that one word of " comprising " is not excluded for other units or step, odd number is not excluded for
Plural number.Multiple units, module or the device stated in system, device or terminal claim can also be by the same unit, moulds
Block or device are realized by software or hardware.The first, the second equal words are used to indicate names, and are not offered as any specific
Sequence.
Finally it should be noted that embodiment of above is only to illustrate the technical solution of the embodiment of the present invention and it is unrestricted,
Although the embodiment of the present invention is described in detail with reference to the above better embodiment, those skilled in the art should
Understand, can modify to the technical solution of the embodiment of the present invention or equivalent replacement should not all be detached from the skill of the embodiment of the present invention
The spirit and scope of art scheme.
Claims (10)
1. a kind of data processing method, which is characterized in that
Receive data processing task；
Determine at least one node tasks that the data processing task includes；
At least one node tasks are distributed to processor unit corresponding at least one node tasks.
2. data processing method as described in claim 1, which is characterized in that the determination data processing task includes
At least one node tasks, specifically include：
Determine the TU task unit in the data processing task；
The TU task unit is split as at least one subtask；Determine at least one subtask includes described at least one
A node tasks.
3. data processing method as described in claim 1, which is characterized in that determination at least one subtask includes
At least one node tasks, specifically include：
Determine at least one subtask at least one section in requisition for execution according at least one subtask type
Point task.
4. data processing method as described in claim 1, which is characterized in that described to distribute at least one node tasks
To processor unit corresponding at least one node tasks, the method further includes：
According to the type and/or operating status of processor unit, the corresponding processor list of at least one node tasks is determined
Member.
5. data processing method as described in claim 1, which is characterized in that the method further includes：
By processor unit treated at least one node tasks combination.
6. data processing method as claimed in claim 4, which is characterized in that
It is described when determining the corresponding processor unit of at least one node tasks according to the operating status of processor unit
The operating status of processor unit includes the idle degrees of at least one processor；
When determining the corresponding processor unit of at least one node tasks according to the type of processor unit, the processing
The type of device unit includes the correspondence of at least one processor and at least one node tasks.
7. data processing method as claimed in any one of claims 1 to 6, which is characterized in that
Dependence is not present between at least one subtask block.
8. a kind of data processing equipment, which is characterized in that including
Receiving module receives data processing task；
Determining module determines at least one node tasks that the data processing task includes；Distribution module：By described in extremely
Few node tasks are distributed to processor unit corresponding at least one node tasks.
9. a kind of terminal, which is characterized in that the terminal includes processor, and the processor is used to execute to store in memory
The step of data processing method as described in any one of claim 1-7 is realized when computer program.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program
It is realized when being executed by processor such as the step of any one of claim 1-7 data processing method.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810175811.3A CN108491263A (en) | 2018-03-02 | 2018-03-02 | Data processing method, data processing equipment, terminal and readable storage medium storing program for executing |
| PCT/CN2018/095369 WO2019165745A1 (en) | 2018-03-02 | 2018-07-12 | Data processing method, data processing device, terminal and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810175811.3A CN108491263A (en) | 2018-03-02 | 2018-03-02 | Data processing method, data processing equipment, terminal and readable storage medium storing program for executing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108491263A true CN108491263A (en) | 2018-09-04 |
Family
ID=63341355
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810175811.3A Withdrawn CN108491263A (en) | 2018-03-02 | 2018-03-02 | Data processing method, data processing equipment, terminal and readable storage medium storing program for executing |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN108491263A (en) |
| WO (1) | WO2019165745A1 (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109495542A (en) * | 2018-10-16 | 2019-03-19 | 深圳壹账通智能科技有限公司 | Load allocation method and terminal device based on performance monitoring |
| CN109960575A (en) * | 2019-03-26 | 2019-07-02 | 深圳市网心科技有限公司 | A kind of computing capability sharing method, system and relevant device |
| CN110175201A (en) * | 2019-04-10 | 2019-08-27 | 阿里巴巴集团控股有限公司 | Business data processing method, system, device and electronic equipment |
| CN110209496A (en) * | 2019-05-20 | 2019-09-06 | 中国平安财产保险股份有限公司 | Task sharding method, device and sliced service device based on data processing |
| CN111858010A (en) * | 2020-07-30 | 2020-10-30 | 中国工商银行股份有限公司 | Batch data processing method and device |
| CN112673352A (en) * | 2018-09-11 | 2021-04-16 | 华为技术有限公司 | Heterogeneous scheduling of sequential compute DAG |
| CN113032119A (en) * | 2021-03-23 | 2021-06-25 | 北京三快在线科技有限公司 | Task scheduling method and device, storage medium and electronic equipment |
| CN113157403A (en) * | 2020-01-07 | 2021-07-23 | 中科寒武纪科技股份有限公司 | Job processing method and device, computer equipment and readable storage medium |
| CN113821311A (en) * | 2020-06-19 | 2021-12-21 | 华为技术有限公司 | Task execution method and storage device |
| CN114090266A (en) * | 2021-12-01 | 2022-02-25 | 中科三清科技有限公司 | Air quality forecast generation method and device |
| CN116069480A (en) * | 2023-04-06 | 2023-05-05 | 杭州登临瀚海科技有限公司 | Processor and computing device |
| CN116070240A (en) * | 2023-02-03 | 2023-05-05 | 广州万协通信息技术有限公司 | Data encryption processing method and device of multi-chip calling mechanism |
| CN116442219A (en) * | 2023-03-24 | 2023-07-18 | 东莞市新佰人机器人科技有限责任公司 | Intelligent robot control system and method |
| WO2023165105A1 (en) * | 2022-03-04 | 2023-09-07 | 深圳海星智驾科技有限公司 | Load balancing control method and apparatus, electronic device, storage medium, and computer program |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5410696A (en) * | 1992-03-16 | 1995-04-25 | Hitachi, Ltd. | Method of processing a program by parallel processing, and a processing unit thereof |
| CN102929723A (en) * | 2012-11-06 | 2013-02-13 | 无锡江南计算技术研究所 | Method for dividing parallel program segment based on heterogeneous multi-core processor |
| CN104375882A (en) * | 2014-11-21 | 2015-02-25 | 北京应用物理与计算数学研究所 | Multistage nested data drive calculation method matched with high-performance computer structure |
| CN104699542A (en) * | 2015-03-31 | 2015-06-10 | 北京奇艺世纪科技有限公司 | Task processing method and system |
-
2018
- 2018-03-02 CN CN201810175811.3A patent/CN108491263A/en not_active Withdrawn
- 2018-07-12 WO PCT/CN2018/095369 patent/WO2019165745A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5410696A (en) * | 1992-03-16 | 1995-04-25 | Hitachi, Ltd. | Method of processing a program by parallel processing, and a processing unit thereof |
| CN102929723A (en) * | 2012-11-06 | 2013-02-13 | 无锡江南计算技术研究所 | Method for dividing parallel program segment based on heterogeneous multi-core processor |
| CN104375882A (en) * | 2014-11-21 | 2015-02-25 | 北京应用物理与计算数学研究所 | Multistage nested data drive calculation method matched with high-performance computer structure |
| CN104699542A (en) * | 2015-03-31 | 2015-06-10 | 北京奇艺世纪科技有限公司 | Task processing method and system |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112673352A (en) * | 2018-09-11 | 2021-04-16 | 华为技术有限公司 | Heterogeneous scheduling of sequential compute DAG |
| CN109495542B (en) * | 2018-10-16 | 2022-08-02 | 深圳壹账通智能科技有限公司 | Load distribution method based on performance monitoring and terminal equipment |
| CN109495542A (en) * | 2018-10-16 | 2019-03-19 | 深圳壹账通智能科技有限公司 | Load allocation method and terminal device based on performance monitoring |
| CN109960575A (en) * | 2019-03-26 | 2019-07-02 | 深圳市网心科技有限公司 | A kind of computing capability sharing method, system and relevant device |
| CN109960575B (en) * | 2019-03-26 | 2023-09-15 | 深圳市网心科技有限公司 | Computing capacity sharing method, system and related equipment |
| CN110175201A (en) * | 2019-04-10 | 2019-08-27 | 阿里巴巴集团控股有限公司 | Business data processing method, system, device and electronic equipment |
| CN110209496A (en) * | 2019-05-20 | 2019-09-06 | 中国平安财产保险股份有限公司 | Task sharding method, device and sliced service device based on data processing |
| CN113157403A (en) * | 2020-01-07 | 2021-07-23 | 中科寒武纪科技股份有限公司 | Job processing method and device, computer equipment and readable storage medium |
| CN113821311A (en) * | 2020-06-19 | 2021-12-21 | 华为技术有限公司 | Task execution method and storage device |
| CN111858010A (en) * | 2020-07-30 | 2020-10-30 | 中国工商银行股份有限公司 | Batch data processing method and device |
| CN113032119A (en) * | 2021-03-23 | 2021-06-25 | 北京三快在线科技有限公司 | Task scheduling method and device, storage medium and electronic equipment |
| CN114090266A (en) * | 2021-12-01 | 2022-02-25 | 中科三清科技有限公司 | Air quality forecast generation method and device |
| WO2023165105A1 (en) * | 2022-03-04 | 2023-09-07 | 深圳海星智驾科技有限公司 | Load balancing control method and apparatus, electronic device, storage medium, and computer program |
| CN116070240A (en) * | 2023-02-03 | 2023-05-05 | 广州万协通信息技术有限公司 | Data encryption processing method and device of multi-chip calling mechanism |
| CN116070240B (en) * | 2023-02-03 | 2024-03-08 | 广州万协通信息技术有限公司 | Data encryption processing method and device of multi-chip calling mechanism |
| CN116442219A (en) * | 2023-03-24 | 2023-07-18 | 东莞市新佰人机器人科技有限责任公司 | Intelligent robot control system and method |
| CN116442219B (en) * | 2023-03-24 | 2023-11-03 | 东莞市新佰人机器人科技有限责任公司 | Intelligent robot control system and method |
| CN116069480A (en) * | 2023-04-06 | 2023-05-05 | 杭州登临瀚海科技有限公司 | Processor and computing device |
| CN116069480B (en) * | 2023-04-06 | 2023-06-13 | 杭州登临瀚海科技有限公司 | Processor and computing device |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2019165745A1 (en) | 2019-09-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108491263A (en) | Data processing method, data processing equipment, terminal and readable storage medium storing program for executing | |
| US10223762B2 (en) | Pipelined approach to fused kernels for optimization of machine learning workloads on graphical processing units | |
| CN108292241B (en) | Processing a computation graph | |
| US10310908B2 (en) | Dynamic usage balance of central processing units and accelerators | |
| US8910173B2 (en) | Datacenter resource allocation | |
| US9152601B2 (en) | Power-efficient nested map-reduce execution on a cloud of heterogeneous accelerated processing units | |
| US8739171B2 (en) | High-throughput-computing in a hybrid computing environment | |
| CN113377540A (en) | Cluster resource scheduling method and device, electronic equipment and storage medium | |
| US9367359B2 (en) | Optimized resource management for map/reduce computing | |
| CN103201719B (en) | Device emulation in a virtualized computing environment | |
| Jangiti et al. | Scalable and direct vector bin-packing heuristic based on residual resource ratios for virtual machine placement in cloud data centers | |
| US11055139B2 (en) | Smart accelerator allocation and reclamation for deep learning jobs in a computing cluster | |
| TW201531997A (en) | Selective utilization of graphics processing unit (GPU) based acceleration in database management | |
| CN108228344B (en) | Multi-process memory processing method and device, computer device and readable storage medium | |
| Hussain et al. | Novel Dynamic Partial Reconfiguration Implementation of K‐Means Clustering on FPGAs: Comparative Results with GPPs and GPUs | |
| US20110307471A1 (en) | Accelerating Database Management System Operations | |
| CN111984400A (en) | Memory allocation method and device of neural network | |
| US11562554B1 (en) | Workload reduction for non-maximum suppression operation | |
| US20170293469A1 (en) | Efficient handling of sort payload in a column organized relational database | |
| CN113672375A (en) | Resource allocation prediction method, device, equipment and storage medium | |
| CN118364918A (en) | Reasoning method, device, equipment and storage medium of large language model | |
| CN113051049A (en) | Task scheduling system, method, electronic device and readable storage medium | |
| US20240005446A1 (en) | Methods, systems, and non-transitory storage media for graphics memory allocation | |
| CN116795492A (en) | Resource scheduling method, device and equipment of cloud platform and readable storage medium | |
| US20230161575A1 (en) | Transformation of a loop within computer code to minimize iterations |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20180904 |