CN106293937A - A kind of for distributing the method that data processing resources processes to data - Google Patents
A kind of for distributing the method that data processing resources processes to data Download PDFInfo
- Publication number
- CN106293937A CN106293937A CN201610625028.3A CN201610625028A CN106293937A CN 106293937 A CN106293937 A CN 106293937A CN 201610625028 A CN201610625028 A CN 201610625028A CN 106293937 A CN106293937 A CN 106293937A
- Authority
- CN
- China
- Prior art keywords
- data
- code
- little code
- distributing
- processing resources
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Devices For Executing Special Programs (AREA)
Abstract
The invention discloses a kind of for distributing the method that data processing resources processes to data.Comprise the steps: when compiling, analyze potential code and data distribution to identify one or more chances of the action selected from the group consisted of: little code migrates and Data Migration；Operationally, migrate little code or data and distribute, with exercise actual code and data, the chance provided；Migration is carrying out maybe performing little code collection.The present invention by by data in storage space, the locality calculated in space, or on time or dependency, close state carries out being distributed, distributing, effective distribution data processing resources processes to data, distributes calculating resource to calculating task by performing following one or more steps: obtain the little code collection realizing task-set；Obtain the data requirement collection of little code request；Construction represents little code locality and they are by the metric space of the data of access.
Description
Technical field
The invention belongs to field of computer technology, particularly relate to a kind of for distributing data processing resources to data process
Method.
Background technology
Modern high end computer configuation comprises thousands of the most millions for the treatment of elements, a large amount of distributed memory,
And non local memorizer, networking component and the storage infrastructure of various rank.These systems are that execution is application consumed
Static state and the dynamic optimization of resource provide huge challenge.Traditionally, computer configuation makes great efforts to provide the most single, simply
The application in space, location, and offer itself is the most semantic to carry out code execution subsequently and data access.The normal form produced
For many years use good, but when by parallel processing rather than being distributed calculating and data and reality by faster clock rate
On realize all hardware accelerate time, described normal form hinders optimized allocation of resources.
Patent ZL201010616076.9 (a kind of system and method for processing data) discloses this system and includes: point
Item metering data storage device；Withhold metering data storage device；Merge device, be used for would correspond to same collecting bank account
Multiple itemized billing amount of money under family are added, and will add up the centralized charging amount of money that result is stored under same collecting bank account
In, withhold proposition file to generate；R-T unit, sends to settlement center for withholding proposition file, and receives in clearing
Withholding that the heart returns carries back into file；Detachment device, for according to withholding the collection flag bit carried in back into file, the multiple subitem of labelling
Corresponding to the collection state of each itemized billing data in charging account.
Summary of the invention
It is an object of the invention to provide a kind of for distributing the method that data processing resources processes to data, by by number
Carry out being distributed, distributing according in state close on storage space, the locality calculated in space, or time or dependency,
Effective distribution data processing resources processes to data, the problem solving the distribution of computer data resource.
For solving above-mentioned technical problem, the present invention is achieved by the following technical solutions:
The present invention is a kind of for distributing the method that data processing resources processes to data, comprises the steps:
P1, when compiling, analyzes potential code and data distribution to identify the one of the action being selected from the group consisted of
Individual or multiple chances: little code migrates and Data Migration；With
P2 operationally, migrates little code or data and distributes, with exercise actual code and data, the chance provided；
P3 migrates and is carrying out maybe performing little code collection, to utilize local memory, particular data and intermediate object program
The locality of resource, and cooperate the locality of little code, thus minimize communication delay.
Preferably, described migration includes that mobile little code is to the processor underused.
Preferably, described at least some data copy to another place from a place, it is contemplated that migrate at least one little
Code.
Preferably, described to little code offer safety sign, when wherein said sign indicates described method to process described little code
By the restriction considered or authority.
The method have the advantages that
The present invention by by data on storage space, the locality calculated in space, or time or dependency close
State in carry out being distributed, distributing, effective distribution data processing resources processes to data, following one or more by performing
Step distributes calculating resource to calculating task: obtain the little code collection realizing task-set；Obtain the data requirement collection of little code request；
Construction represents little code locality and they are by the metric space of the data of access；Obtain quiet for metric space distance of little code-phase
State definition initial arrangement；Metric space is used to represent to come the little code of initial placement or data；Obtain little code and the dynamic of data can use
Operation time resource is asked；Dynamically place with using described metric space to represent or move little code or data.
Certainly, the arbitrary product implementing the present invention it is not absolutely required to reach all the above advantage simultaneously.
Accompanying drawing explanation
In order to be illustrated more clearly that the technical scheme of the embodiment of the present invention, embodiment will be described required use below
Accompanying drawing is briefly described, it should be apparent that, the accompanying drawing in describing below is only some embodiments of the present invention, for ability
From the point of view of the those of ordinary skill of territory, on the premise of not paying creative work, it is also possible to obtain the attached of other according to these accompanying drawings
Figure.
Fig. 1 is a kind of flow chart for distributing the method that data processing resources processes to data of the present invention.
Detailed description of the invention
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Describe, it is clear that described embodiment is only a part of embodiment of the present invention rather than whole embodiments wholely.Based on
Embodiment in the present invention, those of ordinary skill in the art obtained under not making creative work premise all other
Embodiment, broadly falls into the scope of protection of the invention.
The term of the present invention:
Little code: one group generally can be performed continuously after its input is available till instruction.
Little code collection: the little code character that can process as unit relative to dependency analysis or execution.
Referring to shown in Fig. 1, the present invention is a kind of for distributing the method that data processing resources processes to data, including such as
Lower step: P1, when compiling, analyzes potential code and data distribution to identify the one of the action being selected from the group consisted of
Individual or multiple chances: little code migrates and Data Migration；With P2 operationally, little code or data are migrated to practise actual code sum
The chance provided according to distribution；P3 migrate be carrying out maybe performing little code collection, with utilize local memory, particular data and
The locality of the resource of intermediate object program, and cooperate the locality of little code, thus minimize communication delay.
Preferably, described migration includes that mobile little code is to the processor underused.
Preferably, described at least some data copy to another place from a place, it is contemplated that migrate at least one little
Code.
Preferably, described to little code offer safety sign, when wherein said sign indicates described method to process described little code
By the restriction considered or authority.
It should be noted that in said system embodiment, included unit simply carries out drawing according to function logic
Point, but it is not limited to above-mentioned division, as long as being capable of corresponding function；It addition, each functional unit is concrete
Title also only to facilitate mutually distinguish, is not limited to protection scope of the present invention.
It addition, one of ordinary skill in the art will appreciate that all or part of step realizing in the various embodiments described above method
The program that can be by completes to instruct relevant hardware, and corresponding program can be stored in an embodied on computer readable storage and be situated between
In matter, described storage medium, such as ROM/RAM, disk or CD etc..
Present invention disclosed above preferred embodiment is only intended to help to illustrate the present invention.Preferred embodiment is the most detailed
Describe all of details, be also not intended to the detailed description of the invention that this invention is only described.Obviously, according to the content of this specification,
Can make many modifications and variations.These embodiments are chosen and specifically described to this specification, is to preferably explain the present invention
Principle and actual application so that skilled artisan can be best understood by and utilize the present invention.The present invention is only
Limited by claims and four corner thereof and equivalent.
Claims (4)
1. one kind is used for distributing the method that data processing resources processes to data, it is characterised in that comprise the steps:
P1 when compiling, analyze potential code and data distribution identifying selected from one of the action of group consisted of or
Multiple chances: little code migrates and Data Migration；With
P2 operationally, migrates little code or data and distributes, with exercise actual code and data, the chance provided；
P3 migrates and is carrying out maybe performing little code collection, to utilize the resource of local memory, particular data and intermediate object program
Locality, and cooperate the locality of little code, thus minimize communication delay.
It is the most according to claim 1 a kind of for distributing the method that data processing resources processes to data, it is characterised in that
Described migration includes that mobile little code is to the processor underused.
It is the most according to claim 1 a kind of for distributing the method that data processing resources processes to data, it is characterised in that
Described at least some data copy to another place from a place, it is contemplated that migrate at least one little code.
It is the most according to claim 1 a kind of for distributing the method that data processing resources processes to data, it is characterised in that
Described to little code offer safety sign, by the restriction considered or power during the wherein said sign instruction described method described little code of process
Limit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610625028.3A CN106293937A (en) | 2016-08-02 | 2016-08-02 | A kind of for distributing the method that data processing resources processes to data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610625028.3A CN106293937A (en) | 2016-08-02 | 2016-08-02 | A kind of for distributing the method that data processing resources processes to data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106293937A true CN106293937A (en) | 2017-01-04 |
Family
ID=57664218
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610625028.3A Pending CN106293937A (en) | 2016-08-02 | 2016-08-02 | A kind of for distributing the method that data processing resources processes to data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106293937A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030191927A1 (en) * | 1999-05-11 | 2003-10-09 | Sun Microsystems, Inc. | Multiple-thread processor with in-pipeline, thread selectable storage |
| CN101533417A (en) * | 2009-04-28 | 2009-09-16 | 阿里巴巴集团控股有限公司 | A method and system for realizing ETL scheduling |
| US20090259713A1 (en) * | 2001-02-24 | 2009-10-15 | International Business Machines Corporation | Novel massively parallel supercomputer |
| CN102934081A (en) * | 2010-04-13 | 2013-02-13 | Et国际有限公司 | Runspace method, system and apparatus |
-
2016
- 2016-08-02 CN CN201610625028.3A patent/CN106293937A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030191927A1 (en) * | 1999-05-11 | 2003-10-09 | Sun Microsystems, Inc. | Multiple-thread processor with in-pipeline, thread selectable storage |
| US20090259713A1 (en) * | 2001-02-24 | 2009-10-15 | International Business Machines Corporation | Novel massively parallel supercomputer |
| CN101533417A (en) * | 2009-04-28 | 2009-09-16 | 阿里巴巴集团控股有限公司 | A method and system for realizing ETL scheduling |
| CN102934081A (en) * | 2010-04-13 | 2013-02-13 | Et国际有限公司 | Runspace method, system and apparatus |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102279730B (en) | Parallel data processing method, device and system | |
| KR102319172B1 (en) | Platform providing system based on cloud computing and platform provision method using the same | |
| US8726290B2 (en) | System and/or method for balancing allocation of data among reduce processes by reallocation | |
| CN109358965B (en) | Cloud computing cluster resource scheduling method, medium, device and computing equipment | |
| CN103649910A (en) | Virtual machine placement | |
| CN106547627A (en) | The method and system that a kind of Spark MLlib data processings accelerate | |
| Rodrigues et al. | A comparative analysis of load balancing algorithms applied to a weather forecast model | |
| CN104750555B (en) | Process management method and device in a kind of Android program | |
| CN107239329A (en) | Unified resource dispatching method and system under cloud environment | |
| CN103810047A (en) | Dynamically improving memory affinity of logical partitions | |
| GB2508161A (en) | Monitoring applications executing on a virtual machine and allocating the required resources to the virtual machine. | |
| CN106528171B (en) | Method of interface, apparatus and system between a kind of heterogeneous computing platforms subsystem | |
| CN110008018A (en) | A kind of batch tasks processing method, device and equipment | |
| Sood et al. | Matrix based proactive resource provisioning in mobile cloud environment | |
| KR20140014090A (en) | Agile communication operator | |
| CN103425519A (en) | Distributed computing method and distributed computing system | |
| CN104506371A (en) | Method and device for management of application programming interface (API) call records | |
| CN103473115A (en) | Virtual machine placing method and device | |
| CN105528371A (en) | Method, device, and system for executing writing task | |
| Wu et al. | A new data-grouping-aware dynamic data placement method that take into account jobs execute frequency for Hadoop | |
| Giordano et al. | A dynamic load balancing technique for parallel execution of structured grid models | |
| CN106293937A (en) | A kind of for distributing the method that data processing resources processes to data | |
| CN114785693B (en) | Virtual network function migration method and device based on layered reinforcement learning | |
| KR101378348B1 (en) | Basic prototype of hadoop cluster based on private cloud infrastructure | |
| CN110018830A (en) | A kind of large software compilation device based on distributed type assemblies |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170104 |
|
| RJ01 | Rejection of invention patent application after publication |