CN106095552A - A kind of Multi-Task Graph processing method based on I/O duplicate removal and system - Google Patents
A kind of Multi-Task Graph processing method based on I/O duplicate removal and system Download PDFInfo
- Publication number
- CN106095552A CN106095552A CN201610399043.0A CN201610399043A CN106095552A CN 106095552 A CN106095552 A CN 106095552A CN 201610399043 A CN201610399043 A CN 201610399043A CN 106095552 A CN106095552 A CN 106095552A
- Authority
- CN
- China
- Prior art keywords
- task
- diagram data
- shared buffer
- queue
- duplicate removal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The invention discloses a kind of Multi-Task Graph processing method based on I/O duplicate removal and system thereof, belong to Computer Storage field.The method uses special I/O thread to read diagram data from external memory by asynchronous system, and is loaded into the shared buffer of internal memory；Executed in parallel figure processes task, accesses diagram data from shared buffer during tasks carrying.The present invention reuses by providing the unified method accessing external memory diagram data to achieve I/O, the execution speed of parallel task by the Design coordination of shared buffer memory, solve current external memory ideograph processing system when processing concurrent multitask, there is I/O conflict and the problem of Data duplication in request diagram data, optimizes the data latency time of parallel task.
Description
Technical field
The invention belongs to computer memory technical field, more specifically, relate at a kind of Multi-Task Graph based on I/O duplicate removal
Reason method and system thereof, can reduce I/O and repeat.
Background technology
As abstract data structure a kind of in computer science, figure is suitable for describing extensive, irregular, sparse association
Information, is widely used among the data analysis in each field.And nature diagram data is the most irregular, non-structured, and
And nomography lacks locality on data access, randomness highlights, and the most usually needs iteration, causes traditional big data
Computational frame such as Hadoop, MapReduce are inefficient when processing diagram data.
In order to excavate useful information from diagram data, in recent years create be specifically designed for figure calculate field figure process system
System.Substantially being divided into two classes: a kind of is distributed figure processing system based on shared drive, another kind is that figure based on external memory processes
System.Owing to the random access of diagram data has been exclusively carried out centralized optimization, when performing most of nomography than original meter
Calculate framework and have higher execution efficiency.
Present figure processing system can effectively perform single figure and process task, but many processing on same data set
During individual parallel task, there is waste and the prolongation of the time of execution of storage resource.Distributed figure based on shared drive processes system
Uniting when processing parallel task, exchange data are that each figure process task is privately owned, when figure process task transmits message between node
Can competition network bandwidth, and external memory ideograph processing system, although when parallel figure processes task can with share and access diagram data, but
It is to cause I/O contention when accessing external memory diagram data.And during nomography performs, its performance is affected by data access time
Significantly, so being no matter the network bandwidth competition of memorymodel figure processing system or the I/O of external memory pattern competes the number brought
The most all can reflect that figure processed on total execution time of task according to access delay.
Summary of the invention
The present invention provides a kind of Multi-Task Graph processing method based on I/O duplicate removal and system thereof, accesses by providing unified
The method of external memory diagram data achieves I/O and reuses, the execution speed of parallel task by the Design coordination of shared buffer memory, solves
Current external memory ideograph processing system is when processing concurrent multitask, and I/O conflict and Data duplication occurs in request diagram data
Problem, optimizes the data latency time of parallel task.
A kind of Multi-Task Graph processing method based on I/O duplicate removal, particularly as follows:
Use special I/O thread to read diagram data from external memory by asynchronous system, and be loaded into the shared buffer of internal memory
District；
Executed in parallel figure processes task, accesses diagram data from shared buffer during tasks carrying.
Further, process during tasks carrying the diagram data to shared buffer at figure and conduct interviews mark, by institute
It is invalid that the diagram data having figure process task all to access is labeled as, and then replaces to share by the diagram data read in new from external memory and delays
Rush the invalid diagram data in district.
Further, task is processed for completed figure in task queue, it is removed from TU task unit queue, and
Delete it to the access identities of data in internal memory shared buffer；Figure for being not fully complete in task queue processes task, if
Do not receive termination signal, then save it in entrance next iteration in task queue, otherwise, it is moved from task queue
Remove.
Further, the time of calculating is less than more than predetermined computation time threshold and priority the figure of predetermined priority threshold value
Process task temporary respite, recovers when the system free time to perform again.
A kind of Multi-Task Graph processing system based on I/O duplicate removal, including job engine, memory cache and TU task unit；
Job engine, is used for using special I/O thread to read diagram data from external memory by asynchronous system, and is loaded into interior
Deposit relief area；Load and unload TU task unit and executing tasks parallelly unit, access figure number in the process of implementation from relief area
According to；
Memory cache, is used for caching diagram data；
TU task unit, is used for performing figure and processes task.
Further, described job engine is additionally operable to process figure the diagram data that core buffer accessed by task and carries out
Mark, it is invalid to be labeled as by the diagram data that all figures process task all accessed, then by the diagram data read in new from external memory
Replace the invalid diagram data in shared buffer memory.
Further, described job engine is additionally operable to process task for completed figure in task queue, by it from appointing
Business cell queue removes, and deletes it to the access identities of data in memory cache；For the figure being not fully complete in task queue
Process task, if not receiving termination signal, then saves it in task queue entrance next iteration, otherwise, by its from
Task queue removes.
Further, described job engine is additionally operable to be less than the calculating time more than predetermined computation time threshold and priority
The figure of predetermined priority threshold value processes task temporary respite, recovers when the system free time to perform again.
The present invention, compared with current multitask external memory ideograph processing system, accesses external memory diagram data by providing unified
Method achieve I/O and reuse, the execution speed of parallel task by the Design coordination of shared buffer memory, optimize parallel task
Data latency time can effectively accelerate to scheme parallel to process the speed of performing task.
In the case of concurrency is N, total execution time of system is: TG=max (TC-MAX,TW), and to outside current
Depositing ideograph processing system, total execution time of system is: TO=max (TC-MAX,N*TW), wherein TC-MAX=max (TC1,
TC2,...TCN),TCi(i=1,2 ... N) be each figure process task the calculating time, TWIt it is data access time.
When all parallel tasks are all taking as the leading factor with I/O of tasks, the total of native system performs time TG=TW, existing mould
The total of type performs time TO=N*TW, total execution time (N-1) T of optimizingW；When all tasks are all appointing of being calculated as dominating
During business, work as TW<TC-MAX<NTWTime, TG=TC-MAX, TO=NTW, the optimization time is NTW-TC-MAX, work as TC-MAX>NTWTime, TG=TC-MAX
=TO。
Accompanying drawing explanation
Fig. 1 is the schematic flow sheet of the present invention；
Fig. 2 is the schematic flow sheet that figure processes that task loads；
Fig. 3 is the schematic flow sheet that figure processes task unloading.
Fig. 4 is embodiment of the present invention implementation process view, Fig. 4 a) it is original state figure, Fig. 4 b) it is the most repeatedly
For state diagram, Fig. 4 c) it is second time iterative state figure.
Detailed description of the invention
In order to make the purpose of the present invention, technical scheme and advantage clearer, below in conjunction with drawings and Examples, right
The present invention is further elaborated.Should be appreciated that specific embodiment described herein only in order to explain the present invention, and
It is not used in the restriction present invention.If additionally, technical characteristic involved in each embodiment of invention described below
The conflict of not constituting each other just can be mutually combined.
The present invention proposes a kind of Multi-Task Graph processing system based on I/O duplicate removal, including job engine, memory cache and many
Individual TU task unit.
Job engine: be responsible for internal memory shared buffer memory data load with replace and figure process task unloading carry.
Loading for memory cache data and replace, in job engine shields system, figure process task is sent out to operating system
The read-write requests risen, uses the unified responsible I/O of special thread, and this I/O thread reads figure by the way of asynchronous from external memory
The block buffer that data are loaded in internal memory, this relief area is that the figure of all uses same diagram data collection processes task sharing.
Job engine processes task according to figure and the access identities of diagram data completes loading and the replacement of diagram data in shared buffer memory.More
Saying, it is invalid to be labeled as by the diagram data that all figure tasks all accessed body, then replaces slow by the figure information read in external memory
Diagram data invalid in depositing, completes the loading of diagram data.
For figure process task unloading carry, according to system loading conditions and preset dispatching algorithm figure process appoint
Business unloads and carry TU task unit during performing freely.Executed in parallel figure processes task, postpones during tasks carrying
Rush district and access diagram data.According to the embodiment of a kind of optimization, substantial amounts of calculating process is performed for needs and priority is relatively low
Figure process task, job engine can be recalled, it is to avoid this figure processes task and have impact on the figure taken as the leading factor with I/O and process and appoint
The speed of business iteration.Read checkpoint again when the system free time to resume work, i.e. this TU task unit of carry, obtain task real-time
Peace is performed both by temporal balance.
Shared buffer memory: the diagram data read in from external memory for caching.Diagram data amount is very big, uses here and reads in batches,
First load on internal memory shared buffer memory from external memory reading part component data, the diagram data that task all accessed processed for figure,
Replace diagram data invalid in shared buffer memory with the new diagram data read in external memory, complete the loading of diagram data.Shared buffer
The data in district are replaced and are the most disposably completed, and process, simply by the presence of all figures, the invalid diagram data that task accessed, it is possible to
Complete the replacement of diagram data.
TU task unit: TU task unit provides the user DLL, comprises user-defined figure and processes the concrete real of task
Existing, have the state set of oneself and update collection.Wherein, state set processes task for figure and provides the status information being associated with summit
With the set of temporary variable, updating collection is the massage set transmitted during figure processes tasks carrying.
The present invention proposes a kind of Multi-Task Graph processing method based on I/O duplicate removal, as it is shown in figure 1, particularly as follows:
(1) job engine cleaning
Job engine by TU task unit access share storage mark reset, will unit_map (TU task unit to cache
The mapping of side-play amount) in each item be all set to 0, and call reset function and reset each job order in TU task unit queue
Relief area and renewal that unit is corresponding collect.
(2) external memory diagram data loads
Job engine restarts magnetic disc i/o, re-reads external memory diagram data collection, loads diagram data collection to shared buffer memory.
(3) figure processes task loading
The figure only using same data set processes task, could use the mode executed in parallel of shared buffer.
As in figure 2 it is shown, job engine traversal TU task unit queue, process task, work for being complete initialized figure
It is its scatter stage as engine and the gather stage sets up a thread respectively, and add it in thread pool.Job order
During unit's queue not empty, repeat step (3), otherwise, perform step (4).
(4) figure processes tasks carrying
In TU task unit, scatter thread loops calls getNestEdge function and obtains side information from memory cache,
Then perform user-defined algorithm, when needs update summit state when, call addUpdate function add to updating concentration
Adding message, in shared buffer memory, all limits of diagram data collection are terminated when being accessed.Gather step cycle calls getUpdate
Function is collected and is updated the message concentrated, and performs user-defined algorithm, and application updates amendment summit state set, at all message quilts
Terminate when having accessed.
(5) figure processes task unloading
As it is shown on figure 3, job engine traversal TU task unit sequence, task is processed for completed figure, its result is write
Remove from TU task unit queue in output file and by it, and delete its task identification in shared buffer memory；For not
The figure completed processes task, if not receiving termination signal, then saves it in entrance next iteration in task queue, no
Then, it is removed from system.During TU task unit queue not empty, repeat step (5), otherwise, terminate this iterative process.
Below in conjunction with embodiment, the present invention is further described, it is considered to the situation of a degree of parallelism N=2.
Introducing figure as a example by signal source shortest path and process the execution process of task, the diagram data of input is initial with each node
State diagram such as Fig. 4 a) shown in.
Before reading in diagram data, algorithm performs initialization operation, the distance value of source point x is set to 0, remaining vertex distance
Value is set to the most infinite.After initialization completes, each limit, the such as Article 1 that scatter thread processing system successively is read in
Limit (x, y, 12), owing to y summit distance now is the most infinite, then generate more new information (id=y, distance=12,
Pre_vertex=x) joining renewal to concentrate, gather thread receives this more by getUpdate function from updating concentration
New information, finds that the distance in updating is less than in current vertex status field the most infinite, then the distance updating y node is pushed up with forerunner
Point；When processing limit (z, y, 3), owing to the distance value of now z is the most infinite, y.distance<z.distance+ (z->y)
.weight, algorithm does not produce renewal operation；Limit (x, z, 7) can produce the renewal to z summit, first round iteration complete after system
State such as Fig. 4 b) shown in.
Taking turns in iteration second, limit (z, y, 3) can produce a more new information for summit y, is revised as from 12 by distance value
Less 10, and limit (x, y, 12) and limit (x, z, 7) is all without producing more new information, regards as the limit to algorithm " useless ", such as figure
Shown in 4c).
When system finds that in next iteration the renewal message number generated is 0, stop algorithm, export result.
Job engine traversal TU task unit sequence, processes task for completed figure, its result is write output file
In and it is removed from TU task unit queue, and delete its task identification in shared buffer memory；For at the figure that is not fully complete
Reason task, if not receiving termination signal, then saves it in entrance next iteration in task queue, otherwise, by it from being
System removes.
As it will be easily appreciated by one skilled in the art that and the foregoing is only presently preferred embodiments of the present invention, not in order to
Limit the present invention, all any amendment, equivalent and improvement etc. made within the spirit and principles in the present invention, all should comprise
Within protection scope of the present invention.
Claims (8)
1. a Multi-Task Graph processing method based on I/O duplicate removal, it is characterised in that particularly as follows:
Use special I/O thread to read diagram data from external memory by asynchronous system, and be loaded into the shared buffer of internal memory；
Executed in parallel uses the figure of same diagram data collection to process task, visits from shared buffer during figure processes tasks carrying
Ask diagram data.
Multi-Task Graph processing method based on I/O duplicate removal the most according to claim 1, it is characterised in that process at figure and appoint
The diagram data of shared buffer is conducted interviews during performing mark by business, and all figures are processed the diagram data that task all accessed
It is invalid to be labeled as, and then the diagram data read in new from external memory is replaced the invalid diagram data in shared buffer.
Multi-Task Graph processing method based on I/O duplicate removal the most according to claim 1 and 2, it is characterised in that for task
In queue, completed figure processes task, it is removed from TU task unit queue, and deletes it in internal memory shared buffer
The access identities of diagram data；Figure for being not fully complete in task queue processes task, if not receiving termination signal, is then protected
There is entrance next iteration in task queue, otherwise, it is removed from task queue.
Multi-Task Graph processing method based on I/O duplicate removal the most according to claim 1 and 2, it is characterised in that when will calculate
Between process task temporary respite more than predetermined computation time threshold and priority less than the figure of predetermined priority threshold value, empty in system
Idle is recovered to perform again.
5. a Multi-Task Graph processing system based on I/O duplicate removal, it is characterised in that include job engine, memory cache and appoint
Business unit；
Job engine, is used for using special I/O thread to read diagram data from external memory by asynchronous system, and is loaded into internal memory altogether
Enjoy relief area；Load and unload TU task unit and executing tasks parallelly unit, access from shared buffer in the process of implementation
Diagram data；
Memory cache, is used for caching diagram data；
TU task unit, is used for performing figure and processes task.
Multi-Task Graph processing system based on I/O duplicate removal the most according to claim 5, it is characterised in that described work is drawn
Hold up and be additionally operable to process figure the diagram data that core buffer accessed by task and be identified, all figure tasks were all accessed
Diagram data is labeled as invalid, then the diagram data read in new from external memory is replaced the invalid diagram data in shared buffer memory.
Multi-Task Graph processing system based on I/O duplicate removal the most according to claim 5, it is characterised in that described work is drawn
Hold up and be additionally operable to task is processed for completed figure in task queue, it is removed from TU task unit queue, and it is right to delete it
The access identities of data in memory cache；Figure for being not fully complete in task queue processes task, if not receiving termination signal,
Then save it in entrance next iteration in task queue, otherwise, it is removed from task queue.
Multi-Task Graph processing system based on I/O duplicate removal the most according to claim 5, it is characterised in that described work is drawn
Hold up and be additionally operable to less than the figure of predetermined priority threshold value, are processed task more than predetermined computation time threshold and priority the calculating time
Temporary respite, recovers when the system free time to perform again.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610399043.0A CN106095552B (en) | 2016-06-07 | 2016-06-07 | A kind of Multi-Task Graph processing method and system based on I/O duplicate removal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610399043.0A CN106095552B (en) | 2016-06-07 | 2016-06-07 | A kind of Multi-Task Graph processing method and system based on I/O duplicate removal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106095552A true CN106095552A (en) | 2016-11-09 |
| CN106095552B CN106095552B (en) | 2019-06-28 |
Family
ID=57227426
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610399043.0A Active CN106095552B (en) | 2016-06-07 | 2016-06-07 | A kind of Multi-Task Graph processing method and system based on I/O duplicate removal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106095552B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107944044A (en) * | 2017-12-15 | 2018-04-20 | 四川长虹电器股份有限公司 | Message queue big data duplicate removal treatment method |
| CN109522102A (en) * | 2018-09-11 | 2019-03-26 | 华中科技大学 | A kind of multitask external memory ideograph processing method based on I/O scheduling |
| CN110767291A (en) * | 2019-10-15 | 2020-02-07 | 武汉联影医疗科技有限公司 | Medical image processing method, apparatus and storage medium |
| CN112988064A (en) * | 2021-02-09 | 2021-06-18 | 华中科技大学 | Concurrent multitasking-oriented disk image processing method |
| CN113721987A (en) * | 2021-09-02 | 2021-11-30 | 海光信息技术股份有限公司 | Instruction execution method and instruction execution device |
| CN114968102A (en) * | 2022-05-27 | 2022-08-30 | 广州安凯微电子股份有限公司 | Data caching method, device and system, computer equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1672136A (en) * | 2002-07-31 | 2005-09-21 | 思科技术公司 | Method and apparatus for randomized cache entry replacement |
| CN101441655A (en) * | 2008-12-24 | 2009-05-27 | 深圳市迅雷网络技术有限公司 | Method and apparatus for reading data |
| CN104572501A (en) * | 2015-01-08 | 2015-04-29 | 北京航空航天大学 | Access trace locality analysis-based shared buffer optimization method in multi-core environment |
| CN105049374A (en) * | 2015-08-25 | 2015-11-11 | 广东欧珀移动通信有限公司 | Method and device for scheduling download task, and mobile terminal |
| CN105487838A (en) * | 2015-11-23 | 2016-04-13 | 上海交通大学 | Task-level parallel scheduling method and system for dynamically reconfigurable processor |
-
2016
- 2016-06-07 CN CN201610399043.0A patent/CN106095552B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1672136A (en) * | 2002-07-31 | 2005-09-21 | 思科技术公司 | Method and apparatus for randomized cache entry replacement |
| CN101441655A (en) * | 2008-12-24 | 2009-05-27 | 深圳市迅雷网络技术有限公司 | Method and apparatus for reading data |
| CN104572501A (en) * | 2015-01-08 | 2015-04-29 | 北京航空航天大学 | Access trace locality analysis-based shared buffer optimization method in multi-core environment |
| CN105049374A (en) * | 2015-08-25 | 2015-11-11 | 广东欧珀移动通信有限公司 | Method and device for scheduling download task, and mobile terminal |
| CN105487838A (en) * | 2015-11-23 | 2016-04-13 | 上海交通大学 | Task-level parallel scheduling method and system for dynamically reconfigurable processor |
Non-Patent Citations (3)
| Title |
|---|
| KOLLER R,ET AL.,: "I/O Deduplication:Utilizing content similarity to improve I/O performance", 《ACM TRANSACTIONS ON STORAGE》 * |
| MAO B, ET AL.: "POD: Performance Oriented I/O Deduplication for Primary Storage Systems in the Cloud", 《PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM, 2014 IEEE, INTERNATIONAL》 * |
| 李静，: "基于多核的任务调度策略研究", 《中国优秀硕士学位论文全文数据库信息科技辑（月刊）》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107944044A (en) * | 2017-12-15 | 2018-04-20 | 四川长虹电器股份有限公司 | Message queue big data duplicate removal treatment method |
| CN109522102A (en) * | 2018-09-11 | 2019-03-26 | 华中科技大学 | A kind of multitask external memory ideograph processing method based on I/O scheduling |
| CN109522102B (en) * | 2018-09-11 | 2022-12-02 | 华中科技大学 | Multitask external memory mode graph processing method based on I/O scheduling |
| CN110767291A (en) * | 2019-10-15 | 2020-02-07 | 武汉联影医疗科技有限公司 | Medical image processing method, apparatus and storage medium |
| CN112988064A (en) * | 2021-02-09 | 2021-06-18 | 华中科技大学 | Concurrent multitasking-oriented disk image processing method |
| CN112988064B (en) * | 2021-02-09 | 2022-11-08 | 华中科技大学 | Concurrent multitask-oriented disk graph processing method |
| CN113721987A (en) * | 2021-09-02 | 2021-11-30 | 海光信息技术股份有限公司 | Instruction execution method and instruction execution device |
| CN113721987B (en) * | 2021-09-02 | 2022-07-05 | 海光信息技术股份有限公司 | Instruction execution method and instruction execution device |
| CN114968102A (en) * | 2022-05-27 | 2022-08-30 | 广州安凯微电子股份有限公司 | Data caching method, device and system, computer equipment and storage medium |
| CN114968102B (en) * | 2022-05-27 | 2023-10-13 | 广州安凯微电子股份有限公司 | Data caching method, device, system, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106095552B (en) | 2019-06-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106095552B (en) | A kind of Multi-Task Graph processing method and system based on I/O duplicate removal | |
| Zhang et al. | Riffle: Optimized shuffle service for large-scale data analytics | |
| CN110704360B (en) | Graph calculation optimization method based on heterogeneous FPGA data flow | |
| US20220129408A1 (en) | Data actor and data processing method thereof | |
| WO2018120171A1 (en) | Method, device and system for executing stored procedure | |
| Bansal et al. | Dynamic task-scheduling in grid computing using prioritized round robin algorithm | |
| CN110032450B (en) | Large-scale deep learning method and system based on solid-state disk extended memory | |
| CN109656898B (en) | Distributed large-scale complex community detection method and device based on node degree | |
| CN104239134A (en) | Method and device for managing tasks of many-core system | |
| Liu et al. | Optimizing shuffle in wide-area data analytics | |
| US11269687B2 (en) | Map reduce using coordination namespace hardware acceleration | |
| CN109144749A (en) | A method of it is communicated between realizing multiprocessor using processor | |
| CN110222410B (en) | Electromagnetic environment simulation method based on Hadoop MapReduce | |
| CN107992358A (en) | A kind of asynchronous IO suitable for the outer figure processing system of core performs method and system | |
| US20210255793A1 (en) | System and method for managing conversion of low-locality data into high-locality data | |
| Chen et al. | Data prefetching and eviction mechanisms of in-memory storage systems based on scheduling for big data processing | |
| Zhang et al. | A locally distributed mobile computing framework for dnn based android applications | |
| Ranbirsingh et al. | Distributed neural networks using tensorflow over multicore and many-core systems | |
| Huang et al. | Survey of external memory large-scale graph processing on a multi-core system | |
| Sarma et al. | CASH: compiler assisted hardware design for improving DRAM energy efficiency in CNN inference | |
| Huang et al. | Improving speculative execution performance with coworker for cloud computing | |
| Xu et al. | Evaluation and trade-offs of graph processing for cloud services | |
| CN109388733B (en) | Optimization method for graph-oriented data processing engine | |
| Xiu et al. | A memory management strategy based on task requirement for in-memory computing | |
| CN115774736B (en) | NUMA architecture time-varying graph processing method and device for data delay transmission |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |