CN104238999B - A kind of method for scheduling task and device based on horizontal partitioning distributed data base - Google Patents

A kind of method for scheduling task and device based on horizontal partitioning distributed data base Download PDF

Info

Publication number
CN104238999B
CN104238999B CN201310231497.3A CN201310231497A CN104238999B CN 104238999 B CN104238999 B CN 104238999B CN 201310231497 A CN201310231497 A CN 201310231497A CN 104238999 B CN104238999 B CN 104238999B
Authority
CN
China
Prior art keywords
thread
database instance
task
database
numbering
Prior art date
Application number
CN201310231497.3A
Other languages
Chinese (zh)
Other versions
CN104238999A (en
Inventor
李彦超
Original Assignee
阿里巴巴集团控股有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 阿里巴巴集团控股有限公司 filed Critical 阿里巴巴集团控股有限公司
Priority to CN201310231497.3A priority Critical patent/CN104238999B/en
Publication of CN104238999A publication Critical patent/CN104238999A/en
Application granted granted Critical
Publication of CN104238999B publication Critical patent/CN104238999B/en

Links

Abstract

The invention discloses a kind of method for scheduling task and device based on horizontal partitioning distributed data base, including:When receiving waiting task, the first partitions of database ID of extraction waiting task carrying;According to partitions of database ID and database instance mapping relations, the first database instance that the first partitions of database ID belongs to is determined;According to the mapping relations of database instance and thread, first thread corresponding to the first database instance is determined, wherein, a database instance is only corresponding with a thread;Waiting task is distributed into first thread execution realizes the purpose of the mutual wait or interlocking that avoid executing tasks parallelly.

Description

A kind of method for scheduling task and device based on horizontal partitioning distributed data base

Technical field

The present invention relates to database field, more particularly to a kind of task scheduling side based on horizontal partitioning distributed data base Method and device.

Background technology

Horizontal partitioning distributed data base:Refer to splitting data to be stored in different data by horizontal mode Storehouse example(Or SCHEMA, database schema)In a kind of distributed data base solution, in disparate databases example Data are uncorrelated.Typically, a database instance can have multiple partitions of database, and each partitions of database is by unique data Library partition ID is identified, and has certain mapping relations between database instance and the ID of its partitions of database included.

At present, the method for the task scheduling based on horizontal partitioning distributed data base is, when receiving a certain task, meeting Any idle thread is taken out from thread pool and performs the task.Certainly, in order to improve database processing ability, might have a lot Task is assigned in the thread performed parallel and completed.

But for horizontal partitioning distributed data base, because the data in database instance are split as some numbers According to library partition, a usual task is directed to a partitions of database, therefore often occurs for the multiple of database instance Task is individually allocated to the thread performed parallel, and then the causing to perform parallel in a database instance of the task is due to affairs Characteristic and mutually wait or interlock, wait now can influence the execution efficiency of task, so as to influence the handling capacity of parallel task And disposal ability.

The content of the invention

In view of this, it is a primary object of the present invention to provide a kind of task tune based on horizontal partitioning distributed data base Method and device is spent, to avoid the purpose of the mutual wait or interlocking of executing tasks parallelly.

In the one side of the embodiment of the present invention, there is provided a kind of task scheduling based on horizontal partitioning distributed data base Method, for example, this method can include:

When receiving waiting task, the first partitions of database ID of extraction waiting task carrying;

According to partitions of database ID and database instance mapping relations, the first of the first partitions of database ID ownership is determined Database instance;

According to the mapping relations of database instance and thread, first thread corresponding to the first database instance is determined, wherein, One database instance is only corresponding with a thread;

Waiting task is distributed into first thread to perform.

In the other side of the embodiment of the present invention, there is provided a kind of task based on horizontal partitioning distributed data base is adjusted Device is spent, for example, the device can include:

Receiving unit:It is configured to when receiving waiting task, the first database that extraction waiting task carries Partition id;

First computing unit:It is configured to when receiving waiting task, the first number that extraction waiting task carries According to library partition ID;

Second computing unit:The mapping relations according to database instance and thread are configured to, determine the first database reality First thread corresponding to example, wherein, a database instance is only corresponding with a thread;

Allocation unit:It is configured to waiting task distributing to first thread execution.

It can be seen that the present invention has the advantages that:

In the present invention, the Task Assigned Policy of a thread is only corresponded to due to taking a database instance, is being connect When receiving the first waiting task, the first partitions of database ID of extraction the first waiting task carrying, according to partitions of database ID and database instance mapping relations, the first database instance of the first partitions of database ID ownership is determined, and according to data The mapping relations of storehouse example and thread, it is same that the task for same database instance, different pieces of information library partition is assigned to this Performed in thread corresponding with database instance, so as to which the transaction operation that maximum possible allows parallel executory task to be called exists On different database instances, i.e. the parallel task of a database instance be less than or equal to 1, can avoid completely parallel task it Between interlocking and wait.

Brief description of the drawings

In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are only this Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can be with Other accompanying drawings are obtained according to these accompanying drawings.

Fig. 1 is one of method flow schematic diagram provided according to embodiments of the present invention;

Fig. 2 is the two of the method flow schematic diagram provided according to embodiments of the present invention;

Fig. 3 is the three of the method flow schematic diagram provided according to embodiments of the present invention;

Fig. 4 is the apparatus structure schematic diagram provided according to embodiments of the present invention.

Embodiment

In order to facilitate the understanding of the purposes, features and advantages of the present invention, it is below in conjunction with the accompanying drawings and specific real Mode is applied to be described in further detail the embodiment of the present invention.

The method for scheduling task based on horizontal partitioning distributed data base of the embodiment of the present invention, suitable for horizontal partitioning point The scheduling of the batch tasks of cloth database.Below, the application scenarios of horizontal partitioning distributed data base are carried out simply first Introduce:

In horizontal partitioning distributed data base, many database instances can be included, in some instances it may even be possible to more than 50.And Many partitions of database are included again in one database instance, therefore, there is substantial amounts of number in horizontal partitioning distributed data base According to library partition, there are a large amount of processing tasks for being directed to partitions of database.In order to improve the execution efficiency of task, it will usually using some Thread parallel performs these tasks.

And being performed parallel in a database instance for task may mutually be waited or interlocked due to transactional attribute, have In consideration of it, the present invention provides the method for scheduling task as follows based on horizontal partitioning distributed data base:

It is shown in Figure 1, it is a kind of schematic flow sheet of embodiment one of Connection inquiring method provided by the invention, as schemed institute Show, this method embodiment can include:

S101, when receiving waiting task, extraction waiting task carry the first partitions of database ID;

S102, the mapping relations according to partitions of database ID and database instance, determine that the first partitions of database ID belongs to The first database instance;

Wherein, according to the strategy of partitions of database, the mapping relations of partitions of database ID and database instance can be:Number According to quantity complementations of the library partition ID to database instance, its gained remainder is the database instance of partitions of database ID ownership Numbering., can be by quantity complementations of the first partitions of database ID to database instance according to the strategy of the partitions of database, its In, gained remainder is the numbering of the first database instance, so that it is determined that the first partitions of database ID that waiting task carries returns First database instance of category.Certainly, it is not limited to which a kind of this partitions of database strategy, the present invention is not in this regard by any Limitation.

S103, the mapping relations according to database instance and thread, determine first thread corresponding to the first database instance, Wherein, a database instance is only corresponding with a thread;

Wherein, the mapping relations of database instance and thread can be mapped according to needs are implemented, for example, can be according to Certain algorithm calculates mapping relations, in advance can also record the mapping relations of the numbering of database instance and the numbering of thread In mapping table, by inquiring about mapping table, the numbering of first thread corresponding to the numbering of the first database instance is determined, Numbering of the numbering of one of database instance only with a thread is by mapping relations.

It should be noted that a database instance is only corresponding with a thread, i.e. the mapping of database instance and thread Relation can be many-to-one relation, man-to-man relation.

S104, waiting task is distributed to first thread execution.

The method provided using the above embodiment of the present invention, it can will be directed to same database instance, disparate databases point The task in area is assigned in same thread corresponding with database instance and performed, one of database instance only with a thread It is corresponding, so as to which maximum possible allows transaction operation that parallel executory task called on different database instances, i.e., one The parallel task of individual database instance is less than or equal to 1, can avoid the interlocking and wait between parallel task completely.

In an alternative embodiment of the invention, in order to quickly determine database instance corresponding to partitions of database ID, and quickly Thread corresponding to database instance is determined, further to improve the handling capacity of parallel task and disposal ability, the present invention also proposes Following examples, referring to Fig. 2, the flow of embodiment two signal of the method for scheduling task based on horizontal partitioning distributed data base Figure, as shown in fig. 2, the embodiment can include:

S201, when receiving waiting task, extraction waiting task carry the first partitions of database ID;

S202, by quantity complementations of the first partitions of database ID to database instance, wherein, gained remainder is the first data The numbering of storehouse example;

S203, by quantity complementation of the numbering of the first database instance to thread, wherein, gained remainder is first thread Numbering;

It should be noted that in this embodiment, the mapping relations of database instance and thread are by by database instance Quantity complementation of the numbering to thread determine, i.e.,:Quantity complementation of the numbering of database instance to thread, gained remainder are the number According to the numbering of thread corresponding to the numbering of storehouse example.

For example, the quantity of its thread can be configured according to the quantity of database instance and the handling capacity of task, example Such as, when the quantity of database instance is N, the quantity of thread can be the positive integer M less than N, and the numbering of database instance can be with From 0 to N-1 serial numbers, the numbering of thread can be from 0 to M-1 serial numbers.

S204, the numbering according to first thread, waiting task is distributed into first thread and performed.

The method provided using the above embodiment of the present invention, according to the tactful by the first partitions of database ID of partitions of database Quantity complementation to database instance, it may be determined that the database instance of partitions of database ID ownership, take again and database point The consistent thread allocation strategy of area's strategy, by quantity complementation of the numbering of database instance to thread, determine database reality A thread corresponding to example is unique so that for same database instance, different pieces of information library partition task be assigned to it is same with Performed in thread corresponding to database instance, so as to which maximum possible allows the transaction operation that parallel executory task is called not On same database instance, the interlocking and wait between parallel task can be avoided completely.Also, due to distributing the strategy of thread The algorithm to quantity complementation is taken with the strategy of partitions of database, determines database instance corresponding to partitions of database ID, And determine that the process of thread corresponding to database instance is quicker, further increase the handling capacity and processing energy of parallel task Power.

Below, above-described embodiment is illustrated with reference to a possible application scenarios.For example, it is assumed that one is pressed Customer ID (That is partitions of database ID)Carry out the horizontal partitioning distributed data base of partitions of database.In the horizontal partitioning distributed data base In, customer data is distributed in 5 database instances, and database instance numbering is(0、1、2、3、4).The plan of partitions of database For slightly Customer ID to database example quantity complementation, gained remainder is the database where the partitions of database of Customer ID mark Example.The mapping relations of database instance and thread are:Quantity complementation of the numbering of database instance to thread, gained remainder are The numbering of thread corresponding to the database instance.Assuming that number of threads is 4, thread number is(0、1、2、3).

Assuming that the Customer ID that one the received waiting task for freezing client carries is 4, another freezes client's The Customer ID that waiting task carries is 9, and the Customer ID that another waiting task for freezing client carries is 10;

By Customer ID 4 to the remainder of database example quantity 5, the numbering for determining database instance is 4, by database instance The remainders of quantity 4 of 4 pairs of threads of numbering, then it is 4 to freeze treating for client by ID it is determined that corresponding thread is the thread that numbering is 0 Processing task gives the thread process that thread number is 0;

By Customer ID 9 to the remainder of database example quantity 5, the numbering for determining database instance is 4, by database instance The remainders of quantity 4 of 4 pairs of threads of numbering, then it is 9 to freeze treating for client by ID it is determined that corresponding thread is the thread that numbering is 0 Processing task gives the thread process that thread number is 0;

By Customer ID 10 to the remainder of database example quantity 5, the numbering for determining database instance is 1, by database instance 1 pair of thread of numbering quantity 4 remainder, be then 10 to freeze client's by ID it is determined that corresponding thread is the thread that numbering is 1 Waiting task gives the thread process that thread number is 1.

From the example, same and data are assigned to for the task of same database instance, different pieces of information library partition Performed in thread corresponding to the example of storehouse, so as to which maximum possible allows the transaction operation that parallel executory task is called different On database instance, interlocking and wait between parallel task can be avoided completely.

Moreover, the present invention also proposes that following examples to improve the task concurrency of different task type, referring to Fig. 3, are The schematic flow sheet of embodiment three of method for scheduling task provided by the invention based on horizontal partitioning distributed data base, such as 3 figures Shown, the embodiment can include:

S301, when receiving waiting task, extraction waiting task carry the first partitions of database ID, Yi Jiti The first task type for taking waiting task to carry;

S302, the mapping relations according to partitions of database ID and database instance, determine that the first partitions of database ID belongs to The first database instance;

S303, the mapping relations according to task type and thread, determine thread corresponding to first task type;

S304, the mapping relations according to database instance thread corresponding with first task type, determine the first database First thread corresponding to example, wherein, a database instance is only corresponding with a thread in the task type;

For example, the number of thread corresponding to first task type according to the mapping relations of task type and thread, can be determined Amount;By quantity complementation of the numbering of the first database instance to thread corresponding to first task type, wherein, gained remainder is the The numbering of one thread;

S305, waiting task is distributed to first thread execution.

For example, number-mark corresponding with first task type, by first thread can be inquired from thread pool Thread;Waiting task is distributed to the thread of the number-mark corresponding with first task type, by first thread.

The embodiment is distinguished with above-mentioned other embodiment, can also extract the first task class of waiting task carrying Type, task type also have mapping relations with thread, i.e. a task type corresponds to some threads for belonging to the task type.Pin To a kind of task type, a database instance is only corresponding with a thread in the task type.So as to according to task Type, same database instance, different pieces of information library partition the task of same task type is assigned in same thread and held OK, the transaction operation that the task of the parallel execution of same task type is called can be made on different database instances, While improving the task concurrency of different task type, the task of the same task type for same database instance is avoided Interlocking caused by parallel and wait, improve the handling capacity and disposal ability of parallel task.

For example, with reference to above-mentioned possible application scenarios, it is assumed that the quantity for freezing thread corresponding to the task type of client is 4, thread number is(Freeze 0, freeze 1, freeze 2, freeze 3), it is assumed that inquire about the quantity of thread corresponding to the task type of client For 4, thread number is(Inquiry 0, inquiry 1, inquiry 2, inquiry 3).

Assuming that the Customer ID that one the received waiting task for freezing client carries is 4, another inquiry client's The Customer ID that waiting task carries is 9;

By Customer ID 4 to the remainder of database example quantity 5, the numbering for determining database instance is 4, by database instance The remainders of quantity 4 of 4 pairs of threads of numbering, then it is 4 to freeze client by Customer ID it is determined that corresponding thread is the thread that numbering is 0 Waiting task give thread number " to freeze 0 " thread process;

By Customer ID 9 to the remainder of database example quantity 5, the numbering for determining database instance is 4, by database instance The remainder of quantity 4 of 4 pairs of threads of numbering, it is determined that corresponding thread be numbering be 0 thread, then by Customer ID be 9 inquiry client Waiting task give thread number for " inquiry 2 " thread process.

, can be according to unrelated mutually, without waiting for progress it should be noted that the task type described in above-described embodiment three Division, from the example, even if two waiting tasks be directed to same database instance, but what is performed due to it is two Unrelated task type, by the embodiment shown in above-mentioned Fig. 3, the difference that the two can be belonged to same database instance is appointed The waiting task of service type distributes to two threads that can be performed parallel, so as to improve the task of different task type simultaneously Row, the handling capacity and disposal ability of parallel task are improved, while the same task class of same database instance can be avoided again Interlocking caused by the tasks in parallel of type and wait.

It is shown in Figure 4, it is a kind of task scheduling apparatus based on horizontal partitioning distributed data base provided by the invention Structural representation, as illustrated, the device can include:

Receiving unit 401:It may be configured to when receiving waiting task, the first of extraction waiting task carrying Partitions of database ID;

First computing unit 402:It may be configured to when receiving waiting task, extraction waiting task carries First partitions of database ID;

Wherein, according to the strategy of partitions of database, first computing unit 402:It may be configured to the first data Quantity complementations of the library partition ID to database instance, wherein, gained remainder is the numbering of the first database instance.Certainly, not A kind of this partitions of database strategy is confined to, the present invention is unrestricted in this regard.

Second computing unit 403:The mapping relations according to database instance and thread are may be configured to, determine the first number According to first thread corresponding to the example of storehouse, wherein, a database instance is only corresponding with a thread;

Wherein, the mapping relations of database instance and thread can be mapped according to needs are implemented, for example, in order to quick Database instance corresponding to partitions of database ID, and thread corresponding to quick determination database instance are determined, further to improve The handling capacity and disposal ability of parallel task, mapping relations can be calculated according to certain algorithm, e.g., second computing unit 403:It may be configured to quantity complementation of the numbering of the first database instance to thread, wherein, gained remainder is First Line The numbering of journey.

Or reflected for example, it is also possible in advance be recorded in the mapping relations of the numbering of database instance and the numbering of thread Penetrate in relation table, e.g., second computing unit 403:It may be configured to inquire about mapping table, determine the first database reality The numbering of first thread corresponding to the numbering of example, wherein have recorded the numbering and thread of database instance in the mapping table Numbering mapping relations, wherein, the numbering of the numbering of database instance only with a thread has mapping relations.

It should be noted that a database instance is only corresponding with a thread, i.e. the mapping of database instance and thread Relation can be many-to-one relation, man-to-man relation.

Allocation unit 404:It may be configured to waiting task distributing to first thread execution.

The device provided using the above embodiment of the present invention, it can will be directed to same database instance, disparate databases point The task in area is assigned in same thread corresponding with database instance and performed, one of database instance only with a thread It is corresponding, so as to which maximum possible allows transaction operation that parallel executory task called on different database instances, i.e., one The parallel task of individual database instance is less than or equal to 1, can avoid the interlocking and wait between parallel task completely.

In an alternative embodiment of the invention, the receiving unit 401:It can also be configured to receiving waiting task When, the first task type of extraction waiting task carrying;

Correspondingly, second computing unit 403:The mapping relations according to task type and thread are may be configured to, Determine thread corresponding to first task type;According to the mapping relations of database instance thread corresponding with first task type, Determine first thread corresponding to the first database instance, wherein, a database instance only with a line in the task type Journey is corresponding.

For example, second computing unit 403:The mapping relations according to task type and thread are may be configured to, really Determine the quantity of thread corresponding to first task type;By the numbering of the first database instance to thread corresponding to first task type Quantity complementation, wherein, gained remainder be first thread numbering;

The allocation unit 404:May be configured to inquire from thread pool it is corresponding with first task type, by The thread of the number-mark of one thread;It is corresponding with first task type, by first thread that waiting task is distributed to this The thread of number-mark.

, can be according to unrelated mutually, without waiting for being drawn it should be noted that the task type described in the embodiment Point, from the example, even if two waiting tasks be directed to same database instance, but what is performed due to it is two nothings The task type of pass, the waiting task that the two can be belonged to the different task type of same database instance distribute to two The individual thread that can be performed parallel, so as to improve the task concurrency of different task type, improve the handling capacity of parallel task And disposal ability, while can avoid interlocking and waiting caused by the tasks in parallel of the same task type of same database instance again Treat.

As seen through the above description of the embodiments, those skilled in the art can be understood that above-mentioned implementation All or part of step in example method can add the mode of required general hardware platform to realize by software.Based on such Understand, the part that technical scheme substantially contributes to prior art in other words can be in the form of software product Embody, the computer software product can be stored in storage medium, such as ROM/RAM, magnetic disc, CD, including some Instruct to cause a computer equipment(Can be the network services such as personal computer, server, or WMG Equipment, etc.)Perform the method described in some parts of each embodiment of the present invention or embodiment.

It should be noted that each embodiment in this specification is described by the way of progressive, each embodiment it Between identical similar part mutually referring to what each embodiment stressed is the difference with other embodiment. For device embodiment, because it is substantially similar to embodiment of the method, so describe fairly simple, correlation Place illustrates referring to the part of embodiment of the method.Device embodiment described above is only schematical, wherein conduct The unit that separating component illustrates can be or may not be it is physically separate, can be as the part that unit is shown or Person may not be physical location, you can with positioned at a place, or can also be distributed on multiple NEs.Can root Factually border needs to select some or all of module therein realize the purpose of this embodiment scheme.Ordinary skill Personnel are without creative efforts, you can to understand and implement.

Moreover, herein, such as first and second or the like relational terms are used merely to an entity or behaviour Make with another entity or operation make a distinction, and not necessarily require or imply these entities or operate between exist it is any this Kind actual relation or order.Moreover, term " comprising ", "comprising" or its any other variant are intended to nonexcludability Include so that process, method, article or equipment including a series of elements not only include those key elements, but also Including the other element being not expressly set out, or also include for this process, method, article or equipment intrinsic want Element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that wanted including described Other identical element also be present in the process of element, method, article or equipment.

The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the scope of the present invention.It is all Any modification, equivalent substitution and improvements made within the spirit and principles in the present invention etc., are all contained in protection scope of the present invention It is interior.

Claims (12)

  1. A kind of 1. method for scheduling task based on horizontal partitioning distributed data base, it is characterised in that including:
    When receiving waiting task, the first partitions of database ID of extraction waiting task carrying;
    According to partitions of database ID and database instance mapping relations, the first data that the first partitions of database ID belongs to are determined Storehouse example;
    According to the mapping relations of database instance and thread, first thread corresponding to the first database instance is determined, wherein, one Database instance is only corresponding with a thread;
    Waiting task is distributed into first thread to perform.
  2. 2. according to the method for claim 1, it is characterised in that the reflecting according to partitions of database ID and database instance Relation is penetrated, determines that the first database instance corresponding to the first partitions of database ID includes:
    By quantity complementations of the first partitions of database ID to database instance, wherein, gained remainder is the first database instance Numbering.
  3. 3. according to the method for claim 2, it is characterised in that the mapping relations according to database instance and thread, Determine that first thread corresponding to the first database instance includes:
    By quantity complementation of the numbering of the first database instance to thread, wherein, gained remainder is the numbering of first thread.
  4. 4. according to the method for claim 2, it is characterised in that the mapping relations according to database instance and thread, Determine that first thread corresponding to the first database instance includes:
    Mapping table is inquired about, the numbering of first thread corresponding to the numbering of the first database instance is determined, wherein the mapping The mapping relations of the numbering of database instance and the numbering of thread are have recorded in relation table, wherein, the volume of a database instance Numbering number only with a thread has mapping relations.
  5. 5. according to the method described in claim any one of 1-4, it is characterised in that when receiving waiting task, also extract The first task type that waiting task carries;
    The mapping relations according to database instance and thread, determine that first thread corresponding to the first database instance includes:
    According to the mapping relations of task type and thread, thread corresponding to first task type is determined;
    According to the mapping relations of database instance thread corresponding with first task type, determine corresponding to the first database instance First thread, wherein, a database instance is only corresponding with a thread in the task type.
  6. 6. according to the method for claim 5, it is characterised in that the mapping relations according to database instance and thread, Determine that first thread corresponding to the first database instance includes:
    According to the mapping relations of task type and thread, the quantity of thread corresponding to first task type is determined;
    By quantity complementation of the numbering of the first database instance to thread corresponding to first task type, wherein, gained remainder is The numbering of first thread;
    Waiting task, which is distributed to first thread and performed, to be included:
    The thread of number-mark corresponding with first task type, by first thread is inquired from thread pool;
    Waiting task is distributed to the thread of the number-mark corresponding with first task type, by first thread.
  7. A kind of 7. task scheduling apparatus based on horizontal partitioning distributed data base, it is characterised in that including:
    Receiving unit:It is configured to when receiving waiting task, the first partitions of database that extraction waiting task carries ID;
    First computing unit:The mapping relations according to partitions of database ID and database instance are configured to, determine the first data First database instance of library partition ID ownership;
    Second computing unit:The mapping relations according to database instance and thread are configured to, determine the first database instance pair The first thread answered, wherein, a database instance is only corresponding with a thread;
    Allocation unit:It is configured to waiting task distributing to first thread execution.
  8. 8. device according to claim 7, it is characterised in that first computing unit:It is configured to the first data Quantity complementations of the library partition ID to database instance, wherein, gained remainder is the numbering of the first database instance.
  9. 9. device according to claim 8, it is characterised in that second computing unit:It is configured to the first data Quantity complementation of the numbering of storehouse example to thread, wherein, gained remainder is the numbering of first thread.
  10. 10. device according to claim 8, it is characterised in that second computing unit:Inquiry mapping is configured to close It is table, determines the numbering of first thread corresponding to the numbering of the first database instance, wherein have recorded in the mapping table The mapping relations of the numbering of database instance and the numbering of thread, wherein, the numbering of a database instance only with a thread Numbering have mapping relations.
  11. 11. according to the device described in claim any one of 7-10, it is characterised in that the receiving unit:It is also configured to When receiving waiting task, the first task type of extraction waiting task carrying;
    Second computing unit:The mapping relations according to task type and thread are configured to, determine first task type pair The thread answered;According to the mapping relations of database instance thread corresponding with first task type, the first database instance is determined Corresponding first thread, wherein, a database instance is only corresponding with a thread in the task type.
  12. 12. device according to claim 11, it is characterised in that second computing unit:It is configured to according to task The mapping relations of type and thread, determine the quantity of thread corresponding to first task type;By the numbering of the first database instance Quantity complementation to thread corresponding to first task type, wherein, gained remainder is the numbering of first thread;
    The allocation unit:It is configured to inquire volume corresponding with first task type, by first thread from thread pool Number mark thread;Waiting task is distributed into the number-mark corresponding with first task type, by first thread Thread.
CN201310231497.3A 2013-06-09 2013-06-09 A kind of method for scheduling task and device based on horizontal partitioning distributed data base CN104238999B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310231497.3A CN104238999B (en) 2013-06-09 2013-06-09 A kind of method for scheduling task and device based on horizontal partitioning distributed data base

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201310231497.3A CN104238999B (en) 2013-06-09 2013-06-09 A kind of method for scheduling task and device based on horizontal partitioning distributed data base
HK15102900.8A HK1202660A1 (en) 2013-06-09 2015-03-23 Method for task schedule of distributed database with horizontal partitioning and device thereof

Publications (2)

Publication Number Publication Date
CN104238999A CN104238999A (en) 2014-12-24
CN104238999B true CN104238999B (en) 2017-11-17

Family

ID=52227152

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310231497.3A CN104238999B (en) 2013-06-09 2013-06-09 A kind of method for scheduling task and device based on horizontal partitioning distributed data base

Country Status (2)

Country Link
CN (1) CN104238999B (en)
HK (1) HK1202660A1 (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104572262B (en) * 2014-12-27 2018-09-04 北京奇虎科技有限公司 A kind of task executing method and device
CN106406845A (en) * 2015-08-03 2017-02-15 阿里巴巴集团控股有限公司 A task processing method and device
CN105700824B (en) * 2015-12-29 2018-12-07 浪潮(北京)电子信息产业有限公司 A kind of the read operation request processing method and system of storage system mechanical hard disk
CN105912386A (en) * 2016-05-03 2016-08-31 深圳市永兴元科技有限公司 Thread management method and system
CN106095589B (en) * 2016-06-30 2019-04-09 浪潮卓数大数据产业发展有限公司 A kind of method, apparatus and system for distributing subregion
CN108401131A (en) * 2018-05-01 2018-08-14 北京学易科技有限公司 Thread processing method and device, server

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1632794A (en) * 2004-12-27 2005-06-29 中国科学院软件研究所 Method for mapping XML type to RDB table
US20100114826A1 (en) * 2008-10-24 2010-05-06 Microsoft Corporation Configuration management in distributed data systems
US8326825B2 (en) * 2010-11-05 2012-12-04 Microsoft Corporation Automated partitioning in parallel database systems

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
分布式数据库集群系统ZD-DDB的设计与实现;马明理;《CNKI电子数据库》;20060215;第19-20,27,50页和附图3.1 *

Also Published As

Publication number Publication date
CN104238999A (en) 2014-12-24
HK1202660A1 (en) 2015-10-02

Similar Documents

Publication Publication Date Title
Sumbaly et al. The big data ecosystem at linkedin
US8984085B2 (en) Apparatus and method for controlling distributed memory cluster
Afrati et al. Optimizing joins in a map-reduce environment
Zhang et al. Transaction chains: achieving serializability with low latency in geo-distributed storage systems
Papailiou et al. H2RDF: adaptive query processing on RDF data in the cloud.
US8200705B2 (en) Method and apparatus for applying database partitioning in a multi-tenancy scenario
Santos et al. Real-time data warehouse loading methodology
Liao et al. Multi-dimensional index on hadoop distributed file system
Mondal et al. Managing large dynamic graphs efficiently
US8868711B2 (en) Dynamic load balancing in a scalable environment
US20150134796A1 (en) Dynamic partitioning techniques for data streams
EP2801894B1 (en) Adaptive tile framework
US8386473B2 (en) Process architecture for elastic stateful shared nothing system
US20130144866A1 (en) Fault tolerance based query execution
US20140358977A1 (en) Management of Intermediate Data Spills during the Shuffle Phase of a Map-Reduce Job
CN103620601A (en) Joining tables in a mapreduce procedure
US20080222634A1 (en) Parallel processing for etl processes
US20150134626A1 (en) Partition-based data stream processing framework
US9223875B2 (en) Real-time distributed in memory search architecture
US20180189367A1 (en) Data stream ingestion and persistence techniques
Liu et al. Priority-based consolidation of parallel workloads in the cloud
US9276959B2 (en) Client-configurable security options for data streams
US9753980B1 (en) M X N dispatching in large scale distributed system
JP5990192B2 (en) Filtering query data in the data store
KR20060047700A (en) Combining multidimensional expressions and data mining extensions to mine olap cubes

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1202660

Country of ref document: HK

GR01 Patent grant
GR01 Patent grant
REG Reference to a national code

Ref country code: HK

Ref legal event code: GR

Ref document number: 1202660

Country of ref document: HK