CN110347482A - OLTP affairs binding rule and queuing model improved method based on OLTPShare - Google Patents
OLTP affairs binding rule and queuing model improved method based on OLTPShare Download PDFInfo
- Publication number
- CN110347482A CN110347482A CN201910651419.6A CN201910651419A CN110347482A CN 110347482 A CN110347482 A CN 110347482A CN 201910651419 A CN201910651419 A CN 201910651419A CN 110347482 A CN110347482 A CN 110347482A
- Authority
- CN
- China
- Prior art keywords
- affairs
- newly arrived
- queue
- existing
- oltp
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/466—Transaction processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Abstract
OLTP affairs binding rule and queuing model improved method based on OLTPShare, it belongs to OLTP Transaction processing technology field.It is continually changing that the present invention solves the problems, such as that the queuing model for the low efficiency and existing OLTPShare that affairs existing for existing OLTP affairs binding rule are combined and handled cannot cope with well workload.The present invention has abandoned the mode of manual addition affairs binding rule, proposes can automatically generate based on machine learning clustering algorithm and the at runtime binding rule of dynamic adjustment, and the efficiency that affairs can be combined and be handled improves 50% or more；And the invention proposes one kind to add, annexable queuing model, and this queuing model can change to cope with potential workload in automatically adjusting parameter setting at runtime.Present invention could apply to OLTP Transaction processing technology fields.
Description
Technical field
The invention belongs to OLTP Transaction processing technology fields, and in particular to a kind of OLTP (On- based on OLTPShare
Line Transaction Processing, Transaction Processing process) affairs binding rule and queuing model improved method.
Background technique
89% CMDBAC different statement character strings that workload needed for online project possessed of increasing income are extremely limited.Cause
This, OLTP workload, which is shared in reality, possesses huge application potential.
OLTPShare (On-Line Transaction Processing (workloads) Share, OLTP workload
Technology of sharing) a kind of batch processing strategy for OLTP workload is provided, it carries out the affairs of all arrivals based on queue
Batch processing, to make it possess maximum handling capacity.But the judgement institute foundation before the OLTPShare batch processing of current version
Affairs binding rule need manually to formulate, cause affairs combine and processing efficiency it is lower.Moreover, in practical applications
The queuing model of OLTPShare cannot cope with the continually changing situation of workload well.
Summary of the invention
The purpose of the present invention is the efficiency to solve the combination and processing of affairs existing for existing OLTP affairs binding rule
The queuing model of low and existing OLTPShare cannot cope with the continually changing problem of workload well.
The technical solution adopted by the present invention to solve the above technical problem is: the OLTP affairs based on OLTPShare combine
Rule and queuing model improved method, method includes the following steps:
Step 1: checking whether newly arrived affairs are single claimed alternative by SE thread, if inspection when new affairs are reached
The fruit that comes to an end is that newly arrived affairs are not single claimed alternatives, then newly arrived affairs are not combinative affairs, then affairs one by one
It executes；
If it is single claimed alternative that inspection result, which is newly arrived affairs, by the binding rule of SE thread pool existing queue,
Judge whether to be added to newly arrived affairs in existing any queue；
Step 2: if newly arrived affairs are added in existing queue C, newly arrived affairs and present queue C
In affairs carry out batch processing together；
If newly arrived affairs are not added in existing any queue, using newly arrived affairs as individual one
A new queue, the combination of the affairs applied to subsequent arrival；
Step 3: detecting the affairs B's being newly added in the queue when the affairs A in some queue is just when processed
It arrives, checks whether affairs A is compatible with affairs B；If affairs A is compatible with affairs B, step 4 is continued to execute, if affairs A and thing
Business B is incompatible, then continues to execute step 5；
Step 4: if affairs A is compatible with affairs B, parallel processing transaction A and affairs B, and record when affairs B is added
Initial position, if initial position when affairs B is added is not the head of the table of affairs B, when the tail portion for the table for reaching affairs B
When, continuation is started to query from the head of the table of affairs B, is terminated when reaching the initial position of affairs B of record；
Step 5: waiting affairs A after treatment, then start to process affairs B if affairs A and affairs B are incompatible.
The beneficial effects of the present invention are: the invention proposes based on OLTPShare OLTP affairs binding rule and queue
Model refinement method, the present invention have abandoned the mode of manual addition affairs binding rule, have proposed to be based on machine learning clustering algorithm
Can automatically generate and at runtime dynamic adjustment binding rule, can by affairs combine and handle efficiency improve
50% or more；And the invention proposes one kind to add, annexable queuing model, and this queuing model can be at runtime
Automatically adjusting parameter setting changes to cope with potential workload.
Detailed description of the invention
Fig. 1 is the process of the OLTP affairs binding rule based on OLTPShare and queuing model improved method of the invention
Figure.
Specific embodiment
Specific embodiment 1: as shown in Figure 1, the OLTP affairs described in present embodiment based on OLTPShare combine
Rule and queuing model improved method, method includes the following steps:
Step 1: being checked when new affairs are reached by SE thread (SQLExecutor executes the actuator of SQL statement)
Whether newly arrived affairs are single claimed alternative, new to reach if it is not single claimed alternative that inspection result, which is newly arrived affairs,
Affairs be not combinative affairs, then one by one affairs execute；
Affairs, which execute, one by one refers to that is, affairs execute one by one according to traditional OLTP transaction methods realization；
If inspection result is that newly arrived affairs are single claimed alternatives, by the binding rule of SE thread pool existing queue
(merge rules) judges whether to be added to newly arrived affairs in existing any queue；
Existing queue binding rule refers to: affairs binding rule as defined in artificial.It can be determined that by affairs binding rule
Whether one newly arrived affairs and other affairs have similitude, if so, then directly newly arrived affairs can be added
Into queue similar with its, batch processing is carried out together, if it is not, can not directly be added to newly arrived affairs existing
It deposits in queue, is individually performed；
Step 2: if newly arrived affairs are added in existing queue C, newly arrived affairs and present queue C
In affairs carry out batch processing together；
If newly arrived affairs are not added in existing any queue, using newly arrived affairs as individual one
A new queue, the combination of the affairs applied to subsequent arrival；
If continuous n (n is artificial regulation, for example, 30,40) a affairs are all judged as being not belonging to any one existing team
Column, then it can be assumed that the binding rule for existing queue there is a problem that need to modify binding rule, binding rule substantially has
Two kinds of modification modes:
1, change the feature that compares, such as become 3 from 5 or feature quantity is constant, but the spy that replacement is compared
Property, such as change (e, f, g) into from (a, b, c).
2, change the method for calculating distance, such as become manhatton distance etc. from Euclidean distance.
By modified rule, the combination of the new affairs applied to subsequent arrival.
Both the above modification mode can be used all every time, every time after modification, can also need weight only with one of which
It is new to calculate classification.
According to the new feature for being continuously added affairs, binding rule is constantly updated, promotes the processing speed of subsequent similar affairs.
Step 3: detecting the affairs B's being newly added in the queue when the affairs A in some queue is just when processed
It arrives, checks whether affairs A is compatible with affairs B；If affairs A is compatible with affairs B, step 4 is continued to execute, if affairs A and thing
Business B is incompatible, then continues to execute step 5；
Step 4: if affairs A is compatible with affairs B, parallel processing transaction A and affairs B, and record when affairs B is added
Initial position, if initial position when affairs B is added is not the head of the table of affairs B, when the tail portion for the table for reaching affairs B
When, continuation is started to query from the head of the table of affairs B, is terminated when reaching the initial position of affairs B of record；
The table t of affairs B is not actually started to query every time from gauge outfit, looked into when inquiring this table
It is random for asking starting point, so, in order to traverse whole table, the initial position of the table of affairs B is recorded, if initial position is table
Head traversal is over whole table then directly traverse table tail.If the head of initial position not instead of table, in table
Between, then when traversing table tail, the previous section of starting point is there is no traversing, so will be since the head of table time
It goes through, until the initial position of the table of affairs B.
Step 5: waiting affairs A after treatment, then start to process affairs B if affairs A and affairs B are incompatible.
Specific embodiment 2: present embodiment be to described in specific embodiment one based on the OLTP of OLTPShare
Affairs binding rule is further limited with queuing model improved method, described to judge whether to be added to newly arrived affairs now
In any queue deposited, detailed process are as follows:
The similarity that affairs in newly arrived affairs and existing queue are calculated by clustering method, to determine whether will newly arrive
The affairs reached are added in existing any queue.
The transaction characteristics to be compared are set first, for example, if comparing affairs by 5 features, with one
A five dimensional vector indicates each affairs, X={ x1,x2,x3,x4,x5, then in a coordinate system, the thing in each queue
Business, is centainly within close proximity, it can be seen that apparent cluster (cluster).A y value is manually set, represents the maximum cluster of permission
Radius by the center of each cluster of calculating, then calculates the distance (calculating there are many kinds of distance here at new affairs and each cluster center
Method, such as Euclidean distance, manhatton distance etc.), the cluster with minimum range is taken, and the distance is less than y, as this is newly arrived
Classification belonging to the affairs reached.Otherwise, if the distance is greater than y, which classification of newly arrived affairs is all not belonging to, i.e., will be new
The affairs of arrival are put into existing any queue, are individually performed.
Specific embodiment 3: present embodiment be to described in specific embodiment one or two based on OLTPShare's
OLTP affairs binding rule is further limited with queuing model improved method, described to check that newly arrived affairs are by SE thread
No is single claimed alternative, specifically:
Check newly arrived affairs whether only by a SQL (Structured Query Language) by SE thread
Sentence composition, if being only made of a SQL statement, newly arrived affairs are single claimed alternatives, and otherwise, newly arrived affairs are not
It is single claimed alternative.
Specific embodiment 4: present embodiment be to described in specific embodiment three based on the OLTP of OLTPShare
Affairs binding rule is further limited with queuing model improved method, and whether the inspection affairs A is compatible with affairs B, tool
Body are as follows:
If affairs A does not conflict with the data that affairs B is related to, affairs A is compatible with affairs B, otherwise, affairs A and affairs B
It is incompatible.
Same data are read and write as conflict, two write operations write same data as conflict, read same data and do not conflict, write not
The data difference for not conflicting, reading and writing with data does not also conflict.
Specific embodiment 5: present embodiment be to described in specific embodiment four based on the OLTP of OLTPShare
Affairs binding rule is further limited with queuing model improved method, in the step 3, parallel processing between each queue.
Above-mentioned example of the invention only explains computation model and calculation process of the invention in detail, and is not to this
The restriction of the embodiment of invention.It for those of ordinary skill in the art, on the basis of the above description can be with
It makes other variations or changes in different ways, all embodiments can not be exhaustive here, it is all to belong to the present invention
The obvious changes or variations extended out of technical solution still in the scope of protection of the present invention.
Claims (5)
1. OLTP affairs binding rule based on OLTPShare and queuing model improved method, which is characterized in that this method includes
Following steps:
Step 1: checking whether newly arrived affairs are single claimed alternative by SE thread, if checking knot when new affairs are reached
Fruit is that newly arrived affairs are not single claimed alternatives, then newly arrived affairs are not combinative affairs, then affairs execute one by one；
If inspection result is that newly arrived affairs are single claimed alternatives, by the binding rule of SE thread pool existing queue, judgement
Whether newly arrived affairs are added in existing any queue；
Step 2: if newly arrived affairs are added in existing queue C, in newly arrived affairs and present queue C
Affairs carry out batch processing together；
It is new using newly arrived affairs as individual one if newly arrived affairs are not added in existing any queue
Queue, the combination of the affairs applied to subsequent arrival；
Step 3: the arrival for the affairs B being newly added in the queue is detected when the affairs A in some queue is just when processed,
Check whether affairs A is compatible with affairs B；If affairs A is compatible with affairs B, step 4 is continued to execute, if affairs A and affairs B are not
It is compatible, then continue to execute step 5；
Step 4: if affairs A is compatible with affairs B, parallel processing transaction A and affairs B, and record starting when affairs B is added
Position, if initial position when affairs B is added is not the head of the table of affairs B, when reaching the tail portion of table of affairs B, after
It is continuous to be started to query from the head of the table of affairs B, terminate when reaching the initial position of affairs B of record；
Step 5: waiting affairs A after treatment, then start to process affairs B if affairs A and affairs B are incompatible.
2. the OLTP affairs binding rule according to claim 1 based on OLTPShare and queuing model improved method,
It is characterized in that, described to judge whether to be added to newly arrived affairs in existing any queue, detailed process are as follows:
The similarity that affairs in newly arrived affairs and existing queue are calculated by clustering method, to determine whether by newly arrived
Affairs are added in existing any queue.
3. the OLTP affairs binding rule according to claim 1 or 2 based on OLTPShare and queuing model improvement side
Method, which is characterized in that it is described to check whether newly arrived affairs are single claimed alternative by SE thread, specifically:
Check whether newly arrived affairs are only made of a SQL statement by SE thread, if being only made of a SQL statement,
Then newly arrived affairs are single claimed alternatives, and otherwise, newly arrived affairs are not single claimed alternatives.
4. the OLTP affairs binding rule according to claim 3 based on OLTPShare and queuing model improved method,
It is characterized in that, whether the inspection affairs A is compatible with affairs B, specifically:
If affairs A does not conflict with the data that affairs B is related to, affairs A is compatible with affairs B, and otherwise, affairs A and affairs B be not simultaneous
Hold.
5. the OLTP affairs binding rule according to claim 4 based on OLTPShare and queuing model improved method,
It is characterized in that, in the step 3, parallel processing between each queue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910651419.6A CN110347482B (en) | 2019-07-18 | 2019-07-18 | OLTP transaction combination rule and queue model improvement method based on OLTPShare |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910651419.6A CN110347482B (en) | 2019-07-18 | 2019-07-18 | OLTP transaction combination rule and queue model improvement method based on OLTPShare |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110347482A true CN110347482A (en) | 2019-10-18 |
| CN110347482B CN110347482B (en) | 2021-07-23 |
Family
ID=68178919
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910651419.6A Active CN110347482B (en) | 2019-07-18 | 2019-07-18 | OLTP transaction combination rule and queue model improvement method based on OLTPShare |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110347482B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1928872A (en) * | 2005-09-09 | 2007-03-14 | 国际商业机器公司 | Device and method for writing data into disc by dynamic switching |
| US20170193077A1 (en) * | 2015-04-08 | 2017-07-06 | Huawei Technologies Co., Ltd. | Load balancing for large in-memory databases |
| CN107122354A (en) * | 2016-02-24 | 2017-09-01 | 华为技术有限公司 | Affairs perform method, apparatus and system |
| CN108459919A (en) * | 2018-03-29 | 2018-08-28 | 中信百信银行股份有限公司 | A kind of distributed transaction processing method and device |
| CN109451076A (en) * | 2018-12-29 | 2019-03-08 | 乐蜜有限公司 | A kind of the merging treatment method, apparatus and electronic equipment of network request |
| CN109992359A (en) * | 2019-03-28 | 2019-07-09 | 深圳市创联时代科技有限公司 | A kind of transaction scheduling method |
-
2019
- 2019-07-18 CN CN201910651419.6A patent/CN110347482B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1928872A (en) * | 2005-09-09 | 2007-03-14 | 国际商业机器公司 | Device and method for writing data into disc by dynamic switching |
| US20170193077A1 (en) * | 2015-04-08 | 2017-07-06 | Huawei Technologies Co., Ltd. | Load balancing for large in-memory databases |
| CN107122354A (en) * | 2016-02-24 | 2017-09-01 | 华为技术有限公司 | Affairs perform method, apparatus and system |
| CN108459919A (en) * | 2018-03-29 | 2018-08-28 | 中信百信银行股份有限公司 | A kind of distributed transaction processing method and device |
| CN109451076A (en) * | 2018-12-29 | 2019-03-08 | 乐蜜有限公司 | A kind of the merging treatment method, apparatus and electronic equipment of network request |
| CN109992359A (en) * | 2019-03-28 | 2019-07-09 | 深圳市创联时代科技有限公司 | A kind of transaction scheduling method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110347482B (en) | 2021-07-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Kohavi et al. | Useful feature subsets and rough set reducts | |
| CN108600321A (en) | A kind of diagram data storage method and system based on distributed memory cloud | |
| CN103853766B (en) | A kind of on-line processing method and system towards stream data | |
| CN101329676B (en) | Data paralleling abstracting method and apparatus and database system | |
| CN107198884B (en) | Method for efficiently loading game scenes | |
| CN110555506A (en) | gradient self-adaptive particle swarm optimization method based on group aggregation effect | |
| CN105975587A (en) | Method for organizing and accessing memory database index with high performance | |
| CN107798346A (en) | Quick track similarity matching method based on Frechet distance threshold | |
| CN112947300A (en) | Virtual measuring method, system, medium and equipment for processing quality | |
| CN105657064A (en) | Swift load balancing method based on virtual node storage optimization | |
| CN111191786A (en) | Transfer learning algorithm based on active learning | |
| CN112634457A (en) | Point cloud simplification method based on local entropy of Hausdorff distance and average projection distance | |
| CN107153680A (en) | A kind of method and system of the online expanding node of distributed memory database | |
| CN110347482A (en) | OLTP affairs binding rule and queuing model improved method based on OLTPShare | |
| CN115544033B (en) | Method, device, equipment and medium for updating check repeat vector library and checking repeat data | |
| CN116014764B (en) | Distributed energy storage optimization processing method and device | |
| CN108427773B (en) | Distributed knowledge graph embedding method | |
| KR102136111B1 (en) | Particle resampling method for particle filter | |
| CN106648891A (en) | MapReduce model-based task execution method and apparatus | |
| CN103714121A (en) | Index record management method and device | |
| CN105095455A (en) | Data connection optimization method and data operation system | |
| CN115496133A (en) | Density data stream clustering method based on self-adaptive online learning | |
| CN107644422A (en) | Massive point cloud Delaunay triangulation network construction method based on dual space data tissue | |
| CN111368060B (en) | Self-learning method, device and system for conversation robot, electronic equipment and medium | |
| Li et al. | Noisy optimization by evolution strategies with online population size learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |