CN101566986A - Method and device for processing data in online business processing - Google Patents
Method and device for processing data in online business processing Download PDFInfo
- Publication number
- CN101566986A CN101566986A CNA2008100904871A CN200810090487A CN101566986A CN 101566986 A CN101566986 A CN 101566986A CN A2008100904871 A CNA2008100904871 A CN A2008100904871A CN 200810090487 A CN200810090487 A CN 200810090487A CN 101566986 A CN101566986 A CN 101566986A
- Authority
- CN
- China
- Prior art keywords
- sublist
- data
- field
- cut apart
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for processing data in online business processing, which comprises the following steps: a data partitioning field of a target data table is confirmed according to a business characteristic; and the target data table is divided into a plurality of subtables according to the data partitioning field. The invention uses the method that the core data table in an online business processing system is divided into a plurality of subtables to enable the data amount of the subtables after division to be lower than that of the original core data table. When DDL operation is carried out to the subtables, the occupied space of the temporary table is small, and blocked talks are few, thus the pressure born by the system is greatly reduced, and the maintainability and the manageability of the core data table of the database are improved.
Description
Technical field
The present invention relates to the Transaction Processing field, relate in particular to a kind of processing data in online business processing and device, and, data enquire method in a kind of Transaction Processing and device.
Background technology
Transaction Processing (On-Line Transaction Processing, OLTP) be meant and utilize computer network, the business processing computer equipment or the network that are distributed in diverse geographic location are connected with the Service Management Center network, so that on any one network node, can carry out unified, real-time business processing activity or customer service.OLTP is the main application of traditional relevant database, under the OLTP environment, because professional high speed development, in database, there is the data volume of some professional core data tables very big, reach the number more than hundred million grades, and because the concurrent transaction processing feature of OLTP, cause the concurrent visit capacity of these big data quantity tables of data high, thereby when safeguarding these tables of data, all will be chosen in professional low peak period usually, as carrying out attended operation at dead of night, the content of operation comprises: add field, create index, rebuild index or the like.Although selected the window of a professional ebb, but some problems can take place, for example, in the ORACLE database, cause the dependency structure query language (Structured Quevy Language, statement execution plan SQL) changes; Take a large amount of temporary table space (temporary tablespace) when creating index or rebuilding index; Perhaps carry out database definition language (Data DefinitionLanguage, block a large amount of sessions when DDL) operating, and these risks all can be to bringing on a disaster property of core database problem, slow as database response application server requests, and then cause application server responses client's request slow, can't provide stable service for the client.
In the prior art, another very objective reality be: along with the high speed development of the OLTP of enterprise business, some professional core data tables of high concurrent visit concentrate on the separate unit database, these separate unit databases all are to use a storage usually, even high-end storage, per second I/O operation (the Input/Output Per Second that can bear, IOPS) peak value is also extremely limited, therefore storage is easy to become the bottleneck (bottleneck) of entire database system, causes the slack-off result of database read write data; And use in the process of high-end storage in reality, growth along with IOPS, inner buffer memory (Cache) hit rate of storage also can be more and more lower, further increased the weight of the disk read-write pressure of storage, the request time of database response application server is more and more longer, and then has influence on the normal operation of business event.
Current in the application of OLTP, generally there are two kinds of data processing methods, first kind for all to be placed on professional core data in the common table, and second kind for all to be placed on professional core data in the ORACLE partition table.
When adopting first method, because professional core data all is placed in the common table, so carrying out the DDL operation, during such as the interpolation field, to cause all hard parsings of SQL statement generation of Zhang Putong epiphase pass therewith, in the concurrent OLTP of height uses, cause the librarycache pin of occurrence of large-area, library cache load lock waits for, has influence on the visit of all sessions to these table data; Simultaneously, if statistics is inaccurate, the hard parsing produces wrong SQL executive plan, and all sessions that move this SQL all will be subjected to this influence, the high pressure that causes system to bear so; In addition, when creating or rebuild index,, cause the DDL running time long, higher risk is arranged, and in the process of operation, can take a large amount of temporary tables space usually and be used for ordering because data volume is huge, bigger to the database performance impact.
When adopting second method, in existing hash (Hash) partitioned mode, Oracle has carried out subregion for each tables of data, and wherein each subregion (Partition) is exactly a section (segment), can create local index (local index) in these sections; When extraneous data query, Oracle at first carries out the hash computing according to the subregion condition, thereby can determine the subregion that will visit, and then uses local index with data query in these subregions.
The structural representation of second method can be with reference to figure 1, comprise the partition table in application layer and the oracle database, use for the external world, when the user when Oracle submits SQL to, use and do not need to know the data of being visited are at which subregion or on which subregion, only need know the upper strata whole object of these subregions, promptly a table name gets final product.So in Fig. 1, and subregion (promptly be partition 1, partition 2 ..., partition N) be transparent for using, when application layer is visited these data, only need the object of an integral body to get final product.Yet, adopt second method to have following defective: at first, when professional core data table being carried out the DDL operation, with influence all sessions to this tables of data visit, choking phenomenon is serious, and risk is still very high, if and DDL when operation statistics is inaccurate, the hard parsing produces wrong SQL executive plan, and all sessions that move this SQL all will be subjected to this influence, the high pressure that causes system to bear so; Moreover, the hash algorithm of Oracle partition table is a black box, the visit of all data all needs through this unique path, for the concurrent OLTP of height, the visit of this class core table is again extremely frequent, so carry out this hash computing, also will consume the CPU of the database server of a great deal of; In addition, the partition table of Oracle can not integration across database, and when storage became the bottleneck of total system, the horizontal extension of data was relatively poor.
Summary of the invention
Technical matters to be solved by this invention provides processing data in online business processing, carries out data with the core data table to enterprise and splits, and realizes the core data table is split as the function of one or more sublists.
Another object of the present invention has provided the data enquire method in the Transaction Processing, and the described sublist after splitting with inquiry is fast and effeciently obtained target data.
The present invention also provides the data processing in the Transaction Processing, inquiry unit, to realize the function of said method, finishes processing and inquiry to the enterprise key tables of data.
For solving the problems of the technologies described above, the embodiment of the invention provides a kind of processing data in online business processing, comprising: the data of determining target matrix according to service feature are cut apart field; Cut apart field according to described data described target matrix is split into a plurality of sublists.
Preferably, also comprise:
Described sublist is stored in one or more databases.
Preferably, described tables of data splitting step comprises: the data that read certain line item in the described target matrix are cut apart the value of field; The value of described data being cut apart field is carried out modulo operation according to default sublist quantity, and locatees current unloading sublist according to described operation result; With the record unloading of described target matrix to described sublist.
Preferably, described sublist comprises one-level sublist and secondary sublist, and described tables of data splitting step comprises: the data that read certain line item in the described target matrix are cut apart the value of field; The value of described data being cut apart field is carried out modulo operation according to default one-level sublist quantity, and locatees the one-level sublist of current unloading according to described operation result; With described record unloading to the described one-level sublist; If the data volume of described one-level sublist satisfies certain threshold value, the data that then read certain line item in the described one-level sublist are cut apart the value of field; The value of described data being cut apart field is carried out modulo operation according to default secondary sublist quantity, and locatees the secondary sublist of current unloading according to described operation result; With the record unloading of described one-level sublist to described secondary sublist.
Preferably, it is numeric type field or character type field that described data are cut apart field, and described tables of data splitting step comprises: read the value that data in the described target matrix are cut apart field; The value of described data being cut apart field is converted into sexadecimal, locatees the sublist of current unloading according to initial character; To be worth unloading accordingly to described sublist.
Preferably, described data are cut apart field and are comprised the user data identification field.
The embodiment of the invention also provides the data enquire method in a kind of Transaction Processing, comprise: the sublist at locating query data place, described sublist splits acquisition for cutting apart field according to described data to target matrix, described data are cut apart field for determining acquisition according to service feature in target matrix, and described data query is the value that corresponding described data are cut apart field; In described sublist, search the relative recording that satisfies described data query.
Preferably, described positioning step comprises: described data query is carried out modulo operation according to default sublist quantity; According to described operation result location sublist.
The embodiment of the invention also provides the data processing equipment in a kind of Transaction Processing, comprising: determining unit is used for determining that according to service feature the data of target matrix cut apart field; Split cells is used for cutting apart field according to described data described target matrix is split into a plurality of sublists.
Preferably, also comprise: storage unit is used for described sublist is stored to one or more databases.
Preferably, described split cells comprises:
First reads subelement, and the data that are used for reading described target matrix line item are cut apart the value of field;
The first operator unit, the value that is used for described data are cut apart field is carried out modulo operation according to default sublist quantity, and locatees current unloading sublist according to described operation result;
The first unloading subelement is used for the record unloading of described target matrix to described sublist.
Preferably, described sublist comprises one-level sublist and secondary sublist, and described split cells comprises:
First reads subelement, and the data that are used for reading described target matrix line item are cut apart the value of field;
The second operator unit, the value that is used for described data are cut apart field is carried out modulo operation according to default one-level sublist quantity, and locatees the one-level sublist of current unloading according to described operation result;
The second unloading subelement is used for described record unloading to described one-level sublist;
Second reads subelement, is used for when the data volume of described one-level sublist satisfies certain threshold value, and the data that read certain line item in the described one-level sublist are cut apart the value of field;
The 3rd operator unit, the value that is used for described data are cut apart field is carried out modulo operation according to default secondary sublist quantity, and locatees the secondary sublist of current unloading according to described operation result;
The 3rd unloading subelement is used for the record unloading of described one-level sublist to described secondary sublist.
Preferably, it is numeric type field or character type field that described data are cut apart field, and described split cells comprises:
Third reading is got subelement, is used for reading the value that described target matrix data are cut apart field;
The transformant unit, the value that is used for data are cut apart field is converted into sexadecimal, and locatees the sublist of current unloading according to initial character;
The 4th unloading subelement is used for and will be worth unloading accordingly to described sublist.
Preferably, described split cells is positioned at database layer.
The embodiment of the invention also provides the device of the data query in a kind of Transaction Processing, comprising:
Positioning unit, the sublist that is used for locating query data place, to be split cells cut apart field according to described data to described sublist that target matrix is split acquisition, described data are cut apart field for determining acquisition according to service feature in target matrix, and described data query is the value that corresponding described data are cut apart field;
Search the unit, be used for searching the relative recording that satisfies described data query in described sublist.
Preferably, described positioning unit comprises:
The 4th operator unit is used for described data query is carried out modulo operation according to default sublist quantity;
Receive subelement, be used to receive described operation result and sublist is positioned.
Preferably, described positioning unit is positioned at application layer.
Compared with prior art, the present invention has the following advantages:
The present invention is cut apart field by the data of determining target matrix according to service feature, cutting apart field according to described data then splits described target matrix in a plurality of sublists, thereby realize the core data table of enterprise is split the function of sublist, because it is less through the sublist data volume that splits, so the temporary table space hold is less when carrying out the DDL operation, the session of obstruction is also considerably less; And this type of professional core data table is used always the also shortening of safeguarding greatly of running time, make the pressure that bears of system obviously reduce; In addition, the present invention's delivery hash algorithm when positioning can be finished in application layer, having reduced database layer database server CPU consumes, when the executive plan change takes place when, the pressure that effectively reduction system bears, the maintainability and the manageability of professional core data table in the raising database.
Description of drawings
Fig. 1 is the system architecture synoptic diagram of prior art hash subregion;
Fig. 2 is the process flow diagram of processing data in online business processing embodiment 1 of the present invention;
Fig. 3 is the process flow diagram of processing data in online business processing embodiment 2 of the present invention;
Fig. 4 is the process flow diagram of processing data in online business processing embodiment 3 of the present invention;
Fig. 5 is the process flow diagram of processing data in online business processing embodiment 4 of the present invention;
Fig. 6 is the structured flowchart of the data processing equipment embodiment in the Transaction Processing of the present invention;
Fig. 7 is the process flow diagram of the data enquire method embodiment in the Transaction Processing of the present invention;
Fig. 8 is the structured flowchart of the data query device embodiment in the Transaction Processing of the present invention.
Embodiment
For above-mentioned purpose of the present invention, feature and advantage can be become apparent more, the present invention is further detailed explanation below in conjunction with the drawings and specific embodiments.
The present invention can be used in numerous general or special purpose computingasystem environment or the configuration.For example: personal computer, server computer, handheld device or portable set, plate equipment, multicomputer system, the system based on microprocessor, set top box, programmable consumer-elcetronics devices, network PC, small-size computer, mainframe computer, comprise distributed computing environment of above any system or equipment or the like.
The present invention can describe in the general context of the computer executable instructions of being carried out by computing machine, for example program module.Usually, program module comprises the routine carrying out particular task or realize particular abstract, program, object, assembly, data structure or the like.Also can in distributed computing environment, put into practice the present invention, in these distributed computing environment, by by communication network connected teleprocessing equipment execute the task.In distributed computing environment, program module can be arranged in the local and remote computer-readable storage medium that comprises memory device.
One of core idea of the embodiment of the invention is, determines that according to service feature the data of target matrix are cut apart field; And cut apart field described target matrix is split into a plurality of sublists according to described data; Described sublist can be stored in one or more databases then; Process method of the present invention is carried out each sublist after data split, data volume all has only the 1/n (n is the sublist number after splitting) of former target matrix data volume when not splitting, and concurrent visit capacity also has only original 1/n, so when described sublist being used always when safeguarding, not only the running time can shorten, and can significantly reduce the session of obstruction, thus reduced the pressure that system bears, improved the maintainability and the manageability of professional core data table in the database.
With reference to figure 2, show the process flow diagram of a kind of processing data in online business processing embodiment 1 of the present invention, can may further comprise the steps:
Step 201: the data of determining target matrix according to service feature are cut apart field;
Step 202: cut apart field according to described data described target matrix is split into a plurality of sublists.
Because the purpose of among the present invention data being handled is split as a series of sublist with a core business tables of data exactly, therefore be exactly that specified data is cut apart field splitting previous important preliminary work, can be only limited in the sublist when guaranteeing that later data access is most of.
In the present embodiment, the method for cutting apart field according to the service feature specified data can be selected arbitrarily according to actual needs by those skilled in the art.As user data identification field, geographical location information etc.Particularly, described Data Identification field can comprise the field that number ID, user string ID etc. can the unique identification user data.For example, suppose that certain shopping website comprises commodity list, the commodity of seller issue all can be in this commodity list, and are relevant with commodity list, comprise following business function:
(1) commodity on the block;
(2) commodity in the warehouse;
(3) commodity of search oneself;
(4) merchandise news displayed page is showed single-piece merchandise news (commodity ID).
According to the service feature that above-mentioned business function embodied, can determine that promptly it is seller ID that corresponding data are cut apart field.
Perhaps, suppose that the server of certain examination system can only be supported 500 people's online testings, yet have 10000 people to take an examination simultaneously at present, in this case, then can come a minute storehouse,, determine that it is Customs Assigned Number that corresponding data are cut apart field according to this service feature by the user.
Preferably, user data identification field and further feature field combination can also be cut apart field as data, for example, with user ID and geographical location information combination etc.
In actual applications, described target matrix is carried out after data split, preferred, can also comprise step:
Described sublist is stored in one or more databases.
Certainly, sublist not only can be stored in the database, also can be stored in a plurality of databases, and therefore, storage can not become the bottleneck of total system, thus the pressure of mitigation system effectively.
Need to prove, the embodiment that introduces among the present invention is a method of only cutting apart field at numeral, if it is character type or other type that the data after determining are cut apart field, situation and the present invention are similar, can adopt method of the present invention to carry out data processing, the present invention need not to make restriction to this.
With reference to figure 3, show processing data in online business processing embodiment 2 process flow diagrams of the present invention, described data processing method can comprise:
Step 301: the data of determining target matrix according to service feature are cut apart field;
Step 302: cut apart field according to described data described target matrix is split into a plurality of sublists.
Preferably, described step 302 can comprise following substep:
Substep 3021: the data that read certain line item in the described target matrix are cut apart the value of field;
Substep 3022: the value of described data being cut apart field is carried out delivery hash computing according to default sublist quantity, and locatees current unloading sublist according to described operation result;
Substep 3023: with the record unloading of described target matrix to described sublist.
Below by an object lesson present embodiment is described:
The hypothetical target tables of data be defined as test (id varchar2 (32), userid number ...), cutting apart field in the database layer established data is the userid field; The data handling procedure of using present embodiment is:
A1, moving cursor if vernier be a sky, then read the value of this journey record field userid during to certain line item of described target matrix;
The purpose of this step is to split target matrix test according to the userid field value, and reads the value that these data are cut apart field userid, for next step data conversion storage with the test table is prepared to new sublist;
A2, with described userid value delivery n+ 1, calculate v_flag as a result, the computing method of described v_flag are: v_flag=mod (#userid, n+1); Wherein, n+1 is default sublist quantity; Can estimate the quantity that splits sublist in advance according to the size of target matrix data volume in this example.Modulus algorithm of the present invention can be understood as, and in hash to sublist of the data with identical molds value, this mould value is not unique, in practice, and promptly can be in hash to the sublist of class data with same characteristic features.
A3, according to the value of v_flag, with the unloading of this journey record data to new sublist.
After the loading procedure of delegation's record field is finished among the target matrix test, can circulate and read record data in the former test table, be empty until described record data, show test show in all data that need split all unloading finish.
When data level being divided into a plurality of sublist through said method, because the sublist data volume after cutting apart is less, therefore to create index, rebuild index and will become than being easier to, the temporary table space hold is also less, so shortened the time of attended operation commonly used greatly, greatly reduced risk; Simultaneously, after the core data table is split as n sublist, visit capacity for these core data tables can be shunted, therefore the visit capacity on every sublist will be the 1/n of the tables of data before not splitting, in this case sublist being added the field ratio is easier to, the session of blocking is also few, and is also less relatively to the influence of using.Simultaneously, when a sublist being carried out the DDL operation, because the data of this sublist statistics is inaccurate, to resolve the new executive plan of generation firmly be wrong to the SQL that sublist is relevant therewith, but simultaneously because visit capacity only is the 1/n of original visit capacity, the pressure of system is just in the scope that can bear, thereby the number of users of influence significantly reduces, and makes the manageability of system strengthen greatly.
Adopt the data processing method among the present invention, after the core business tables of data carried out splitting the first time, if the data volume of every sublist is still bigger, can also proceed to split for the second time to described sublist, this fractionation can be called second layer delivery hash algorithm.Wherein, the sublist that splits is for the first time called the one-level sublist, the sublist after splitting is for the second time called the secondary sublist.Specifically can show the process flow diagram of processing data in online business processing embodiment 3 of the present invention with reference to figure 4, described sublist comprises one-level sublist and secondary sublist, and described data processing method can may further comprise the steps:
Step 401: the data of determining target matrix according to service feature are cut apart field;
Step 402: cut apart field according to described data described target matrix is split into a plurality of sublists.
Preferably, described step 402 can comprise following substep:
Substep 4021: the data that read certain line item in the described target matrix are cut apart the value of field;
Substep 4022: the value of described data being cut apart field is carried out modulo operation according to default one-level sublist quantity, and locatees the one-level sublist of current unloading according to described operation result;
Substep 4023: with described record unloading to the described one-level sublist;
Substep 4024: if the data volume of described one-level sublist satisfies certain threshold value, the data that then read certain line item in the described one-level sublist are cut apart the value of field;
Substep 4025: the value of described data being cut apart field is carried out modulo operation according to default secondary sublist quantity, and locatees the secondary sublist of current unloading according to described operation result;
Substep 4026: with the record unloading of described one-level sublist to described secondary sublist.
Below by an object lesson present embodiment is described:
The hypothetical target tables of data be defined as test (id varchar2 (32), userid number ...), cutting apart field in the database layer established data is the userid field; The data handling procedure of using present embodiment is:
B1, default one-level sublist quantity are 16, according to the method in the foregoing description 2 data segmented word section userid field value are carried out delivery 16 computings, locate 16 one-level sublists of current unloading then according to operation result;
B2, will be to described 16 one-level sublists through the data conversion storage in the target matrix that splits, as, sublist _ 00, sublist _ 01, sublist _ 02 ... sublist _ 15;
B3, to the one-level sublist split finish after, judge whether the data volume of described one-level sublist satisfies certain threshold value, described threshold value with split after the number of one-level sublist relevant with data volume, can determine to obtain according to the situation of the data volume grow of one-level sublist;
In the practical application, the user sets this threshold value according to the sublist number and the contained data volume of sublist of own needs.If satisfy this threshold value then represent that the data volume grow of one-level sublist is still very fast, continue the one-level sublist is carried out the secondary deconsolidation process with regard to needs, enter step B4; If the data volume grow that does not satisfy then represent the one-level sublist just finishes this data handling procedure in the scope that can allow;
B4, suppose under the one-level sublist, to continue to be provided with 3 secondary sublists, then can carry out mould 3 computings to the userid field value of described 16 sublists respectively with reference to the method among the embodiment 2 equally, and locate the secondary sublist of current unloading according to operation result; Wherein, described secondary sublist quantity also is to come as the case may be to set flexibly, among the different embodiment different quantity can be arranged, and the present invention does not do particular determination to the quantity of one-level sublist and secondary sublist;
After the secondary fractionation through present embodiment, then one-level sublist _ 00 can be split as secondary sublist 00_00,00_01 and 00_02, one-level sublist _ 01 can be split as secondary sublist 01_00,01_01 and 01_02 ..., one-level sublist _ 15 can be split as secondary sublist 15_00,15_01 and 15_02;
B5, with the record unloading of described one-level sublist to described secondary sublist.
Need to prove, in the present embodiment to the one-level sublist read and the unloading process also can circulate, be sky up to the one-level sublist that reads, showing that the one-level sublist has split finishes, all unloading is to corresponding secondary sublist for the one-level sublist that need to split, and this moment, the data handling procedure of present embodiment finished.From abovementioned steps we as can be seen, described data processing method has good extendability, the quantity of one-level sublist can split once more according to the growth rate of what and data volume of sublist data volume.
In actual applications, sometimes target matrix can be huger, after described target matrix being carried out the fractionation second time, the still very fast phenomenon of Data Growth that occurs the secondary sublist probably, at this moment, can described secondary sublist be split as three grades of sublists according to preceding method, described split process is called fractionation for the third time, the rest may be inferred, sublist can comprise one-level sublist, secondary sublist, three grades of sublists after splitting target matrix, ..., N level sublist, the present invention does not do special qualification to the progression of sublist.
Certainly, the method that those skilled in the art adopt any data to split all is feasible, for example, cut apart under the situation that field is numeric type field or character type field in described data, the present invention can be converted into sexadecimal with data segmented word segment value, and the initial character according to conversion data carries out data processing then.Specifically can show the process flow diagram of processing data in online business processing embodiment 4 of the present invention with reference to figure 5, when adopting this method, described data processing method can may further comprise the steps:
Step 501: the data of determining target matrix according to service feature are cut apart field;
Step 502: cut apart field according to described data described target matrix is split into a plurality of sublists.
Preferably, step 502 can comprise following substep:
Substep 5021: read the value that data in the described target matrix are cut apart field;
Substep 5022: the value of described data being cut apart field is converted into sexadecimal, locatees the sublist of current unloading according to initial character;
Substep 5023: will be worth unloading accordingly to described sublist.
Below by an object lesson present embodiment is described:
The hypothetical target tables of data is table t, and wherein to cut apart field be nick to data, promptly is that the nick field according to table t splits, and nick is 32 CHARs, and wherein each character all uses 16 systems to represent.
Data among the table t are as follows:
mysql＞select nick from t；
| nick |
| 8f37218b86d69cced1f1c63be326449b 26885d4e325de0e7a62e0a2cb1b302d9 0cc175b9c0f1b6a831c399e269772661 900150983cd24fb0d6963f7d28e17f72 e2fc714c4727ee9395f324cd2e7f331f |
5 rows in set(0.00sec)
Data characteristics according to the nick field can know that the quantity that should preset sublist according to initial character to the fractionation of described table t is 16, wherein, and with ' 0 ', ' 1 ', ' 2 ' ..., ' e ', ' f ' is each corresponding sublist of beginning, therefore formulates following data processing rule:
Nick field initial character is 0, and deposit data is at sublist t_00;
Nick field initial character is 1, and deposit data is at sublist t_01;
Nick field initial character is 2, and deposit data is at sublist t_02;
Nick field initial character is 3, and deposit data is at sublist t_03;
...
Nick field initial character is f's, and deposit data is at sublist t_15;
To show data conversion storage described sublist t_00 extremely among the t according to described data processing rule, t_01 ..., t_15 can finish the process that target matrix t is split according to the nick field value.
With reference to figure 6, show the structured flowchart of using the device embodiment of data processing method in the Transaction Processing of the present invention, comprising:
Determining unit 601 is used for determining that according to service feature the data of target matrix cut apart field;
When carrying out data processing, at first need service feature according to the user, as user data identification field etc., specified data is cut apart field, after determining unit determines that described data are cut apart field, just send it to split cells, described split cells is cut apart field according to described data, just target matrix is split into a plurality of sublists that meet the demands.
Preferably, described device can also comprise:
Storage unit is used for described sublist is stored to one or more databases.
In the practical application, after the sublist fractionation, to store described sublist in one or more databases into, when storing a database into, the inquiry of using so sublist also is only limited in the database, when storing a plurality of database into, at first locate the database at sublist place in the time of application query, more described sublist is positioned and inquires about.
Preferably, described split cells comprises:
First reads subelement, and the data that are used for reading described target matrix line item are cut apart the value of field;
The first operator unit, the value that is used for described data are cut apart field is carried out modulo operation according to default sublist quantity, and locatees current unloading sublist according to described operation result;
The first unloading subelement is used for the record unloading of described target matrix to described sublist.
In this case, using the method that preferred embodiment shown in Figure 6 carries out data processing can may further comprise the steps:
C1, determining unit at first determine that according to service feature the data of target matrix cut apart field;
C2, first reads the value that data that subelement reads certain line item in the described target matrix are cut apart field;
The value that described data are cut apart field in C3, the first operator unit is carried out modulo operation according to default sublist quantity, and locatees current unloading sublist according to described operation result;
C4, the first unloading subelement with the record unloading of described target matrix to described sublist.
Preferably, when the sublist data volume grow after splitting is still bigger, can carry out splitting the second time to the sublist after splitting for the first time, at this moment, sublist after splitting for the first time is called the one-level sublist, and the sublist after splitting for the second time is called the secondary sublist, and described split cells comprises:
First reads subelement, and the data that are used for reading described target matrix line item are cut apart the value of field;
The second operator unit, the value that is used for described data are cut apart field is carried out modulo operation according to default one-level sublist quantity, and locatees the one-level sublist of current unloading according to described operation result;
The second unloading subelement is used for described record unloading to described one-level sublist;
Second reads subelement, is used for when the data volume of described one-level sublist satisfies certain threshold value, and the data that read certain line item in the described one-level sublist are cut apart the value of field;
The 3rd operator unit, the value that is used for described data are cut apart field is carried out modulo operation according to default secondary sublist quantity, and locatees the secondary sublist of current unloading according to described operation result;
The 3rd unloading subelement is used for the record unloading of described one-level sublist to described secondary sublist.
When the one-level sublist is split, need to satisfy a condition, promptly be that the data volume grow of one-level sublist exceeds certain threshold value, this threshold value can be set and change according to actual conditions are autonomous by the user.At this moment, second reads subelement will read delegation's record data in the one-level sublist, and the method for splitting according to aforementioned introduction splits the one-level sublist then.
In this case, using the method that preferred embodiment shown in Figure 6 carries out data processing can may further comprise the steps:
D1, first reads the value that data that subelement reads certain line item in the described target matrix are cut apart field;
The value that described data are cut apart field in D2, the second operator unit is carried out modulo operation according to default one-level sublist quantity, and locatees the one-level sublist of current unloading according to described operation result;
D3, the second unloading subelement with described record unloading to described one-level sublist;
D4, second reads subelement when the data volume of described one-level sublist satisfies certain threshold value, and the data that read certain line item in the described one-level sublist are cut apart the value of field;
The value that described data are cut apart field in D5, the 3rd operator unit is carried out modulo operation according to default secondary sublist quantity, and locatees the secondary sublist of current unloading according to described operation result;
D6, the 3rd unloading subelement with the record unloading of described one-level sublist to described secondary sublist.
Preferably, data are cut apart field and are comprised that numeral cuts apart field or Character segmentation field, data are cut apart field be converted into sexadecimal, and described split cells comprises:
Third reading is got subelement, is used for reading the value that described target matrix data are cut apart field;
The transformant unit, the value that is used for data are cut apart field is converted into sexadecimal, and locatees the sublist of current unloading according to initial character;
The 4th unloading subelement is used for and will be worth unloading accordingly to described sublist.
In this case, using the method that preferred embodiment shown in Figure 6 carries out data processing can may further comprise the steps:
E1, third reading are got subelement and are read the value that data in the described target matrix are cut apart field;
The value that field is cut apart with data in E2, transformant unit is converted into sexadecimal, and locatees the sublist of current unloading according to initial character;
E3, the 4th unloading subelement will be worth unloading accordingly to described sublist.
Data are cut apart field can also be converted into sexadecimal by the transformant unit, when described data are cut apart field and are sexadecimal, conversion unit need be resolved the initial character of the sexadecimal number after transforming, the initial character value that parses is sent to positioning unit, described positioning unit splits target matrix according to the initial character value, and its concrete split process is identical with preceding method.
Preferably, described split cells is positioned at database layer, and the sublist after the fractionation can be arranged in same database, also can be arranged in different databases.
With reference to shown in Figure 7, show the process flow diagram of the embodiment 1 of the data enquire method in the Transaction Processing of the present invention, can comprise the steps:
Step 701: the sublist at locating query data place;
Wherein, described sublist splits acquisition for cutting apart field according to described data to target matrix, and described data are cut apart field for determining acquisition according to service feature in target matrix, and described data query is the value that corresponding described data are cut apart field;
Step 702: in described sublist, search the relative recording that satisfies described data query.
Preferably, described positioning step comprises:
Described data query is carried out modulo operation according to default sublist quantity;
According to described operation result location sublist.
Use present embodiment, suppose that the sublist number is n+1, wherein, the data of application layer accessing database layer can be represented with following false code:
v_flag＝mod(#userid，n+1)；
If v_flag＝0then
Deposit data is carried out various operations in sublist 00 to it;
Else if v_flag＝1 then
Deposit data is carried out various operations in sublist 01 to it;
Else if v_flag＝2 then
Deposit data is carried out various operations in sublist 02 to it;
...
Else
Deposit data is carried out various operations at sublist n to it;
End if；
As can be seen, after target matrix split, when the system queries user data is cut apart the data of field usesid, at first can adopt the delivery hash computing among the aforementioned data disposal route embodiment, leave on which sublist to navigate to this user data, can carry out the inquiry and the operation of data subsequently.
Below by two concrete examples present embodiment is described:
One, supposes n=15 in the aforementioned data disposal route, promptly be that the sublist number that splits is 16, when being applied in visit data, if the data of visit userid=1 at first need be carried out modulo operation in application layer, obtain v_flag=mod (1,16)=1, then can learn from operation result needs visit sublist _ 01, can carry out corresponding data manipulation language (DML) (DataManipulation Language, DML) operation to sublist _ 01 immediately;
Similarly, if the data of visit userid=20, carry out modulo operation in application layer, obtain v_flag=mod (20,16)=4, needing then can from operation result, obtain visit sublist _ 04, can carry out corresponding D ML operation to sublist _ 04 immediately, ..., the rest may be inferred, can use the same method always the location and inquire about to sublist _ 15.
Two, in the present embodiment, no matter need how many users of inquiry is, can carry out similar modulo operation and finish position fixing process, and different user data may leave on the sublist, for example, when the user inquired about, if need inquiry 100 these user's data, then query steps was:
Modulo operation v_flag=mod (100,16)=4; Calculating the result is 4, can learn that deposit data is on sublist _ 04;
Navigate on sublist _ 04, service data is to inquire about; As inquiring about: select id, type, status from sublist _ 04 where userid=100;
Previous embodiment just splits in the individual data storehouse, if when splitting target matrix, sublist _ 04 is placed on the another one database test_db, when being applied in visit so, only need go to sublist _ 04 of accessing database test_db to get final product.
Need to prove for how navigating to sublist, perhaps how to navigate to database, the rule searching of location is by the fractionation rule decision of data, and the data when target matrix is split are cut apart field, promptly are the data query fields of using when positioning; And inquiry mode also is that the fractionation rule by data decides, and deposit data which in which database opens on sublist, and then Dui Ying application just need be inquired about the described sublist of described database.
Preferably, when described target matrix carried out splitting the second time, described data enquire method comprised by one-level sublist location and inquires about secondary sublist under the described one-level sublist.In the present embodiment, described sublist is divided into one-level sublist and secondary sublist, the embodiment 3 of the corresponding processing data in online business processing of the present invention of this querying method.When the method that splits the second time of describing in the target matrix Application Example 3, described data enquire method also is divided into the location and inquires about two steps of secondary sublist under the described one-level sublist.
Below by an object lesson present embodiment is described:
At first suppose (id at target matrix test, userid, type, status, other field) inquires about in, the data query field is userid, and preset when splitting for the first time still according to mould 16 compute modes, promptly be that the fractionation sublist number of presetting is 16, because it is still too fast to split the data volume grow of back sublist for the first time, so need once more the userid field value of every sublist is carried out mould 3 computings at database layer, promptly be split as 16 the one-level sublist each be split as 3 secondary sublists again.
Wherein, when using present embodiment, the data in the application layer Query Database can be represented with following false code:
V_flag=mod (#userid, 16);--ground floor hash result
Vv_flag=mod (#userid, 3);--second layer hash result
If v_flag＝0then
Deposit data is in sublist _ 00;
If vv_flag＝0then
Deposit data is carried out various operations at sublist 00_00 to it;
Else if vv_flag＝1then
Deposit data is carried out various operations at sublist 00_01 to it;
Else
Deposit data is carried out various operations at sublist 00_02 to it;
End if；
Else if v_flag＝1 then
Deposit data is in sublist _ 01;
If vv_flag＝0 then
Deposit data is carried out various operations at sublist 01_00 to it;
Else if vv_flag＝1 then
Deposit data is carried out various operations at sublist 01_01 to it;
Else
Deposit data is carried out various operations at sublist 01_02 to it;
End if；
...
Else
Deposit data is in sublist _ 15;
If vv_flag＝0 then
Deposit data is carried out various operations at sublist 15_00 to it;
Else if vv_flag＝1 then
Deposit data is carried out various operations at sublist 15_01 to it;
Else
Deposit data is carried out various operations at sublist 15_02 to it;
End if；
End if；
Need to prove, after data split the secondary sublist, above-mentioned one-level sublist 00,01 ..., 15 in fact do not existed.That is to say that under the situation that has subordinate's sublist, its higher level's sublist is exactly the sublist on the logical meaning, promptly a kind of empty sublist, but not the existence on the physical significance.
Certainly, it all is feasible that those skilled in the art adopt the method for any data query, for example, corresponding to the embodiment 4 of aforesaid processing data in online business processing, preferred, when carrying out data query, can be according to first character of #nick among the target matrix t, at first judge data query and deposit in which corresponding sublist, navigate to described sublist then, the data that inquiry needs.Data processing in aforesaid Transaction Processing and querying method, all be based on described sublist is dumped to a basis on the database, if when carrying out the step of data processing, sublist is stored on other database, when so data being inquired about, need at first inquire about the database at sublist place, and then navigate to the described sublist in the described database.From the embodiment of aforementioned data processing, querying method as can be seen, data processing in this Transaction Processing, querying method have good extendability, one-level sublist number after splitting can be carried out fractionation second time once more according to what of one-level sublist data volume, then can navigate at an easy rate during inquiry on the secondary sublist after the fractionation.Need to prove that in this case, the secondary sublist just has data, the one-level sublist is an empty table, is a path that arrives the secondary sublist.
With reference to shown in Figure 8, show the structural drawing of the data query device in the Transaction Processing of the present invention, described data query device is in order to realize the function of data query, described device comprises:
In the present embodiment, described target data is through splitting, and is stored in the different sublists, therefore when positioning unit locating query data, directly navigates in the sublist at described data query place and gets final product.
Preferably, described positioning unit comprises:
The 4th operator unit is used for described data query is carried out modulo operation according to default sublist quantity;
Receive subelement, be used to receive described operation result and sublist is positioned.
In this case, using the method that preferred embodiment shown in Figure 8 carries out data query can may further comprise the steps:
F1, the just described data query in the 4th operator unit carry out modulo operation according to default sublist quantity;
F2, reception subelement receive described operation result and sublist are positioned;
F3, search the unit and in described sublist, search the relative recording that satisfies described data query.
Preferably, described positioning unit is positioned at application layer, and described location can be understood as the data of application layer visit and Query Database layer.
In the above-described embodiments, the description of each embodiment is all emphasized particularly on different fields, do not have the part that describes in detail among certain embodiment, can get final product referring to the associated description of aforementioned part.Above-mentionedly arbitrarily enumerated several embodiment of the present invention, those skilled in the art are appropriate combination, selection as the case may be, can bring into play technology effect of the present invention fully.Combination in any based on the foregoing description all is embodiment of the present invention, but this instructions has not just described in detail one by one at this as space is limited.
Because the device that provides among the present invention can correspondence be applicable among the aforesaid the whole bag of tricks embodiment that so description is comparatively simple, not detailed part can be referring to the description of this instructions front appropriate section.
More than the data processing in the data processing in the Transaction Processing provided by the present invention, querying method and the Transaction Processing, inquiry unit are described in detail, used specific case herein principle of the present invention and embodiment are set forth, the explanation of above embodiment just is used for helping to understand method of the present invention and core concept thereof; Simultaneously, for one of ordinary skill in the art, according to thought of the present invention, the part that all can change in specific embodiments and applications, in sum, this description should not be construed as limitation of the present invention.
Claims (17)
1, a kind of processing data in online business processing is characterized in that, comprising:
The data of determining target matrix according to service feature are cut apart field;
Cut apart field according to described data described target matrix is split into a plurality of sublists.
2, method according to claim 1 is characterized in that, also comprises:
Described sublist is stored in one or more databases.
3, method according to claim 1 and 2 is characterized in that, described tables of data splitting step comprises:
The data that read certain line item in the described target matrix are cut apart the value of field;
The value of described data being cut apart field is carried out modulo operation according to default sublist quantity, and locatees current unloading sublist according to described operation result;
With the record unloading of described target matrix to described sublist.
4, method according to claim 1 and 2 is characterized in that, described sublist comprises one-level sublist and secondary sublist, and described tables of data splitting step comprises:
The data that read certain line item in the described target matrix are cut apart the value of field;
The value of described data being cut apart field is carried out modulo operation according to default one-level sublist quantity, and locatees the one-level sublist of current unloading according to described operation result;
With described record unloading to the described one-level sublist;
If the data volume of described one-level sublist satisfies certain threshold value, the data that then read certain line item in the described one-level sublist are cut apart the value of field;
The value of described data being cut apart field is carried out modulo operation according to default secondary sublist quantity, and locatees the secondary sublist of current unloading according to described operation result;
With the record unloading of described one-level sublist to described secondary sublist.
5, method according to claim 1 and 2 is characterized in that, it is numeric type field or character type field that described data are cut apart field, and described tables of data splitting step comprises:
Read data in the described target matrix and cut apart the value of field;
The value of described data being cut apart field is converted into sexadecimal, locatees the sublist of current unloading according to initial character;
To be worth unloading accordingly to described sublist.
6, method according to claim 1 is characterized in that, described data are cut apart field and comprised the user data identification field.
7, the data enquire method in a kind of Transaction Processing is characterized in that, comprising:
The sublist at locating query data place, described sublist splits acquisition for cutting apart field according to described data to target matrix, described data are cut apart field for determining acquisition according to service feature in target matrix, and described data query is the value that corresponding described data are cut apart field;
In described sublist, search the relative recording that satisfies described data query.
8, method according to claim 7 is characterized in that, described positioning step comprises:
Described data query is carried out modulo operation according to default sublist quantity;
According to described operation result location sublist.
9, the data processing equipment in a kind of Transaction Processing is characterized in that, comprising:
Determining unit is used for determining that according to service feature the data of target matrix cut apart field;
Split cells is used for cutting apart field according to described data described target matrix is split into a plurality of sublists.
10, device according to claim 9 is characterized in that, also comprises:
Storage unit is used for described sublist is stored to one or more databases.
11, according to claim 9 or 10 described devices, it is characterized in that described split cells comprises:
First reads subelement, and the data that are used for reading described target matrix line item are cut apart the value of field;
The first operator unit, the value that is used for described data are cut apart field is carried out modulo operation according to default sublist quantity, and locatees current unloading sublist according to described operation result;
The first unloading subelement is used for the record unloading of described target matrix to described sublist.
According to claim 9 or 10 described devices, it is characterized in that 12, described sublist comprises one-level sublist and secondary sublist, described split cells comprises:
First reads subelement, and the data that are used for reading described target matrix line item are cut apart the value of field;
The second operator unit, the value that is used for described data are cut apart field is carried out modulo operation according to default one-level sublist quantity, and locatees the one-level sublist of current unloading according to described operation result;
The second unloading subelement is used for described record unloading to described one-level sublist;
Second reads subelement, is used for when the data volume of described one-level sublist satisfies certain threshold value, and the data that read certain line item in the described one-level sublist are cut apart the value of field;
The 3rd operator unit, the value that is used for described data are cut apart field is carried out modulo operation according to default secondary sublist quantity, and locatees the secondary sublist of current unloading according to described operation result;
The 3rd unloading subelement is used for the record unloading of described one-level sublist to described secondary sublist.
According to claim 9 or 10 described devices, it is characterized in that 13, it is numeric type field or character type field that described data are cut apart field, described split cells comprises:
Third reading is got subelement, is used for reading the value that described target matrix data are cut apart field;
The transformant unit, the value that is used for data are cut apart field is converted into sexadecimal, and locatees the sublist of current unloading according to initial character;
The 4th unloading subelement is used for and will be worth unloading accordingly to described sublist.
14, device according to claim 9 is characterized in that, described split cells is positioned at database layer.
15, the data query device in a kind of Transaction Processing is characterized in that, comprising:
Positioning unit, the sublist that is used for locating query data place, to be split cells cut apart field according to described data to described sublist that target matrix is split acquisition, described data are cut apart field for determining acquisition according to service feature in target matrix, and described data query is the value that corresponding described data are cut apart field;
Search the unit, be used for searching the relative recording that satisfies described data query in described sublist.
16, device according to claim 15 is characterized in that, described positioning unit comprises:
The 4th operator unit is used for described data query is carried out modulo operation according to default sublist quantity;
Receive subelement, be used to receive described operation result and sublist is positioned.
17, device according to claim 15 is characterized in that, described positioning unit is positioned at application layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2008100904871A CN101566986A (en) | 2008-04-21 | 2008-04-21 | Method and device for processing data in online business processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2008100904871A CN101566986A (en) | 2008-04-21 | 2008-04-21 | Method and device for processing data in online business processing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101566986A true CN101566986A (en) | 2009-10-28 |
Family
ID=41283141
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNA2008100904871A Pending CN101566986A (en) | 2008-04-21 | 2008-04-21 | Method and device for processing data in online business processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101566986A (en) |
Cited By (44)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102193986A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Method of implementing online transaction in graphic database |
| CN102193987A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Method and system for increasing node data relationship based on OLTP (online transaction processing) |
| CN102193985A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Method for canceling node data relation in graphical database offline transaction |
| CN102193975A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Method for realizing transaction committing mechanism in online transaction of graphic database |
| CN102193978A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Control method of deleting data in offline transaction of pattern database |
| CN102193980A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Control method for inserting data into offline transaction of graphic database |
| CN102193977A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Method for increasing node data relation in off-line transactions of graphic database |
| CN102193979A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Control method for inquiring data from offline transaction of graphic database |
| CN102193976A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Method for implementing transaction rollback mechanism in online transaction of graphic database |
| CN102262626A (en) * | 2010-05-24 | 2011-11-30 | 阿里巴巴集团控股有限公司 | Method and device for storing data in database |
| CN102867071A (en) * | 2012-10-19 | 2013-01-09 | 烽火通信科技股份有限公司 | Management method for massive network management historical data |
| CN102999526A (en) * | 2011-09-16 | 2013-03-27 | 阿里巴巴集团控股有限公司 | Splitting and inquiring method and system of database relational table |
| CN103020227A (en) * | 2012-12-13 | 2013-04-03 | 中国银行股份有限公司 | Data processing method and system in computer equipment |
| CN103093324A (en) * | 2011-10-27 | 2013-05-08 | 镇江雅迅软件有限责任公司 | Inventory management system based on data sheet splitting technology |
| CN103377211A (en) * | 2012-04-20 | 2013-10-30 | 上海梅山钢铁股份有限公司 | High-frequency data storage and reading method during hot continuous rolling production process |
| CN103473271A (en) * | 2013-08-20 | 2013-12-25 | 苏州迈科网络安全技术股份有限公司 | Optimized storing method for mass data |
| CN104615684A (en) * | 2015-01-22 | 2015-05-13 | 北京彩云动力教育科技有限公司 | Mass data communication concurrent processing method and system |
| CN104714945A (en) * | 2013-12-11 | 2015-06-17 | 世纪禾光科技发展（北京）有限公司 | Commodity information system establishing method and system |
| CN104715076A (en) * | 2015-04-13 | 2015-06-17 | 东信和平科技股份有限公司 | Multi-threaded data processing method and device |
| CN104781814A (en) * | 2012-10-01 | 2015-07-15 | 甲骨文国际公司 | Reference data segmentation from single to multiple tables |
| CN105045877A (en) * | 2015-07-20 | 2015-11-11 | 深圳市深信服电子科技有限公司 | Database data fragmentation storage method and apparatus and data query method and apparatus |
| CN105224596A (en) * | 2015-08-27 | 2016-01-06 | 浪潮集团有限公司 | A kind of method of visit data and device |
| CN105279198A (en) * | 2014-07-24 | 2016-01-27 | 北京古盘创世科技发展有限公司 | Data table storage method, data table modification method, data table query method and data table statistical method |
| CN105930502A (en) * | 2012-10-22 | 2016-09-07 | 北京奇虎科技有限公司 | System, client terminal and method for collecting data |
| CN106294740A (en) * | 2016-08-10 | 2017-01-04 | 北京创锐文化传媒有限公司 | Data processing method, device and server |
| CN106294191A (en) * | 2015-05-26 | 2017-01-04 | 华为技术有限公司 | The method processing table, the method and apparatus accessing table |
| CN106326241A (en) * | 2015-06-15 | 2017-01-11 | 阿里巴巴集团控股有限公司 | Method and apparatus for reading/writing data table in data table splitting process |
| CN106547784A (en) * | 2015-09-22 | 2017-03-29 | 阿里巴巴集团控股有限公司 | A kind of data split storage method and device |
| CN106802891A (en) * | 2015-11-26 | 2017-06-06 | 中国电信股份有限公司 | The querying method of the non-burst field of distributed data base, system and equipment |
| CN106933936A (en) * | 2015-12-31 | 2017-07-07 | 远光软件股份有限公司 | The application method and device of entity in a kind of management software system |
| CN106933903A (en) * | 2015-12-31 | 2017-07-07 | 北京国双科技有限公司 | It is applied to the storage method and device of distributed storage |
| CN107015919A (en) * | 2017-04-13 | 2017-08-04 | 济南浪潮高新科技投资发展有限公司 | Nand flash storage array Mapping management methods |
| CN107766459A (en) * | 2017-09-27 | 2018-03-06 | 天翼电子商务有限公司 | A kind of high-performance and high availability divide table method and its system |
| CN107798030A (en) * | 2017-02-17 | 2018-03-13 | 平安科技（深圳）有限公司 | The method for splitting and device of tables of data |
| CN109325050A (en) * | 2018-08-01 | 2019-02-12 | 吉林盘古网络科技股份有限公司 | Data query method, apparatus and terminal device |
| CN109344152A (en) * | 2018-08-22 | 2019-02-15 | 中国平安人寿保险股份有限公司 | Data processing method, device, electronic equipment and storage medium |
| CN104933066B (en) * | 2014-03-19 | 2019-03-15 | 北京畅游天下网络技术有限公司 | A kind of method and system of data processing |
| CN110059306A (en) * | 2019-04-11 | 2019-07-26 | 北京字节跳动网络技术有限公司 | Processing method, device, equipment and the computer readable storage medium of online table |
| CN110196854A (en) * | 2019-06-11 | 2019-09-03 | 中国科学院寒区旱区环境与工程研究所 | Data processing method and device |
| CN110263105A (en) * | 2019-05-21 | 2019-09-20 | 北京百度网讯科技有限公司 | Inquiry processing method, query processing system, server and computer-readable medium |
| CN110781258A (en) * | 2019-09-16 | 2020-02-11 | 北京三快在线科技有限公司 | Packet query method and device, electronic equipment and readable storage medium |
| CN110825739A (en) * | 2019-10-30 | 2020-02-21 | 京东数字科技控股有限公司 | Table building statement generation method, device, equipment and storage medium |
| CN112685402A (en) * | 2019-10-17 | 2021-04-20 | 拉扎斯网络科技（上海）有限公司 | Data storage and query method and device, electronic equipment and storage medium |
| CN113111066A (en) * | 2021-04-20 | 2021-07-13 | 长沙市到家悠享网络科技有限公司 | Automatic online method, device and system for database operation work order and computer equipment |
-
2008
- 2008-04-21 CN CNA2008100904871A patent/CN101566986A/en active Pending
Cited By (69)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102262626A (en) * | 2010-05-24 | 2011-11-30 | 阿里巴巴集团控股有限公司 | Method and device for storing data in database |
| CN102262626B (en) * | 2010-05-24 | 2013-08-07 | 阿里巴巴集团控股有限公司 | Method and device for storing data in database |
| CN102193975B (en) * | 2011-03-25 | 2013-01-02 | 北京世纪互联宽带数据中心有限公司 | Method for realizing transaction committing mechanism in online transaction of graphic database |
| CN102193985A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Method for canceling node data relation in graphical database offline transaction |
| CN102193978A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Control method of deleting data in offline transaction of pattern database |
| CN102193986A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Method of implementing online transaction in graphic database |
| CN102193977A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Method for increasing node data relation in off-line transactions of graphic database |
| CN102193979A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Control method for inquiring data from offline transaction of graphic database |
| CN102193976A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Method for implementing transaction rollback mechanism in online transaction of graphic database |
| CN102193987B (en) * | 2011-03-25 | 2013-03-20 | 北京世纪互联宽带数据中心有限公司 | Method and system for increasing node data relationship based on OLTP (online transaction processing) |
| CN102193979B (en) * | 2011-03-25 | 2012-09-05 | 北京世纪互联工程技术服务有限公司 | Control method for inquiring data from offline transaction of graphic database |
| CN102193978B (en) * | 2011-03-25 | 2012-12-05 | 北京世纪互联宽带数据中心有限公司 | Control method of deleting data in offline transaction of pattern database |
| CN102193986B (en) * | 2011-03-25 | 2013-01-02 | 北京世纪互联宽带数据中心有限公司 | Method of implementing online transaction in graphic database |
| CN102193976B (en) * | 2011-03-25 | 2013-01-02 | 北京世纪互联宽带数据中心有限公司 | Method for implementing transaction rollback mechanism in online transaction of graphic database |
| CN102193977B (en) * | 2011-03-25 | 2013-01-02 | 北京世纪互联宽带数据中心有限公司 | Method for increasing node data relation in off-line transactions of graphic database |
| CN102193987A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Method and system for increasing node data relationship based on OLTP (online transaction processing) |
| CN102193980B (en) * | 2011-03-25 | 2013-01-02 | 北京世纪互联宽带数据中心有限公司 | Control method for inserting data into offline transaction of graphic database |
| CN102193985B (en) * | 2011-03-25 | 2013-01-02 | 北京世纪互联宽带数据中心有限公司 | Method for canceling node data relation in graphical database offline transaction |
| CN102193980A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Control method for inserting data into offline transaction of graphic database |
| CN102193975A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Method for realizing transaction committing mechanism in online transaction of graphic database |
| CN102999526B (en) * | 2011-09-16 | 2016-04-06 | 阿里巴巴集团控股有限公司 | A kind of fractionation of database relational table, querying method and system |
| CN102999526A (en) * | 2011-09-16 | 2013-03-27 | 阿里巴巴集团控股有限公司 | Splitting and inquiring method and system of database relational table |
| CN103093324A (en) * | 2011-10-27 | 2013-05-08 | 镇江雅迅软件有限责任公司 | Inventory management system based on data sheet splitting technology |
| CN103377211A (en) * | 2012-04-20 | 2013-10-30 | 上海梅山钢铁股份有限公司 | High-frequency data storage and reading method during hot continuous rolling production process |
| CN104781814A (en) * | 2012-10-01 | 2015-07-15 | 甲骨文国际公司 | Reference data segmentation from single to multiple tables |
| CN102867071B (en) * | 2012-10-19 | 2015-04-29 | 烽火通信科技股份有限公司 | Management method for massive network management historical data |
| WO2014059808A1 (en) * | 2012-10-19 | 2014-04-24 | 烽火通信科技股份有限公司 | Method for managing mass historical data of network management |
| CN102867071A (en) * | 2012-10-19 | 2013-01-09 | 烽火通信科技股份有限公司 | Management method for massive network management historical data |
| CN105930502A (en) * | 2012-10-22 | 2016-09-07 | 北京奇虎科技有限公司 | System, client terminal and method for collecting data |
| CN105930502B (en) * | 2012-10-22 | 2020-04-10 | 北京奇虎科技有限公司 | System, client and method for collecting data |
| CN103020227A (en) * | 2012-12-13 | 2013-04-03 | 中国银行股份有限公司 | Data processing method and system in computer equipment |
| CN103020227B (en) * | 2012-12-13 | 2016-06-29 | 中国银行股份有限公司 | Data processing method in computer equipment and system |
| CN103473271A (en) * | 2013-08-20 | 2013-12-25 | 苏州迈科网络安全技术股份有限公司 | Optimized storing method for mass data |
| CN104714945A (en) * | 2013-12-11 | 2015-06-17 | 世纪禾光科技发展（北京）有限公司 | Commodity information system establishing method and system |
| CN104933066B (en) * | 2014-03-19 | 2019-03-15 | 北京畅游天下网络技术有限公司 | A kind of method and system of data processing |
| CN105279198A (en) * | 2014-07-24 | 2016-01-27 | 北京古盘创世科技发展有限公司 | Data table storage method, data table modification method, data table query method and data table statistical method |
| CN105279198B (en) * | 2014-07-24 | 2019-03-26 | 北京古盘创世科技发展有限公司 | Tables of data storage, modification, inquiry and statistical method |
| CN104615684A (en) * | 2015-01-22 | 2015-05-13 | 北京彩云动力教育科技有限公司 | Mass data communication concurrent processing method and system |
| CN104615684B (en) * | 2015-01-22 | 2018-06-12 | 北京彩云动力教育科技有限公司 | A kind of mass data communication concurrent processing method and system |
| CN104715076A (en) * | 2015-04-13 | 2015-06-17 | 东信和平科技股份有限公司 | Multi-threaded data processing method and device |
| CN106294191B (en) * | 2015-05-26 | 2019-07-09 | 华为技术有限公司 | The method for handling table, the method and apparatus for accessing table |
| CN106294191A (en) * | 2015-05-26 | 2017-01-04 | 华为技术有限公司 | The method processing table, the method and apparatus accessing table |
| CN106326241A (en) * | 2015-06-15 | 2017-01-11 | 阿里巴巴集团控股有限公司 | Method and apparatus for reading/writing data table in data table splitting process |
| CN105045877B (en) * | 2015-07-20 | 2018-10-12 | 深信服科技股份有限公司 | Database data fragment storage method and device, data query method and apparatus |
| CN105045877A (en) * | 2015-07-20 | 2015-11-11 | 深圳市深信服电子科技有限公司 | Database data fragmentation storage method and apparatus and data query method and apparatus |
| CN105224596A (en) * | 2015-08-27 | 2016-01-06 | 浪潮集团有限公司 | A kind of method of visit data and device |
| CN106547784A (en) * | 2015-09-22 | 2017-03-29 | 阿里巴巴集团控股有限公司 | A kind of data split storage method and device |
| CN106547784B (en) * | 2015-09-22 | 2020-04-28 | 阿里巴巴集团控股有限公司 | Data splitting and storing method and device |
| CN106802891A (en) * | 2015-11-26 | 2017-06-06 | 中国电信股份有限公司 | The querying method of the non-burst field of distributed data base, system and equipment |
| CN106933903A (en) * | 2015-12-31 | 2017-07-07 | 北京国双科技有限公司 | It is applied to the storage method and device of distributed storage |
| CN106933903B (en) * | 2015-12-31 | 2020-02-07 | 北京国双科技有限公司 | Storage method and device applied to distributed storage |
| CN106933936B (en) * | 2015-12-31 | 2021-03-16 | 远光软件股份有限公司 | Method and device for using entity in management software system |
| CN106933936A (en) * | 2015-12-31 | 2017-07-07 | 远光软件股份有限公司 | The application method and device of entity in a kind of management software system |
| CN106294740A (en) * | 2016-08-10 | 2017-01-04 | 北京创锐文化传媒有限公司 | Data processing method, device and server |
| CN107798030A (en) * | 2017-02-17 | 2018-03-13 | 平安科技（深圳）有限公司 | The method for splitting and device of tables of data |
| CN107015919A (en) * | 2017-04-13 | 2017-08-04 | 济南浪潮高新科技投资发展有限公司 | Nand flash storage array Mapping management methods |
| CN107766459A (en) * | 2017-09-27 | 2018-03-06 | 天翼电子商务有限公司 | A kind of high-performance and high availability divide table method and its system |
| CN107766459B (en) * | 2017-09-27 | 2021-03-02 | 天翼商业保理有限公司 | Table dividing method and system |
| CN109325050A (en) * | 2018-08-01 | 2019-02-12 | 吉林盘古网络科技股份有限公司 | Data query method, apparatus and terminal device |
| CN109344152A (en) * | 2018-08-22 | 2019-02-15 | 中国平安人寿保险股份有限公司 | Data processing method, device, electronic equipment and storage medium |
| CN110059306A (en) * | 2019-04-11 | 2019-07-26 | 北京字节跳动网络技术有限公司 | Processing method, device, equipment and the computer readable storage medium of online table |
| CN110263105A (en) * | 2019-05-21 | 2019-09-20 | 北京百度网讯科技有限公司 | Inquiry processing method, query processing system, server and computer-readable medium |
| CN110263105B (en) * | 2019-05-21 | 2021-09-10 | 北京百度网讯科技有限公司 | Query processing method, query processing system, server, and computer-readable medium |
| US11194807B2 (en) | 2019-05-21 | 2021-12-07 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Query processing method, query processing system, server and computer readable medium |
| CN110196854A (en) * | 2019-06-11 | 2019-09-03 | 中国科学院寒区旱区环境与工程研究所 | Data processing method and device |
| CN110781258A (en) * | 2019-09-16 | 2020-02-11 | 北京三快在线科技有限公司 | Packet query method and device, electronic equipment and readable storage medium |
| CN112685402A (en) * | 2019-10-17 | 2021-04-20 | 拉扎斯网络科技（上海）有限公司 | Data storage and query method and device, electronic equipment and storage medium |
| CN110825739A (en) * | 2019-10-30 | 2020-02-21 | 京东数字科技控股有限公司 | Table building statement generation method, device, equipment and storage medium |
| CN113111066A (en) * | 2021-04-20 | 2021-07-13 | 长沙市到家悠享网络科技有限公司 | Automatic online method, device and system for database operation work order and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101566986A (en) | Method and device for processing data in online business processing | |
| US11176132B2 (en) | Processing database queries using format conversion | |
| US10402421B2 (en) | Systems and methods for interest-driven data sharing in interest-driven business intelligence systems | |
| US10762071B2 (en) | Value-ID-based sorting in column-store databases | |
| CA2824319C (en) | Column smart mechanism for column based database | |
| US11520760B2 (en) | System and method for providing bottom-up aggregation in a multidimensional database environment | |
| US8862540B2 (en) | Replica placement strategy for distributed data persistence | |
| US20170116311A1 (en) | System and method for use of automatic slice merge in a multidimensional database environment | |
| US20080201296A1 (en) | Partitioning of nested tables | |
| US9208180B2 (en) | Determination of database statistics using application logic | |
| US7814045B2 (en) | Semantical partitioning of data | |
| US20150088806A1 (en) | Supporting multi-tenancy in a federated data management system | |
| Schaffner et al. | A hybrid row-column OLTP database architecture for operational reporting | |
| US20100235344A1 (en) | Mechanism for utilizing partitioning pruning techniques for xml indexes | |
| US20220067026A1 (en) | System and method for dependency analysis in a multidimensional database environment | |
| CN105164673A (en) | Query integration across databases and file systems | |
| US9229969B2 (en) | Management of searches in a database system | |
| US10810219B2 (en) | Top-k projection | |
| US20180232416A1 (en) | Distribute execution of user-defined function | |
| US20230376485A1 (en) | Distributed query plan generation | |
| US10997178B2 (en) | Implicit partitioning | |
| Alam | Data Migration: Relational Rdbms To Non-Relational Nosql | |
| Chinchilla et al. | MCSA SQL 2016 BI Development Exam Ref 2-pack | |
| CN116521941A (en) | Semi-structured data processing method, electronic device and storage medium | |
| CN115729930A (en) | Using self-maintained structure information for faster data access |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1137243 Country of ref document: HK |
|
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20091028 |
|
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: WD Ref document number: 1137243 Country of ref document: HK |