CN103559281B - The treating method and apparatus of partition data - Google Patents
The treating method and apparatus of partition data Download PDFInfo
- Publication number
- CN103559281B CN103559281B CN201310551737.8A CN201310551737A CN103559281B CN 103559281 B CN103559281 B CN 103559281B CN 201310551737 A CN201310551737 A CN 201310551737A CN 103559281 B CN103559281 B CN 103559281B
- Authority
- CN
- China
- Prior art keywords
- value
- data
- partition
- statistical indicant
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000005192 partition Methods 0.000 title claims abstract description 127
- 238000000034 method Methods 0.000 title claims abstract description 24
- 238000003672 processing method Methods 0.000 claims abstract description 18
- 230000000694 effects Effects 0.000 abstract description 6
- 238000004458 analytical method Methods 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24554—Unary operations; Data partitioning operations
- G06F16/24557—Efficient disk access during query execution
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind for the treatment of method and apparatus of partition data.Wherein, the processing method of partition data includes：Value of statistical indicant is obtained, the wherein initial value of value of statistical indicant is null value；In target partition, the data at value of statistical indicant in start to process target partition, wherein, target partition is the corresponding subregion of newly-increased data；And value of statistical indicant is updated to the largest key value of newly-increased data.By the present invention, the problem of partition data treatment effeciency is relatively low is solved, and then reached the effect for improving partition data treatment effeciency.
Description
Technical field
The present invention relates to database field, in particular to a kind for the treatment of method and apparatus of partition data.
Background technology
For the processing of zone object in Microsoft Analysis Services, it is often necessary to inquire about nearest one or many
Individual active partition, if metric group monthly subregion, then if the data of processing this month, rather than all subregions are done
Go out processing, and the active partition residing for this month includes historical data and newly-increased data, wherein, only increasing data newly needs place
Reason, but Analysis Services can handle whole active partition in data handling procedure is carried out.Whenever data have change
During change, Analysis Services can be again loaded into all data of data correspondence subregion, and for having treated
Partial data, again processing can cause unnecessary expense, treatment effeciency is low.
For the problem of partition data treatment effeciency is low in correlation technique, effective solution is not yet proposed at present.
The content of the invention
It is a primary object of the present invention to provide a kind for the treatment of method and apparatus of partition data, to solve in the prior art
The problem of partition data treatment effeciency is low.
To achieve these goals, according to an aspect of the invention, there is provided a kind of processing method of partition data, bag
Include：Value of statistical indicant is obtained, wherein, the initial value of value of statistical indicant is null value；In target partition, the start to process target point at value of statistical indicant
Data in area, wherein, target partition is the corresponding subregion of newly-increased data；And value of statistical indicant is updated to the maximum of newly-increased data
Key assignments.
Further, after value of statistical indicant to be updated to the largest key value of newly-increased data, processing method also includes：Storage mark
Will value to the target location of target partition, wherein, obtain value of statistical indicant include：Value of statistical indicant is obtained from target location.
Further, target partition is multiple, and obtaining value of statistical indicant includes：Obtained from the target location of each target partition
Value of statistical indicant is taken, in target partition, the data at value of statistical indicant in start to process target partition include：In each target partition
In, proceed by data processing at corresponding value of statistical indicant.
Further, target location is determined by increasing the attribute of target partition.
To achieve these goals, should according to another aspect of the present invention there is provided a kind of processing unit of partition data
Processing unit is mainly used in performing the processing method for any partition data that the above of the present invention is provided.
To achieve these goals, there is provided a kind of processing unit of partition data, bag according to another aspect of the present invention
Include：Acquiring unit, for obtaining value of statistical indicant, wherein, the initial value of value of statistical indicant is null value；Processing unit, in target partition
In, the data at value of statistical indicant in start to process target partition, wherein, target partition is the corresponding subregion of newly-increased data；And
Updating block, the largest key value for value of statistical indicant to be updated to newly-increased data.
Further, processing unit also includes：Memory cell, the target location for storage mark value to target partition.
Further, target partition is multiple, and acquiring unit includes multiple acquisition subelements, wherein, it is each to obtain sub single
Member is used to obtain value of statistical indicant from the target location of corresponding target partition, and processing unit includes multiple processing subelements, often
Individual processing subelement is used in corresponding target partition, and data processing is proceeded by corresponding value of statistical indicant.
Further, processing unit also includes：Adding unit, target position is determined for increasing the attribute of target partition
Put.
The present invention is using value of statistical indicant is obtained, and the wherein initial value of value of statistical indicant is null value；In target partition, at value of statistical indicant
Data in start to process target partition, wherein, target partition is the corresponding subregion of newly-increased data；And be updated to value of statistical indicant
The largest key value of newly-increased data, it is to avoid when having newly-increased data in database, all data to whole subregion are handled, and
The position simply terminated from last time data processing proceeds by data processing, solves that partition data treatment effeciency is relatively low to ask
Topic, and then reached the effect for improving partition data treatment effeciency.
Brief description of the drawings
The accompanying drawing for constituting the part of the application is used for providing a further understanding of the present invention, schematic reality of the invention
Apply example and its illustrate to be used to explain the present invention, do not constitute inappropriate limitation of the present invention.In the accompanying drawings：
Fig. 1 is the flow chart of the processing method of partition data according to embodiments of the present invention；And
Fig. 2 is the structural representation of the processing unit of partition data according to embodiments of the present invention.
Embodiment
It should be noted that in the case where not conflicting, the feature in embodiment and embodiment in the application can phase
Mutually combination.Describe the present invention in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
The embodiment of the present invention provides a kind of processing method of partition data, below to the partition data of the embodiment of the present invention
Processing method is specifically introduced：
Fig. 1 is the flow chart of the partition data processing method of the present invention, as shown in figure 1, the processing method of the partition data
Including steps S102 to step S106：
Step S102, obtains value of statistical indicant, the wherein initial value of value of statistical indicant is null value.Ident value be stored in one it is self-defined
In attribute, when first time data processing is carried out, this Custom Attributes should be not present, also just without value of statistical indicant
Record, so to proceed by data processing at first key assignments, then the initial value of value of statistical indicant be null value.
Step S104, in target partition, the data at value of statistical indicant in start to process target partition, wherein, target point
Area is the corresponding subregion of newly-increased data.If not first time carry out data processing, when there is newly-increased data in database, then from
Data processing is proceeded by the position that last data processing terminates in the subregion.Data in so-called processing target partition are
It is by newly-increased data storage to target partition.
Step S106, value of statistical indicant is updated to the largest key value of newly-increased data（Key values）.It is complete per a collection of newly-increased data processing
Cheng Hou, the largest key value of the newly-increased data of record, and the occurrence of value of statistical indicant is updated to the largest key value of newly-increased data.Value of statistical indicant
It is the largest key value in the last newly-increased data handled, the position that last data processing terminates is represented with value of statistical indicant.
By obtaining value of statistical indicant, in target partition, the data at value of statistical indicant in start to process target partition will indicate
Value is updated to the largest key value of newly-increased data, it is to avoid when having newly-increased data in database, all data to whole subregion are entered
Row processing, and the position simply terminated from last time data processing proceeds by data processing, solves partition data treatment effeciency
Relatively low the problem of, and then reached the effect for improving partition data treatment effeciency.
Wherein, before value of statistical indicant is obtained, the processing method of the partition data of the embodiment of the present invention also needs first to determine target
Subregion, specific determination mode can be：Judge whether there is the subregion of data variation in database, wherein, in data at judgement
In the case of the subregion for having data variation, the subregion for determining data variation is target partition.
Further, the processing method of the partition data of the embodiment of the present invention also includes：Value of statistical indicant is being updated to increase newly
After the largest key value of data, the target location of storage mark value to target partition, when being identified the acquisition of value again next time,
It is then to obtain value of statistical indicant from target location.
Further, a Custom Attributes can be increased in the attribute list of target partition, i.e. by increasing target
The attribute of subregion determines target location, wherein, Custom Attributes has attribute-name and property value, attribute-name can arbitrarily determine
Justice, value of statistical indicant is property value.Indicated when obtaining value of statistical indicant by the property value of the acquisition Custom Attributes
Value.
Further, multiple subregions have newly-increased data, i.e. target partition to include in the case of multiple, to obtain value of statistical indicant
Value of statistical indicant is obtained from the target location of each target partition.It is in this mesh accordingly for any one target partition
Mark in subregion, the data from this target partition of start to process at its value of statistical indicant, i.e. from this in each target partition
Data processing is proceeded by the value of statistical indicant of subregion.
The embodiment of the present invention also provides a kind of processing unit of partition data, and the processing unit is mainly used in performing the present invention
The processing method for any partition data that embodiment the above is provided, below to the partition data of this method embodiment
Processing unit is specifically introduced：
Fig. 2 is the structural representation of the partitionable data processing arrangement of the present invention, as shown in Fig. 2 the processing of the partition data
Device includes：Acquiring unit 10, processing unit 20 and updating block 30.
Acquiring unit 10 is used to obtain value of statistical indicant, and the wherein initial value of value of statistical indicant is null value.Ident value is stored in one
In Custom Attributes, when first time data processing is carried out, this Custom Attributes should be not present, and also just not have
The record of value of statistical indicant, so to proceed by data processing at first key assignments, then the initial value of value of statistical indicant is null value.
Processing unit 20 is used in target partition, the data at value of statistical indicant in start to process target partition, wherein, mesh
It is the corresponding subregion of newly-increased data to mark subregion.Data processing is carried out if not first time, when there is newly-increased data in database,
Data processing is proceeded by the position that then last data processing terminates from the subregion.Number in so-called processing target partition
According to being by newly-increased data storage to target partition.
Updating block 30 is used for the largest key value that value of statistical indicant is updated to newly-increased data（Key values）.Per a collection of newly-increased data
After the completion of processing, the largest key value of the newly-increased data of record, and the occurrence of value of statistical indicant is updated to the largest key value of newly-increased data.
Value of statistical indicant is the largest key value in the newly-increased data of last processing, and the position that last data processing terminates is represented with value of statistical indicant
Put.
By obtaining value of statistical indicant, in target partition, the data at value of statistical indicant in start to process target partition will indicate
Value is updated to the largest key value of newly-increased data, it is to avoid when having newly-increased data in database, all data to whole subregion are entered
Row processing, and the position simply terminated from last time data processing proceeds by data processing, solves partition data treatment effeciency
Relatively low the problem of, and then reached the effect for improving partition data treatment effeciency.
Wherein, the processing unit of the partition data of the embodiment of the present invention also includes the determining unit for determining target partition, tool
Body, determining unit mainly includes judgment sub-unit and determination subelement, wherein, judgment sub-unit be used to judge in database be
The no subregion for having a data variation, in the case of having the subregion of data variation in data at judgement, determination subelement determines data
The subregion of change is target partition.
Further, the processing unit of the partition data of the embodiment of the present invention also includes：Memory cell, for storage mark
It is worth to target location.After value of statistical indicant to be updated to updating block 30 largest key value of newly-increased data, pass through memory cell
Effect stores value of statistical indicant to target location, when being identified the acquisition of value again next time, then is to obtain to mark from target location
Will value.
Further, the processing unit of the partition data of the embodiment of the present invention also includes：Adding unit, for increasing target
The attribute of subregion determines target location.Specifically, adding unit increase in the attribute list of target partition one it is self-defined
Attribute, i.e. determine target location by increasing the attribute of target partition, wherein, Custom Attributes has attribute-name and attribute
Value, attribute-name can be defined arbitrarily, and value of statistical indicant is property value.Acquiring unit 10 is somebody's turn to do when value of statistical indicant is obtained by obtaining
The property value of Custom Attributes obtains value of statistical indicant.
Further, in embodiments of the present invention, the quantity of target partition is multiple, and acquiring unit 10 includes multiple obtain
Subelement, one obtains the mark that subelement accordingly obtains this target partition from the target location of a target partition
Value, i.e. each obtains subelement and is used to obtain value of statistical indicant from the target location of corresponding target partition.Processing unit 20
Including multiple processing subelements, a processing subelement accordingly acts on a target partition, i.e. each processing subelement
For in a target partition, data processing to be proceeded by the value of statistical indicant of this target partition, that is, each is handled
Subelement is used in corresponding target partition, and data processing is proceeded by the value of statistical indicant of the subregion.
Specifically, multiple subregions have newly-increased data, i.e. target partition in the case of multiple, it is necessary to from each target
The target location of subregion obtains the value of statistical indicant of the subregion, and proceeds by data processing at the value of statistical indicant of the subregion.This
In the case of, subelement and processing subelement collaboration one target partition of processing can be obtained by one, i.e. for each
For target partition, there are a corresponding acquisition subelement and processing subelement, and obtain subelement acquisition value of statistical indicant
Method and data processing subelement processing target partition data method with only one of which target partition when method it is identical.
As can be seen from the above description, the problem of partition data treatment effeciency that the present invention is solved is relatively low, reaches
Improve the effect of partition data treatment effeciency.
It should be noted that can be in such as one group computer executable instructions the step of the flow of accompanying drawing is illustrated
Performed in computer system, and, although logical order is shown in flow charts, but in some cases, can be with not
The order being same as herein performs shown or described step.
Obviously, those skilled in the art should be understood that above-mentioned each module of the invention or each step can be with general
Computing device realize that they can be concentrated on single computing device, or be distributed in multiple computing devices and constituted
Network on, alternatively, the program code that they can be can perform with computing device be realized, it is thus possible to they are stored
Performed in the storage device by computing device, either they are fabricated to respectively each integrated circuit modules or by they
In multiple modules or step single integrated circuit module is fabricated to realize.So, the present invention is not restricted to any specific
Hardware and software is combined.
The preferred embodiments of the present invention are the foregoing is only, are not intended to limit the invention, for the skill of this area
For art personnel, the present invention can have various modifications and variations.Within the spirit and principles of the invention, that is made any repaiies
Change, equivalent substitution, improvement etc., should be included in the scope of the protection.
Claims (8)
1. a kind of processing method of partition data, it is characterised in that including：
Value of statistical indicant is obtained, wherein, the initial value of the value of statistical indicant is null value；
In target partition, the data from target partition described in start to process at the value of statistical indicant, wherein, the target partition
To increase the corresponding subregion of data newly；And
The value of statistical indicant is updated to the largest key value of the newly-increased data,
Wherein, before value of statistical indicant is obtained, the processing method also includes：Judge point for whether having data variation in database
Area, wherein, in the case where having the subregion of data variation in judging the database, the subregion for determining data variation is described
Target partition,
Wherein, the value of statistical indicant is the largest key value in the newly-increased data of last processing.
2. the processing method of partition data according to claim 1, it is characterised in that the value of statistical indicant is being updated to institute
After the largest key value for stating newly-increased data, the processing method also includes：
The value of statistical indicant is stored to the target location of the target partition,
Wherein, obtaining value of statistical indicant includes：The value of statistical indicant is obtained from the target location.
3. the processing method of partition data according to claim 2, it is characterised in that the target partition is multiple, is obtained
Value of statistical indicant is taken to include：The value of statistical indicant is obtained from the target location of each target partition,
In target partition, include from the data in target partition described in start to process at the value of statistical indicant：Described in each
In target partition, data processing is proceeded by the corresponding value of statistical indicant.
4. the processing method of partition data according to claim 2, it is characterised in that by increasing the target partition
Attribute determines the target location.
5. a kind of processing unit of partition data, it is characterised in that including：
Acquiring unit, for obtaining value of statistical indicant, wherein, the initial value of the value of statistical indicant is null value；
Processing unit, in target partition, the data from target partition described in start to process at the value of statistical indicant, its
In, the target partition is the corresponding subregion of newly-increased data；And
Updating block, the largest key value for the value of statistical indicant to be updated to the newly-increased data,
Wherein, the processing unit also includes：Judging unit, for before value of statistical indicant is obtained, judging whether have in database
The subregion of data variation, wherein, in the case where having the subregion of data variation in judging the database, determine data variation
Subregion be the target partition,
Wherein, the value of statistical indicant is the largest key value in the newly-increased data of last processing.
6. the processing unit of partition data according to claim 5, it is characterised in that the processing unit also includes：
Memory cell, for storing the value of statistical indicant to the target location of the target partition.
7. the processing unit of partition data according to claim 6, it is characterised in that the target partition be it is multiple,
The acquiring unit includes multiple acquisition subelements, wherein, each acquisition subelement is used to from corresponding described
The target location of target partition obtains the value of statistical indicant,
The processing unit includes multiple processing subelements, and each processing subelement is used in the corresponding target point
Qu Zhong, data processing is proceeded by the corresponding value of statistical indicant.
8. the processing unit of partition data according to claim 6, it is characterised in that the processing unit also includes：
Adding unit, the target location is determined for increasing the attribute of the target partition.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310551737.8A CN103559281B (en) | 2013-11-07 | 2013-11-07 | The treating method and apparatus of partition data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310551737.8A CN103559281B (en) | 2013-11-07 | 2013-11-07 | The treating method and apparatus of partition data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103559281A CN103559281A (en) | 2014-02-05 |
| CN103559281B true CN103559281B (en) | 2017-08-01 |
Family
ID=50013527
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310551737.8A Active CN103559281B (en) | 2013-11-07 | 2013-11-07 | The treating method and apparatus of partition data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103559281B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102004797A (en) * | 2010-12-24 | 2011-04-06 | 深圳市同洲电子股份有限公司 | Method, device and system for processing data |
| CN102591749A (en) * | 2011-12-30 | 2012-07-18 | 华中科技大学 | Continuous data storage method |
| CN103064843A (en) * | 2011-10-20 | 2013-04-24 | 北京中搜网络技术股份有限公司 | Data processing device and data processing method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7346609B2 (en) * | 2004-11-16 | 2008-03-18 | International Business Machines Corporation | Streaming XPath algorithm for XPath value index key generation |
| US7917474B2 (en) * | 2005-10-21 | 2011-03-29 | Isilon Systems, Inc. | Systems and methods for accessing and updating distributed data |
-
2013
- 2013-11-07 CN CN201310551737.8A patent/CN103559281B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102004797A (en) * | 2010-12-24 | 2011-04-06 | 深圳市同洲电子股份有限公司 | Method, device and system for processing data |
| CN103064843A (en) * | 2011-10-20 | 2013-04-24 | 北京中搜网络技术股份有限公司 | Data processing device and data processing method |
| CN102591749A (en) * | 2011-12-30 | 2012-07-18 | 华中科技大学 | Continuous data storage method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103559281A (en) | 2014-02-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109583594B (en) | Deep learning training method, device, equipment and readable storage medium | |
| WO2015074486A1 (en) | Method and apparatus for calculating user retention ratio | |
| JP2001188865A5 (en) | ||
| CN107301091A (en) | Resource allocation methods and device | |
| CN106570027A (en) | Transactional task processing method and device | |
| CN108365989A (en) | Event-handling method and device | |
| CN104462447A (en) | Access timeout data processing method and device | |
| Chen | A branch and bound procedure for the reentrant permutation flow-shop scheduling problem | |
| CN110955390A (en) | Data processing method and device and electronic equipment | |
| CN103559281B (en) | The treating method and apparatus of partition data | |
| CN108023905A (en) | Internet of things application system and method | |
| CN113127327B (en) | Test method and device for performance test | |
| CN106126670B (en) | Operation data sorting processing method and device | |
| CN106790258B (en) | A kind of method and system of screening server network request | |
| CN110968770B (en) | Method and device for stopping crawling of crawler tool | |
| CN110232143A (en) | Pass through the question and answer data query method and its equipment of block chain | |
| CN108268379B (en) | Distributed automatic testing method and device | |
| CN107818537A (en) | Data processing method and device | |
| KR20210065792A (en) | Method and device for end-to-end monitoring situation of massive transaction efficiently in order to input and output trace information at high speed | |
| CN106156210A (en) | A kind of method and apparatus determining application identities list of matches | |
| CN107320119A (en) | Image acquiring method, device and medical supply | |
| CN103544317B (en) | The treating method and apparatus of dimension table data | |
| CN110420458A (en) | Game request processing method and processing device | |
| CN112131179A (en) | Task state detection method and device, computer equipment and storage medium | |
| CN111158798A (en) | Service data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Method and device for processing partitioned data Effective date of registration: 20190531 Granted publication date: 20170801 Pledgee: Shenzhen Black Horse World Investment Consulting Co.,Ltd. Pledgor: BEIJING GRIDSUM TECHNOLOGY Co.,Ltd. Registration number: 2019990000503 |
|
| CP02 | Change in the address of a patent holder | ||
| CP02 | Change in the address of a patent holder |
Address after: 100083 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing Patentee after: BEIJING GRIDSUM TECHNOLOGY Co.,Ltd. Address before: 100086 Beijing city Haidian District Shuangyushu Area No. 76 Zhichun Road cuigongfandian 8 layer A Patentee before: BEIJING GRIDSUM TECHNOLOGY Co.,Ltd. |
|
| PP01 | Preservation of patent right | ||
| PP01 | Preservation of patent right |
Effective date of registration: 20240604 Granted publication date: 20170801 |