CN109376148A - The data processing method of slowly varying dimension table, device, electronic equipment - Google Patents
The data processing method of slowly varying dimension table, device, electronic equipment Download PDFInfo
- Publication number
- CN109376148A CN109376148A CN201810962478.0A CN201810962478A CN109376148A CN 109376148 A CN109376148 A CN 109376148A CN 201810962478 A CN201810962478 A CN 201810962478A CN 109376148 A CN109376148 A CN 109376148A
- Authority
- CN
- China
- Prior art keywords
- data
- major key
- subregion
- time
- target service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure belongs to technical field of data processing, it is related to the data processing method and device, computer readable storage medium, electronic equipment of slowly varying dimension table, this method comprises: creating the first data subregion corresponding with the time range according to preset time range, the first data subregion is for storing the value before the major key of the slowly varying dimension table updates in the time range, and the number that the major key updates in the time range is no more than once；The second data subregion is created, the second data subregion is for storing the updated latest value that the major key corresponds to the current queries time；The newest value of major key described in the first data subregion before the extraction target service time, to obtain the history full dose data of the target service time；The value of major key described in the second data subregion is extracted, to obtain the full dose data of current queries time.On the one hand this method has saved a large amount of memory space, on the other hand can reappear daily full dose data.
Description
Technical field
This disclosure relates to technical field of data processing, in particular to a kind of data processing side of slowly varying dimension table
Method, the data processing equipment of slowly varying dimension table, computer storage medium and electronic equipment.
Background technique
With economic growth and social progress, the terminal devices such as computer, tablet computer at for people's lives, the master of work
Want tool, people on the terminal device operate the data warehouse that can be all stored in data mode in terminal device each time
In, these data can be used for analyzing user behavior or program mal reason.
The main demand of one of data warehouse is to retain historical data, while to guarantee data correlation efficiency, generally requiring
One ID of dimension table only corresponds to a record.Thus a kind of irreconcilable contradiction is just produced, if an ID can only correspond to one
Record, it can only just record the newest attribute value of the ID, can not record attribute value variation.Daily full dose is often used in the industry
Dimension table data are extracted, is put into same day subregion, is saved historical data with this full dose.But when dimension table amount is very big, such as the quotient of electric business
Product data, more than one hundred million user data, this scheme will use a large amount of memory space.
Therefore, this field needs the data processing method and device of a kind of slowly varying dimension table.
It should be noted that information is only used for reinforcing the reason to the background of the disclosure disclosed in above-mentioned background technology part
Solution, therefore may include the information not constituted to the prior art known to persons of ordinary skill in the art.
Summary of the invention
The data processing method for being designed to provide a kind of slowly varying dimension table, the data of slowly varying dimension table of the disclosure
Processing unit, computer storage medium and electronic equipment, and then a large amount of memory spaces are saved at least to a certain extent, and
The full dose data snapshot of business hours can be reappeared.
According to one aspect of the disclosure, a kind of data processing method of slowly varying dimension table is provided, which is characterized in that packet
It includes:
The first data subregion corresponding with the time range, first data point are created according to preset time range
Area is for storing the value before the major key of the slowly varying dimension table updates in the time range, and the major key is when described
Between the number that updates in range it is no more than once；
The second data subregion is created, the second data subregion corresponds to the current queries time more for storing the major key
Latest value after new；
The newest value of major key described in the first data subregion before the extraction target service time, described in obtaining
The history full dose data of target service time；The value of major key described in the second data subregion is extracted, to obtain current queries
The full dose data of time.
In an exemplary embodiment of the disclosure, the quantity of the major key is multiple；It is described according to preset time range
The first data subregion corresponding with the time range is created, the first data subregion is for storing the slowly varying dimension table
Major key updated in the time range before value, comprising:
When the value of the part major key does not update, in the first data subregion with the master that does not update
The corresponding position of key is sky.
In an exemplary embodiment of the disclosure, it is provided in the first data subregion and the second data subregion
Multiple data storage areas, the data storage area are used to store the data of the corresponding different major keys.
In an exemplary embodiment of the disclosure, institute in the first data subregion before one business hours of the extraction
The newest value of major key is stated, to obtain the history full dose data of the target service time, comprising:
Obtain the target service time；
All first data point according to the target service time, before extracting the target service time
Area, and and renewal time corresponding with each major key is obtained in the first data subregion near the target service time
The corresponding and renewal time with each major key is determined as the mesh near the data of the target service time by data
Mark the history full dose data of business hours.
In an exemplary embodiment of the disclosure, the method also includes:
When the major key is deleted, set empty for the position for corresponding to the major key in the second data subregion.
In an exemplary embodiment of the disclosure, the quantity of the major key is multiple；The method also includes:
When each major key does not update in the time range, then corresponds to the time range and do not create described
One data subregion.
According to one aspect of the disclosure, a kind of data processing equipment of slowly varying dimension table is provided, which is characterized in that packet
It includes:
First data partition creating module, for according to preset time range creation corresponding with the time range the
One data subregion, the major key that the first data subregion is used to store the slowly varying dimension table update in the time range
Preceding value, and the number that the major key updates in the time range is no more than once；
Second data partition creating module, for creating the second data subregion, the second data subregion is for storing institute
State the latest value that major key corresponds to the current queries time；
Full dose data acquisition module, major key described in the first data subregion before the extraction target service time is most
New value, to obtain the history full dose data of the target service time；Extract major key described in the second data subregion
Value, to obtain the full dose data of current queries time.
In an exemplary embodiment of the disclosure, the full dose data acquisition module includes:
Target service time acquisition unit, for obtaining the target service time；
History full dose data capture unit, for according to the target service time, extract the target service time it
Preceding all the first data subregions, and obtain and renewal time corresponding with each major key in the first data subregion
Near the data of the target service time, by and the renewal time corresponding with each major key near the target industry
The data of business time are determined as the history full dose data of the target service time.
According to one aspect of the disclosure, a kind of computer readable storage medium is provided, computer program is stored thereon with,
The computer program realizes the data processing method of slowly varying dimension table described in above-mentioned any one when being executed by processor.
According to one aspect of the disclosure, a kind of electronic equipment is provided, comprising:
Processor；And
Memory, for storing the executable instruction of the processor；
Wherein, the processor is configured to execute via the executable instruction is executed described in above-mentioned any one and delay
The data processing method of slow variation dimension table.
The data processing method of the slowly varying dimension table of the disclosure is by creating the first data according to preset time range
Subregion, while the second data subregion is created, wherein the first data subregion is used to store the major key of slowly varying dimension table in time model
Value before enclosing interior update, the second data subregion is for storing the updated value of major key, by extracting before the target service time the
The newest value of major key is in one data subregion to obtain the history full dose data of target service time, or extracts the second data subregion
The updated value of middle major key is to obtain the full dose data of current queries time.The data processing side of slowly varying dimension table in the disclosure
On the one hand method can save a large amount of memory space；On the other hand it can reappear business hours corresponding full dose data snapshot.
It should be understood that above general description and following detailed description be only it is exemplary and explanatory, not
The disclosure can be limited.
Detailed description of the invention
The drawings herein are incorporated into the specification and forms part of this specification, and shows the implementation for meeting the disclosure
Example, and together with specification for explaining the principles of this disclosure.It should be evident that the accompanying drawings in the following description is only the disclosure
Some embodiments for those of ordinary skill in the art without creative efforts, can also basis
These attached drawings obtain other attached drawings.
Fig. 1 schematically shows a kind of flow chart of the data processing method of slowly varying dimension table；
Fig. 2 schematically shows a kind of Application Scenarios-Example figure of the data processing method of slowly varying dimension table；
Fig. 3 schematically shows a kind of structural schematic diagram of data subregion；
Fig. 4 schematically shows a kind of structural schematic diagram for reappearing full dose data；
Fig. 5 schematically shows a kind of structural schematic diagram of the data processing equipment of slowly varying dimension table；
Fig. 6 schematically shows a kind of electronic equipment example frame of data processing method for realizing slowly varying dimension table
Figure；
Fig. 7 schematically shows a kind of computer-readable storage medium of data processing method for realizing slowly varying dimension table
Matter.
Specific embodiment
Example embodiment is described more fully with reference to the drawings.However, example embodiment can be with a variety of shapes
Formula is implemented, and is not understood as limited to example set forth herein；On the contrary, thesing embodiments are provided so that the disclosure will more
Fully and completely, and by the design of example embodiment comprehensively it is communicated to those skilled in the art.Described feature, knot
Structure or characteristic can be incorporated in any suitable manner in one or more embodiments.In the following description, it provides perhaps
More details fully understand embodiment of the present disclosure to provide.It will be appreciated, however, by one skilled in the art that can
It is omitted with technical solution of the disclosure one or more in the specific detail, or others side can be used
Method, constituent element, device, step etc..In other cases, be not shown in detail or describe known solution to avoid a presumptuous guest usurps the role of the host and
So that all aspects of this disclosure thicken.
In addition, attached drawing is only the schematic illustrations of the disclosure, it is not necessarily drawn to scale.Identical attached drawing mark in figure
Note indicates same or similar part, thus will omit repetition thereof.Some block diagrams shown in the drawings are function
Energy entity, not necessarily must be corresponding with physically or logically independent entity.These function can be realized using software form
Energy entity, or these functional entitys are realized in one or more hardware modules or integrated circuit, or at heterogeneous networks and/or place
These functional entitys are realized in reason device device and/or microcontroller device.
A kind of data processing method of slowly varying dimension table is provided firstly in this example embodiment, the slowly varying dimension
The data processing method of table can run on server, can also run on server cluster or Cloud Server etc., certainly, ability
Field technique personnel can also run disclosed method in other platforms according to demand, and spy is not to this in the present exemplary embodiment
It is different to limit.Refering to what is shown in Fig. 1, the method for the adjustment winning probability may comprise steps of:
Step S110. creates the first data subregion corresponding with the time range according to preset time range, described
First data subregion is for storing the value before the major key of the slowly varying dimension table updates in the time range, and the master
The number that key updates in the time range is no more than once；
Step S120. creates the second data subregion, and the second data subregion is currently looked into for storing the major key correspondence
Ask the latest value of time；
The newest value of major key described in the first data subregion before the step S130. extraction target service time,
To obtain the history full dose data of the target service time；The value of major key described in the second data subregion is extracted, to obtain
Take the full dose data of current queries time.
In the data processing method of above-mentioned slowly varying dimension table, by creating the first data subregion and the second data subregion,
Major key for storing slowly varying dimension table respectively updated in preset time range before value and latest value, and pass through pumping
The newest value of major key in the first data subregion before the target service time is taken, to obtain the history full dose number of target service time
According to, or the value of the major key of the second data subregion is extracted, to obtain the full dose data of current queries time.It is on the one hand saved in the disclosure
On the other hand about a large amount of memory space can be convenient and obtain business hours corresponding full dose data.
In the following, Fig. 2 will be combined to each in the data processing method of slowly varying dimension table above-mentioned in this example embodiment
Step carries out detailed explanation and explanation.
In step s 110, the first data subregion corresponding with the time range is created according to preset time range,
The first data subregion is for storing the value before the major key of the slowly varying dimension table updates in the time range, and institute
It is no more than once to state the number that major key updates in the time range.
In an exemplary embodiment of the disclosure, it can create in server 201 or terminal 202 for storing reality first
The fact that the real time data on border table, define the major key and external key of true table, and the record of factual data is inserted into true table；
Multiple dimension tables for being used to store the data for describing attribute from different perspectives are created simultaneously, define the major key of dimension table, and will description
The record of the data of attribute is inserted into dimension table；Finally using the external key of true table and the major key of multiple dimension tables, to true table
It is associated with dimension table.As those skilled in the art should know each dimension table only has a major key.
In an exemplary embodiment of the disclosure, due in slowly varying dimension table data renewal speed it is slow, may daily more
It is new primary, it is also possible to which that two weeks updates primary, etc..Therefore when creating the first data subregion according to time range, Ke Yigen
Created according to different time range, such as daily, by week, monthly create the first data subregion, and guarantee major key is in the time model
The number for enclosing interior update is no more than once.It, can be from the 00:00:00-23 on the same day such as when daily creating the first data subregion:
59:59 creates the first data subregion.The first data subregion includes multiple data storage areas, and each data storage area corresponds to each slow
The major key for changing dimension table corresponds to the value before time range updates for storing the major key.
In the step s 120, the second data subregion is created, the second data subregion is for storing the major key to should
The latest value of preceding query time.
In an exemplary embodiment of the disclosure, the second data subregion can be in addition created, which is used for
Store the updated latest value of major key.It for example, is the first dimension table D1, the second dimension table D2, the with the true associated dimension table of table F
The major key of three-dimensional table D3, the first dimension table D1 are A, and the major key of the second dimension table D2 is B, and the major key of third dimension table D3 is C, correspondingly,
Three data storage areas are separately included in first data subregion and the second data subregion, which respectively corresponds master
Key A, B and C, when one or more of A, B, C are sometime being updated, then before then updating the major key of update
Value is stored in the first data subregion in corresponding data storage area, updated value is stored in the second data subregion corresponding
Data storage area in.
In an exemplary embodiment of the disclosure, if major key does not update, then in the first data of corresponding time range
The data storage area that the major key is corresponded in subregion is set as empty, can save memory space in this way；Further, if all masters
Key does not update, then then corresponding to the time range does not create the first data subregion, that is to say, that only update in major key
When, just create the first data subregion.
Fig. 3 shows a kind of structural schematic diagram of data subregion, as shown in figure 3, using day as the first number of time range creation
According to subregion, major key does not create the first data subregion, the first data subregion and the second data subregion (New-DB) then if not updating
It include three data storage areas.Assuming that three major keys are respectively A, B, C, it is corresponding with major key A, B, C in the second data subregion
Data are respectively a1, b1, c1, if on April 1st, 2018, the corresponding data of major key A are changed, are then stored in new value a2
New data subregion corresponds in the position of major key A, and previous data a1 is then stored in the first data subregion that number is 20180401,
And corresponding major key A is placed；If on May 1st, 2018, the corresponding data of major key B are changed, then new value b2 is stored in new number
In the position for corresponding to major key B according to subregion, previous data b1 is then stored in the first data subregion that number is 20180501, and right
Major key B is answered to place；If on May 25th, 2018, the corresponding data of major key C are changed, then new value c2 is stored in new data
Subregion corresponds in the position of major key C, and previous data c1 is then stored in the first data subregion that number is 20180525, and corresponding
Major key C is placed.
In step s 130, major key described in the first data subregion before the extraction target service time is newest
Value, to obtain the history full dose data of the target service time；Major key described in the second data subregion is extracted simultaneously
Value, to obtain the full dose data of current queries time.
In an exemplary embodiment of the disclosure, in order to extract vector data and be analyzed, after completing data processing,
The newest value of major key in the first data subregion before the target service time can be extracted, to obtain going through for target service time
History full dose data；The value of major key in the second data subregion is extracted, to obtain the full dose data of current queries time.
In an exemplary embodiment of the disclosure, since what is stored in the second data subregion is the last updated of each major key
Value, therefore the value by extracting the second data subregion can obtain newest full dose data；If wanting to obtain the target service time pair
The full dose data answered, then the first all data subregion before can extracting the target service time obtain each first data subregion
In each major key it is corresponding and renewal time is near the data of target service time, can be obtained target service time corresponding history
Full dose data.
Fig. 4 shows the structural schematic diagram of reproduction full dose data, as shown in figure 4, storing 20180101- in database
First data subregion of 20180525 time ranges extracts each master if thinking the history full dose data in acquisition on May 1st, 2018
The key corresponding time, for example major key A was near first data point on May 1st, 2018 near the data on May 1st, 2018
Area is the first data subregion on March 15th, 2018, then corresponding data a2 is the latest value of major key A, and similarly, major key B is corresponding
Latest data be b3, the corresponding latest data of major key C is c3, i.e., the history full dose data in 2018 on May 1, are a2b3c3.
In an exemplary embodiment of the disclosure, in the ETL stage, data may be newly-increased data, the data of modification and
Newly-increased or modification data are then stored in the second data subregion when data are newly-increased or modification data by the data of deletion
In in data storage area corresponding with each major key, the data for each major key stored before the second data subregion are stored in data hair
In the corresponding data storage area of each major key in the corresponding first data subregion of the time range for changing；When data are to delete
When data, then the data that the second data subregion is stored are stored in the first data subregion when data are deleted, and the second number
Then it is set as empty according to the corresponding data storage area of subregion.
The data processing method of the slowly varying dimension table of the disclosure substantially increases data space, can be protected with full dose
Historical data is deposited, and corresponding full dose data can be obtained according to the business hours, is analyzed for data.
The disclosure additionally provides a kind of data processing equipment of slowly varying dimension table.Fig. 5 shows slowly varying dimension table
The structural schematic diagram of data processing equipment, as shown in figure 5, the data processing equipment of the slowly varying dimension table may include the first number
According to partition creating module 510, the second data partition creating module 520 and full dose data acquisition module 530.Wherein:
First data partition creating module 510, it is corresponding with the time range for being created according to preset time range
The first data subregion, the first data subregion is used to store the major key of the slowly varying dimension table in the time range
Value before update, and the number that the major key updates in the time range is no more than once；
Second data partition creating module 520, for creating the second data subregion, the second data subregion is for storing
The major key corresponds to the latest value of current queries time；
Full dose data acquisition module 530, for described in the first data subregion before extracting the target service time
The newest value of major key, to obtain the history full dose data of the target service time；Extract institute in the second data subregion
The value of major key is stated, to obtain the full dose data of current queries time.
The detail of each module is corresponding slowly varying in the data processing equipment of above-mentioned slowly varying dimension table
It is described in detail in the data processing method of dimension table, therefore details are not described herein again.
It should be noted that although being referred to several modules or list for acting the equipment executed in the above detailed description
Member, but this division is not enforceable.In fact, according to embodiment of the present disclosure, it is above-described two or more
Module or the feature and function of unit can embody in a module or unit.Conversely, an above-described mould
The feature and function of block or unit can be to be embodied by multiple modules or unit with further division.
In addition, although describing each step of method in the disclosure in the accompanying drawings with particular order, this does not really want
These steps must be executed in this particular order by asking or implying, or having to carry out step shown in whole could realize
Desired result.Additional or alternative, it is convenient to omit multiple steps are merged into a step and executed by certain steps, and/
Or a step is decomposed into execution of multiple steps etc..
Through the above description of the embodiments, those skilled in the art is it can be readily appreciated that example described herein is implemented
Mode can also be realized by software realization in such a way that software is in conjunction with necessary hardware.Therefore, according to the disclosure
The technical solution of embodiment can be embodied in the form of software products, which can store non-volatile at one
Property storage medium (can be CD-ROM, USB flash disk, mobile hard disk etc.) in or network on, including some instructions are so that a calculating
Equipment (can be personal computer, server, mobile terminal or network equipment etc.) is executed according to disclosure embodiment
Method.
In an exemplary embodiment of the disclosure, a kind of electronic equipment that can be realized the above method is additionally provided.
Person of ordinary skill in the field it is understood that various aspects of the disclosure can be implemented as system, method or
Program product.Therefore, various aspects of the disclosure can be with specific implementation is as follows, it may be assumed that complete hardware embodiment, complete
The embodiment combined in terms of full Software Implementation (including firmware, microcode etc.) or hardware and software, can unite here
Referred to as circuit, " module " or " system ".
The electronic equipment 600 of this embodiment according to the disclosure is described referring to Fig. 6.The electronics that Fig. 6 is shown
Equipment 600 is only an example, should not function to the embodiment of the present disclosure and use scope bring any restrictions.
As shown in fig. 6, electronic equipment 600 is showed in the form of universal computing device.The component of electronic equipment 600 can wrap
It includes but is not limited to: at least one above-mentioned processing unit 610, at least one above-mentioned storage unit 620, the different system components of connection
The bus 630 of (including storage unit 620 and processing unit 610).
Wherein, the storage unit is stored with program code, and said program code can be held by the processing unit 610
Row, so that various according to the disclosure described in the execution of the processing unit 610 above-mentioned " illustrative methods " part of this specification
The step of illustrative embodiments.For example, the processing unit 610 can execute step S110 as shown in fig. 1: according to pre-
If time range create corresponding with the time range the first data subregion, the first data subregion is for described in storing
The major key of slowly varying dimension table update in the time range before value, and the major key updates in the time range
Number is no more than once；Step S120: the second data subregion of creation, the second data subregion are corresponding for storing the major key
The latest value of current queries time；Step S130: main described in the first data subregion before extracting the target service time
The newest value of key, to obtain the history full dose data of the target service time；It extracts described in the second data subregion
The value of major key, to obtain the full dose data of current queries time.
Storage unit 620 may include the readable medium of volatile memory cell form, such as Random Access Storage Unit
(RAM) 6201 and/or cache memory unit 6202, it can further include read-only memory unit (ROM) 6203.
Storage unit 620 can also include program/utility with one group of (at least one) program module 6205
6204, such program module 6205 includes but is not limited to: operating system, one or more application program, other program moulds
It may include the realization of network environment in block and program data, each of these examples or certain combination.
Bus 630 can be to indicate one of a few class bus structures or a variety of, including storage unit bus or storage
Cell controller, peripheral bus, graphics acceleration port, processing unit use any bus structures in a variety of bus structures
Local bus.
Electronic equipment 600 can also be with one or more external equipments 1100 (such as keyboard, sensing equipment, bluetooth equipment
Deng) communication, can also be enabled a user to one or more equipment interact with the electronic equipment 600 communicate, and/or with make
Any equipment (such as the router, modulation /demodulation that the electronic equipment 600 can be communicated with one or more of the other calculating equipment
Device etc.) communication.This communication can be carried out by input/output (I/O) interface 650.Also, electronic equipment 600 can be with
By network adapter 660 and one or more network (such as local area network (LAN), wide area network (WAN) and/or public network,
Such as internet) communication.As shown, network adapter 660 is communicated by bus 630 with other modules of electronic equipment 600.
It should be understood that although not shown in the drawings, other hardware and/or software module can not used in conjunction with electronic equipment 600, including but not
Be limited to: microcode, device driver, redundant processing unit, external disk drive array, RAID system, tape drive and
Data backup storage system etc..
Through the above description of the embodiments, those skilled in the art is it can be readily appreciated that example described herein is implemented
Mode can also be realized by software realization in such a way that software is in conjunction with necessary hardware.Therefore, according to the disclosure
The technical solution of embodiment can be embodied in the form of software products, which can store non-volatile at one
Property storage medium (can be CD-ROM, USB flash disk, mobile hard disk etc.) in or network on, including some instructions are so that a calculating
Equipment (can be personal computer, server, terminal installation or network equipment etc.) is executed according to disclosure embodiment
Method.
In an exemplary embodiment of the disclosure, a kind of computer readable storage medium is additionally provided, energy is stored thereon with
Enough realize the program product of this specification above method.In some possible embodiments, various aspects of the disclosure may be used also
In the form of being embodied as a kind of program product comprising program code, when described program product is run on the terminal device, institute
Program code is stated for executing the terminal device described in above-mentioned " illustrative methods " part of this specification according to this public affairs
The step of opening various illustrative embodiments.
Refering to what is shown in Fig. 7, describing the program product for realizing the above method according to embodiment of the present disclosure
700, can using portable compact disc read only memory (CD-ROM) and including program code, and can in terminal device,
Such as it is run on PC.However, the program product of the disclosure is without being limited thereto, in this document, readable storage medium storing program for executing can be with
To be any include or the tangible medium of storage program, the program can be commanded execution system, device or device use or
It is in connection.
Described program product can be using any combination of one or more readable mediums.Readable medium can be readable letter
Number medium or readable storage medium storing program for executing.Readable storage medium storing program for executing for example can be but be not limited to electricity, magnetic, optical, electromagnetic, infrared ray or
System, device or the device of semiconductor, or any above combination.The more specific example of readable storage medium storing program for executing is (non exhaustive
List) include: electrical connection with one or more conducting wires, portable disc, hard disk, random access memory (RAM), read-only
Memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read only memory
(CD-ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.
Computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
In carry readable program code.The data-signal of this propagation can take various forms, including but not limited to electromagnetic signal,
Optical signal or above-mentioned any appropriate combination.Readable signal medium can also be any readable Jie other than readable storage medium storing program for executing
Matter, the readable medium can send, propagate or transmit for by instruction execution system, device or device use or and its
The program of combined use.
The program code for including on readable medium can transmit with any suitable medium, including but not limited to wirelessly, have
Line, optical cable, RF etc. or above-mentioned any appropriate combination.
Can with any combination of one or more programming languages come write for execute the disclosure operation program
Code, described program design language include object oriented program language-Java, C++ etc., further include conventional
Procedural programming language-such as " C " language or similar programming language.Program code can be fully in user
It calculates and executes in equipment, partly executes on a user device, being executed as an independent software package, partially in user's calculating
Upper side point is executed on a remote computing or is executed in remote computing device or server completely.It is being related to far
Journey calculates in the situation of equipment, and remote computing device can pass through the network of any kind, including local area network (LAN) or wide area network
(WAN), it is connected to user calculating equipment, or, it may be connected to external computing device (such as utilize ISP
To be connected by internet).
In addition, above-mentioned attached drawing is only the schematic theory of the processing according to included by the method for disclosure exemplary embodiment
It is bright, rather than limit purpose.It can be readily appreciated that the time that above-mentioned processing shown in the drawings did not indicated or limited these processing is suitable
Sequence.In addition, be also easy to understand, these processing, which can be, for example either synchronously or asynchronously to be executed in multiple modules.
Those skilled in the art after considering the specification and implementing the invention disclosed here, will readily occur to its of the disclosure
His embodiment.The disclosure is intended to cover any variations, uses, or adaptations of the disclosure, these modifications, purposes or
Adaptive change follow the general principles of this disclosure and including the undocumented common knowledge in the art of the disclosure or
Conventional techniques.The description and examples are only to be considered as illustrative, and the true scope and spirit of the disclosure are by claim
It points out.
Claims (10)
1. a kind of data processing method of slowly varying dimension table characterized by comprising
The first data subregion corresponding with the time range is created according to preset time range, the first data subregion is used
Value before the major key for storing the slowly varying dimension table updates in the time range, and the major key is in the time model
The number for enclosing interior update is no more than once；
The second data subregion is created, the second data subregion corresponds to the newest of current queries time for storing the major key
Value；
The newest value of major key described in the first data subregion before the extraction target service time, to obtain the target
The history full dose data of business hours；The value of major key described in the second data subregion is extracted, to obtain the current queries time
Full dose data.
2. the data processing method of slowly varying dimension table according to claim 1, which is characterized in that the quantity of the major key
It is multiple；
It is described that the first data subregion corresponding with the time range, first data point are created according to preset time range
Area is for storing the value before the major key of the slowly varying dimension table updates in the time range, comprising:
When the value of the part major key does not update, in the first data subregion with the major key pair that does not update
The position answered is sky.
3. the data processing method of slowly varying dimension table according to claim 1, which is characterized in that first data point
Multiple data storage areas are provided in area and the second data subregion, the data storage area is for storing corresponding different institutes
State the data of major key.
4. the data processing method of slowly varying dimension table according to claim 1, which is characterized in that the extraction target industry
The newest value of major key described in the first data subregion before the business time, to obtain the history of the target service time
Full dose data, comprising:
Obtain the target service time；
According to the target service time, all the first data subregions before extracting the target service time, and
The data of and renewal time corresponding with each major key in the first data subregion near the target service time are obtained,
The corresponding and renewal time with each major key is determined as the target industry near the data of the target service time
The history full dose data of business time.
5. the data processing method of slowly varying dimension table according to claim 1, which is characterized in that the method is also wrapped
It includes:
When the major key is deleted, set empty for the position for corresponding to the major key in the second data subregion.
6. the data processing method of slowly varying dimension table according to claim 1, which is characterized in that the quantity of the major key
It is multiple；
The method also includes:
When each major key does not update in the time range, then corresponds to the time range and do not create first number
According to subregion.
7. a kind of data processing equipment of slowly varying dimension table characterized by comprising
First data partition creating module, for according to preset time range creation the first number corresponding with the time range
According to subregion, before the major key that the first data subregion is used to store the slowly varying dimension table updates in the time range
Value, and the number that the major key updates in the time range is no more than once；
Second data partition creating module, for creating the second data subregion, the second data subregion is for storing the master
Key corresponds to the updated latest value of current queries time；
Full dose data acquisition module, extract the target service time before the first data subregion described in major key it is newest
Value, to obtain the history full dose data of the target service time；The value of major key described in the second data subregion is extracted, with
Obtain the full dose data of current queries time.
8. the data processing equipment of slowly varying dimension table according to claim 7, which is characterized in that the full dose data obtain
Modulus block, comprising:
Target service time acquisition unit, for obtaining the target service time；
History full dose data capture unit, for before according to the target service time, extracting the target service time
All the first data subregions, and obtain in the first data subregion and renewal time corresponding with each major key near
The data of the nearly target service time, when by and the renewal time corresponding with each major key near the target service
Between data be determined as the history full dose data of the target service time.
9. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program quilt
Processor realizes the data processing method of slowly varying dimension table described in any one of claims 1-6 when executing.
10. a kind of electronic equipment characterized by comprising
Processor；And
Memory, for storing the executable instruction of the processor；
Wherein, the processor is configured to require 1-6 described in any item via executing the executable instruction and carry out perform claim
The data processing method of slowly varying dimension table.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810962478.0A CN109376148B (en) | 2018-08-22 | 2018-08-22 | Data processing method and device for slow change dimension table and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810962478.0A CN109376148B (en) | 2018-08-22 | 2018-08-22 | Data processing method and device for slow change dimension table and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109376148A true CN109376148A (en) | 2019-02-22 |
| CN109376148B CN109376148B (en) | 2023-07-18 |
Family
ID=65404468
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810962478.0A Active CN109376148B (en) | 2018-08-22 | 2018-08-22 | Data processing method and device for slow change dimension table and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109376148B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111274253A (en) * | 2020-01-10 | 2020-06-12 | 北京奇艺世纪科技有限公司 | Generation method and device of full-scale partition view, storage medium and electronic device |
| CN112306999A (en) * | 2020-10-19 | 2021-02-02 | 亚信科技（中国）有限公司 | Data auditing method and device, electronic equipment and computer-readable storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080256029A1 (en) * | 2007-04-13 | 2008-10-16 | Acei Ab | Partition management system |
| CN102141963A (en) * | 2010-01-28 | 2011-08-03 | 阿里巴巴集团控股有限公司 | Method and equipment for analyzing data |
| US8260822B1 (en) * | 2008-08-12 | 2012-09-04 | United Services Automobile Association (Usaa) | Systems and methods for storing and querying slowly changing dimensions |
| CN103577474A (en) * | 2012-08-03 | 2014-02-12 | 阿里巴巴集团控股有限公司 | Method and system for updating database |
| CN106709269A (en) * | 2017-03-13 | 2017-05-24 | 山东众阳软件有限公司 | Creation method and system for medical big data warehouse |
| CN107861989A (en) * | 2017-10-17 | 2018-03-30 | 平安科技（深圳）有限公司 | Partitioned storage method, apparatus, computer equipment and the storage medium of data |
| CN112579692A (en) * | 2019-09-29 | 2021-03-30 | 杭州海康威视数字技术股份有限公司 | Data synchronization method, device, system, equipment and storage medium |
-
2018
- 2018-08-22 CN CN201810962478.0A patent/CN109376148B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080256029A1 (en) * | 2007-04-13 | 2008-10-16 | Acei Ab | Partition management system |
| US8260822B1 (en) * | 2008-08-12 | 2012-09-04 | United Services Automobile Association (Usaa) | Systems and methods for storing and querying slowly changing dimensions |
| CN102141963A (en) * | 2010-01-28 | 2011-08-03 | 阿里巴巴集团控股有限公司 | Method and equipment for analyzing data |
| CN103577474A (en) * | 2012-08-03 | 2014-02-12 | 阿里巴巴集团控股有限公司 | Method and system for updating database |
| CN106709269A (en) * | 2017-03-13 | 2017-05-24 | 山东众阳软件有限公司 | Creation method and system for medical big data warehouse |
| CN107861989A (en) * | 2017-10-17 | 2018-03-30 | 平安科技（深圳）有限公司 | Partitioned storage method, apparatus, computer equipment and the storage medium of data |
| CN112579692A (en) * | 2019-09-29 | 2021-03-30 | 杭州海康威视数字技术股份有限公司 | Data synchronization method, device, system, equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 李海龙 等: "易制毒领域数据仓库数据模型设计", no. 07, pages 54 - 56 * |
| 王涛;: "使用JSON模型优化数据拉链表", no. 03, pages 90 - 91 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111274253A (en) * | 2020-01-10 | 2020-06-12 | 北京奇艺世纪科技有限公司 | Generation method and device of full-scale partition view, storage medium and electronic device |
| CN112306999A (en) * | 2020-10-19 | 2021-02-02 | 亚信科技（中国）有限公司 | Data auditing method and device, electronic equipment and computer-readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109376148B (en) | 2023-07-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109614427A (en) | The access method and device of Various database, storage medium and electronic equipment | |
| CN107895049A (en) | Data processing method and device, computer-readable recording medium, electronic equipment | |
| CN106933837A (en) | A kind of database table model and creation method | |
| CN109491989A (en) | Data processing method and device, electronic equipment, storage medium | |
| CN105706041A (en) | Technique for global deduplication across datacenters with minimal coordination | |
| CN111708557B (en) | Method, device and storage medium for updating configuration file | |
| CN109144785A (en) | Method and apparatus for Backup Data | |
| CN114244595B (en) | Authority information acquisition method and device, computer equipment and storage medium | |
| CN107832448A (en) | Database operation method, device and equipment | |
| CN108733787A (en) | Database operation method, device, electronic equipment and storage medium | |
| EP3388957A1 (en) | Method and system for optimizing database system, electronic device, and storage medium | |
| US20200082026A1 (en) | Graph data processing | |
| CN109376148A (en) | The data processing method of slowly varying dimension table, device, electronic equipment | |
| CN110162518A (en) | Data grouping method, apparatus, electronic equipment and storage medium | |
| CN109271428A (en) | Data pick-up method and method for exhibiting data based on geography information | |
| CN108763577A (en) | node processing method and device, storage medium and electronic equipment | |
| CN110162537A (en) | Data query method and device, storage medium and electronic equipment | |
| CN104881454A (en) | Updating method and system of parameter | |
| CN109086382A (en) | A kind of method of data synchronization, device, equipment and storage medium | |
| US20190354576A1 (en) | Atom-based sensible synchronization for information indexing | |
| CN103177092A (en) | Data updating method and system of knowledge base and knowledge base | |
| CN110321694A (en) | Operating right distribution method and relevant device based on tag update system | |
| CN110460647A (en) | Dispatching method, device, electronic equipment and the storage medium of network node | |
| CN109493077A (en) | Activity recognition method and device, electronic equipment, storage medium | |
| CN105190601B (en) | The sequence based on region on mobile device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |