CN106407351A - Data processing method and device applied to database - Google Patents
Data processing method and device applied to database Download PDFInfo
- Publication number
- CN106407351A CN106407351A CN201610805597.6A CN201610805597A CN106407351A CN 106407351 A CN106407351 A CN 106407351A CN 201610805597 A CN201610805597 A CN 201610805597A CN 106407351 A CN106407351 A CN 106407351A
- Authority
- CN
- China
- Prior art keywords
- data
- acquisition system
- data acquisition
- pond
- update
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2358—Change logging, detection, and notification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data processing method and device applied to a database. The database comprises multiple same data sets. The method comprises the steps of receiving an update instruction for the database, and selecting a first data set from the multiple same data sets; updating the first data set, and marking the first data set to be in an un-queriable state; and receiving a query instruction for the database, and selecting a second data set for data query, wherein the second data set is marked to be in a queriable state. Both the data query operation and the data update operation can be carried out on the database at the same time, the data query operation cannot be influenced by the data update operation, and the query stability of the database when the database is updated can be effectively ensured.
Description
Technical field
The present invention relates to the information processing technology, more particularly, to a kind of data processing method being applied to data base, Yi Jiyi
Plant the data processing equipment being applied to data base.
Background technology
In the application scenarios of the Internet big data, there is a kind of big data real time inquiry system, please for a certain inquiry
Ask, need to inquire rapidly satisfactory one or more data from more than one hundred million data records, inquiry system needs simultaneously
Meet the query demand of high concurrent.Because data updates very fast it is therefore desirable to do periodically more to the background data base of inquiry system
Newly, for ensureing the ageing of Query Result it is desirable to the renewal of database operates within the time as short as possible.
Every data in data base is provided with unique mark, when writing data, if there is a certain number in tables of data
According to mark, then support to carry out write operation to this data it is seen that the existing method updating the data storehouse meets data base's
Upgrade demand, the performance of data base has obtained certain lifting.
But, the existing method updating the data storehouse still deposits problems with：Data general record in data base is in data
In table, if be updated to the data in tables of data, in the presence of data base's " write lock " mechanism, do not allow to data
Data in storehouse carries out inquiry operation, thus leading to the inquiry less stable of data base.
Content of the invention
The embodiment provides a kind of data processing method being applied to data base, and one kind is applied to data
The data processing equipment in storehouse, in order to solve the problems, such as to update the data the inquiry less stable of data base during storehouse.
According to one aspect of the present invention, there is provided a kind of data processing method being applied to data base, described data base
Include multiple identical data acquisition systems, methods described includes：
Receive the update instruction to described data base, from the plurality of identical data acquisition system, select the first data set
Close；
Update described first data acquisition system, the first data acquisition system described in labelling is to inquire about state；
Receive the inquiry instruction to described data base, selected marker has the second data acquisition system that can inquire about state to carry out data
Inquiry.
According to another aspect of the present invention, additionally provide a kind of data processing equipment being applied to data base, described dress
Put including：
Update instruction receiver module, for receiving the update instruction to described data base, described data base includes multiple
Identical data acquisition system；
First data acquisition system selecting module, for selecting the first data acquisition system from the plurality of identical data acquisition system；
First data acquisition system update module, for updating described first data acquisition system, the first data acquisition system described in labelling is
State can not be inquired about；
Inquiry instruction receiver module, for receiving the inquiry instruction to described data base, selected marker has can inquire about state
The second data acquisition system carry out data query.
Compared with prior art, in the embodiment of the present invention, data base includes multiple identical data acquisition systems, right receiving
After the update instruction of data base, optionally go out the first data acquisition system from multiple identical data acquisition systems, update the first data acquisition system,
Labelling first data acquisition system is state of can not inquiring about, and receives the inquiry instruction to data base, recognizing the first data acquisition system is
The first data acquisition system can not carried out with data query, but selected marker has the second number that can inquire about state after inquiry state
Carry out data query according to set it is achieved that data base is carried out with data query operation data update operation, data updates simultaneously
Operation will not have undesirable effect to data query operation, and the inquiry stability of when updating the data storehouse data base has been effectively ensured.
Described above is only the general introduction of technical solution of the present invention, in order to better understand the technological means of the present invention,
And can be practiced according to the content of description, and in order to allow the above and other objects of the present invention, feature and advantage can
Become apparent, below especially exemplified by the specific embodiment of the present invention.
Brief description
By reading the detailed description of hereafter preferred implementation, various other advantages and benefit are common for this area
Technical staff will be clear from understanding.Accompanying drawing is only used for illustrating the purpose of preferred implementation, and is not considered as to the present invention
Restriction.And in whole accompanying drawing, it is denoted by the same reference numerals identical part.In the accompanying drawings：
The flow chart that Fig. 1 shows according to embodiments of the present invention 1 data processing method being applied to data base；
The flow chart that Fig. 2 shows according to embodiments of the present invention 2 data processing method being applied to data base；
The flow chart that Fig. 3 shows according to embodiments of the present invention 3 data processing method being applied to data base；
The flow chart that Fig. 4 shows the data processing method being applied to data base according to one example of the present invention；
Fig. 5 shows the schematic diagram of the data processing method being applied to data base according to one example of the present invention；
Fig. 6 shows that the change procedure of the first state table of the first data acquisition system according to one example of the present invention is illustrated
Figure；
Fig. 7 shows the structured flowchart of according to embodiments of the present invention 1 data processing equipment being applied to data base；
Fig. 8 shows the structured flowchart of according to embodiments of the present invention 2 data processing equipment being applied to data base；
Fig. 9 shows the structured flowchart of according to embodiments of the present invention 3 data processing equipment being applied to data base.
Specific embodiment
It is more fully described the exemplary embodiment of the disclosure below with reference to accompanying drawings.Although showing the disclosure in accompanying drawing
Exemplary embodiment it being understood, however, that may be realized in various forms the disclosure and should not be by embodiments set forth here
Limited.On the contrary, these embodiments are provided to be able to be best understood from the disclosure, and can be by the scope of the present disclosure
Complete conveys to those skilled in the art.
Data base is the warehouse to organize, store and to manage data according to data structure.Big due to storing in data base
Amount data, so data base can carry out data query as data source for data query side.Generally the data in data base is
Change, need periodically or non-periodically to be updated, data renewal side can extract data the number with extracting from file system
Carry out data renewal according to data base.
The present invention only improves to the data process method of data base, need not build other hardware, therefore has very strong
Compatibility, all suitable to Mongo DB data base, MSQL data base, Redis data base, Cassandra database data base
With.
Embodiment 1
Reference Fig. 1, the flow chart showing according to embodiments of the present invention 1 data processing method being applied to data base,
The data base of the embodiment of the present invention includes multiple identical data acquisition systems, and the method specifically may comprise steps of：
Step 101, the update instruction to described data base for the reception, select first from the plurality of identical data acquisition system
Data acquisition system.
Prior art is in order to ensure the accuracy of the data query result for data base it will usually write to data lab setting
Enter lock mechanism, specifically, if data base is in write operation when updating operation, do not allow data base is carried out with inquiry behaviour
Make.Although existing method ensure that the accuracy of data query result, for same data base data query operation and
Data updates operation and can not carry out, thus leading to data query performance to be greatly lowered simultaneously.
The embodiment of the present invention, in order to overcome defect present in prior art, is changed to the data composition in data base
Enter.Specifically, all business datums in data base are backed up, different business data is combined into a data acquisition system,
Therefore data base includes multiple identical data acquisition systems.The number of identical data set can be set according to actual, example
As data base includes the first data acquisition system, the second data acquisition system and the 3rd data acquisition system.There is data query pond, number in data base
It is investigated that asking pond is the place that data base is carried out with data query, before receiving the update instruction to data base, in data base
Multiple identical data acquisition systems are respectively positioned in data query pond.
After receiving the update instruction to data base, because data base includes multiple identical data acquisition systems, thus first
First needing to select a data acquisition system from multiple data acquisition systems is the first data acquisition system as data upgating object.
Step 102, renewal the first data acquisition system, the first data acquisition system described in labelling is to inquire about state.
Data base includes multiple identical data acquisition systems, when updating the first data acquisition system, in order to stop to the first number
Carry out data query according to set it is ensured that data query speed, this method carries out status indication to data acquisition system, by the first data set
Conjunction is labeled as inquiring about state.Wherein it is possible to be marked to state using label symbol, label symbol can include multiple
Form, for example, be used numeral as label symbol, different digital instruction data acquisition system is in different conditions；Word is used as mark
Note symbol etc..
When data query side detect the first data acquisition system be marked with can not inquiry state after, the first data set will not be selected
Conjunction carries out data query, but selected marker has other data acquisition systems that can inquire about state to carry out data query, it is achieved thereby that
Data base is carried out with data renewal operation data inquiry operation simultaneously and two kinds of operations do not interfere with each other, thus ensure that data
Inquiry velocity.
Data base includes multiple identical data acquisition systems, in data base updating procedure, can be to multiple data acquisition systems
Updated one by one；Data query can also be used for, to other multiple data acquisition systems simultaneously by remaining one or several data acquisition systems
It is updated.
Step 103, the inquiry instruction to described data base for the reception, selected marker has the second data acquisition system that can inquire about state
Carry out data query.
During updating the first data acquisition system, if receiving the inquiry instruction to data base, can be in data base
The flag state of data acquisition system is identified, and is determining that the first data acquisition system is can not be after inquiry state, not in the first data set
Carry out data query in conjunction, but selected marker has the second data acquisition system that can inquire about state to carry out data query.
The improved method built with respect to other hardware, the method described in the embodiment of the present invention is only on data process method
Improve, therefore cost of implementation is relatively low, have the characteristics that simultaneously compatibility stronger it is adaptable to multitype database.
Because the data base of the embodiment of the present invention includes multiple identical data acquisition systems, a data acquisition system can be selected
Carry out the foundation of database index, select other identical data acquisition systems to carry out data query operation, therefore can be effectively ensured
Set up the inquiry stability of data base during database index.
According to the embodiment of the present invention, data base includes multiple identical data acquisition systems, is receiving the renewal to data base
After instruction, optionally go out the first data acquisition system from multiple identical data acquisition systems, update the first data acquisition system, labelling first data
Collection is combined into state of can not inquiring about, and receives the inquiry instruction to data base, is to inquire about state recognizing the first data acquisition system
Afterwards, the first data acquisition system is not carried out with data query, but selected marker has the second data acquisition system that can inquire about state to enter line number
It is investigated that asking it is achieved that data base is carried out with data query operation data update operation, data updates operation will not logarithm simultaneously
Have undesirable effect according to inquiry operation, the inquiry stability of when updating the data storehouse data base has been effectively ensured.
Embodiment 2
Reference Fig. 2, the flow chart showing according to embodiments of the present invention 2 data processing method being applied to data base,
Data base includes multiple identical data acquisition systems, and the method specifically may comprise steps of：
Step 201, the update instruction to data base for the reception, select the first data set from multiple identical data acquisition systems
Close.
Step 202, by the first data acquisition system from data query pond move to data update pond, by the first data acquisition system from number
It is investigated that before inquiry pond moves to data renewal pond, the first data acquisition system carries first that the first data acquisition system is located at data query pond
Confidence ceases.
Data base includes data query pond data and updates pond, and data query pond is to carry out data query to data base
Place, it is the place that data base is carried out with data renewal that data updates pond.Before receiving the update instruction to data base, number
It is located in data query pond according to the multiple identical data acquisition systems in storehouse, after receiving to data base's update instruction, from multiple
Optionally go out the first data acquisition system in data acquisition system, and the first data acquisition system is moved to data from data query pond and update pond, counting
According to updating in pond, the first data acquisition system is updated.
The embodiment of the present invention is provided with positional information to each data acquisition system in data base, entrained by a certain data acquisition system
Positional information can indicate the species of the data pool residing for this data acquisition system, that is, this data acquisition system is in data query pond
Or data updates in pond.Determine that the positional information that instruction data acquisition system is located at data query pond is primary importance information, determine
Indicate that the positional information that data acquisition system is located at data renewal pond is second position information.Wherein, positional information can be word letter
Other service forms such as breath, preset numbers.This method can be by the positional information entrained by identification and change data acquisition system
Species, controls the multiple operating procedures updating the data set to carry out in order and accurately.
Because the embodiment of the present invention is before receiving the update instruction to data base, multiple data acquisition systems are located at data query
Chi Zhong, the positional information that the therefore first data acquisition system carries is the primary importance information in instruction data query pond.
Step 203, after the current queries of the first data acquisition system terminate, change primary importance information be the first data acquisition system
Update the second position information in pond positioned at data.
First data acquisition system be moved to data update behind pond, prepare the first data acquisition system to be updated operate, in order to protect
The current queries demonstrate,proving the first data acquisition system normally complete, and or not first data acquisition system are not carried out any during being limited to current queries
Operation.After current queries terminate, positional information entrained by change the first data acquisition system, due at this moment the first data acquisition system position
Update in pond in data, therefore be changed to indicate the second position information that data updates pond by its positional information.
First data acquisition system can be moved to by number according to the inquiry duration needed for the operation of actual set current queries in practice
It is investigated that after asking pond and being spaced default inquiry duration, then the positional information of the first data acquisition system is modified.For example, by the first number
Move to data query pond according to set, after the default 10s in interval, the positional information of the first data acquisition system is modified.
Step 204, monitor after the first data acquisition system carries second position information, update in pond to the first data in data
Set carries out data renewal, and labelling first data acquisition system is to inquire about state.
Second position information is started mark as what data acquisition system renewal operated by the embodiment of the present invention.
After data updates bearing data storehouse transmission update instruction, data can be updated entrained by newly-increased data acquisition system in pond
Positional information is monitored, if monitoring the first newly-increased data acquisition system to carry second position information, starts to the first number
It is updated operating according to set；If monitoring the first newly-increased data acquisition system, but do not monitor the first data acquisition system and carry
Two positional informationes, then be not updated to the first data acquisition system operating.
In concrete operations, if monitoring the first data acquisition system in data renewal Chi Zhongwei to carry second position information,
Illustrate that move to data to update pond and the positional information of change the first data acquisition system by the first data acquisition system is second position information
One of or two operating procedures go wrong, at this moment can execute warning operation, can be to data query in warning period
The second data acquisition system in pond carries out data query.Therefore this method, while ensureing that data query operation is normally carried out, is
Operation maintenance personnel provides grace time to solve failure problems.
Data base includes multiple identical data acquisition systems, in order to realize data base is made a look up with operation and update simultaneously
Operation, and two kinds of operations do not interfere with each other, the embodiment of the present invention when updating the first data acquisition system, the shape to the first data acquisition system
State is marked, and is marked as to inquire about state, even if so receiving to data during updating the first data acquisition system
The inquiry instruction in storehouse, is also marked with and can not select the first data acquisition system to enter after inquiry state recognizing the first data acquisition system
Row data query, but select other data acquisition systems to carry out data query.
Step 205, during updating the first data acquisition system, if received to the inquiry of data base instruction, select mark
Note has the second data acquisition system that can inquire about state to carry out data query.
During updating the first data acquisition system, after receiving the inquiry instruction to data base, due to the first data acquisition system
It is marked with state of can not inquiring about, therefore selected marker the second data acquisition system that can inquire about state can be had to carry out data query.
Step 206, first data acquisition system update terminate after, by the first data acquisition system from data update pond move back to number
It is investigated that in inquiry pond, and to change the second position information that the first data acquisition system carries be primary importance information.
It is only the place that data base is carried out with data renewal that data updates pond, when completing the renewal to the first data acquisition system
Afterwards, the first data acquisition system can be moved back in data query pond.Because the data pool residing for the first data acquisition system there occurs change,
It is thus desirable to being altered to the positional information entrained by the first data acquisition system, it is changed to indicate first of data query pond
Confidence ceases.
In concrete operations, after completing the renewal to the first data acquisition system, the flag state of the first data acquisition system can be entered
Row adjustment, the first data acquisition system is re-flagged as inquiring about state, and functional module is detecting in data renewal pond
First data acquisition system is can to move back to the first data acquisition system in data query pond after inquiry state, thus completing to the first number
Renewal operation according to set.According to the method described above data renewal can be carried out to other data acquisition systems in data base.
The present invention, before data acquisition system renewal, the first data acquisition system is moved to data from data query pond and updates pond, and
And the primary importance information carrying the first data acquisition system is changed to second position information, using second position information as data set
Close update operation start identify, recognizing after the first data acquisition system carries second position information, starting to the first data
Set carries out data and updates operation, after the first data acquisition system updates end and moved back to data query pond, by the first number
More change back primary importance information according to the second position information that set carries, the embodiment of the present invention is by the position letter to data acquisition system
The identification of breath and change operation, complete in order update the data the required multiple operating procedures of set it is ensured that data acquisition system more
The accuracy of new operation.
The present invention also has very strong fault-tolerance, if the operation updating the first data acquisition system goes wrong, can execute
Report to the police and operate, at this moment still can carry out data query to other data acquisition systems of data base, normally enter in guarantee data query operation
While row, grace time is provided to solve failure problems for operation maintenance personnel.
According to the embodiment of the present invention, data base includes multiple identical data acquisition systems, is receiving the renewal to data base
After instruction, optionally go out the first data acquisition system from multiple identical data acquisition systems, update the first data acquisition system, labelling first data
Collection is combined into state of can not inquiring about, and receives the inquiry instruction to data base, is to inquire about state recognizing the first data acquisition system
Afterwards, the first data acquisition system is not carried out with data query, but selected marker has the second data acquisition system that can inquire about state to enter line number
It is investigated that asking it is achieved that data base is carried out with data query operation data update operation, data updates operation will not logarithm simultaneously
Have undesirable effect according to inquiry operation, the inquiry stability of when updating the data storehouse data base has been effectively ensured.
Embodiment 3
Reference Fig. 3, the flow chart showing according to embodiments of the present invention 3 data processing method being applied to data base,
Data base includes multiple identical data acquisition systems, and the method specifically may comprise steps of：
Step 301, the update instruction to data base for the reception, select the first data set from multiple identical data acquisition systems
Close.
Step 302, inquiring about of change the first data acquisition system are designated preparation renewal request mark, are receiving to data base
Update instruction before, the status indicator of the first data acquisition system is the inquired about mark that instruction can inquire about state.
For multiple operating procedures that are orderly and accurately completing in data acquisition system renewal process, the embodiment of the present invention is to data base
In each data acquisition system all marked status indicator, in data acquisition system renewal process, the status indicator of data acquisition system there occurs
Repeatedly change.Because status indicator can indicate the data processing stage residing for data acquisition system, identification state therefore can be passed through
The species of mark, determines species and the operation initial time of pre-execution operation, thus the renewal behaviour of accurately complete paired data set
Make.
In the embodiment of the present invention, the change procedure of status indicator can include：Mark can be inquired about prepare to update request mark
Know to prepare to update and confirm that identifying more new logo can inquire about mark.Before receiving the update instruction to data base,
First data acquisition system is located in data query pond, and at this moment the status indicator of the first data acquisition system is to inquire about mark, data query
Side, after the inquired about mark recognizing the first data acquisition system, can select the first data acquisition system to carry out data query.
After receiving update instruction to data base, pre- data renewal is carried out to the first data acquisition system in data base, by the
The inquired about mark of one data acquisition system is changed to prepare to update request mark, and at this moment the first data acquisition system is still located at data query pond
Interior.This preparation updates request mark instruction the first data acquisition system and receives update instruction and be located in data query pond.
Step 303, detect the first data acquisition system status indicator be prepare update request mark after, by the first data set
Close and move to data renewal pond from data query pond.
After being to prepare to update request mark in the status indicator the first data acquisition system is detected, determine the first data acquisition system
Ready, wait next step operation.Mark is asked to move to data as by data acquisition system from data query pond using preparing to update
Update the sign in pond, after the preparation the first data acquisition system is detected updates request mark, by the first data acquisition system from number
Update pond it is investigated that asking pond and moving to data, update in pond in data and the business datum in the first data acquisition system is updated.
Step 304, the preparation of change the first data acquisition system update request mark and extremely prepare to update to confirm mark.
Detect after data updates newly-increased first data acquisition system in pond, the status indicator of change the first data acquisition system, by the
The status indicator of one data acquisition system is changed to prepare to update confirm mark.This preparation updates and confirms mark instruction the first data acquisition system
Update in pond and carry out renewal to prepare positioned at data.
Step 305, detect the first data acquisition system preparation update confirm mark after, update the first data acquisition system, simultaneously
The renewal for preparing changing the first data acquisition system confirms to be designated the more new logo that instruction can not inquire about state.
Confirm mark using preparing to update as the sign updating the data set, in the standard the first data acquisition system is detected
After standby renewal confirms mark, start the first data acquisition system to be updated operate, and by the status indicator of the first data acquisition system more
It is changed to more new logo.The initial period that can update in data is modified to the status indicator of the first data acquisition system, should
More new logo indicates that the first data acquisition system is in renewal process and can not be queried.
In the renewal process of the first data acquisition system, the status indicator of the first data acquisition system is more new logo, if
The first data acquisition system is detected and update and terminate, then the more new logo changing the first data acquisition system is changed to inquire about mark.
The status indicator of the first data acquisition system is detected, when the status indicator the first data acquisition system is detected is to look into
After asking mark, determine that the first data acquisition system updates and terminate, the first data acquisition system is updated pond from data and moves back to data query pond
For the inquiry of data query side, thus completing the renewal to the first data acquisition system.Using said method to other numbers in data base
It is updated according to set.
Step 306, the inquiry instruction to data base for the reception, selected marker has the second data acquisition system that can inquire about mark to carry out
Data query.
In the first data acquisition system renewal process, if receiving the inquiry instruction to data base, to number in data base
Status indicator according to set is identified, and after the status indicator recognizing the first data acquisition system is more new logo, determines
First data acquisition system is in renewal, afterwards the status indicator of the second data acquisition system is identified, if recognizing the second number
Status indicator according to set is to inquire about mark, then select the second data acquisition system to carry out data query.
The embodiment of the present invention is provided with status indicator to each data acquisition system, from receiving to the update instruction of data base to the
One data acquisition system updates during terminating, and the status indicator of the first data acquisition system there occurs multiple change, can be by identification
Different conditions identify, and accurately complete the renewal operation to the first data acquisition system.
According to the embodiment of the present invention, data base includes multiple identical data acquisition systems, is receiving the renewal to data base
After instruction, optionally go out the first data acquisition system from multiple identical data acquisition systems, update the first data acquisition system, labelling first data
Collection is combined into state of can not inquiring about, and receives the inquiry instruction to data base, is to inquire about state recognizing the first data acquisition system
Afterwards, the first data acquisition system is not carried out with data query, but selected marker has the second data acquisition system that can inquire about state to enter line number
It is investigated that asking it is achieved that data base is carried out with data query operation data update operation, data updates operation will not logarithm simultaneously
Have undesirable effect according to inquiry operation, the inquiry stability of when updating the data storehouse data base has been effectively ensured.
In actual mechanical process, can in conjunction with the embodiments 2 and the method for embodiment 3 data renewal is carried out to data base,
For making those skilled in the art be more clearly understood from associated methods, now by following examples, associated methods are carried out specifically
Bright.
Example
This example adopts Mongo DB data base as data source, select HTTP that C++ writes (HTTP,
Hypertext transfer protocol) service as data query side；Select the program conduct write by python language
Data renewal side, regularly extracts number from distributed file system (HDFS, Hadoop Distributed File System)
According to and with extract business datum the business datum in data base is updated.
In this example, Mongo DB data base includes two identical data acquisition systems, the business in each data acquisition system
Data record is in tables of data.Before receiving the update instruction to data base, two identical data acquisition systems are looked into positioned at data
Ask in pond, after receiving the inquiry to data base and indicating, from two datasets are closed, optionally go out a data acquisition system as number
Carry out data query according to query object.
This example is provided with a state machine to each data acquisition system, and each state machine corresponds to a state table, state table
The information of middle storage includes：1st, the read-write state of data acquisition system, is represented with status indicator；2nd, the connection of data acquisition system and data pool
Mark, is represented with positional information.Specifically, using numeral mark status indicator, including 0- readable state, 1- prepares renewal please
Ask, 2- prepares renewal confirmation, 3- is updating four kinds；Using the positional information of alphabetic flag data acquisition system, including A- data set
Close in data query pond, B- data acquisition system is located at data and updates in pond.
The flow chart that Fig. 4 shows the data processing method being applied to data base according to one example of the present invention；Fig. 5 shows
Go out the schematic diagram of the data processing method being applied to data base according to one example of the present invention；Fig. 6 shows according to this
The change procedure schematic diagram of the first state table of the first data acquisition system of a bright example.Can be clearer in conjunction with Fig. 5 and Fig. 6
Ground understands the method described in Fig. 4, and Fig. 4 methods described specifically may comprise steps of：
S1, the update instruction to data base for the reception, select the first data acquisition system, change first number from two datasets are closed
According to the first state table of set, status indicator is changed to 1 from 0.
After S2, the status indicator 1 detecting in the first tables of data, the first data acquisition system is moved to data from data query pond
Update in pond, after the default 10s in interval, delete the positional information A in first state table, that is, delete the first data acquisition system and look into data
Ask the connection mark in pond, first state table increases newly positional information B, that is, increase by the first data acquisition system and data updates the company in pond
Connect mark.
After S3, the positional information B detecting in first state table, status indicator is changed to 2 from 1；Further in detection
After status indicator 2 in first state table, start the first data acquisition system is updated, status indicator is changed to 3 from 2.
During updating the first data acquisition system, data renewal side obtains new data from distributed file system, and uses
The new data obtaining carries out full dose renewal to the data in the first data acquisition system, writes new data into the data of the first data acquisition system
In table.
During updating the first data acquisition system, if receiving the inquiry instruction that data query side sends to data base,
Then after the status indicator 3 in recognizing first state table, determine that the first data acquisition system is in more new state, the meeting of data query side
The data acquisition system carrying status indicator 0 is selected to carry out data query.
When updating the first data acquisition system, if adopting traditional Data Identification to know the business datum in the first data acquisition system
The data-updating method of not-renewal, then need to carry out substantial amounts of Data Identification identification operation.Due to the identification of substantial amounts of Data Identification
Operation needs take a substantial amount of time, and data acquisition system therefore can be led to update and take longer, data acquisition system renewal is less efficient etc.
The generation of problem.In order to solve the above problems, this example adopts full dose update method to the business datum in the first data acquisition system
It is updated, eliminate substantial amounts of Data Identification identification operation, simplify the operating procedure that data acquisition system updates operation, improve
The renewal efficiency of data acquisition system.
Full dose update method can effectively delete the fail data in data base simultaneously, saves failure number in inquiry data base
According to time, thus improve data query speed.A large number of services data in data acquisition system can be recorded in tables of data, can
So that full dose renewal is carried out to the tables of data corresponding to the first data acquisition system, effectively delete the failure indication in tables of data, thus carrying
The high data query speed for the first data acquisition system.
S4, treat first data acquisition system update terminate, the status indicator 3 in first state table is changed to 0；Further in inspection
After measuring the status indicator 0 in first state table, the first data acquisition system is moved back in data query pond.
S5, in data query pond, by the positional information B in the first state table of the first data acquisition system be changed to position letter
Breath A.
S6, according to the method for above-mentioned steps S1-S5 the second data acquisition system is updated operate.
This example is provided with status indicator and positional information for data acquisition system, can be by status indicator and position letter
The identification of breath and change operation, realize the accurate control to the multiple operating procedures updating the first data acquisition system, accurately complete
Renewal operation to the first data acquisition system.
Embodiment 4
With reference to Fig. 7, show the structural frames of according to embodiments of the present invention 1 data processing equipment being applied to data base
Figure, specifically can include：
Update instruction receiver module 401, for receiving the update instruction to data base, data base includes multiple identicals
Data acquisition system.
First data acquisition system selecting module 402, for selecting the first data set from the plurality of identical data acquisition system
Close.
First data acquisition system update module 403, for updating described first data acquisition system, labelling first data acquisition system is not
State can be inquired about.
Inquiry instruction receiver module 404, for receiving the inquiry instruction to data base, selected marker has can inquire about state
Second data acquisition system carries out data query.
According to the embodiment of the present invention, data base includes multiple identical data acquisition systems, is receiving the renewal to data base
After instruction, optionally go out the first data acquisition system from multiple identical data acquisition systems, update the first data acquisition system, labelling first data
Collection is combined into state of can not inquiring about, and receives the inquiry instruction to data base, is to inquire about state recognizing the first data acquisition system
Afterwards, the first data acquisition system is not carried out with data query, but selected marker has the second data acquisition system that can inquire about state to enter line number
It is investigated that asking it is achieved that data base is carried out with data query operation data update operation, data updates operation will not logarithm simultaneously
Have undesirable effect according to inquiry operation, the inquiry stability of when updating the data storehouse data base has been effectively ensured.
Embodiment 5
With reference to Fig. 8, show the structural frames of according to embodiments of the present invention 2 data processing equipment being applied to data base
Figure, specifically can include：
Update instruction receiver module 501, for receiving the update instruction to data base, data base includes multiple identicals
Data acquisition system.
First data acquisition system selecting module 502, for selecting the first data set from the plurality of identical data acquisition system
Close.
First data pool mobile module 503, updates pond for the first data acquisition system is moved to data from data query pond,
By the first data acquisition system before data query pond moves to data renewal pond, the first data acquisition system carries the first data acquisition system and is located at
The primary importance information in data query pond.
Primary importance information changes module 504, after the current queries for treating the first data acquisition system terminate, changes first
Positional information is that the first data acquisition system is located at the second position information that data updates pond.
Positional information monitoring module 505, is used for monitoring the first data acquisition system and carries second position information.
First data acquisition system update module 506, carries out data renewal for updating in pond in data to the first data acquisition system,
Labelling first data acquisition system is to inquire about state.
Inquiry instruction receiver module 507, for receiving the inquiry instruction to data base, selected marker has can inquire about state
Second data acquisition system carries out data query.
Second data pool mobile module 508, for first data acquisition system update terminate after, by the first data acquisition system from
Data updates pond and moves back in data query pond.
Second position information changes module 509, is first for changing the second position information that the first data acquisition system carries
Positional information.
It is preferable that described device also includes in the embodiment of the present invention：
Warning operation executing module, if carry for monitoring described first data acquisition system in described data renewal Chi Zhongwei
There is described second position information, then execution is reported to the police and operated, can be to described second in described data query pond in warning period
Data acquisition system carries out data query.
According to the embodiment of the present invention, data base includes multiple identical data acquisition systems, is receiving the renewal to data base
After instruction, optionally go out the first data acquisition system from multiple identical data acquisition systems, update the first data acquisition system, labelling first data
Collection is combined into state of can not inquiring about, and receives the inquiry instruction to data base, is to inquire about state recognizing the first data acquisition system
Afterwards, the first data acquisition system is not carried out with data query, but selected marker has the second data acquisition system that can inquire about state to enter line number
It is investigated that asking it is achieved that data base is carried out with data query operation data update operation, data updates operation will not logarithm simultaneously
Have undesirable effect according to inquiry operation, the inquiry stability of when updating the data storehouse data base has been effectively ensured.
And, before data acquisition system updates, the first data acquisition system is moved to data from data query pond and updates pond, and
The primary importance information that first data acquisition system is carried is changed to second position information, using second position information as data acquisition system
Update operation start identify, recognizing after the first data acquisition system carries second position information, starting to the first data set
Conjunction carries out data and updates operation, after the first data acquisition system updates end and moved back to data query pond, by the first data
Gather the second position information that carries and more change back primary importance information, the change based on diverse location information for the present invention and identification behaviour
Make it is ensured that the multistep of data acquisition system is updated with the carrying out in order it is ensured that data updates the accuracy of operation of operation.
In addition, the present invention also has very strong fault-tolerance, if the operation updating the data first data acquisition system in storehouse occurs
Problem, then can execute warning operation, at this moment still can carry out data query to other data acquisition systems of data base, is ensureing that data looks into
While inquiry operation is normally carried out, grace time is provided to solve failure problems for operation maintenance personnel.
Embodiment 6
With reference to Fig. 9, show the structural frames of according to embodiments of the present invention 3 data processing equipment being applied to data base
Figure, specifically can include：
Update instruction receiver module 601, for receiving the update instruction to data base, data base includes multiple identicals
Data acquisition system.
First data acquisition system selecting module 602, for selecting the first data set from the plurality of identical data acquisition system
Close.
First mark change module 603, is designated preparation renewal request mark for changing inquiring about of the first data acquisition system
Know, before receiving the update instruction to data base, the status indicator of the first data acquisition system is that instruction can inquire about looking into of state
Ask mark.
First label detection module 604, the status indicator for the first data acquisition system is detected is to prepare to update request mark
Know.
First data pool mobile module 605, updates pond for the first data acquisition system is moved to data from data query pond.
Second mark changes module 606, and the preparation for changing the first data acquisition system updates request mark and updates to preparation
Confirm mark.
Second label detection module 607, the preparation for the first data acquisition system is detected updates and confirms mark.
First data acquisition system update module 608, for updating the first data acquisition system, changes the standard of the first data acquisition system simultaneously
Standby renewal confirms to be designated the more new logo that instruction can not inquire about state.
Inquiry instruction receiver module 609, for receiving the inquiry instruction to data base, selected marker has can inquire about mark
Second data acquisition system carries out data query.
According to the embodiment of the present invention, data base includes multiple identical data acquisition systems, is receiving the renewal to data base
After instruction, optionally go out the first data acquisition system from multiple identical data acquisition systems, update the first data acquisition system, labelling first data
Collection is combined into state of can not inquiring about, and receives the inquiry instruction to data base, is to inquire about state recognizing the first data acquisition system
Afterwards, the first data acquisition system is not carried out with data query, but selected marker has the second data acquisition system that can inquire about state to enter line number
It is investigated that asking it is achieved that data base is carried out with data query operation data update operation, data updates operation will not logarithm simultaneously
Have undesirable effect according to inquiry operation, the inquiry stability of when updating the data storehouse data base has been effectively ensured.
In addition, the present invention is also provided with status indicator to each data acquisition system, from receiving the update instruction to data base
Update during terminating to the first data acquisition system, the status indicator of the first data acquisition system there occurs multiple change, can pass through
Identification different conditions mark, accurately completes the renewal operation to the first data acquisition system, further ensures data and update operation
Order and accuracy.
Because described device embodiment essentially corresponds to the embodiment of the method shown in earlier figures 1-6, therefore the retouching of the present embodiment
In stating, not detailed part, may refer to the related description in previous embodiment, and here does not just repeat.
Algorithm and display be not inherently related to any certain computer, virtual system or miscellaneous equipment provided herein.
Various general-purpose systems can also be used together with based on teaching in this.As described above, construct required by this kind of system
Structure be obvious.Additionally, the present invention is also not for any certain programmed language.It is understood that, it is possible to use various
Programming language realizes the content of invention described herein, and the description above language-specific done is to disclose this
Bright preferred forms.
In description mentioned herein, illustrate a large amount of details.It is to be appreciated, however, that the enforcement of the present invention
Example can be put into practice in the case of not having these details.In some instances, known method, structure are not been shown in detail
And technology, so as not to obscure the understanding of this description.
Similarly it will be appreciated that in order to simplify the disclosure and help understand one or more of each inventive aspect,
Above in the description to the exemplary embodiment of the present invention, each feature of the present invention is grouped together into single enforcement sometimes
In example, figure or descriptions thereof.However, the method for the disclosure should be construed to reflect following intention：I.e. required guarantor
The application claims of shield more features than the feature being expressly recited in each claim.More precisely, it is such as following
Claims reflected as, inventive aspect is all features less than single embodiment disclosed above.Therefore,
The claims following specific embodiment are thus expressly incorporated in this specific embodiment, wherein each claim itself
All as the separate embodiments of the present invention.
Those skilled in the art are appreciated that and the module in the equipment in embodiment can be carried out adaptively
Change and they are arranged in one or more equipment different from this embodiment.Can be the module in embodiment or list
Unit or assembly be combined into a module or unit or assembly, and can be divided in addition multiple submodule or subelement or
Sub-component.In addition to such feature and/or at least some of process or unit exclude each other, can adopt any
Combination is to all features disclosed in this specification (including adjoint claim, summary and accompanying drawing) and so disclosed
Where method or all processes of equipment or unit are combined.Unless expressly stated otherwise, this specification (includes adjoint power
Profit requires, summary and accompanying drawing) disclosed in each feature can carry out generation by the alternative features providing identical, equivalent or similar purpose
Replace.
Although additionally, it will be appreciated by those of skill in the art that some embodiments described herein include other embodiments
In included some features rather than further feature, but the combination of the feature of different embodiment means to be in the present invention's
Within the scope of and form different embodiments.For example, in the following claims, embodiment required for protection appoint
One of meaning can in any combination mode using.
The all parts embodiment of the present invention can be realized with hardware, or to run on one or more processor
Software module realize, or with combinations thereof realize.It will be understood by those of skill in the art that can use in practice
Microprocessor or digital signal processor (DSP) are realizing the according to embodiments of the present invention data processing being applied to data base
The some or all functions of some or all parts in device.The present invention is also implemented as being retouched here for execution
Some or all equipment of the method stated or program of device (for example, computer program and computer program).
Such program realizing the present invention can store on a computer-readable medium, or can have one or more signal
Form.Such signal can be downloaded from internet website and obtain, or on carrier signal provide, or with any its
He provides form.
It should be noted that above-described embodiment the present invention will be described rather than limits the invention, and ability
Field technique personnel can design alternative embodiment without departing from the scope of the appended claims.In the claims,
Any reference markss between bracket should not be configured to limitations on claims.Word "comprising" does not exclude the presence of not
Element listed in the claims or step.Word "a" or "an" before element does not exclude the presence of multiple such
Element.The present invention can come real by means of the hardware including some different elements and by means of properly programmed computer
Existing.If in the unit claim listing equipment for drying, several in these devices can be by same hardware branch
To embody.The use of word first, second, and third does not indicate that any order.These words can be explained and run after fame
Claim.
Claims (10)
1. a kind of data processing method being applied to data base is it is characterised in that described data base includes multiple identical numbers
According to set, methods described includes：
Receive the update instruction to described data base, from the plurality of identical data acquisition system, select the first data acquisition system；
Update described first data acquisition system, the first data acquisition system described in labelling is to inquire about state；
Receive to the inquiry of described data base instruction, selected marker has the second data acquisition system that can inquire about state to carry out data to look into
Ask.
2. method according to claim 1 is it is characterised in that receive to the business object in described data base described
After update instruction, methods described also includes：
Described first data acquisition system is moved to data from data query pond and updates pond；
Described first data acquisition system of described renewal includes：
Update in pond in described data and data renewal is carried out to described first data acquisition system.
3. method according to claim 2 it is characterised in that described by described first data acquisition system from data query pond
Move to before data updates pond, described first data acquisition system carries that described first data acquisition system is located at described data query pond
One positional information；
Described by described first data acquisition system from data query pond move to data update pond after, methods described also includes：
After the current queries of described first data acquisition system terminate, the described primary importance information of change is described first data acquisition system
Update the second position information in pond positioned at described data；
Before described first data acquisition system of described renewal, methods described also includes：
Monitor described first data acquisition system and carry described second position information；
After described first data acquisition system renewal terminates, methods described also includes：
Described first data acquisition system is updated pond from described data move back to described data query pond, and change described first
The second position information that data acquisition system carries is described primary importance information.
4. method according to claim 2 is it is characterised in that receive to the business object in described data base described
Before update instruction, the status indicator of described first data acquisition system is the inquired about mark that instruction can inquire about state, connects described
After receiving the update instruction to the business object in described data base, methods described also includes：
Change inquiring about of described first data acquisition system and be designated preparation renewal request mark；
Described, described first data acquisition system is moved to before described data updates pond from described data query pond, methods described is also
Including：
The status indicator described first data acquisition system is detected is to prepare to update request mark；
Described by described first data acquisition system from data query pond move to data update pond after, methods described also includes：
The preparation renewal request mark changing described first data acquisition system confirms mark to preparing renewal；
Before described first data acquisition system of described renewal, methods described also includes：
The preparation described first data acquisition system is detected updates confirmation mark.
5. method according to claim 4 is it is characterised in that described first data acquisition system of described renewal, described in labelling
One data acquisition system is the state of inquiring about to include：
Update described first data acquisition system, change simultaneously described first data acquisition system preparation update confirmation be designated instruction can not
The more new logo of inquiry state；
Described selected marker has the second data acquisition system that can inquire about state to carry out data query to include：
Selected marker has the second data acquisition system that can inquire about mark to carry out data query.
6. a kind of data processing equipment being applied to data base is it is characterised in that described device includes：
Update instruction receiver module, for receiving the update instruction to described data base, described data base includes multiple identical
Data acquisition system；
First data acquisition system selecting module, for selecting the first data acquisition system from the plurality of identical data acquisition system；
First data acquisition system update module, for updating described first data acquisition system, the first data acquisition system described in labelling is can not
Inquiry state；
Inquiry instruction receiver module, for receiving to the inquiry of described data base instruction, selected marker has can inquire about the of state
Two data acquisition systems carry out data query.
7. device according to claim 6 is it is characterised in that described device also includes：
First data pool mobile module, for after the described update instruction receiving to described data base, by the described first number
Move to data according to set from data query pond and update pond；
Described first data acquisition system update module, is carried out to described first data acquisition system specifically for updating in pond in described data
Data updates.
8. device according to claim 7 is it is characterised in that described device also includes：
Primary importance information changes module, for described first data acquisition system being moved to data renewal from data query pond described
After pond, after the current queries of described first data acquisition system terminate, the described primary importance information of change is described first data
Set is located at the second position information that described data updates pond, moves to described first data acquisition system from data query pond described
Before data updates pond, described first data acquisition system carries first that described first data acquisition system is located at described data query pond
Confidence ceases；
Described device also includes：
Positional information monitoring module, for, before described first data acquisition system of described renewal, monitoring described first data set
Conjunction carries described second position information；
Described device also includes：
Second data pool mobile module, after terminating in described first data acquisition system renewal, by described first data acquisition system
Update pond from described data to move back to described data query pond；
Second position information changes module, is described first for changing the second position information that described first data acquisition system carries
Positional information.
9. device according to claim 7 is it is characterised in that described device also includes：
First mark change module, for, after the described update instruction receiving to described data base, change described first counts
It is designated preparation renewal request mark according to inquiring about of set, in the described renewal receiving to the business object in described data base
Before instruction, the status indicator of described first data acquisition system is the inquired about mark that instruction can inquire about state；
Described device also includes：
First label detection module, for moving to described data by described first data acquisition system from described data query pond described
Before updating pond, the status indicator described first data acquisition system is detected is to prepare to update request mark；
Described device also includes：
Second mark change module, for described by described first data acquisition system from data query pond move to data update pond it
Afterwards, the preparation renewal request mark changing described first data acquisition system confirms mark to preparing renewal；
Described device also includes：
Second label detection module, for, before described first data acquisition system of described renewal, detecting described first data set
The preparation closed updates and confirms mark.
10. device according to claim 9 it is characterised in that：
Described first data acquisition system update module, specifically for updating described first data acquisition system, changes described first number simultaneously
The renewal for preparing according to set confirms to be designated the more new logo that instruction can not inquire about state；
Described inquiry instruction receiver module, has specifically for selected marker the second data acquisition system that can inquire about mark to carry out data and looks into
Ask.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610805597.6A CN106407351A (en) | 2016-09-05 | 2016-09-05 | Data processing method and device applied to database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610805597.6A CN106407351A (en) | 2016-09-05 | 2016-09-05 | Data processing method and device applied to database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106407351A true CN106407351A (en) | 2017-02-15 |
Family
ID=57998710
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610805597.6A Pending CN106407351A (en) | 2016-09-05 | 2016-09-05 | Data processing method and device applied to database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106407351A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1928873A (en) * | 2006-09-25 | 2007-03-14 | 华为技术有限公司 | System and method for database access for implementing load sharing |
| CN103955486A (en) * | 2014-04-14 | 2014-07-30 | 五八同城信息技术有限公司 | Distributed service system as well as data updating method and data query method thereof |
| CN104778267A (en) * | 2015-04-22 | 2015-07-15 | 无锡天脉聚源传媒科技有限公司 | Searching and index updating method and device |
-
2016
- 2016-09-05 CN CN201610805597.6A patent/CN106407351A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1928873A (en) * | 2006-09-25 | 2007-03-14 | 华为技术有限公司 | System and method for database access for implementing load sharing |
| CN103955486A (en) * | 2014-04-14 | 2014-07-30 | 五八同城信息技术有限公司 | Distributed service system as well as data updating method and data query method thereof |
| CN104778267A (en) * | 2015-04-22 | 2015-07-15 | 无锡天脉聚源传媒科技有限公司 | Searching and index updating method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2019170175A3 (en) | System and method for executing different types of blockchain contracts | |
| CN102768639B (en) | Operating system kernel-level error positioning method and device | |
| US10887171B2 (en) | Routing configuration method of view files, storage medium, terminal device and apparatus | |
| CN102033948A (en) | Method and device for updating data | |
| US20170270183A1 (en) | Sensor data management apparatus, sensor data management method, and computer program product | |
| CN111324781A (en) | Data analysis method, device and equipment | |
| JP2015088191A5 (en) | ||
| CN109783509A (en) | SQL scenario generation method and device | |
| KR20170136636A (en) | Graphical indexing for lifecycle management of drilling system assets | |
| US11023451B2 (en) | Change detection apparatus, maintenance and management system, change detection method, program, and recording medium | |
| CN112783749A (en) | Static code scanning optimization method and device, electronic equipment and storage medium | |
| CN102222033A (en) | Method and device for saving small computer system interface access error | |
| CN106021046A (en) | Hard disk performance comparison method, device and server | |
| CN106407351A (en) | Data processing method and device applied to database | |
| US20140181599A1 (en) | Task server and method for allocating tasks | |
| CN106469175A (en) | Data processing method and device | |
| CN112987679A (en) | Method, system and computer program product for commissioning field devices in a control system | |
| CN104268244A (en) | Method and device for processing form | |
| CN102682038A (en) | Database change method and device | |
| CN111221894B (en) | Time sequence database storage method, device and server based on configuration | |
| CN115309334A (en) | Disk management method, device, equipment and storage medium | |
| KR20170071825A (en) | Application Failure Trace Support Method for IT Service and Management System applying the same | |
| US8769173B2 (en) | Systems and methods for detecting supported small form-factor pluggable (SFP) devices | |
| CN115204810A (en) | Project management method, device and storage medium based on RPA and automation script | |
| CN103838845A (en) | Universal Excel data importing implementing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20170215 |