CN108460070A - A kind of data processing method, device and equipment based on database - Google Patents
A kind of data processing method, device and equipment based on database Download PDFInfo
- Publication number
- CN108460070A CN108460070A CN201711396827.9A CN201711396827A CN108460070A CN 108460070 A CN108460070 A CN 108460070A CN 201711396827 A CN201711396827 A CN 201711396827A CN 108460070 A CN108460070 A CN 108460070A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- business datum
- backup
- business
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
- G06F11/1451—Management of the data involved in backup or backup restore by selection of backup contents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/278—Data partitioning, e.g. horizontal or vertical partitioning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This specification embodiment discloses a kind of data processing method, device and equipment based on database, each database is no longer deployed as master library for the framework in library, corresponding data management system can divide business datum, and the data after division are distributed into each database, business datum assigned by disparate databases simultaneously differs, and each database can provide data service according to the business datum being assigned to for corresponding business.At the same time, the business datum of other databases is also stored in each database, to be backed up.
Description
Technical field
This application involves a kind of field of computer technology more particularly to data processing method, device and equipment.
Background technology
With the development of Internet technology, more and more data storages are in the database.
In the prior art, business provider does not lose in order to ensure data, and database would generally use active and standby mould when disposing
Formula, that is, deployment primary database (subsequently may be simply referred to as：Master library) and standby database (subsequently may be simply referred to as：Standby library).In reality
When operation, the business datum of generation is usually preferably stored in master library, subsequently can be same by the data in master library by asynchronous task
Step is to for library.To, the data stored in active and standby library can be consistent, data will not because a certain database exception (such as：It delays
Machine) and lose.
Based on above-mentioned deployment mode, if master library delay machine, standby library can be enabled, to provide corresponding business support,
Switchback master library after master library restores normal, and the incremental data in standby library is synchronized to master library.In the process of synchronous incremental data
In, master library and standby library are unavailable.
Based on the prior art, it would be desirable to a kind of more effectively data processing method based on database.
Invention content
This specification embodiment provides a kind of data processing method, device and equipment based on database, to provide one
The significantly more efficient data processing method of kind.
One kind that this specification embodiment provides is based on database data processing method, including：
Divide business datum；
Business datum difference after division is distributed into each database, so that each database was obtained according to distribution respectively
Different business data provide data service and backup.
The another kind that this specification embodiment provides is based on database data processing method, including：
Receive the business datum of distribution；Wherein, the business datum received is by dividing and carrying out difference distribution institute
It obtains, the business datum of distribution includes operation data and Backup Data；
Store the operation data and Backup Data, so as to according to the operation data provide data service, and according to
The Backup Data provides backup services.
A kind of data processing equipment based on database that this specification embodiment also provides, including：
Data division module divides business datum；
Business datum difference after division is distributed to each database by data allocation module, so that each database is distinguished
Data service and backup are provided according to the different business data that distribution obtains.
Another data processing equipment based on database that this specification embodiment provides, including：
Data reception module receives the business datum of distribution；Wherein, the business datum received is gone forward side by side by division
Obtained by the distribution of row difference, the business datum of distribution includes operation data and Backup Data；
Memory module stores the operation data and Backup Data, to provide data clothes according to the operation data
Business, and provide backup services according to the Backup Data.
A kind of data processing equipment based on database that this specification embodiment also provides, including：
Memory stores data processor；
Processor calls the data processor stored in memory, and executes：
Divide business datum；
Business datum difference after division is distributed into each database, so that each database was obtained according to distribution respectively
Different business data provide data service and backup.
Another data processing equipment based on database that this specification embodiment provides, including：
Receive the business datum of distribution；Wherein, the business datum received is by dividing and carrying out difference distribution institute
It obtains, the business datum of distribution includes operation data and Backup Data；
Store the operation data and Backup Data, so as to according to the operation data provide data service, and according to
The Backup Data provides backup services.
Above-mentioned at least one technical solution that this specification embodiment uses can reach following advantageous effect：
Each database in this specification embodiment is no longer deployed as the framework in the standby library of master library-, corresponding data management system
System can divide business datum, and the data after division are distributed to each database, assigned by disparate databases
Business datum simultaneously differs, and each database can provide data service according to the business datum being assigned to for corresponding business.
At the same time, the business datum of other databases is also stored in each database, to be backed up, so, when certain
One database is abnormal (such as：Delay machine) when, other databases still are able to normal operation, further, enable to big
Partial service normal operation.Obviously, it using the above method in this specification embodiment, can be promoted to a certain extent fault-tolerant
Property, and ensure the stabilization of data, to realize the disaster tolerance of database.
Description of the drawings
Attached drawing described herein is used for providing further understanding of the present application, constitutes part of this application, this Shen
Illustrative embodiments and their description please do not constitute the improper restriction to the application for explaining the application.In the accompanying drawings：
Fig. 1 is the configuration diagram that the data processing method that this specification embodiment provides is based on；
Fig. 2 is the data handling procedure in data management system side that this specification embodiment provides；
Fig. 3 is the data handling procedure in database side that this specification embodiment provides；
Fig. 4 is the data processing schematic diagram in a kind of application example that this specification embodiment provides；
Fig. 5 is the data processing equipment structural diagram in data management system side that this specification embodiment provides；
Fig. 6 be this specification embodiment provide in database side data processing equipment structural diagram.
Specific implementation mode
To keep the purpose, technical scheme and advantage of the application clearer, below in conjunction with the application specific embodiment and
Technical scheme is clearly and completely described in corresponding attached drawing.Obviously, described embodiment is only the application one
Section Example, instead of all the embodiments.Based on the embodiment in the application, those of ordinary skill in the art are not doing
Go out the every other embodiment obtained under the premise of creative work, shall fall in the protection scope of this application.
In one or more embodiments in the present specification, the data processing method can be used as shown in Figure 1
Framework.At least it may include in Fig. 1：Operation system, data management system and service database.
Wherein, the operation system is regarded as system of the business provider backstage for providing different business service,
It can be usually made of service server or other are capable of providing the equipment of business service (such as：Computer, supercomputer
Deng) constituted, here and it is not especially limited.Such as：For providing the payment system of on-line payment function, for providing order pipe
Manage the form ordering system etc. of function.When operation system is during providing business service, it will usually generate corresponding business datum
(such as：Payment data, order data etc., do not enumerate here), the business datum of generation can be stored in service database
In.
The data management system, the business datum for being generated operation system are deployed in each service database
In.It should be noted here that by the business datum in the be deployed in different business database of data management system and differing.Separately
Outside, the business datum being deployed in a certain service database, a part is used as operation data, for being carried for the operation of operation system
For supporting, another part is as Backup Data, in order to provide backup.
In this specification embodiment, the data management system can be a kind of entity device (such as：With data pipe
Manage the server of function), it can also be software/service, be set in corresponding background devices, in a kind of possible mode,
Data management function can be used as a kind of software service and be set in operation system, in this fashion, can realize number by operation system
According to distribution.Certainly, which kind of mode is specifically used, will be determined according to the needs of practical application.
The service database, the business datum generated for storage service system.It should be noted that the industry in Fig. 1
Business database does not simultaneously differentiate between master library or standby library, and in other words, each service database can be used as master library and standby library simultaneously.In Fig. 1,
Each service server can both characterize a single server, can also characterize a server cluster (with single in Fig. 1
Server indicate, should not be construed as the restriction to the application), the service database specifically may be used such as：It is open
Data processing service (Open Data Processing Service, ODPS) database, distributed data base
(Distributed Data Base, DDB), Hbase databases, MySQL database etc., are not especially limited here.For ease of
Description, hereinafter referred to as：Database.
Based on above-mentioned framework shown in FIG. 1, it is based on data management system side, the data processing side in this specification embodiment
Method is as shown in Fig. 2, specifically include following steps：
Step S201：Divide business datum.
As a kind of feasible pattern in this specification embodiment, the division to business datum can be directed to have generated
Business datum divided.That is, generated business datum, which is divided into several pieces, (obtains several business datums
Collection), so that subsequent allocations are stored to corresponding database.Certainly, distributing to the business datum of a certain database can be
At least one business datum subset, is not defined here.
As another feasible pattern in this specification embodiment, the division to business datum can be directed to and not give birth to also
At business datum corresponding division rule, so, the business datum subsequently generated by operation system, by basis are set
The division rule carries out the division of business datum.
Certainly, there are many above-mentioned dividing modes to business datum：
In a kind of feasible mode, business datum can be divided based on database information, database here
Information includes but not limited to：At least one of the quantity information of database, storage space information, running state information.It is based on
Database information, it may be determined that the quantity of the business datum subset divided needed for going out, to distribute to each database.
In another feasible mode, the service feature of business datum itself can be combined with (such as：Temporal information, number
According to mark etc.), carry out the division of business datum.
Certainly, foregoing teachings are only two kinds of feasible dividing modes, and when specific implementation can be combined with using above two
Dividing mode should not constitute the restriction to the application here alternatively, using other dividing modes.
Step S203：Business datum difference after division is distributed into each database, so that each database basis respectively
It distributes obtained different business data and data service and backup is provided.
As foregoing teachings, in this specification embodiment, service server and master library or standby library are not differentiated between, then,
That is after the business datum after division is distributed to each database, each database is phase by the business datum based on distribution
The service operation answered provides data service.It should be noted here that the business datum for distributing to each database is typically difference
, that is, difference is distributed.Each database holds a part of business datum, to provide corresponding number for different business
According to service.
Meanwhile for each database, additionally it is possible to the business datum in other databases is stored, to be backed up.
Through the above steps, each database in this specification embodiment is no longer deployed as the framework in the standby library of master library-, accordingly
Data management system business datum can be divided, and the data after division are distributed into each database, different data
Business datum assigned by library simultaneously differs, and each database can be that corresponding business is carried according to the business datum being assigned to
For data service.At the same time, the business datum of other databases is also stored in each database, to be backed up, this
Sample one, when a certain database is abnormal (such as：Delay machine) when, other databases still are able to normal operation, further,
Enable to most of business normal operation.It obviously, can be to a certain degree using the above method in this specification embodiment
Upper promotion fault-tolerance, and ensure the stabilization of data, to realize the disaster tolerance of database.
For the above, it should be noted that it can either be corresponding just to allow for each database in practical applications
Business provide data support again can be backed up, therefore, when dividing business datum, business datum can be divided into fortune
Row data and Backup Data.In other words, database can be based on operation data and provide data support for corresponding business, and can be based on
Backup Data realizes backup functionality.
For two kinds above-mentioned divide data mode for, specifically for, knowing database information
On the basis of, hash algorithm can be used to realize the division to business datum.Such as：Hash subregions, that is, business datum can be directed to
Using hash algorithm, so that it is determined that going out the subregion that business datum should belong to, correspondingly, can will belong to same in same database
The business datum of subregion is divided into a business datum subset, and the data in the follow-up business datum subset will uniformly be stored in this
In subregion.
So, the business datum is divided according to the database information for the above method, so that it may including：According to institute
Database information is stated, the business datum is divided by multiple business datum subsets using hash algorithm.
It is, of course, also possible to which the service feature based on business datum itself realizes the division to business datum.Such as：Based on business
The temporal information of data is divided into multiple business datum subsets of specified quantity.That is, for the above method, according to the number
According to library information, the business datum is divided, specifically may include：The service feature corresponding to the business datum is determined, according to institute
Database information and service feature are stated, the business datum is divided.
Further, the service feature includes at least：Temporal information corresponding to business datum, on this basis, root
According to the database information and service feature, the business datum is divided, so that it may to include：According to the database information,
The quantity of the business datum subset divided needed for determining, number of matches is divided into according to the temporal information by the business datum
Business datum subset.
In general, obtained business datum subset can have corresponding mark after division, and data structure can be such as
Shown in lower：
The partition identification of the Database Identification+database
Specifically for example：The mark of one business datum subset can be " A (Database Identification)+a1 (partition identification) ", table
Show that the data set will be subsequently stored in the subregion a1 of database A.Certainly, only it is here a kind of simple examples, and should not be used as
Restriction to the application.
Certainly, partition process is typically divided for operation data, and for Backup Data, data management system can
The operation data that a certain database is distributed will be directed to, another database is distributed to, the Backup Data as another database
It repeats to store.
The above content is the explanations based on data management system side, in this specification embodiment, are based on database one
The data processing method of side, can be as shown in figure 3, specifically include following steps：
Step S301：Receive the business datum of distribution；The business datum received is by dividing and carrying out difference point
With acquired, the business datum of distribution includes operation data and Backup Data.
In this specification embodiment, the business datum of database is distributed to, divides to obtain by data management system.And
And for each database, the business datum of distribution includes：With the business number in the business datum subset corresponding to the database
According to (that is, operation data), and, distribute to the operation data (that is, Backup Data) of other databases.Certainly, the related industry
Data of being engaged in divide and assigning process, can refer to foregoing teachings, just no longer excessively repeat here.
Step S303：The operation data and Backup Data are stored, to provide data clothes according to the operation data
Business, and provide backup services according to the Backup Data.
It is believed that for any database, operation data is to be divided for the database and distribute to the number
According to the business datum in library, and Backup Data is to be divided for other databases and distribute to the business datum of other databases
(allowing the database to make a backup store, in service operation and not use Backup Data).
For the ease of understanding the above, it is described in detail below by specific example：
In this example, it is assumed that business provider backstage has two databases A and B.Based on this, accordingly
Data management system business datum can be divided, and distribute to database A and B.And it is assumed that divide business datum in number
According in library A and B, stored according to mode as shown in Figure 4.
That is, for database A, operation data is stored in the storage section 1 and 2 of database A, Backup Data
It is stored in the backup section 3 and 4 of database A.
For database B, operation data is stored in the storage section a and b of database B, backup data store
In the backup section c and d of database B.
Wherein, the backup section 3 and 4 of database A, the business in storage section a and b for backing up storage database B
Data, correspondingly, the backup section c and d of database B, the business number in storage section 1 and 2 for backing up storage database A
According to.
On this basis, it is assumed that database A delay machines, then, database B still can be based on the business number in storage section a and b
Data service is provided according to for corresponding business.And after database A restores, database B can will store the increasing in section a and b
Amount data are synchronized in the backup section 3 and 4 of database A.
As it can be seen that even if a certain database delay machine, another database can not also be discontinuously on the basis of above-mentioned example
Half business, which provides data service, correspondingly also need to only synchronize the incremental data of half business datum in synchronizing process, to
With effectively reduce synchronize it is time-consuming.
Further, in the case where increasing database, the quantity of database is more, then effect is more notable.Such as：Such as
Fruit disposes three databases, then still can guarantee 2/3 service nondestructive, incremental data synchronization time after single database delay machine
It is reduced to the 1/3 of total amount.
It is the data processing method that this specification embodiment provides above, same thinking is based on, in data management system
Side, this specification embodiment also provides a kind of data processing equipment based on database, as shown in figure 5, described device includes：
Data division module 501 divides business datum；
Business datum difference after division is distributed to each database by data allocation module 502, so that each database point
Data service and backup are not provided according to the different business data that distribution obtains.
Further, the data division module 501 obtains database information, according to the database information, divides institute
State business datum；
Wherein, the database information includes：In the quantity information of database, storage space information, running state information
At least one.
The data division module 501 is divided the business datum using hash algorithm according to the database information
For multiple business datum subsets.
The data division module 501, determines the service feature corresponding to the business datum, is believed according to the database
Breath and service feature, divide the business datum；
The business spy includes：At least one of temporal information, field name, business number corresponding to business datum.
The data division module 501 determines the number of the required business datum subset divided according to the database information
The business datum, the business datum subset of number of matches is divided into according to the service feature by amount.
The data allocation module 502 will divide business obtain, corresponding with the database for any database
Business datum in data subset distributes to the database as operation data；
The operation data of some or all of other databases will be distributed to, as Backup Data, distributes to the database.
Based on device as shown in Figure 5, this specification embodiment also provides a kind of data processing equipment based on database
It (specifically can be as：Server, computer etc.), including：
Memory stores data processor；
Processor calls the data processor stored in memory, and executes：
Divide business datum；
Business datum difference after division is distributed into each database, so that each database was obtained according to distribution respectively
Different business data provide data service and backup.
In database side, this specification embodiment also provides a kind of data processing equipment based on database, such as Fig. 6 institutes
Show, described device includes：
Data reception module 601 receives the business datum of distribution；Wherein, the business datum received is by dividing
And obtained by carrying out difference distribution, the business datum of distribution includes operation data and Backup Data；
Memory module 602 stores the operation data and Backup Data, to provide data according to the operation data
Service, and provide backup services according to the Backup Data.
Further, described device further includes：Data processing module 603, backup reception data instruction, according to the data
Backup instruction, the incremental data that will be generated based on the operation data are synchronized in specified data library, or
According to the data backup command, the incremental data generated in specified database is obtained, the backup of itself is updated
Data.
Based on device as shown in FIG. 6, this specification embodiment also provides a kind of data processing equipment based on database
It (specifically can be as：Server, computer etc.), including：
Memory stores data processor；
Processor calls the data processor stored in memory, and executes：
Receive the business datum of distribution；Wherein, the business datum received is by dividing and carrying out difference distribution institute
It obtains, the business datum of distribution includes operation data and Backup Data；
Store the operation data and Backup Data, so as to according to the operation data provide data service, and according to
The Backup Data provides backup services.
Each embodiment in this specification is described in a progressive manner, identical similar portion between each embodiment
Point just to refer each other, and each embodiment focuses on the differences from other embodiments.Especially for device,
For equipment and medium class embodiment, since it is substantially similar to the method embodiment, so description is fairly simple, related place
Illustrate referring to the part of embodiment of the method, just no longer repeats one by one here.
It is above-mentioned that this specification specific embodiment is described.Other embodiments are in the scope of the appended claims
It is interior.In some cases, the action recorded in detail in the claims or step or module can be according to different from embodiments
Sequence executes and desired result still may be implemented.In addition, the process described in the accompanying drawings is not necessarily required and is shown
Particular order or consecutive order could realize desired result.In some embodiments, multitasking and parallel processing
It is also possible or it may be advantageous.
In the 1990s, the improvement of a technology can be distinguished clearly be on hardware improvement (for example,
Improvement to circuit structures such as diode, transistor, switches) or software on improvement (improvement for method flow).So
And with the development of technology, the improvement of current many method flows can be considered as directly improving for hardware circuit.
Designer nearly all obtains corresponding hardware circuit by the way that improved method flow to be programmed into hardware circuit.Cause
This, it cannot be said that the improvement of a method flow cannot be realized with hardware entities module.For example, programmable logic device
(Programmable Logic Device, PLD) (such as field programmable gate array (Field Programmable Gate
Array, FPGA)) it is exactly such a integrated circuit, logic function determines device programming by user.By designer
Voluntarily programming comes a digital display circuit " integrated " on a piece of PLD, designs and makes without asking chip maker
Dedicated IC chip.Moreover, nowadays, substitution manually makes IC chip, this programming is also used instead mostly " patrols
Volume compiler (logic compiler) " software realizes that software compiler used is similar when it writes with program development,
And the source code before compiling also write by handy specific programming language, this is referred to as hardware description language
(Hardware Description Language, HDL), and HDL is also not only a kind of, but there are many kind, such as ABEL
(Advanced Boolean Expression Language)、AHDL(Altera Hardware Description
Language)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL
(Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(Ruby
Hardware Description Language) etc., VHDL (Very-High-Speed are most generally used at present
Integrated Circuit Hardware Description Language) and Verilog.Those skilled in the art also answer
This understands, it is only necessary to method flow slightly programming in logic and is programmed into integrated circuit with above-mentioned several hardware description languages,
The hardware circuit for realizing the logical method flow can be readily available.
Controller can be implemented in any suitable manner, for example, controller can take such as microprocessor or processing
The computer for the computer readable program code (such as software or firmware) that device and storage can be executed by (micro-) processor can
Read medium, logic gate, switch, application-specific integrated circuit (Application Specific Integrated Circuit,
ASIC), the form of programmable logic controller (PLC) and embedded microcontroller, the example of controller includes but not limited to following microcontroller
Device：ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, are deposited
Memory controller is also implemented as a part for the control logic of memory.It is also known in the art that in addition to
Pure computer readable program code mode is realized other than controller, can be made completely by the way that method and step is carried out programming in logic
Controller is obtained in the form of logic gate, switch, application-specific integrated circuit, programmable logic controller (PLC) and embedded microcontroller etc. to come in fact
Existing identical function.Therefore this controller is considered a kind of hardware component, and to including for realizing various in it
The device of function can also be considered as the structure in hardware component.Or even, it can will be regarded for realizing the device of various functions
For either the software module of implementation method can be the structure in hardware component again.
System, device, module or the unit that above-described embodiment illustrates can specifically realize by computer chip or entity,
Or it is realized by the product with certain function.It is a kind of typically to realize that equipment is computer.Specifically, computer for example may be used
Think personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media play
It is any in device, navigation equipment, electronic mail equipment, game console, tablet computer, wearable device or these equipment
The combination of equipment.
For convenience of description, it is divided into various units when description apparatus above with function to describe respectively.Certainly, implementing this
The function of each unit is realized can in the same or multiple software and or hardware when application.
It should be understood by those skilled in the art that, the embodiment of the present invention can be provided as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the present invention
Apply the form of example.Moreover, the present invention can be used in one or more wherein include computer usable program code computer
The computer program production implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
The form of product.
The present invention be with reference to according to the method for the embodiment of the present invention, the flow of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that can be realized by computer program instructions every first-class in flowchart and/or the block diagram
The combination of flow and/or box in journey and/or box and flowchart and/or the block diagram.These computer programs can be provided
Instruct the processor of all-purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine so that the instruction executed by computer or the processor of other programmable data processing devices is generated for real
The device for the function of being specified in present one flow of flow chart or one box of multiple flows and/or block diagram or multiple boxes.
These computer program instructions, which may also be stored in, can guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works so that instruction generation stored in the computer readable memory includes referring to
Enable the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one box of block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device so that count
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, in computer or
The instruction executed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in a box or multiple boxes.
In a typical configuration, computing device includes one or more processors (CPU), input/output interface, net
Network interface and memory.
Memory may include computer-readable medium in volatile memory, random access memory (RAM) and/or
The forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable medium
Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology realizes information storage.Information can be computer-readable instruction, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase transition internal memory (PRAM), static RAM (SRAM), moves
State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable
Programmable read only memory (EEPROM), fast flash memory bank or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM),
Digital versatile disc (DVD) or other optical storages, magnetic tape cassette, tape magnetic disk storage or other magnetic storage apparatus
Or any other non-transmission medium, it can be used for storage and can be accessed by a computing device information.As defined in this article, it calculates
Machine readable medium does not include temporary computer readable media (transitory media), the data letter number and carrier wave of such as modulation.
It should also be noted that, the terms "include", "comprise" or its any other variant are intended to nonexcludability
Including so that process, method, commodity or equipment including a series of elements include not only those elements, but also wrap
Include other elements that are not explicitly listed, or further include for this process, method, commodity or equipment intrinsic want
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that wanted including described
There is also other identical elements in the process of element, method, commodity or equipment.
It will be understood by those skilled in the art that embodiments herein can be provided as method, system or computer program product.
Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in the application
Form.It is deposited moreover, the application can be used to can be used in the computer that one or more wherein includes computer usable program code
The shape for the computer program product implemented on storage media (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
The application can describe in the general context of computer-executable instructions executed by a computer, such as program
Module.Usually, program module includes routine, program, object, the group for executing particular transaction or realizing particular abstract data type
Part, data structure etc..The application can also be put into practice in a distributed computing environment, in these distributed computing environments, by
Affairs are executed by the connected remote processing devices of communication network.In a distributed computing environment, program module can be with
In the local and remote computer storage media including storage device.
Each embodiment in this specification is described in a progressive manner, identical similar portion between each embodiment
Point just to refer each other, and each embodiment focuses on the differences from other embodiments.Especially for system reality
For applying example, since it is substantially similar to the method embodiment, so description is fairly simple, related place is referring to embodiment of the method
Part explanation.
Above is only an example of the present application, it is not intended to limit this application.For those skilled in the art
For, the application can have various modifications and variations.It is all within spirit herein and principle made by any modification, equivalent
Replace, improve etc., it should be included among the interest field of the application.
Claims (18)
1. a kind of data processing method based on database, including：
Divide business datum；
Business datum difference after division is distributed into each database, so that the difference that each database is obtained according to distribution respectively
Business datum provides data service and backup.
2. the method as described in claim 1 divides business datum, specifically includes：
Obtain database information；
According to the database information, the business datum is divided；
Wherein, the database information includes：In the quantity information of database, storage space information, running state information extremely
Few one kind.
3. method as claimed in claim 2 divides the business datum, specifically includes according to the database information：
According to the database information, the business datum is divided by multiple business datum subsets using hash algorithm.
4. method as claimed in claim 2 divides the business datum, specifically includes according to the database information：
Determine the service feature corresponding to the business datum；
According to the database information and service feature, the business datum is divided；
The business spy includes：At least one of temporal information, field name, business number corresponding to business datum.
5. method as claimed in claim 4 divides the business datum according to the database information and service feature,
It specifically includes：
According to the database information, the quantity of the required business datum subset divided is determined；
The business datum is divided into the business datum subset of number of matches according to the service feature.
6. the business datum after division is distributed to each database, specifically included by the method as described in claim 1：
For any database, the business datum divided in business datum subset obtain, corresponding with the database is distributed
The database is given, as operation data；
The operation data for distributing to some or all of other databases is distributed into the database, as Backup Data.
7. a kind of data processing method based on database, including：
Receive the business datum of distribution；Wherein, obtained by the business datum received is by division and carrying out difference distribution,
The business datum of distribution includes operation data and Backup Data；
The operation data and Backup Data are stored, to provide data service according to the operation data, and according to described
Backup Data provides backup.
8. the method for claim 7, the method further includes：
Backup reception data instructs；
According to the data backup command, the incremental data that will be generated based on the operation data is synchronized to specified data
In library, or
According to the data backup command, the incremental data generated in specified database is obtained, the Backup Data of itself is updated.
9. a kind of data processing equipment based on database, including：
Data division module divides business datum；
Business datum difference after division is distributed to each database by data allocation module, so that each database basis respectively
It distributes obtained different business data and data service and backup is provided.
10. device as claimed in claim 9, the data division module obtain database information；
According to the database information, the business datum is divided；
Wherein, the database information includes：In the quantity information of database, storage space information, running state information extremely
Few one kind.
11. device as claimed in claim 10, the data division module is calculated according to the database information using Hash
The business datum is divided into multiple business datum subsets by method.
12. device as claimed in claim 10, the data division module determine that the business corresponding to the business datum is special
Sign, according to the database information and service feature, divides the business datum；
The business spy includes：At least one of temporal information, field name, business number corresponding to business datum.
13. device as claimed in claim 12, the data division module determines required draw according to the database information
The business datum, the business datum of number of matches is divided into according to the service feature by the quantity for the business datum subset divided
Subset.
14. device as claimed in claim 9, the data allocation module, for any database, will divide it is obtaining, with
Business datum in the corresponding business datum subset of the database distributes to the database, as operation data,；
The operation data for distributing to some or all of other databases is distributed into the database, as Backup Data.
15. a kind of data processing equipment based on database, including：
Data reception module receives the business datum of distribution；Wherein, the business datum received is by dividing and carrying out difference
Not Fen Pei obtained by, the business datum of distribution includes operation data and Backup Data；
Memory module stores the operation data and Backup Data, to provide data service according to the operation data, and
Backup is provided according to the Backup Data.
16. device as claimed in claim 15, described device further include：Data processing module, backup reception data instruction, root
According to the data backup command, the incremental data that will be generated based on the operation data is synchronized in specified data library, or
According to the data backup command, the incremental data generated in specified database is obtained, the Backup Data of itself is updated.
17. a kind of data processing equipment based on database, including：
Memory stores data processor；
Processor calls the data processor stored in memory, and executes：
Divide business datum；
Business datum difference after division is distributed into each database, so that the difference that each database is obtained according to distribution respectively
Business datum provides data service and backup.
18. a kind of data processing equipment based on database, including：
Memory stores data processor；
Processor calls the data processor stored in memory, and executes：
Receive the business datum of distribution；Wherein, obtained by the business datum received is by division and carrying out difference distribution,
The business datum of distribution includes operation data and Backup Data；
The operation data and Backup Data are stored, to provide data service according to the operation data, and according to described
Backup Data provides backup services.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711396827.9A CN108460070A (en) | 2017-12-21 | 2017-12-21 | A kind of data processing method, device and equipment based on database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711396827.9A CN108460070A (en) | 2017-12-21 | 2017-12-21 | A kind of data processing method, device and equipment based on database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108460070A true CN108460070A (en) | 2018-08-28 |
Family
ID=63220578
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711396827.9A Pending CN108460070A (en) | 2017-12-21 | 2017-12-21 | A kind of data processing method, device and equipment based on database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108460070A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109542866A (en) * | 2018-11-20 | 2019-03-29 | 深圳市博实结科技有限公司 | Vehicle location monitor cloud framework, method, server and medium |
| CN109783275A (en) * | 2018-12-17 | 2019-05-21 | 北京春鸿科技有限公司 | The method and terminal of file backup are carried out to multiple wifi storage equipment |
| CN109947846A (en) * | 2018-11-30 | 2019-06-28 | 网联清算有限公司 | Transaction data processing method, device, computer equipment and storage medium |
| CN110074767A (en) * | 2019-05-30 | 2019-08-02 | 江苏富林医疗设备有限公司 | Can temperature-compensating electronic sphygmomanometer |
| CN110298666A (en) * | 2019-06-21 | 2019-10-01 | 网联清算有限公司 | Abnormality eliminating method and device during trading processing |
| CN114637736A (en) * | 2022-03-09 | 2022-06-17 | 北京金堤科技有限公司 | Database splitting method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110184915A1 (en) * | 2010-01-28 | 2011-07-28 | Microsoft Corporation | Cluster restore and rebuild |
| CN102938001A (en) * | 2012-12-10 | 2013-02-20 | 曙光信息产业(北京)有限公司 | Data loading device and data loading method |
| CN103716343A (en) * | 2012-09-29 | 2014-04-09 | 重庆新媒农信科技有限公司 | Distributed service request processing method and system based on data cache synchronization |
| CN104317800A (en) * | 2014-09-19 | 2015-01-28 | 山东大学 | Hybrid storage system and method for mass intelligent power utilization data |
| CN105138638A (en) * | 2015-08-24 | 2015-12-09 | 浪潮通用软件有限公司 | Database distribution method based on application layer |
| CN106844721A (en) * | 2017-02-09 | 2017-06-13 | 济南浪潮高新科技投资发展有限公司 | Date storage method, device, system, computer-readable recording medium and storage control |
| CN106888245A (en) * | 2016-06-07 | 2017-06-23 | 阿里巴巴集团控股有限公司 | A kind of data processing method, apparatus and system |
-
2017
- 2017-12-21 CN CN201711396827.9A patent/CN108460070A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110184915A1 (en) * | 2010-01-28 | 2011-07-28 | Microsoft Corporation | Cluster restore and rebuild |
| CN103716343A (en) * | 2012-09-29 | 2014-04-09 | 重庆新媒农信科技有限公司 | Distributed service request processing method and system based on data cache synchronization |
| CN102938001A (en) * | 2012-12-10 | 2013-02-20 | 曙光信息产业(北京)有限公司 | Data loading device and data loading method |
| CN104317800A (en) * | 2014-09-19 | 2015-01-28 | 山东大学 | Hybrid storage system and method for mass intelligent power utilization data |
| CN105138638A (en) * | 2015-08-24 | 2015-12-09 | 浪潮通用软件有限公司 | Database distribution method based on application layer |
| CN106888245A (en) * | 2016-06-07 | 2017-06-23 | 阿里巴巴集团控股有限公司 | A kind of data processing method, apparatus and system |
| CN106844721A (en) * | 2017-02-09 | 2017-06-13 | 济南浪潮高新科技投资发展有限公司 | Date storage method, device, system, computer-readable recording medium and storage control |
Non-Patent Citations (1)
| Title |
|---|
| 许振霞: "《计算机网络安全技术》", 31 May 2015, 天津科学技术出版社 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109542866A (en) * | 2018-11-20 | 2019-03-29 | 深圳市博实结科技有限公司 | Vehicle location monitor cloud framework, method, server and medium |
| CN109947846A (en) * | 2018-11-30 | 2019-06-28 | 网联清算有限公司 | Transaction data processing method, device, computer equipment and storage medium |
| CN109947846B (en) * | 2018-11-30 | 2021-06-15 | 网联清算有限公司 | Transaction data processing method and device, computer equipment and storage medium |
| CN109783275A (en) * | 2018-12-17 | 2019-05-21 | 北京春鸿科技有限公司 | The method and terminal of file backup are carried out to multiple wifi storage equipment |
| CN110074767A (en) * | 2019-05-30 | 2019-08-02 | 江苏富林医疗设备有限公司 | Can temperature-compensating electronic sphygmomanometer |
| CN110298666A (en) * | 2019-06-21 | 2019-10-01 | 网联清算有限公司 | Abnormality eliminating method and device during trading processing |
| CN110298666B (en) * | 2019-06-21 | 2022-08-30 | 网联清算有限公司 | Exception handling method and device in transaction processing process |
| CN114637736A (en) * | 2022-03-09 | 2022-06-17 | 北京金堤科技有限公司 | Database splitting method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108460070A (en) | A kind of data processing method, device and equipment based on database | |
| CN107450979B (en) | Block chain consensus method and device | |
| CN107391527B (en) | Data processing method and device based on block chain | |
| CN108628688B (en) | Message processing method, device and equipment | |
| CN111885082B (en) | Web application package processing method, device and equipment | |
| CN108345977A (en) | A kind of method for processing business and device | |
| US11074246B2 (en) | Cluster-based random walk processing | |
| CN110401700A (en) | Model loading method and system, control node and execution node | |
| CN110019062A (en) | Method of data synchronization and system | |
| CN111966631A (en) | Mirror image file generation method, system, equipment and medium capable of being rapidly distributed | |
| CN109002357A (en) | Resource allocation methods, device and Internet of things system | |
| CN108268289A (en) | Method for parameter configuration, the apparatus and system of web applications | |
| CN105991463B (en) | Method, message main node, token server and system for realizing flow control | |
| CN108959330A (en) | A kind of processing of database, data query method and apparatus | |
| CN109446271B (en) | Data synchronization method, device, equipment and medium | |
| US10901971B2 (en) | Random walking and cluster-based random walking method, apparatus and device | |
| TWI687820B (en) | Random walk, cluster-based random walk method, device and equipment | |
| CN110244964A (en) | It is a kind of based on O&M application O&M method, O&M method, device and equipment | |
| CN113391882B (en) | Virtual machine memory management method and device, storage medium and electronic equipment | |
| CN117314512A (en) | Resource allocation method and device, and coupon allocation method and device | |
| CN109582388A (en) | One parameter configuration method, device and equipment | |
| CN109101514A (en) | Data lead-in method and device | |
| CN108681492A (en) | Data processing method and device | |
| CN107645541B (en) | Data storage method and device and server | |
| CN117914721A (en) | Java-based call substitute service system construction method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20201020 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20201020 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180828 |