CN108595490A - A kind of construction method and system of database retrieval mechanism - Google Patents
A kind of construction method and system of database retrieval mechanism Download PDFInfo
- Publication number
- CN108595490A CN108595490A CN201810213892.1A CN201810213892A CN108595490A CN 108595490 A CN108595490 A CN 108595490A CN 201810213892 A CN201810213892 A CN 201810213892A CN 108595490 A CN108595490 A CN 108595490A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- storage
- partition
- relational
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The present invention relates to a kind of construction methods and system of database retrieval mechanism, including step：It determines that using data storage method be relevant database or non-relational database, data access layer is established to relevant database and non-relational database respectively；Data partition is carried out to the data in the relevant database or the non-relational database, obtains partition data；Service request is analyzed, correspondence will be built between different requests and the partition data；On the basis of the correspondence built, database retrieval is built.The beneficial effects of the invention are as follows：Data retrieval and the efficiency of data management greatly improved, simplify the complexity problem of database retrieval mechanism in relational data and non-relational data handling procedure.
Description
Technical field
The invention belongs to the construction method of technical field of network storage more particularly to a kind of database retrieval mechanism and it is
System.
Background technology
With the development of Internet technology, the service solution of high in the clouds the whole series is also growing with each passing hour.Traditional KTV applications
The landing of data and operational version are more dull inefficient, lack the support of advanced tools and advanced design framework thought, simply
Persistent storage and simply divide table handling, has made " the driving service logic by applying data " this soft project revolutionary character
Thought be necessarily required in the more and more various backward data application form of the application of KTV internet industries it is abundant and first
Into storage tool and design pattern support.
Storage engines are in Database Systems, a method of by storage management in data set, existing storage engines are main
It to be directed to relevant database, this storage engines are by the tables of data in relevant database according to storage method, index type
And locking horizontal partitioning, and the tables of data after subregion is loaded into memory.
But there is no release the relationship being highly coupled between data, the not storage of change data for existing storage engines
Method so that user devotes a tremendous amount of time when retrieving data and carrying out data management and searched, compared, and is existing
Storage engines do not support non-relational database.Therefore, existing storage engines are in face of the cloud storage distribution side of isomery
When formula, it is difficult to adapt to that data variation is big, the high new feature of Data Concurrent amount.
Invention content
Present invention aim to address existing storage engines when in face of the cloud storage distribution mode of isomery, it is difficult to adapt to
The deficiency that data variation is big, Data Concurrent amount is high proposes a kind of database retrieval mechanism construction method towards subregion, this method
Using the thought towards subregion, the operation to data is taken out, shield to bottom relevant database and non-relational data
Difference, unified management and access control to data.
In order to achieve the above objectives, on the one hand the embodiment of the present invention provides a kind of database retrieval mechanism construction method, including
Following steps：
Determine that using data storage method be relevant database or non-relational database, respectively to relevant database
Data access layer is established with non-relational database；
Data partition is carried out to the data in the relevant database or the non-relational database, obtains the number of partitions
According to；
Service request is analyzed, correspondence will be built between different requests and the partition data；
On the basis of the correspondence built, database retrieval is built.
Optionally, the process to relevant database layering includes the following steps：
All raw data tables in ergodic data library, are numbered every table, ensure that number does not repeat；
The table numbering is stored separately in another table；
It to the data of raw data table, extracts, the data of extraction is individually stored in new tables of data by column, then will be new
Tables of data in data number, all data extracted by column from raw data table are corresponding in all new tables of data
The consistent and new tables of data of number in all data numbers do not repeat.
Optionally, the process to non-relational database layering includes the following steps：
The data key of data is stored in a table, the corresponding data value storage of data key obtains in another table
Two new tables of data.
Optionally, it is described data access layer is established to relevant database and non-relational database respectively before, also
Including：
A point library is carried out to the data of persistent storage and divides table；
The Fen Ku and the table is divided to include：
A point library is carried out according to file ID；And a point table is carried out according to the business demand.
Optionally, further include：
By the high data buffer storage of access frequency to anti-amount silver bullet.
On the other hand, the embodiment of the present invention provides a kind of database retrieval mechanism construction system, including：
Access layer construction unit, for determining that using data storage method be relevant database or non-relational data
Data access layer is established in library to relevant database and non-relational database respectively；
Data partition unit, for the data in the relevant database or the non-relational database into line number
According to subregion, partition data is obtained；
Relationship construction unit, for analyzing service request, by different requests pass corresponding with structure between the partition data
System；
Construction unit is retrieved, on the basis of the correspondence built, building database retrieval.
Optionally, access layer construction unit includes relevant database delaminating units, is used for：
All raw data tables in ergodic data library, are numbered every table, ensure that number does not repeat；
The table numbering is stored separately in another table；
It to the data of raw data table, extracts, the data of extraction is individually stored in new tables of data by column, then will be new
Tables of data in data number, all data extracted by column from raw data table are corresponding in all new tables of data
The consistent and new tables of data of number in all data numbers do not repeat.
Optionally, access layer construction unit includes non-relational database delaminating units, is used for：
The data key of data is stored in a table, the corresponding data value storage of data key obtains in another table
Two new tables of data.
Optionally, further include that table unit is divided in a point library, be used for：
A point library is carried out to the data of persistent storage and divides table；
The Fen Ku and the table is divided to include：
A point library is carried out according to file ID；And a point table is carried out according to the business demand.
Optionally, further include：
Buffer unit is used for the high data buffer storage of access frequency to anti-amount silver bullet.
The beneficial effects of the invention are as follows：The present invention is compared with traditional technology, new to be added in existing cloud storage framework
The concept of data partition, the data partition greatly improved data retrieval and the efficiency of data management, simplify relational data
With the complexity problem of database retrieval mechanism in non-relational data handling procedure.
Description of the drawings
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
Other attached drawings are obtained according to these attached drawings.
Fig. 1 is a kind of flow diagram of database retrieval mechanism construction method of the present invention.
Fig. 2 is a kind of structure diagram of database retrieval mechanism construction system of the present invention.
Specific implementation mode
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation describes, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
The data application of KTV industries is various informative, and most common example is to apply to requesting song data
In order to promote the query capability of database, the present invention does a point library to database and table handling is divided such as to request a song information bank,
We can also remove a point table according to business demand in addition to going to a point library to divide table according to song ID dimensions according to KTV dimensions, or according to
A point table is removed in region, language, singer, requesting song time etc..
In addition to point library to persistent storage data divides table, the isomery of data storage facility multifarious for data application
Support is even more conclusive
For the song information often accessed, needed according to specific service logic, it is this that we can be buffered in redis
On anti-amount silver bullet, it is not only possible to realize quickly access, still significantly improve concurrency, improve system effectiveness
The logic support of search need may be implemented in ES (ElasticSearch), can also persistent storage using it this
The advantage and characteristic of body, can simply using JSON by HTTP come index data, it is intended that our search server
Always it can be used, it is therefore desirable to be able to since one and expand to hundreds, it is perfect to carry out database retrieval mechanism by ES
Solves the problems, such as this
KTV can use various database retrieval mechanism and storage service by cloud server, upload requesting song letter
When breath, by the way that in the direct persistent storage to Mysql of asynchronous message queue, the waiting for reducing the ends Client inside KTV is prolonged
When, improve the usage experience of client.
1. cloud storage framework
Cloud storage framework, by adding standard hardware and shared standard network (public internet or privately owned enterprises
Net) access of the offer to data, cloud capacity and performance have good scalability.Hundreds cloud storage service device externally provides clothes
Business, is a single, large-scale storage pool equipment in logic.
Cloud storage system according to the present invention, structural model are formed by following five layers from bottom to top, wherein first layer
(accumulation layer), third layer (basic management layer), the 4th layer (access layer), the cloud storage knot that layer 5 (application-interface layer) is standard
Structure model, the second layer are the present invention to build new database retrieval mechanism, and the data access layer newly introduced, the data are visited
Ask that the structure of layer has used the thought programmed towards subregion.Each layer is described further below：
11. accumulation layer
Accumulation layer is the most basic part of cloud storage.Storage device can be FC fibre channel storages, can be NAS
Can also be the DAS storage devices such as SCSI or SAS with the IP storage devices such as iSCSI.Storage device in cloud storage often quantity
The huge and more different geographicals of distribution, are linked together by wide area network, internet or FC fiber channel networks each other.It deposits
Reservoir can include storage engines, and storage engines be a kind of data to be stored in file (or memory) with a variety of different technologies
In construction method.Data storage method, index skill, the difference of locking level are divided according to being used in various different technologies
Qu Hou finally provides extensive function.
The present invention is to build the scheme of storage engines.
12. data access layer
Data access layer is using the data to relationship type and non-relational database realized towards subregion programming idea
Operation.Data access layer is encapsulated into specific routine call upwards, shows as identical program name, realizes downwards to not
Identical program name is mapped as the concrete operations to disparate databases by the access of same type database.
13. basic management layer
Basic management layer is the most crucial part of cloud storage.Basic management layer passes through cluster, distributed file system and net
The technologies such as lattice calculating realize the collaborative work between multiple storage devices in cloud storage, keep multiple storage devices external
Same service is provided, and provides bigger stronger better data access performance.
Identity identifying technology, data encryption technology ensure that the data in cloud storage will not be accessed by the user of unauthorized,
Meanwhile can ensure that the data in cloud storage will not lose by various data backups and disaster tolerance technology and measure, ensure that cloud is deposited
Store up itself safety and stabilization.
14. access layer
Any one authorized user can log in cloud storage system by the public application interface of standard, enjoy cloud and deposit
Storage service.Cloud storage unit of operation is different, and the access type and addressing means that cloud storage provides are also different.
15. application-interface layer
The most flexible part of cloud storage.Different cloud storage units of operation can develop different according to practical business type
Application service interface, different application services is provided.
After fully understanding the relevant dedicated concept of aforementioned present invention and theory, introduce the present invention's further below
Specific implementation process：
As shown in Figure 1, a kind of database retrieval mechanism construction method, includes the following steps：
Step 101, determine that using data storage method be relevant database or non-relational database, respectively to relationship
Type database and non-relational database establish data access layer；
Determine that using data storage method be relevant database or non-relational database, respectively to relevant database
Data access layer is established with non-relational database；Specific design is as follows：
A. access mode workable for relevant database includes：
ODBC(Open Database Connectivity)：Open Database Connection,
JDBC(Java Database Connectivity)：Java databases connect,
ADO(ActiveX Data Objects)：ActiveX Data Objects,
OLEDB(Object Linking and Embedding Database)：Object links embedding data library；
B. access mode workable for non-relational database includes：
Key-Value：Storage mode based on key-value pair；
Since the above-mentioned access mode referred to is the state of the art, it is not developed in details description.
Step 102, data partition is carried out to the data in the relevant database or the non-relational database, obtained
To partition data；
In relevant database, data are stored with the form of table, and every a line of table is exactly a data, table it is every
One row are exactly attribute that data are possessed, since relevant database is just for data itself, without attribute for data,
So subregion can be carried out to the attribute of data.
The process of relational data library partition is as follows：
All raw data tables in ergodic data library, are numbered every table, ensure that number does not repeat；
Table numbering in previous step is stored separately in another table；
To the data of raw data table, (one attribute of a row) extracts by column, and the data of extraction are individually stored in new number
According in table, then by the data number in new tables of data, all data extracted by column from raw data table, all new
Corresponding number is consistent in tables of data, this number can be the major key in raw data table, can also generate in other ways,
But ensure that all data numbers in new tables of data do not repeat.
In non-relational database, data stores with the form of key-value pair, each data can all have data key with
Corresponding data value, so when to non-relational database subregion, it is only necessary to which the data key of data is stored in one
In table, the corresponding data value storage of data key obtains two new tables of data in another table.
The data stored above with new tables of data are exactly partition data.
Step 103, service request is analyzed, correspondence will be built between different requests and the partition data；
Service request can be a kind of abstract behavior, as the operation of data can be abstracted as inquiry, newly-increased, modification, deletion, day
Will etc., each operation can be used as a service request；Present embodiment is illustrated for inquiring.
In Database Systems, the access point of data manipulation is extracted first, and this data manipulation corresponds to specific database
Operating procedure, specific database manipulation step need specific database access mode again, by specific database access side
Formula is packaged into a program, this program can also may have access to non-relational database, this program with access relation type database
As notice.
Application-interface layer can send out request to server, and the program for sending out request is access point, and request is sent out from access point
Afterwards, division operation Intercept and capture service is asked, and captures the program of access point as point of contact, the request captured by point of contact will be by cutting
Point judges the validity of request, is then intercepted if it is invalidation request, will just ask to give at notice if it is effective request
Reason, notice can turn to specific database object for specific request.
Specific database object becomes partition data by the first step, and at this moment, partition data is exactly target object,
If target object itself is the encapsulation to data, these encapsulation only contain operation to truthful data, then containing
Agent object is known as to the data object of truthful data operation, when division operation will handle partition data, will first handle generation
Object is managed, then by agent object, realizes the operation to partition data.
Above by operation agent object, the process of indirect operation partition data is known as being woven into.
Step 104, on the basis of the correspondence built, database retrieval is built.
Storage itself due to data still needs dependence type database and non-relational database, but due to the two
Architectural difference is big, and when data exchange is easy to cause access error, therefore during building database retrieval mechanism, uses step
The data access mode of rapid 1 structure accesses database respectively.The influence brought due to the difference of database is reduced with this.
The construction process of relevant database storage engines includes the following steps：
A. the deletion of data, update and insertion operation are mixed, so that data manipulation generates more the size of data
Few fragmentation；By merging adjacent deleted memory block, if as soon as and memory block be deleted, expand to next deposit
The mode of storage block can improve the efficiency of storage.
B. the largest index number of each table is set.It is 64 that setting, which each indexes maximum columns,.Maximum key length is
1024 bytes.The case where being more than 250 byte for key length, use a key block more than 1024 bytes.
C. it if the table among tables of data does not have the block of suitable size, increases a table newly and carrys out auxiliary storage.
D. a mark is set in index file, it shows whether table is correctly closed or not.
E. lock mechanism is established.It is locked when accessing data, the process for detecting deadlock generates after deadlock very in systems
The presence of deadlock is detected by short time.Once detect and produce deadlock in system, just by judge request for data when
Between select to generate two of deadlock operations, allow the operation of first to file to first carry out.If the time of application is identical, overtime machine is used
System release lock, two operations do not execute.
Non-relational database construction storage engines construction process includes the following steps：
A. available key assignments data structure is selected：
Lists (list),
Sets (set),
Sorted Sets (ordered set),
Hashes (Hash table),
The data type of key assignments determines the operation that the key assignments is supported, such as the intersection, simultaneously of list, set or ordered set
Collect, look into the advanced atomic operations such as collection.
B. data key is first stored into high byte, to allow the index compression of a higher address, when record is with sorted
It is sequentially inserted into, index tree is reconstructed, so that high byte only includes a key.To improve the storage efficiency of key assignments data structure.
C. data value is first stored into low byte, then stores high byte, to reduce the data that data value and data value are run
The server of the dependence of lab environment, operation data library indicates data value using complement code and floating-point format, the word in data line
Section is usually not united, reads not united byte from a direction, server can be made to have more when obtaining train value in this way
High efficiency.
D. master-slave synchronisation：The storage medium of non-relational database can be deployed in the system of isomery, so being stored in different
Data in construction system database need to synchronize, to ensure the consistency of data.
The system of isomery is got up by network connection, and in a network, each heterogeneous system is exactly a memory node, storage
Data in node need to synchronize, and needing will a global timing mode so that the memory node in network is in the regular hour
Interior progress data synchronization, the method for synchronization can have the timing of primary storage node and from two kinds of memory node timings.Main timing working side
Formula is the working method using this node clock source as timing.From timing working mode be with a certain reference data time be mark
Standard, the working method of timing again after being locked to this node clock source.
After completing above-mentioned steps, division operation is completed, and the partition data after being processed using soap protocol is converted into XML
Format, and need to return to the corresponding data of application-interface layer according to request.
Optionally, it is described data access layer is established to relevant database and non-relational database respectively before, also
Including：
A point library is carried out to the data of persistent storage and divides table；
The Fen Ku and the table is divided to include：
A point library is carried out according to file ID；And a point table is carried out according to the business demand.
Optionally, table can be divided with singing-hall dimension, divides table with singer's dimension, divide table with language etc..
Optionally, further include：
By the high data buffer storage of access frequency to anti-amount silver bullet.
To realize quickly access, and reply high concurrent amount, it can need, be buffered according to specific service logic
In Redis.
Technical scheme of the present invention is further described with reference to one embodiment：Assuming that user needs selection one
A song table needs to select suitable song in terms of type, language, singer, temperature three.Present use is programmed towards subregion
Database retrieval mechanism handle screening request.
Step 1：Build data access layer.
Since all songs are not or not the same region, the song data in each region is stored in different servers
On, and only store in region server type, language, singer information, and temperature information storage is beyond the clouds on server, and
Different server uses different database software, so to establish a set of access data for this isomeric data storage
Mechanism.
Step 2：Subregion is carried out to data
After establishing access mechanism, the master data in disparate databases is read from table, by type, language, song
The all properties such as hand, temperature individually extract and cut mark, and song ID is used in combination to be associated with.Data in type, language, singer, Thermometer are
For partition data, partition data is exactly target object.
Step 3, service request and partition data are established into correspondence
Service request includes the operations such as reading, deleting, sort, it is assumed that read, all information of the read requests are analyzed,
Itself and above-mentioned partition data are established into correspondence.
Step 4：After institute's data in need have been obtained according to screening conditions, the partition data of dispersion is reformulated and is closed
The independent information of connection, is presented in face of screening request person, is the complete song information of combination.
On the other hand, the embodiment of the present invention provides a kind of database retrieval mechanism construction system, as shown in Fig. 2, including：
Access layer construction unit 201, for determining that using data storage method be relevant database or non-relational number
According to library, data access layer is established to relevant database and non-relational database respectively；
Data partition unit 202, for the data in the relevant database or the non-relational database into
Row data partition, obtains partition data；
Relationship construction unit 203, it is for analyzing service request, different requests are corresponding with structure between the partition data
Relationship；
Construction unit 204 is retrieved, on the basis of the correspondence built, building database retrieval.
Optionally, access layer construction unit includes relevant database delaminating units, is used for：
All raw data tables in ergodic data library, are numbered every table, ensure that number does not repeat；
The table numbering is stored separately in another table；
It to the data of raw data table, extracts, the data of extraction is individually stored in new tables of data by column, then will be new
Tables of data in data number, all data extracted by column from raw data table are corresponding in all new tables of data
The consistent and new tables of data of number in all data numbers do not repeat.
Optionally, access layer construction unit includes non-relational database delaminating units, is used for：
The data key of data is stored in a table, the corresponding data value storage of data key obtains in another table
Two new tables of data.
Optionally, further include that table unit is divided in a point library, be used for：
A point library is carried out to the data of persistent storage and divides table；
The Fen Ku and the table is divided to include：
A point library is carried out according to file ID；And a point table is carried out according to the business demand.
Optionally, further include：
Buffer unit is used for the high data buffer storage of access frequency to anti-amount silver bullet.
Technical scheme of the present invention is further described with reference to one embodiment：Assuming that user needs selection one
A song table needs to select suitable song in terms of type, language, singer, temperature three.Present use is programmed towards subregion
Database retrieval mechanism handle screening request.
Step 1：Build data access layer.
Since all songs are not or not the same region, the song data in each region is stored in different servers
On, and only store in region server type, language, singer information, and temperature information storage is beyond the clouds on server, and
Different server uses different database software, so to establish a set of access data for this isomeric data storage
Mechanism.
Step 2：Subregion is carried out to data
After establishing access mechanism, the master data in disparate databases is read from table, by type, language, song
The all properties such as hand, temperature individually extract and cut mark, and song ID is used in combination to be associated with.Data in type, language, singer, Thermometer are
For partition data, partition data is exactly target object.
Step 3, service request and partition data are established into correspondence
Service request includes the operations such as reading, deleting, sort, it is assumed that read, all information of the read requests are analyzed,
Itself and above-mentioned partition data are established into correspondence.
Step 4：After institute's data in need have been obtained according to screening conditions, the partition data of dispersion is reformulated and is closed
The independent information of connection, is presented in face of screening request person, is the complete song information of combination.
The beneficial effects of the invention are as follows：The present invention is compared with traditional technology, new to be added in existing cloud storage framework
The concept of data partition, the data partition greatly improved data retrieval and the efficiency of data management, simplify relational data
With the complexity problem of database retrieval mechanism in non-relational data handling procedure.
Those skilled in the art will also be appreciated that the various illustrative components, blocks that the embodiment of the present invention is listed
(illustrative logical block), unit and step can pass through the knot of electronic hardware, computer software, or both
Conjunction is realized.To clearly show that the replaceability (interchangeability) of hardware and software, above-mentioned various explanations
Property component (illustrative components), unit and step universally describe their function.Such work(
Can be that the design requirement for depending on specific application and whole system is realized by hardware or software.Those skilled in the art
Can be for each specific function of applying, the realization of various methods can be used described, but this realization is understood not to
Range beyond protection of the embodiment of the present invention.
Various illustrative logical blocks or unit described in the embodiment of the present invention can by general processor,
Digital signal processor, application-specific integrated circuit (ASIC), field programmable gate array or other programmable logic systems, discrete gate
Or described function is realized or is operated in transistor logic, the design of discrete hardware components or any of the above described combination.General place
It can be microprocessor to manage device, and optionally, which may be any traditional processor, controller, microcontroller
Device or state machine.Processor can also be realized by the combination of computing system, such as digital signal processor and microprocessor,
Multi-microprocessor, one or more microprocessors combine a digital signal processor core or any other like configuration
To realize.
The step of construction method or algorithm described in the embodiment of the present invention, can be directly embedded into hardware, processor executes
Software module or the two combination.Software module can be stored in RAM memory, flash memory, ROM memory, EPROM
Other any form of storages in memory, eeprom memory, register, hard disk, moveable magnetic disc, CD-ROM or this field
In medium.Illustratively, storaging medium can be connect with processor, so that processor can read letter from storaging medium
Breath, and can be to storaging medium stored and written information.Optionally, storaging medium can also be integrated into processor.Processor and storage
Medium can be set in ASIC, and ASIC can be set in user terminal.Optionally, processor and storaging medium can also be set
It is placed in the different components in user terminal.
In one or more illustrative designs, above-mentioned function described in the embodiment of the present invention can be in hardware, soft
Part, firmware or the arbitrary of this three combine to realize.If realized in software, these functions can store and computer-readable
On medium, or with one or more instruction or code form be transmitted on the medium of computer-readable.Computer readable medium includes electricity
Brain storaging medium and convenient for allow computer program to be transferred to from a place telecommunication media in other places.Storaging medium can be with
It is that any general or special computer can be with the useable medium of access.For example, such computer readable media may include but
It is not limited to RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage systems or other
What can be used for carry or store with instruct or data structure and it is other can be by general or special computer or general or specially treated
The medium of the program code of device reading form.In addition, any connection can be properly termed computer readable medium, example
Such as, if software is to pass through a coaxial cable, fiber optic cables, double from a web-site, server or other remote resources
Twisted wire, Digital Subscriber Line (DSL) are defined with being also contained in for the wireless way for transmitting such as example infrared, wireless and microwave
In computer readable medium.The disk (disk) and disk (disc) includes compress disk, radium-shine disk, CD, DVD, floppy disk
And Blu-ray Disc, disk is usually with magnetic duplication data, and disk usually carries out optical reproduction data with laser.Combinations of the above
It can also be included in computer readable medium.
Above-described specific implementation mode has carried out further the purpose of the present invention, technical solution and advantageous effect
It is described in detail, it should be understood that the foregoing is merely the specific implementation mode of the present invention, is not intended to limit the present invention
Protection domain, all within the spirits and principles of the present invention, any modification, equivalent substitution, improvement and etc. done should all include
Within protection scope of the present invention.
Claims (10)
1. a kind of database retrieval mechanism construction method, includes the following steps：
Determine that using data storage method be relevant database or non-relational database, respectively to relevant database and non-
Relevant database establishes data access layer；
Data partition is carried out to the data in the relevant database or the non-relational database, obtains partition data；
Service request is analyzed, correspondence will be built between different requests and the partition data；
On the basis of the correspondence built, database retrieval is built.
2. according to the method described in claim 1, it is characterized in that, the process to relevant database layering includes as follows
Step：
All raw data tables in ergodic data library, are numbered every table, ensure that number does not repeat；
The table numbering is stored separately in another table；
It to the data of raw data table, extracts, the data of extraction is individually stored in new tables of data by column, then by new number
According to the data number in table, all data extracted by column from raw data table, the corresponding volume in all new tables of data
It is number consistent, and all data numbers in new tables of data do not repeat.
3. according to the method described in claim 1, it is characterized in that, the process to non-relational database layering includes as follows
Step：
The data key of data is stored in a table, the corresponding data value storage of data key obtains two in another table
New tables of data.
4. according to the method described in claim 1, it is characterized in that, described respectively to relevant database and non-relational number
Before establishing data access layer according to library, further include：
A point library is carried out to the data of persistent storage and divides table；
The Fen Ku and the table is divided to include：
A point library is carried out according to file ID；And a point table is carried out according to the business demand.
5. according to the method described in claim 1, it is characterized in that, further including：
By the high data buffer storage of access frequency to anti-amount silver bullet.
6. a kind of database retrieval mechanism construction system, including：
Access layer construction unit divides for determining that using data storage method be relevant database or non-relational database
It is other that data access layer is established to relevant database and non-relational database；
Data partition unit, for carrying out data point to the data in the relevant database or the non-relational database
Area obtains partition data；
Relationship construction unit will build correspondence for analyzing service request between different requests and the partition data；
Construction unit is retrieved, on the basis of the correspondence built, building database retrieval.
7. system according to claim 6, which is characterized in that access layer construction unit includes that relevant database layering is single
Member is used for：
All raw data tables in ergodic data library, are numbered every table, ensure that number does not repeat；
The table numbering is stored separately in another table；
It to the data of raw data table, extracts, the data of extraction is individually stored in new tables of data by column, then by new number
According to the data number in table, all data extracted by column from raw data table, the corresponding volume in all new tables of data
It is number consistent, and all data numbers in new tables of data do not repeat.
8. system according to claim 6, which is characterized in that access layer construction unit includes non-relational database layering
Unit is used for：
The data key of data is stored in a table, the corresponding data value storage of data key obtains two in another table
New tables of data.
9. system according to claim 6, which is characterized in that further include that table unit is divided in a point library, be used for：
A point library is carried out to the data of persistent storage and divides table；
The Fen Ku and the table is divided to include：
A point library is carried out according to file ID；And a point table is carried out according to the business demand.
10. system according to claim 6, which is characterized in that further include：
Buffer unit is used for the high data buffer storage of access frequency to anti-amount silver bullet.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810213892.1A CN108595490A (en) | 2018-03-15 | 2018-03-15 | A kind of construction method and system of database retrieval mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810213892.1A CN108595490A (en) | 2018-03-15 | 2018-03-15 | A kind of construction method and system of database retrieval mechanism |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108595490A true CN108595490A (en) | 2018-09-28 |
Family
ID=63626251
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810213892.1A Pending CN108595490A (en) | 2018-03-15 | 2018-03-15 | A kind of construction method and system of database retrieval mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108595490A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110717074A (en) * | 2019-09-26 | 2020-01-21 | 上海依图网络科技有限公司 | Object persistence methods, apparatus, media and systems |
| CN111552740A (en) * | 2020-04-28 | 2020-08-18 | 腾讯科技（深圳）有限公司 | Data processing method and device |
| CN113486005A (en) * | 2021-06-09 | 2021-10-08 | 中国科学院空天信息创新研究院 | Space science satellite big data organization and query method under heterogeneous structure |
| US20220405417A1 (en) * | 2021-06-17 | 2022-12-22 | International Business Machines Corporation | Sensitive data classification in non-relational databases |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102495853A (en) * | 2011-11-17 | 2012-06-13 | 成都康赛电子科大信息技术有限责任公司 | Aspect-oriented cloud storage engine construction method |
| CN105468720A (en) * | 2015-11-20 | 2016-04-06 | 北京锐安科技有限公司 | Method for integrating distributed data processing systems, corresponding systems and data processing method |
| CN106649328A (en) * | 2015-10-30 | 2017-05-10 | 宁波轩悦行电动汽车服务有限公司 | Time-sharing lease system data access layer interface establishment method |
| US9886483B1 (en) * | 2010-04-29 | 2018-02-06 | Quest Software Inc. | System for providing structured query language access to non-relational data stores |
-
2018
- 2018-03-15 CN CN201810213892.1A patent/CN108595490A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9886483B1 (en) * | 2010-04-29 | 2018-02-06 | Quest Software Inc. | System for providing structured query language access to non-relational data stores |
| CN102495853A (en) * | 2011-11-17 | 2012-06-13 | 成都康赛电子科大信息技术有限责任公司 | Aspect-oriented cloud storage engine construction method |
| CN106649328A (en) * | 2015-10-30 | 2017-05-10 | 宁波轩悦行电动汽车服务有限公司 | Time-sharing lease system data access layer interface establishment method |
| CN105468720A (en) * | 2015-11-20 | 2016-04-06 | 北京锐安科技有限公司 | Method for integrating distributed data processing systems, corresponding systems and data processing method |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110717074A (en) * | 2019-09-26 | 2020-01-21 | 上海依图网络科技有限公司 | Object persistence methods, apparatus, media and systems |
| CN110717074B (en) * | 2019-09-26 | 2023-04-07 | 上海依图网络科技有限公司 | Object persistence methods, apparatus, media and systems |
| CN111552740A (en) * | 2020-04-28 | 2020-08-18 | 腾讯科技（深圳）有限公司 | Data processing method and device |
| CN111552740B (en) * | 2020-04-28 | 2023-09-12 | 腾讯科技（深圳）有限公司 | Data processing method and device |
| CN113486005A (en) * | 2021-06-09 | 2021-10-08 | 中国科学院空天信息创新研究院 | Space science satellite big data organization and query method under heterogeneous structure |
| CN113486005B (en) * | 2021-06-09 | 2024-04-19 | 中国科学院空天信息创新研究院 | Space science satellite big data organization and query method under heterogeneous structure |
| US20220405417A1 (en) * | 2021-06-17 | 2022-12-22 | International Business Machines Corporation | Sensitive data classification in non-relational databases |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7410181B2 (en) | Hybrid indexing methods, systems, and programs | |
| US10764045B2 (en) | Encrypting object index in a distributed storage environment | |
| CN110799960B (en) | System and method for database tenant migration | |
| US10191932B2 (en) | Dependency-aware transaction batching for data replication | |
| CN106462545B (en) | Scalable file storage service | |
| Padhy et al. | RDBMS to NoSQL: reviewing some next-generation non-relational database’s | |
| CN106462544B (en) | Session management in distributed storage systems | |
| US7302520B2 (en) | Method and apparatus for data storage using striping | |
| CN106462601B (en) | Atomic writing for multiple extent operations | |
| CN108595490A (en) | A kind of construction method and system of database retrieval mechanism | |
| CN103353867B (en) | Distributed memory system with network service client interface | |
| Thanh et al. | A taxonomy and survey on distributed file systems | |
| CN102495853B (en) | Aspect-oriented cloud storage engine construction method | |
| TW200303468A (en) | Deferred copy-on-write of a snapshot | |
| CN109298835B (en) | Data archiving processing method, device, equipment and storage medium of block chain | |
| US20220083504A1 (en) | Managing snapshotting of a dataset using an ordered set of b+ trees | |
| CN111782134A (en) | Data processing method, device, system and computer readable storage medium | |
| JP5236129B2 (en) | Storage service providing apparatus, system, service providing method, and service providing program | |
| CN116467275A (en) | Shared remote storage method, apparatus, system, electronic device and storage medium | |
| US20190370382A1 (en) | Unbalanced partitioning of database for application data | |
| Rammer et al. | Atlas: A distributed file system for spatiotemporal data | |
| CN110245037B (en) | Hive user operation behavior restoration method based on logs | |
| US9292523B1 (en) | Managing data storage | |
| Azim et al. | Offsite 2-Way Data Replication toward Improving Data Refresh Performance | |
| Lu et al. | Research on Cassandra data compaction strategies for time-series data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180928 |