CN110134681A - Data storage and querying method, device, computer equipment and storage medium - Google Patents
Data storage and querying method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN110134681A CN110134681A CN201910299922.XA CN201910299922A CN110134681A CN 110134681 A CN110134681 A CN 110134681A CN 201910299922 A CN201910299922 A CN 201910299922A CN 110134681 A CN110134681 A CN 110134681A
- Authority
- CN
- China
- Prior art keywords
- data
- tree
- node
- major key
- stored
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of date storage method, the following steps are included: in response to instruction data storage, generate the abstract tree for characterizing the index relative between multiple tables of data, the abstract tree includes multiple nodes, each node includes information and map information, the information characterizes the type for the data to be stored on the node, and the map information is used to characterize the other nodes for having mapping relations with present node；Data to be stored is obtained, the data to be stored is stored onto the corresponding node of the abstract tree according to the type of the data to be stored, forms full dose tree；The major key information and map information of each node are extracted from the full dose tree, major key tree are formed, wherein the major key information is corresponding with the information；The full dose tree is stored to database, and by the major key tree and is stored as inverted index.
Description
Technical field
The present invention relates to technical field of data processing more particularly to a kind of high-speed data independent of database performance to deposit
Store up querying method, device, computer equipment and storage medium.
Background technique
Widely available with database technology, Various types of data information grows at top speed trend in magnanimity.Current most of meters
The application scenarios of calculation machine software are all relevant databases, in the inquiry and storage of relationship complex condition, can be consumed a large amount of
Database resource, including memory space and allocating time, therefore under current line sparetime university data background, with the increasing of data volume
It is long, it is easy to which that the bottleneck for reaching database seriously affects the efficiency of data storage and inquiry.
Summary of the invention
The object of the present invention is to provide it is a kind of independent of the high-speed data processing querying method of database performance, system,
Computer equipment and storage medium, it is of the existing technology for solving the problems, such as.
To achieve the above object, the present invention provides a kind of high-speed data processing method, comprising the following steps:
In response to instruction data storage, the abstract tree for characterizing the index relative between multiple tables of data, institute are generated
Stating abstract tree includes multiple nodes, and each node includes information and map information, and the information characterizes institute
The type for the data to be stored on node is stated, the map information is used to characterize the other sections for having mapping relations with present node
Point；
Data to be stored is obtained, is stored the data to be stored to described abstract according to the type of the data to be stored
In the corresponding node of tree, full dose tree is formed；
The major key information and map information of each node are extracted from the full dose tree, form the tree-shaped knot of major key
Structure, wherein the major key information is corresponding with the information；
The full dose tree is stored to database, and by the major key tree and is stored as inverted index.
Further, the acquisition data to be stored, according to the type of the data to be stored by the data to be stored
In the corresponding node for storing the abstract tree, formed full dose tree the step of include:
Each of abstract tree node is disassembled, tables of data to be updated is obtained, according to the data to be stored
Type increases data into every tables of data to be updated.
Further, non-master key index is also preserved in the major key tree, for characterizing the multiple tables of data
In non-primary key and major key between corresponding relationship.
To achieve the above object, the present invention provides a kind of high-speed data querying method, comprising the following steps:
It is instructed in response to data query, obtains the major key tree saved in a manner of inverted index, the major key is tree-shaped
Structure includes multiple nodes, the corresponding one or more tables of data with identical major key of each node；
The field that will be inquired inputs the major key tree, obtains node corresponding with the field；
Target data is inquired from tables of data corresponding with the node；
The target data is exported by output mode.
Further, the field that will be inquired inputs the major key tree, obtains corresponding with the field
Node the step of any one of include the following steps:
The field is major key, according to major key query node；
The field is non-primary key, inquires corresponding with non-primary key major key from concordance list, and according to major key from master
Query node in key tree.
Further, the output mode output data of pressing includes pressing designated mode output data or defeated by default mode
Data out.
To achieve the above object, the present invention provides a kind of high-speed data processing system, comprising:
Abstract tree generation module, suitable for generating the abstract tree for characterizing the index relative between multiple tables of data,
The abstract tree includes multiple nodes, and each node includes information and map information, the information characterization
The type for the data to be stored on the node, the map information, which is used to characterize, has the other of mapping relations with present node
Node；
Full dose tree generation module is suitable for obtaining data to be stored, according to the type of the data to be stored will it is described to
In storing data storage to the corresponding node of the abstract tree, form full dose tree；
Major key tree generation module, major key information and mapping suitable for extracting each node from the full dose tree
Information forms major key tree, wherein the major key information is corresponding with the information；
Data memory module is suitable for storing the full dose tree to oracle database, and by the major key
Tree is stored as inverted index.
To achieve the above object, the present invention provides a kind of data query system, comprising:
Major key tree obtains module, is adapted to respond to instruct in data query, obtains the major key stored in a manner of inverted index
Tree, the major key tree include multiple nodes, the corresponding one or more numbers with identical major key of each node
According to table；
Node obtains module, and the field suitable for that will inquire inputs the major key tree, obtains and the field
Corresponding node；
Data inquiry module, suitable for inquiring target data from tables of data corresponding with the node；
Data outputting module is suitable for exporting the target data by output mode.
To achieve the above object, it the present invention also provides a kind of computer equipment, including memory, processor and is stored in
On memory and the computer program that can run on a processor, the processor are realized above-mentioned when executing the computer program
The step of method.
To achieve the above object, the present invention also provides computer readable storage mediums, are stored thereon with computer program, institute
State the step of above method is realized when computer program is executed by processor.
High-speed data processing provided by the invention and querying method, device, computer equipment and storage medium are suitable for number
According to processing technology field, a kind of automation realization side independent of database performance is provided for the storage and inquiry of mass data
Case.Database in the present invention will no longer be required to safeguard a large amount of relation data and index, and be the introduction of big data background
High speed inverted index frame ES (elasticsearch) carrys out the major key information of storage service relation data, only retains in database
The backup of the details and relationship of data.When having inquiry request, the present invention can retrieve first inverted index frame according to condition
Frame ES inquires major key information, then executes inquiry from major key to database, shields all non-primary key database requests, pole
Recall precision is improved greatly.
Detailed description of the invention
Fig. 1 is the flow chart of date storage method embodiment one of the invention；
Fig. 2 is the program module schematic diagram of data storage device embodiment one of the invention；
Fig. 3 is the hardware structural diagram of data storage device embodiment one of the invention；
Fig. 4 is the flow chart of data query embodiment of the method one of the invention；
Fig. 5 is the program module schematic diagram of data query device embodiment one of the invention.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that described herein, specific examples are only used to explain the present invention, not
For limiting the present invention.Based on the embodiments of the present invention, those of ordinary skill in the art are not before making creative work
Every other embodiment obtained is put, shall fall within the protection scope of the present invention.
High-speed data processing provided by the invention and querying method, device, computer equipment and storage medium are suitable for number
According to processing technology field, a kind of automation realization side independent of database performance is provided for the storage and inquiry of mass data
Case.The invention mainly comprises data storage and data query two large divisions.In data storage procedure, the present invention obtains respectively first
Logical relation and its major key between a tables of data generate the abstract tree comprising multiple nodes according to logical relation and major key come table
Above-mentioned logical relation is levied, wherein each node has unique major key, and correspondingly corresponding with one or more tables of data.?
When carrying out data storage, the present invention stores the data information in data to be updated to corresponding each according to the structure of abstract tree
In tables of data under node, while the abstract tree (i.e. major key tree) for only containing major key information is stored as inverted index (ES)
Form, and other data informations in addition to major key are saved to oracle database.When carrying out data query, the present invention
Can first from transferred in inverted index major key tree with clear band inquiry data where node, then further according to nodal information from
Inquiry and one or more tables of data corresponding to the node in oracle database.Above scheme of the invention enormously simplifies
Data query process, the tables of data without traversing all are inquired, to effectively increase the efficiency of data storage and query.
Embodiment one
Referring to Fig. 1, in a kind of high-speed data processing method of the present embodiment, comprising the following steps:
S1: in response to instruction data storage, generating the abstract tree for characterizing the index relative between multiple tables of data,
The abstract tree includes multiple nodes, and each node includes information and map information, the information characterization
The type for the data to be stored on the node, the map information, which is used to characterize, has the other of mapping relations with present node
Node.
This step is used to build the frame of a progress data storage first, this frame has arranged different data properly
Mapping relations between the storage location and tables of data of table, so as to subsequent toward storing specific data on corresponding position.This hair
The bright frame by above-mentioned carry out data storage is known as abstract tree, which includes multiple nodes, here
What node indicated is the position of tables of data storage, each node corresponding storage one or multiple storage tables.Section in the present invention
Point is divided according to the type of data to be stored, and specific type for example can be customer information, account information, bank
The contents such as card information.There are mapping relations, such as account information is generated on the basis of customer information between node and node
, first by the basic condition information of client, then re-record the account information of each client.In this case by customer information
The node at place is as father node A, using the node where account information as child node B.Between interior joint and node of the present invention
Mapping relations node in other words map information, can be by adding the father being associated with its own section on each node
Information or child node information are put to determine.It should be pointed out that when by way of father node to indicate map information, by
No longer there is father node in the node of top layer, therefore the parent information of the node of the top layer is emptied；It is saved when by son
The mode of point is come when indicating map information, since undermost node no longer has child node, by the undermost node
Child node information empty.
By taking bank's background data base system as an example, client's essential information is generally preserved, client applies for account information, client
The multinomial content such as bank card information includes one or multiple tables of data in each content.Assuming that the database on certain bank backstage
In data related with client are divided into following several tables of data: client information table (A table), customer information supplementary table (A1
Table), client's application record table (C table), customer account information table (B table), account amount control table (B1 table) and account bank
Card table (D table).Wherein client information table (A table) and customer information supplementary table (A1 table) major key custNo having the same, client
Account information table (B table) and the control table of account amount (B1 table) major key accountNo having the same, client's application record table (C
Table) there is major key applyNo, account bank card table (D table) has major key cardNo.
The present invention according to logic index relative to above-mentioned several tables of data re-started divide and by tree into
Row is shown.Firstly, to describe the number comprising client, name, identification card number, address etc. most basic for client information table (A table)
Information, therefore as father's layer.In addition there is the also customer information supplementary table of identical major key with client information table (A table)
(A1 table), as the further supplement to customer information in client information table, such as the Income situation of client.In order to simplify knot
Table with identical major key is classified as same node by structure, the present invention, is denoted as node A, and using it as father's layer.With client information table
Table with index relative includes client's application record table (C table) and customer account information table (B table), therefore this two tables are made
For the first sublayer.It is associated with since client's application record table (C table) does not have each other with customer account information table (B table)
System, therefore client's application record table (C table) is denoted as node C respectively, customer account information table (B table) is denoted as node B.Separately
Outside, also there are direct index relative, and account amount control between account amount control table (B1 table) and client information table (A table)
Tabulate (B1 table) has same major key accountNo between customer account information table (B table), therefore for simplicity by account
Family amount control table (B1 table) is also included into node B.Next, having the table of index relative with customer account information table (B table)
It is account bank card table (D table), therefore account bank card table (D table) is incorporated into as the second sublayer, and is labeled as node D.Such as Fig. 2
It is shown, it is above-mentioned to contain many levels and the tree of multiple nodes is known as abstract tree of the invention, it can intuitively show
Index relative between each tables of data out.
Above-mentioned father's layer, the hierarchical relationship between the first sublayer, the second sublayer can indicate with child node information, such as
The child node information that there are mapping relations with the node is recorded in each node.Above, the child node information of node A includes section
The child node information of point B and node C, node B include node D.It is saved after node C and node D without the son with mapping relations
Point can then empty the child node information in node C and node D.
When implementation, the abstract tree of generation is saved into configuration file, dataSchema is named as.
S2: obtaining data to be stored, is stored the data to be stored to described according to the type of the data to be stored
In the corresponding node of abstract tree, full dose tree is formed.
This step adds the data information to be stored on the basis of abstract tree, becomes full dose tree.This hair
Include multiple nodes in bright abstract book, the corresponding information of each node, under each information comprising one or
Multiple tables of data.Data record is filled into the corresponding data table of corresponding node according to type, is just realized tree-shaped by being abstracted
Conversion of the structure to full dose tree.Preferably, the information in the present invention can be determined according to the major key of tables of data.
For example, it is desired to by following data entry storage to database:
(1) client information table
custNo00123；Customer ID: 210123190101019987；
(2) account information table
Coding of accounts: acc01；
(3) account bank card table
Card number: card662266；Card details: abc
(4) application record table
Application number: apl0012333552, application country: CN；
(5) application record table
Application number: apl0012334663, mobile phone: 13633331804, applicant city: Shanghai.
When implementation, the present invention obtains abstract tree dataSchema from configuration file, carries out dismantling to dataSchema and obtains
Corresponding tables of data under each node and node is obtained, the data entry that will be stored is stored by field contents to different numbers
According in table.
This step be on the basis of abstract tree dataSchema into the corresponding data table under each node storing data,
Corresponding with abstract tree, the tree for being stored with partial data is known as full dose tree.
S3: extracting the major key information and map information of each node from the full dose tree, and it is tree-shaped to form major key
Structure, wherein the major key information is corresponding with the information.
This step extracts major key on the basis of full dose tree, closes to establish and describe to index between multiple tables of data
The data of the major key tree of system.By taking the data entry stored in step S2 as an example, on the basis of only extracting major key data,
The data entry information stored in ES is as follows:
(1) client information table
Customer number: custNo00123；
(2) account information table
Coding of accounts: acc01；
(3) account bank card table
Card number: card662266；
(4) application record table
Application number: apl0012333552；
(5) application record table
Application number: apl0012334663.
Compared with full dose tree, the major key tree generated in this step retains on eliminating mass data content basis
The index informations of necessary major key data, those skilled in the art only need can by the index information of the major key data
The storage location of data is obtained, therefore can effectively improve storage and the search efficiency of data.
S4: the full dose tree is stored and is stored as down to oracle database, and by the major key tree
Row's index.
Full dose tree in the present invention is exactly the partial data record according to index relative storage, this step will be complete
Data record be saved in oracle database, facilitate those skilled in the art to store and inquiry operation.With the prior art
Unlike, the oracle database in the present invention is not connected to each other with conventional application program directly, i.e., when application program needs
When data in database are stored or inquired, it is not directly facing oracle database and is operated, but go first
The major key tree that characterize data table index relationship is extracted in ES index database, obtains each tables of data from major key tree
Between relationship and its location information, then transferred from oracle database again according to the relationship and its location information corresponding
Tables of data is operated.The present invention no longer needs to traverse all tables of data from oracle database, therefore can be greatly
Improve data storage and query efficiency.Simultaneously as the index relative of tables of data individually stores, therefore oracle database
It no longer needs to safeguard a large amount of relation data and index, so that the limit of data storage is not in the property for depending on oracle database
Energy.
In addition, the present invention carries out the major key tree after extraction in a manner of inverted index (elasticsearch)
It saves.So-called inverted index refers to position and frequency of occurrence of the storage keyword in article, to improve searching keyword
Efficiency.
Please continue to refer to Fig. 2, a kind of high-speed data processing device, in the present embodiment, data storage device 10 are shown
It may include or be divided into one or more program modules, one or more program module is stored in storage medium,
And as performed by one or more processors, to complete the present invention, and above-mentioned automatic update method can be realized.The present invention is so-called
Program module is the series of computation machine program instruction section for referring to complete specific function, than program itself more suitable for describing number
According to implementation procedure of the storage device 10 in storage medium.The function of each program module of the present embodiment will specifically be introduced by being described below
Can:
Abstract tree generation module 11 is abstracted tree-shaped knot suitable for generates the index relative between the multiple tables of data of characterization
Structure, the abstract tree include multiple nodes, and each node includes information and map information, the information table
Levy the type for the data to be stored on the node, the map information be used to characterize with present node with mapping relations its
Its node；
Full dose tree generation module 12 is suitable for obtaining data to be stored, will be described according to the type of the data to be stored
In data to be stored storage to the corresponding node of the abstract tree, form full dose tree；
Major key tree generation module 13, suitable for extracting the major key information of each node from the full dose tree and reflecting
Information is penetrated, major key tree is formed, wherein the major key information is corresponding with the information；
Data memory module 14 is suitable for storing the full dose tree to oracle database, and by the master
Key tree is stored as inverted index.
The present embodiment also provides a kind of computer equipment, can such as execute the smart phone, tablet computer, notebook of program
Computer, desktop computer, rack-mount server, blade server, tower server or Cabinet-type server are (including independent
Server cluster composed by server or multiple servers) etc..The computer equipment 20 of the present embodiment includes at least but not
It is limited to: memory 21, the processor 22 of connection can be in communication with each other by system bus, as shown in Figure 3.It is pointed out that Fig. 3
The computer equipment 20 with component 21-22 is illustrated only, it should be understood that being not required for implementing all groups shown
Part, the implementation that can be substituted is more or less component.
In the present embodiment, memory 21 (i.e. readable storage medium storing program for executing) includes flash memory, hard disk, multimedia card, card-type memory
(for example, SD or DX memory etc.), random access storage device (RAM), static random-access memory (SRAM), read-only memory
(ROM), electrically erasable programmable read-only memory (EEPROM), programmable read only memory (PROM), magnetic storage, magnetic
Disk, CD etc..In some embodiments, memory 21 can be the internal storage unit of computer equipment 20, such as the calculating
The hard disk or memory of machine equipment 20.In further embodiments, memory 21 is also possible to the external storage of computer equipment 20
The plug-in type hard disk being equipped in equipment, such as the computer equipment 20, intelligent memory card (Smart Media Card, SMC), peace
Digital (Secure Digital, SD) card, flash card (Flash Card) etc..Certainly, memory 21 can also both include meter
The internal storage unit for calculating machine equipment 20 also includes its External memory equipment.In the present embodiment, memory 21 is commonly used in storage
It is installed on the operating system and types of applications software of computer equipment 20, such as the high-speed data processing device 10 of embodiment one
Program code etc..In addition, memory 21 can be also used for temporarily storing the Various types of data that has exported or will export.
Processor 22 can be in some embodiments central processing unit (Central Processing Unit, CPU),
Controller, microcontroller, microprocessor or other data processing chips.The processor 22 is commonly used in control computer equipment
20 overall operation.In the present embodiment, program code or processing data of the processor 22 for being stored in run memory 21,
Such as operation high-speed data processing device 10, to realize the automatic update method of embodiment one.
The present embodiment also provides a kind of computer readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory
(for example, SD or DX memory etc.), random access storage device (RAM), static random-access memory (SRAM), read-only memory
(ROM), electrically erasable programmable read-only memory (EEPROM), programmable read only memory (PROM), magnetic storage, magnetic
Disk, CD, server, App are stored thereon with computer program, phase are realized when program is executed by processor using store etc.
Answer function.The computer readable storage medium of the present embodiment is for storing automatic renewing device 10, realization when being executed by processor
The automatic update method of embodiment one.
Embodiment two
Referring to Fig. 4, the high-speed data querying method of the present embodiment is based on embodiment one, comprising the following steps:
S1: instructing in response to data query, obtains the major key tree saved with the examination of inverted index side, the major key tree
Shape structure includes multiple nodes, the corresponding one or more tables of data with identical major key of each node.
The present embodiment is directed to the data query operation process in database.When needs inquire a certain item from database
When data, the present invention does not access oracle database directly, but goes in ES index database to search between characterization tables of data first
The tree of index relative e.g. contains the major key tree (abbreviation major key tree) of major key information.The tree
Many levels are divided into according to the index relative between tables of data, include multiple nodes, each node corresponding one in each level
A or multiple tables of data.
Assuming that data related with client are divided into following several tables of data: Ke Huxin in the database on certain bank backstage
Cease table (A table), customer information supplementary table (A1 table), client's application record table (C table), customer account information table (B table), account volume
Spend control table (B1 table) and account bank card table (D table).Wherein client information table (A table) and customer information supplementary table (A1 table)
Major key custNo having the same, customer account information table (B table) and the control table of account amount (B1 table) major key having the same
AccountNo, client's application record table (C table) have major key applyNo, and account bank card table (D table) has major key cardNo.
The present invention according to logic index relative to above-mentioned several tables of data re-started divide and by tree into
Row is shown.Firstly, to describe the number comprising client, name, identification card number, address etc. most basic for client information table (A table)
Information, therefore as father's layer.In addition there is the also customer information supplementary table of identical major key with client information table (A table)
(A1 table), as the further supplement to customer information in client information table, such as the Income situation of client.In order to simplify knot
Table with identical major key is classified as same node by structure, the present invention, is denoted as node A.There is index relative with client information table
Table includes client's application record table (C table) and customer account information table (B table), therefore using this two tables as the first sublayer.By
Do not have incidence relation each other in client's application record table (C table) and customer account information table (B table), therefore respectively will be objective
Family application record table (C table) is denoted as node C, and customer account information table (B table) is denoted as node B.In addition, account amount control table
Also there are direct index relative, and account amount control table (B1 table) and client between (B1 table) and client information table (A table)
There is same major key accountNo between account information table (B table), therefore for simplicity by account amount control table (B1
Table) it is also included into node B.Next, having the table of index relative with customer account information table (B table) is account bank card table (D
Table), therefore account bank card table (D table) is incorporated into as the second sublayer, and be labeled as node D.
S2: the field that will be inquired inputs the major key tree, obtains node corresponding with the field.
On the basis of obtaining major key tree, the data epitope where the data to be searched need to be further determined that
It sets.The major key information of each tables of data is contained in major key tree, ideally, when the data field to be inquired is major key,
Nodal information corresponding with the major key then can be directly got from major key tree, then only needed from the data under the node
The data that needs are inquired in table are inquired without the tables of data for traversing all.
When the data field to be inquired is non-primary key, then need first to search from concordance list opposite with the non-master key field
Then the major key information answered searches corresponding node further according to the major key information found from major key tree.
S3: target data is inquired from tables of data corresponding with the node.
Include one or more tables of data under each node of the invention, has between these tables of data and be parallel to each other
Index relative, and major key having the same.On the basis of node has been determined, the present invention is only needed from being under the jurisdiction of the node
Target data is inquired in tables of data, it is no longer necessary to all tables of data in convenience data library, therefore it is remarkably improved data
Search efficiency.
S4: the target data is exported by output mode.
May be comprising multinomial content using the data that a certain field is found as target, such as searching customer number is 001
Data, the name of the client, coding of accounts, name on account, application numbers, loan code etc. letter may be inquired simultaneously
Breath, therefore need exist for being defined output format.Content to be shown can be needed in the present embodiment taking human as setting, such as
The data that customer number is 001 are inquired, it is specified that several information such as display customer name, name on account, loan code.It can also select
Default form is selected, such as when not set need particular content to be shown, then defaults all data informations that display inquires.
Please continue to refer to Fig. 5, the high-speed data inquiry unit 30 of the present embodiment is based on embodiment one, to realize reality
Apply the high-speed data querying method of example two comprising each program module function:
Major key tree obtains module 31, is adapted to respond to instruct in data query, obtains the master stored in a manner of inverted index
Key tree, the major key tree include multiple nodes, the corresponding one or more with identical major key of each node
Tables of data；
Node obtains module 32, and the field suitable for that will inquire inputs the major key tree, obtains and the word
The corresponding node of section；
Data inquiry module 33, suitable for inquiring target data from tables of data corresponding with the node；
Data outputting module 34 is suitable for exporting the target data by output mode.
In conclusion high-speed data processing provided by the invention and querying method, device, computer equipment and storage are situated between
Matter, has the function of the data storage and query of high-speed and high-efficiency, and the present invention utilizes high speed inverted index frame ES component, significant increase
The performance of data storage and search, reduces the access pressure of database, reduces when developer writes program to data
The possible hidden danger of performance influences.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
Any process or the method description described in other ways in flow chart or herein is construed as, and expression includes
It is one or more for realizing specific logical function or process the step of executable instruction code module, segment or portion
Point, and the range of the preferred embodiment of the present invention includes other realization, wherein can not press shown or discussed suitable
Sequence, including according to related function by it is basic simultaneously in the way of or in the opposite order, Lai Zhihang function, this should be of the invention
Embodiment person of ordinary skill in the field understood.
Those skilled in the art are appreciated that all or part of step for realizing that above-described embodiment method carries
It suddenly is that relevant hardware can be instructed to complete by program, the program can store in a kind of computer-readable medium
In, which when being executed, includes the steps that one or a combination set of embodiment of the method.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means particular features, structures, materials, or characteristics described in conjunction with this embodiment or example
It is included at least one embodiment or example of the invention.In the present specification, schematic expression of the above terms are different
Surely identical embodiment or example is referred to.Moreover, particular features, structures, materials, or characteristics described can be any
It can be combined in any suitable manner in one or more embodiment or examples.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases
The former is more preferably embodiment.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills
Art field, is included within the scope of the present invention.
Claims (10)
1. a kind of date storage method, comprising the following steps:
In response to instruction data storage, the abstract tree for characterizing the index relative between multiple tables of data, the pumping are generated
As tree includes multiple nodes, each node includes information and map information, and the information characterizes the section
The type for the data to be stored on point, the map information are used to characterize the other nodes for having mapping relations with present node；
Data to be stored is obtained, is stored the data to be stored to the abstract tree according to the type of the data to be stored
In corresponding node, full dose tree is formed；
The major key information and map information of each node are extracted from the full dose tree, form major key tree,
Described in major key information it is corresponding with the information；
The full dose tree is stored to database, and by the major key tree and is stored as inverted index.
2. date storage method according to claim 1, which is characterized in that the acquisition data to be stored, according to described
The type of data to be stored stores the data to be stored onto the corresponding node of the abstract tree, forms full dose tree
The step of include:
Each of abstract tree node is disassembled, tables of data to be updated is obtained, according to the type of the data to be stored
Increase data into every tables of data to be updated.
3. date storage method according to claim 1, which is characterized in that also preserved in the major key tree non-
Major key index, for characterizing the non-primary key in the multiple tables of data and the corresponding relationship between major key.
4. a kind of data query method, which is characterized in that be based on claim 1-3 any data storage method, including following step
It is rapid:
It is instructed in response to data query, obtains the major key tree saved in a manner of inverted index, the major key tree
Including multiple nodes, the corresponding one or more tables of data with identical major key of each node；
The field that will be inquired inputs the major key tree, obtains node corresponding with the field；
Target data is inquired from tables of data corresponding with the node；
The target data is exported by output mode.
5. data query method according to claim 4, which is characterized in that the field that will be inquired inputs the master
Key tree, the step of obtaining node corresponding with the field, any one of include the following steps:
The field is major key, according to major key query node；
The field is non-primary key, inquires corresponding with non-primary key major key from concordance list, and according to major key from major key tree
Middle query node.
6. data query method according to claim 4, which is characterized in that it is described by output mode output data include press
Designated mode output data presses default mode output data.
7. a kind of data-storage system characterized by comprising
Abstract tree generation module, it is described suitable for generating the abstract tree for characterizing the index relative between multiple tables of data
Abstract tree includes multiple nodes, and each node includes information and map information, described in the information characterization
The type for the data to be stored on node, the map information are used to characterize the other sections for having mapping relations with present node
Point；
Full dose tree generation module is suitable for obtaining data to be stored, according to the type of the data to be stored by described wait store
In data storage to the corresponding node of the abstract tree, form full dose tree；
Major key tree generation module, major key information and mapping letter suitable for extracting each node from the full dose tree
Breath forms major key tree, wherein the major key information is corresponding with the information；
Data memory module is suitable for storing the full dose tree to oracle database, and the major key is tree-shaped
Structure is stored as inverted index.
8. a kind of data query system, which is characterized in that be based on data-storage system as claimed in claim 7, comprising:
Major key tree obtains module, is adapted to respond to instruct in data query, it is tree-shaped to obtain the major key stored in a manner of inverted index
Structure, the major key tree include multiple nodes, the corresponding one or more tables of data with identical major key of each node；
Node obtains module, and the field suitable for that will inquire inputs the major key tree, obtains opposite with the field
The node answered；
Data inquiry module, suitable for inquiring target data from tables of data corresponding with the node；
Data outputting module is suitable for exporting the target data by output mode.
9. a kind of computer equipment, can run on a memory and on a processor including memory, processor and storage
Computer program, which is characterized in that the processor realizes any one of claim 1 to 6 institute when executing the computer program
The step of stating method.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program
The step of any one of claim 1 to 6 the method is realized when being executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910299922.XA CN110134681B (en) | 2019-04-15 | 2019-04-15 | Data storage and query method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910299922.XA CN110134681B (en) | 2019-04-15 | 2019-04-15 | Data storage and query method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110134681A true CN110134681A (en) | 2019-08-16 |
| CN110134681B CN110134681B (en) | 2023-10-13 |
Family
ID=67569712
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910299922.XA Active CN110134681B (en) | 2019-04-15 | 2019-04-15 | Data storage and query method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110134681B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110990403A (en) * | 2019-11-26 | 2020-04-10 | 深圳壹账通智能科技有限公司 | Business data storage method, system, computer equipment and storage medium |
| CN111475511A (en) * | 2020-04-03 | 2020-07-31 | 弦子科技(北京)有限公司 | Data storage method, data access method, data storage device, data access device and data access equipment based on tree structure |
| CN112583761A (en) * | 2019-09-27 | 2021-03-30 | 奇安信科技集团股份有限公司 | Management method and device of security entity, computer equipment and storage medium |
| CN114373173A (en) * | 2022-01-12 | 2022-04-19 | 中国平安人寿保险股份有限公司 | Data processing method and device, terminal equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6266660B1 (en) * | 1998-07-31 | 2001-07-24 | Unisys Corporation | Secondary index search |
| CN102646111A (en) * | 2012-02-16 | 2012-08-22 | 中国测绘科学研究院 | Knowledge base-based fast construction method of common correlation information query tree |
| CN103412897A (en) * | 2013-07-25 | 2013-11-27 | 中国科学院软件研究所 | Parallel data processing method based on distributed structure |
| CN109388636A (en) * | 2017-08-11 | 2019-02-26 | 中国移动通信集团重庆有限公司 | Business datum is inserted into database method, apparatus, computer equipment and storage medium |
| CN109558128A (en) * | 2018-10-25 | 2019-04-02 | 平安科技（深圳）有限公司 | Json data analysis method, device and computer readable storage medium |
-
2019
- 2019-04-15 CN CN201910299922.XA patent/CN110134681B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6266660B1 (en) * | 1998-07-31 | 2001-07-24 | Unisys Corporation | Secondary index search |
| CN102646111A (en) * | 2012-02-16 | 2012-08-22 | 中国测绘科学研究院 | Knowledge base-based fast construction method of common correlation information query tree |
| CN103412897A (en) * | 2013-07-25 | 2013-11-27 | 中国科学院软件研究所 | Parallel data processing method based on distributed structure |
| CN109388636A (en) * | 2017-08-11 | 2019-02-26 | 中国移动通信集团重庆有限公司 | Business datum is inserted into database method, apparatus, computer equipment and storage medium |
| CN109558128A (en) * | 2018-10-25 | 2019-04-02 | 平安科技（深圳）有限公司 | Json data analysis method, device and computer readable storage medium |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112583761A (en) * | 2019-09-27 | 2021-03-30 | 奇安信科技集团股份有限公司 | Management method and device of security entity, computer equipment and storage medium |
| CN112583761B (en) * | 2019-09-27 | 2023-03-28 | 奇安信科技集团股份有限公司 | Management method and device of security entity, computer equipment and storage medium |
| CN110990403A (en) * | 2019-11-26 | 2020-04-10 | 深圳壹账通智能科技有限公司 | Business data storage method, system, computer equipment and storage medium |
| CN111475511A (en) * | 2020-04-03 | 2020-07-31 | 弦子科技(北京)有限公司 | Data storage method, data access method, data storage device, data access device and data access equipment based on tree structure |
| CN114373173A (en) * | 2022-01-12 | 2022-04-19 | 中国平安人寿保险股份有限公司 | Data processing method and device, terminal equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110134681B (en) | 2023-10-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110134681A (en) | Data storage and querying method, device, computer equipment and storage medium | |
| US11068441B2 (en) | Caseless file lookup in a distributed file system | |
| US8200684B2 (en) | Method and system for dynamic templatized query language in software | |
| CN101405728B (en) | Relational database architecture with dynamic load capability | |
| CN103577440B (en) | A kind of data processing method and device in non-relational database | |
| CN100468402C (en) | Sort data storage and split catalog inquiry method based on catalog tree | |
| US20200387480A1 (en) | Path resolver for client access to distributed file systems | |
| CN109815228A (en) | Creation method, device, computer equipment and the readable storage medium storing program for executing of database table | |
| CN112559481A (en) | Data storage method and device based on distributed system and relational database | |
| CN102243660A (en) | Data access method and device | |
| US10262055B2 (en) | Selection of data storage settings for an application | |
| CN109614411B (en) | Data storage method, device and storage medium | |
| CN112835638A (en) | Configuration information management method and device based on embedded application program | |
| CN110955658B (en) | Data organization and storage method based on Java intelligent contract | |
| CN110020001A (en) | Storage, querying method and the corresponding equipment of string data | |
| CN111125216A (en) | Method and device for importing data into Phoenix | |
| CN116414935A (en) | Method for distributed Search space vector data based on Elastic Search | |
| CN112527792A (en) | Data storage method, device, equipment and storage medium | |
| CN111259003B (en) | Database establishment method and device | |
| CN115543993A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN112861495A (en) | Method for generating impala SQL statement based on Excel template file | |
| US10380090B1 (en) | Nested object serialization and deserialization | |
| CN110019448A (en) | A kind of data interactive method and device | |
| CN115905415A (en) | Database operation method and device, electronic equipment and medium | |
| CN117235203A (en) | Data storage method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |