CN107357794A - Optimize the method and apparatus of the data store organisation of key value database - Google Patents
Optimize the method and apparatus of the data store organisation of key value database Download PDFInfo
- Publication number
- CN107357794A CN107357794A CN201610305828.7A CN201610305828A CN107357794A CN 107357794 A CN107357794 A CN 107357794A CN 201610305828 A CN201610305828 A CN 201610305828A CN 107357794 A CN107357794 A CN 107357794A
- Authority
- CN
- China
- Prior art keywords
- prefix
- keyword
- list
- database
- hit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application discloses the method and apparatus of the data store organisation of optimization key value database.One embodiment of methods described includes：The keyword of predetermined number is read from database and each keyword to reading performs analytical procedure up to all keywords in analytical database, obtains prefix list；Obtain the quantity and memory space of the keyword of the prefix in hit prefix list；The quantity and memory space of the keyword of prefix in hit prefix list, optimize the data store organisation of database.The embodiment improves the storage performance of database and accelerates the reading efficiency of database.
Description
Technical field
The application is related to field of computer technology, and in particular to Internet technical field, especially relates to
And the method and apparatus of the data store organisation of optimization key value database.
Background technology
With the complexity more and more higher of Distributed Storage business, in order to avoid single common
It is excessive to enjoy reading pressure on internal memory, shared storage can be set by Hash (hash) scheduling algorithm
It is standby to carry out load balancing, but this also brings the sharp increase of key assignments table quantity.
At present, during key assignments table in monitoring distributed data storage, all keys are generally obtained
Word and the memory space and data structure for showing keyword one by one, or for pass that user specifies
Key word obtains the data structure of memory space and keyword, or should by line real-time monitoring
With read-write keyword, and the service condition such as reading performance of analysis keyword in real time.
However, it is only certain data line in key assignments table to monitor data acquired during key assignments table at present
The information such as the memory space of occupancy or read-write are frequent, can not be deposited according to the data-optimized data of acquisition
Storage.
The content of the invention
The purpose of the application is to propose a kind of data storage knot of improved optimization key value database
The method and apparatus of structure, to solve the technical problem that background section above is mentioned.
In a first aspect, this application provides a kind of data store organisation for optimizing key value database
Method, methods described include：The keyword of predetermined number is read from database and to reading
Each keyword performs analytical procedure up to all keywords in analytical database, described
Analytical procedure includes：The keyword read is analyzed to be segmented, to the participle perform detection
Step, the detecting step include：Last participle is deleted to obtain predicting prefix, identification
Whether the prediction prefix hits the prefix in prefix list, if hit, the prefix is arranged
The number that is hit of prefix in table adds 1, if miss, the prediction prefix is added to
The prefix list, and identify whether the prediction prefix is single participle, if so, then to reading
The next keyword taken performs analytical procedure, if it is not, then to the participle in the prediction prefix
Perform the detecting step；Obtain the quantity for the keyword for hitting the prefix in the prefix list
And memory space；The quantity of the keyword of prefix in the hit prefix list and
Memory space, optimize the data store organisation of database.
In certain embodiments, it is described to obtain the keyword for hitting the prefix in the prefix list
Quantity and memory space include：Before number is hit in the prefix list less than 2 times
Keyword corresponding to sewing re-executes analytical procedure, the prefix list after being updated；Obtain life
Described in update after prefix list in prefix keyword quantity and memory space；And
The quantity of the keyword of the prefix in the hit prefix list and storage are empty
Between, optimizing the data store organisation of database includes：Arranged according to the prefix after the hit renewal
The quantity and memory space of the keyword of prefix in table, optimize the data store organisation of database.
In certain embodiments, the keyword of predetermined number is read from database and to reading
Each keyword performs analytical procedure until all keywords in analytical database include：
The keyword of predetermined number is read from database and analysis is performed to each keyword of reading
Step, at the end of analytical procedure is performed to the keyword of predetermined number, by the prefix list
Middle prefix of the hit mark less than 2 times is moved in individual Hash table as individual prefix until having divided
Analyse all keywords in database；It is described to obtain the pass for hitting the prefix in the prefix list
The quantity and memory space of key word include：After all keywords in identifying the database,
After pair key corresponding with the individual prefix re-executes the analytical procedure to be updated
The quantity of the keyword of the prefix in the prefix list after the renewal is hit in prefix list, acquisition
And memory space；And the keyword of the prefix in the hit prefix list
Quantity and memory space, optimizing the data store organisation of database includes：According to hit
The quantity and memory space of the keyword of the prefix in prefix list after renewal, optimize database
Data store organisation.
In certain embodiments, the database includes example and/or memory image file.
In certain embodiments, the keyword that the analysis is read is included with obtaining participle：
In certain embodiments, the pass of the prefix in the hit prefix list
The quantity and memory space of key word, the data store organisation of the optimization database include：According to
The quantity and memory space of the keyword of prefix in the hit prefix list, to user
The option of optimization data store organisation is presented；In response to receiving selection of the user to the option,
Optimize the data store organisation of the database.
Second aspect, this application provides a kind of data store organisation for optimizing key value database
Device, described device include：Analytic unit, for reading the pass of predetermined number from database
Key word simultaneously performs analytical procedure until the institute in analytical database to each keyword of reading
There is keyword, the analytical procedure includes：The keyword read is analyzed to be segmented, to institute
Participle perform detection step is stated, the detecting step includes：Last participle is deleted to obtain
Prefix is predicted, identifies whether the prediction prefix hits the prefix in prefix list, if hit,
The number that is hit of the prefix in the prefix list is then added 1, if miss, by described in
Prediction prefix is added to the prefix list, and identifies whether the prediction prefix is single participle,
If so, analytical procedure then is performed to next keyword of reading, if it is not, then to the prediction
Participle in prefix performs the detecting step；Acquiring unit, the prefix is hit for obtaining
The quantity and memory space of the keyword of prefix in list；Optimize unit, for according to
The quantity and memory space of the keyword of the prefix in the prefix list are hit, optimizes database
Data store organisation.
In certain embodiments, the acquiring unit is further used for：To in the prefix list
Keyword corresponding to being hit prefix of the number less than 2 times re-executes analytical procedure, obtains more
Prefix list after new；Obtain the keyword of the prefix in the prefix list after hitting the renewal
Quantity and memory space；And the optimization unit is further used for：According to hit more
The quantity and memory space of the keyword of the prefix in prefix list after new, optimize database
Data store organisation.
In certain embodiments, the analytic unit is further used for：Read from database pre-
If the keyword of quantity simultaneously performs analytical procedure to each keyword of reading, to present count
At the end of the keyword of amount performs analytical procedure, hit mark in the prefix list is less than 2
Secondary prefix is moved in individual Hash table as individual prefix until all in analytical database
Keyword；The acquiring unit is further used for：Pair key weight corresponding with the individual prefix
The analytical procedure is newly performed with the prefix list after being updated, is obtained after hitting the renewal
Prefix list in prefix keyword quantity and memory space；And the optimization unit
It is further used for：The number of the keyword of prefix in the prefix list after the hit renewal
Amount and memory space, optimize the data store organisation of database.
In certain embodiments, in some optional implementations of the present embodiment, the analysis
The keyword that unit is used to read predetermined number from the database includes：The analytic unit
For reading the keyword of predetermined number from example and/or memory image file.
In certain embodiments, the analytic unit is used to analyze the keyword read to be divided
Word includes：The analytic unit be used for according to it is following it is one or more analysis read keywords with
Segmented：Separator, capital and small letter change and digital alphabet change.
In certain embodiments, the optimization module is further used for：According to the hit
The quantity and memory space of the keyword of prefix in prefix list, optimization data are presented to user
The option of storage organization；In response to receiving selection of the user to the option, optimize the data
The data store organisation in storehouse.
The method and apparatus of the data store organisation for the optimization key value database that the application provides, it is first
The keyword of predetermined number is first read from database and each keyword to reading is performed and divided
Step is analysed until all keywords in analytical database, the analytical procedure include：Analysis
The keyword of reading is to be segmented, to the participle perform detection step, the detecting step
Including：Last participle is deleted to obtain predicting prefix, identifies whether the prediction prefix orders
Prefix in middle prefix list, if hit, by being hit for the prefix in the prefix list
Number adds 1, if miss, the prediction prefix is added into the prefix list, and know
Whether not described prediction prefix is single participle, if so, then being held to next keyword of reading
Row analytical procedure, if it is not, then performing the detecting step to the participle in the prediction prefix；
The quantity and memory space for the keyword for hitting the prefix in the prefix list are obtained afterwards；Most
The quantity and memory space of the keyword of prefix in the hit prefix list afterwards,
Optimize the data store organisation of key value database, so as to improve the storage performance of database and add
The speed reading efficiency of database.
Brief description of the drawings
Retouched by reading with reference to the detailed of being made to non-limiting example of being made of the following drawings
State, other features, objects and advantages will become more apparent upon：
Fig. 1 is that the application can apply to exemplary system architecture figure therein；
Fig. 2 is one according to the method for the data store organisation of the optimization key value database of the application
The flow chart of individual embodiment；
Fig. 3 be according to the method for the data store organisation of the optimization key value database of the application again
The flow chart of one embodiment；
Fig. 4 is according to the method for the data store organisation of the optimization key value database of the application
The flow chart of three embodiments；
Fig. 5 is one according to the device of the data store organisation of the optimization key value database of the application
The structural representation of individual embodiment；
Fig. 6 is adapted for for realizing the terminal device of the embodiment of the present application or the computer of server
The structural representation of system.
Embodiment
The application is described in further detail with reference to the accompanying drawings and examples.It is appreciated that
, specific embodiment described herein is used only for explaining related invention, rather than to the hair
Bright restriction.It also should be noted that for the ease of description, illustrate only in accompanying drawing with
About the related part of invention.
It should be noted that in the case where not conflicting, embodiment and embodiment in the application
In feature can be mutually combined.Describe this in detail below with reference to the accompanying drawings and in conjunction with the embodiments
Application.
Fig. 1 is shown can be using the data store organisation of the optimization key value database of the application
The example system of the embodiment of the device of the data store organisation of method or optimization key value database
Framework 100.
As shown in figure 1, system architecture 100 can include terminal device 101,102,103,
Network 104 and server 105,106.Network 104 is in terminal device 101,102,103
The medium of communication link is provided between server 105.Network 104 can include various connections
Type, such as wired, wireless communication link or fiber optic cables etc..
User 110 can pass through network 104 and service with using terminal equipment 101,102,103
Device 105,106 interacts, to receive or send message etc..Terminal device 101,102,103
On various telecommunication customer end applications can be installed, such as the application of data base administration class, shopping class
Using, searching class application, JICQ, mailbox client, social platform software etc..
Terminal device 101,102,103 can be the various electronic equipments for having display screen, bag
Include but be not limited to smart mobile phone, tablet personal computer, E-book reader, MP3 player (Moving
Picture Experts Group Audio Layer III, dynamic image expert's compression standard audio layer
Face 3), (Moving Picture Experts Group Audio Layer IV, dynamic image are special by MP4
Family's compression standard audio aspect 4) player, pocket computer on knee and desktop computer etc..
Server 105,106 can be to provide the server of various database services, such as to end
End equipment 101,102,103 provides the background server supported.Background server can dock
The data such as the request received are carried out the processing such as analyzing, and result is fed back into terminal device.
It should be noted that the data for the optimization key value database that the embodiment of the present application is provided are deposited
The method of storage structure is typically performed by terminal device 101,102,103 or server 105,106,
Correspondingly, the device for optimizing the data store organisation of key value database is generally positioned at terminal device
101st, 102,103 or server 105,106 in.
It should be understood that the number of the terminal device, network and server in Fig. 1 is only to illustrate
Property.According to needs are realized, can have any number of terminal device, network and server.
It should be appreciated that the key value database in the application refers to key-value (Key-Value) data
Storehouse, the Key-Value databases in the prior art and in the technology of future development can be included.
Below by taking the Redis databases in Key-Value databases as an example, illustrate to optimize database
The method and apparatus of data store organisation.
Fig. 2 is refer to, Fig. 2 is shown to be deposited according to the data of the optimization key value database of the application
The flow 200 of one embodiment of the method for storage structure.The data of the optimization key value database are deposited
The flow of the method for storage structure, by taking Redis databases as an example, specifically include following steps：
In step 210, the keyword (Key) of predetermined number is read from Redis databases
And analytical procedure is performed until having analyzed in Redis databases to each keyword of reading
All keywords.
In the present embodiment, analytical procedure includes：The keyword read is analyzed to be segmented, it is right
Perform detection step is segmented, detecting step includes：Before last participle is deleted to obtain prediction
Sew, whether identification prediction prefix hits the prefix in prefix list, if hit, prefix is arranged
The number that is hit of prefix in table adds 1, if miss, prediction prefix is added into prefix
List, and whether identification prediction prefix is single participle, if so, then to next pass of reading
Key word performs analytical procedure, if it is not, then to the participle perform detection step in prediction prefix.
When reading the keyword of predetermined number from Redis databases, in order to prevent what is read
Redis data volumes are excessive, cause application tool internal memory to exhaust, can every time from Redis examples and
/ or Redis memory image files in read predetermined number keyword.
, can be according to following one or more points when the keyword that analysis is read is to obtain participle
The keyword read is analysed to be segmented：Separator, capital and small letter change and digital alphabet change.
For example, it can be segmented according to separator analysis keyword Customer-username
" Customer " and " username "；Can be according to capital and small letter mutation analysis keyword
CustomerSaveKeyUsername, segmented：“Customer”、“Save”、“Key”
" Username "；Participle keyword " abc1231 " can be changed according to digital alphabet, obtained
Segment " abc ", " 123 " and " 1 ".Herein, it is also possible to which the keyword of reading is the overall situation
User identity proves (UID) class keyword, then global user identity can be proved into class is crucial
Word is as a kind of keyword individually recorded.
It should be appreciated that above-mentioned detecting step prefix list when performing first time is sky, now in advance
The prefix surveyed in the miss prefix list of prefix, prediction prefix is added to prefix list afterwards,
Whether then follow-up participle can hit prefix in perform detection step with identification prediction prefix
Prefix in list.
In a step 220, obtain the quantity of the keyword of the prefix in hit prefix list and deposit
Store up space.
In the present embodiment, based on all keys identified in step 210 in Redis databases
The prefix list that word obtains, Key all in Redis is traveled through, obtained in hit prefix list
Prefix Key and Key value (Value), and record each Key and Value and take
Memory headroom.
, can in the quantity and memory space of the keyword of the prefix in obtaining hit prefix list
To obtain the quantity and memory space of the keyword for hitting all prefixes in prefix list, so as to
Improve the comprehensive of the data of acquisition；The number of the prefix in hit prefix list can also be obtained
More than the quantity and memory space of the keyword of pre-determined number (such as 2 times), before hit
Sew quantity of the number not less than the keyword of pre-determined number (such as 2 times) of the prefix in list
And memory space, this partial data individually can be obtained or abandon, so as to improve the effect of acquisition
Rate.
In step 230, according to hit prefix list in prefix keyword quantity and deposit
Space is stored up, optimizes the data store organisation of Redis databases.
In the present embodiment, based on the prefix in the hit prefix list obtained in a step 220
Keyword quantity and memory space, optimize Redis databases data store organisation.Show
Example property, can according to the quantity of the keyword of the prefix in the hit prefix list of acquisition with deposit
Space is stored up, the load pressure of caching system is prompted, to carry out load balancing；Can also basis
The quantity and memory space of the keyword of prefix in the hit prefix list of acquisition, optimize data
Storage, such as delete junk data or remove duplicate data etc..
In the quantity and memory space of the keyword of the prefix in hit prefix list, optimization
, can be in the key of the prefix in hitting prefix list during the data store organisation of Redis databases
When the quantity and memory space of word meet the preparatory condition of preset rules, complete to limit in preset rules
Fixed operation, can also so as to be optimized to the data store organisation in Redis databases
The quantity and memory space of the keyword of prefix in above-mentioned hit prefix list meet default
During condition, the option of optimization data store organisation is presented to user, in response to receiving user to choosing
The selection of item, optimize the data store organisation of Redis databases.
The above embodiments of the present application optimize the data store organisation of Redis databases, improve
The storage performance of Redis databases and the reading efficiency for accelerating Redis databases.
With further reference to Fig. 3, it illustrates the data of the optimization key value database according to the application
The flow chart 300 of another embodiment of the method for storage organization.The optimization key value database
The flow 300 of the method for data store organisation, still by taking Redis databases as an example, specifically include with
Lower step：
In the step 310, the keyword of predetermined number is read from Redis databases and to reading
Each keyword taken performs analytical procedure until the institute analyzed in Redis databases is relevant
Key word.
In the present embodiment, analytical procedure includes：The keyword read is analyzed to be segmented, it is right
Perform detection step is segmented, detecting step includes：Before last participle is deleted to obtain prediction
Sew, whether identification prediction prefix hits the prefix in prefix list, if hit, prefix is arranged
The number that is hit of prefix in table adds 1, if miss, prediction prefix is added into prefix
List, and whether identification prediction prefix is single participle, if so, then to next pass of reading
Key word performs analytical procedure, if it is not, then to the participle perform detection step in prediction prefix.
It should be appreciated that the step 310 in the present embodiment is identical with the step 210 in Fig. 2, because
This operation being described in step 210 and feature are equally applicable to step 310, no longer superfluous herein
State.
In step 320, to being hit in prefix list corresponding to prefix of the number less than 2 times
Keyword re-executes analytical procedure, the prefix list after being updated.
In the present embodiment, in order to improve identification Redis databases in all keywords standard
Exactness, can be to being hit keyword weight corresponding to prefix of the number less than 2 times in prefix list
It is new to perform analytical procedure, with the prefix list after being updated.
In a step 330, the keyword of the prefix in the prefix list after hit renewal is obtained
Quantity and memory space.
In the present embodiment, based on the prefix pair for being less than 2 times in step 320 to being hit number
The keyword answered re-executes the prefix list after the renewal that analytical procedure obtains, and travels through Redis
In all keyword Key, obtain the Key of prefix in the prefix list after hit renewal with
And Key value Value, and record the memory headroom that each Key and Value take.
In step 340, the keyword of the prefix in prefix list after being updated according to hit
Quantity and memory space, optimize the data store organisation of Redis databases.
In the present embodiment, based on the prefix list after the hit renewal obtained in a step 330
In prefix keyword quantity and memory space, the data of Redis databases can be optimized
Storage organization.It is exemplary, can be according to before in the prefix list after the hit renewal of acquisition
The quantity and memory space for the keyword sewed, the load pressure of caching system is prompted, to carry out
Load balancing；The key of the prefix in prefix list after can also being updated according to the hit of acquisition
The quantity and memory space of word, optimize data storage, such as delete junk data or remove and repeat
Data etc..
From figure 3, it can be seen that compared with embodiment corresponding to Fig. 2, it is excellent in the present embodiment
The flow 300 for changing the method for caching Redis data store organisation is highlighted to being hit number
Keyword corresponding to prefix less than 2 times re-executes analytical procedure with the prefix after being updated
The step of list.Thus, after the scheme of the present embodiment description can improve identification hit renewal
The quantity of the keyword of prefix in prefix list and the precision of memory space, to optimize Redis
The data store organisation of database provides more accurately data.
With further reference to Fig. 4, it illustrates the data of the optimization key value database according to the application
The flow chart 400 of 3rd embodiment of the method for storage organization.The optimization key value database
The flow 400 of the method for data store organisation, continues by taking Redis databases as an example, including following
Step：
In step 410, the keyword of predetermined number is read from Redis databases and to reading
Each keyword taken performs analytical procedure.
In the present embodiment, analytical procedure includes：The keyword read is analyzed to be segmented, it is right
Perform detection step is segmented, detecting step includes：Before last participle is deleted to obtain prediction
Sew, whether identification prediction prefix hits the prefix in prefix list, if hit, prefix is arranged
The number that is hit of prefix in table adds 1, if miss, prediction prefix is added into prefix
List, and whether identification prediction prefix is single participle, if so, then to next pass of reading
Key word performs analytical procedure, if it is not, then to the participle perform detection step in prediction prefix.
When reading the keyword of predetermined number from Redis databases, in order to prevent what is read
Redis data volumes are excessive, cause application tool internal memory to exhaust, can every time from Redis examples and
/ or Redis memory image files in read predetermined number keyword.
, can be according to following one or more points when the keyword that analysis is read is to obtain participle
The keyword read is analysed to be segmented：Separator, capital and small letter change and digital alphabet change.
For example, it is " Customer " that keyword Customer-username can segment according to separator
" username "；Keyword CustomerSaveKeyUsername can become according to capital and small letter
Changing participle is：" Customer ", " Save ", " Key " and " Username "；Keyword
It is " abc ", " 123 " and " 1 " that " abc1231 " can change participle according to digital alphabet.
Here, global user identity proves (UID) class, and a kind of keyword (Key) can be used as single
Solely record.
It should be appreciated that above-mentioned detecting step prefix list when performing first time is sky, now in advance
The prefix surveyed in the miss prefix list of prefix, prediction prefix is added to prefix list afterwards,
Whether then follow-up participle can hit prefix in perform detection step with identification prediction prefix
Prefix in list.
At step 420, will at the end of analytical procedure is performed to the keyword of predetermined number
Prefix of the hit mark less than 2 times is moved in individual Hash table as individual prefix in prefix list
Until all keywords in Redis databases are analyzed.
In the present embodiment, at the end of the keyword recognition of each group of predetermined number, can incite somebody to action
Hit mark is moved in individual Hash table less than the prefix of 2 times as individual prefix, so as to
After all keywords in Redis databases complete identification, step 430 is performed to be updated
Prefix list afterwards.The random character that can so avoid some programs from directly being generated by algorithm
String is added in prefix list, causes the long detection for influenceing detecting step of queue in prefix list
Efficiency.
In step 430, a pair key corresponding with individual prefix re-executes analytical procedure to obtain
Prefix list after to renewal.
In the present embodiment, after all keywords in Redis databases complete identification,
After the keyword in individual Hash table can be taken out one by one and perform analytical procedure to obtain renewal
Prefix list.Herein, can also be according to length one by one by solely in order to improve recognition efficiency
Keyword in vertical Hash table takes out and performs analytical procedure.
In step 440, the keyword of the prefix in the prefix list after hit renewal is obtained
Quantity and memory space.
In the present embodiment, based on the prefix list after the renewal obtained in step 430, traversal
All keyword Key in Redis, obtain hit prefix list in prefix Key and
Key value Value, and record the memory headroom that each Key and Value take.
In step 450, the keyword of the prefix in prefix list after being updated according to hit
Quantity and memory space, optimize the data store organisation of Redis databases.
In the present embodiment, based on the prefix list after the hit renewal obtained in step 440
In prefix keyword quantity and memory space, optimize Redis databases data storage
Structure.It is exemplary, can be according to the prefix in the prefix list after the hit renewal of acquisition
The quantity and memory space of keyword, the load pressure of caching system is prompted, to be loaded
It is balanced；The keyword of the prefix in prefix list after can also being updated according to the hit of acquisition
Quantity and memory space, optimize data storage, such as delete junk data or remove duplicate data
Deng.
Figure 4, it is seen that compared with embodiment corresponding to Fig. 2, it is excellent in the present embodiment
The flow 400 for changing the method for caching Redis data store organisation highlights pair and individual prefix
The step of corresponding key re-executes analytical procedure with prefix list after being updated.Thus,
The scheme of the present embodiment description can improve the prefix in the prefix list after identification hit renewal
The efficiency of keyword, provided more accurately to optimize the data store organisation of Redis databases
Data.
With further reference to Fig. 5, as the realization to method shown in above-mentioned each figure, the application provides
A kind of one embodiment of the device for the data store organisation for optimizing key value database.Here still
By taking Redis databases as an example, the device embodiment is corresponding with the embodiment of the method shown in Fig. 2,
The device specifically can apply in various electronic equipments.
As shown in figure 5, the device 500 of the optimization Redis of the present embodiment data store organisation
Including：Analytic unit 510, acquiring unit 520 and optimization unit 530.
Wherein, analytic unit 510, for reading the key of predetermined number from Redis databases
Word simultaneously performs analytical procedure until having analyzed in Redis databases to each keyword of reading
All keywords, analytical procedure includes：The keyword read is analyzed to be segmented, to dividing
Word perform detection step, detecting step include：Last participle is deleted to obtain predicting prefix,
Whether identification prediction prefix hits the prefix in prefix list, if hit, by prefix list
The number that is hit of prefix add 1, if miss, prediction prefix is added to prefix list,
And whether identification prediction prefix is single participle, if so, then being held to next keyword of reading
Row analytical procedure, if it is not, then to the participle perform detection step in prediction prefix.
Acquiring unit 520, for obtaining the quantity for the keyword for hitting the prefix in prefix list
And memory space.
Optimize unit 530, the quantity for the keyword of the prefix in hit prefix list
And memory space, the data store organisation of optimization Redis databases.
In some optional implementations of the present embodiment, acquiring unit is further used for：To preceding
Sew keyword corresponding to prefix of the number less than 2 times is hit in list and re-execute analysis step
Suddenly, the prefix list after being updated；Obtain the prefix in the prefix list after hit renewal
The quantity and memory space of keyword；And optimization unit is further used for：Updated according to hit
The quantity and memory space of the keyword of the prefix in prefix list afterwards, optimize Redis data
The data store organisation in storehouse.
In some optional implementations of the present embodiment, analytic unit is further used for：From
The keyword of predetermined number is read in Redis databases and each keyword to reading performs
Analytical procedure, at the end of analytical procedure is performed to the keyword of predetermined number, by prefix list
Middle prefix of the hit mark less than 2 times is moved in individual Hash table as individual prefix until having divided
Analyse all keywords in Redis databases；Acquiring unit is further used for：Pair with independence before
Sew corresponding key and re-execute analytical procedure with the prefix list after being updated, obtain hit
The quantity and memory space of the keyword of the prefix in prefix list after renewal；And optimization is single
Member is further used for：The quantity of the keyword of the prefix in prefix list after being updated according to hit
And memory space, the data store organisation of optimization Redis databases.
In some optional implementations of the present embodiment, analytic unit is used for from Redis data
The keyword of predetermined number is read in storehouse to be included：Analytic unit be used for from Redis examples and/or
The keyword of predetermined number is read in Redis memory image files.
In some optional implementations of the present embodiment, analytic unit is used to analyze the pass read
Key word is included with obtaining participle：Analytic unit is used for what is read according to following one or more analyses
Keyword is to be segmented：Separator, capital and small letter change and digital alphabet change.
In some optional implementations of the present embodiment, optimization module is further used for：According to
The quantity and memory space of the keyword of the prefix in prefix list are hit, presents and optimizes to user
The option of data store organisation；In response to receiving selection of the user to option, optimize Redis numbers
According to the data store organisation in storehouse.
It will be understood by those skilled in the art that the data store organisation of above-mentioned optimization key value database
Device 500 also include some other known features, such as processor, memory etc., in order to
Embodiment of the disclosure is unnecessarily obscured, these known structures are not shown in Figure 5.
It should be appreciated that all units described in device 500 with reference to figure 2, Fig. 3 and Fig. 4 with retouching
Each step in the method stated is corresponding.Thus, above with respect to the number of optimization key value database
The operation and feature described according to the method for storage organization is equally applicable to device 500 and wherein included
Unit, will not be repeated here.Corresponding units in device 500 can be with the list in server
Member cooperates to realize the scheme of the embodiment of the present application.
Below with reference to Fig. 6, it illustrates suitable for for realizing the terminal device of the embodiment of the present application
Or the structural representation of the computer system 600 of server.
As shown in fig. 6, computer system 600 includes CPU (CPU) 601, its
Can according to the program being stored in read-only storage (ROM) 602 or from storage part 608
The program that is loaded into random access storage device (RAM) 603 and perform various appropriate actions
And processing.In RAM 603, also it is stored with system 600 and operates required various program sums
According to.CPU 601, ROM 602 and RAM 603 are connected with each other by bus 604.Input
/ output (I/O) interface 605 is also connected to bus 604.
I/O interfaces 605 are connected to lower component：Importation 606 including keyboard, mouse etc.；
Including cathode-ray tube (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.
Output par, c 607；Storage part 608 including hard disk etc.；And including such as LAN card,
The communications portion 609 of the NIC of modem etc..Communications portion 609 is via such as
The network of internet performs communication process.Driver 610 is also according to needing to be connected to I/O interfaces
605.Detachable media 611, such as disk, CD, magneto-optic disk, semiconductor memory etc.,
Be arranged on as needed on driver 610, in order to the computer program that reads from it according to
Need to be mounted into storage part 608.
Especially, in accordance with an embodiment of the present disclosure, can be with above with reference to the process of flow chart description
It is implemented as computer software programs.For example, embodiment of the disclosure includes a kind of computer journey
Sequence product, it includes being tangibly embodied in the computer program on machine readable media, computer
Program bag contains the program code for being used for the method shown in execution flow chart.In such embodiments,
The computer program can be downloaded and installed by communications portion 609 from network, and/or from
Detachable media 611 is mounted.
Flow chart and block diagram in accompanying drawing, it is illustrated that according to the system of the various embodiments of the application,
Architectural framework in the cards, function and the operation of method and computer program product.This point
On, each square frame in flow chart or block diagram can represent a unit, program segment or code
A part, the part of unit, program segment or code, which includes, one or more to be used to realize
The executable instruction of defined logic function.It should also be noted that at some as the realization replaced
In, the function of being marked in square frame can also be with different from the order marked in accompanying drawing generation.
For example, two square frames succeedingly represented can essentially perform substantially in parallel, they are sometimes
It can also perform in the opposite order, this is depending on involved function.It is also noted that
The group of each square frame and block diagram in block diagram and/or flow chart and/or the square frame in flow chart
Close, function or the special hardware based system of operation can be realized as defined in execution,
Or it can be realized with the combination of specialized hardware and computer instruction.
Being described in unit involved in the embodiment of the present application can be real by way of software
It is existing, it can also be realized by way of hardware.Described unit can also be arranged on processing
In device, for example, can be described as：A kind of processor includes analytic unit, acquiring unit and excellent
Change unit.Wherein, the title of these units is not formed to the unit in itself under certain conditions
Restriction, for example, analytic unit is also described as " reading predetermined number from database
Keyword and analytical procedure is performed to each keyword of reading until in analytical database
All keywords unit ".
As on the other hand, present invention also provides a kind of nonvolatile computer storage media,
The nonvolatile computer storage media can be non-easy included in device in above-described embodiment
The property lost computer-readable storage medium；Can also be individualism, without non-volatile in supplying terminal
Property computer-readable storage medium.Above-mentioned nonvolatile computer storage media is stored with one or more
Individual program, when one or more program is performed by an equipment so that equipment：From data
It is straight to perform analytical procedure for the keyword of reading predetermined number and each keyword to reading in storehouse
All keywords into analytical database, analytical procedure include：Analyze the keyword read
To be segmented, to segmenting perform detection step, detecting step includes：Delete last point
For word to obtain predicting prefix, whether identification prediction prefix hits the prefix in prefix list, if life
In, then the number that is hit of the prefix in prefix list is added 1, will prediction if miss
Prefix is added to prefix list, and whether identification prediction prefix is single participle, if so, then right
The next keyword read performs analytical procedure, if it is not, then being held to the participle in prediction prefix
Row detecting step；Obtain the quantity and memory space of the keyword of the prefix in hit prefix list；
The quantity and memory space of the keyword of prefix in hit prefix list, optimize database
Data store organisation.
Above description is only the preferred embodiment of the application and saying to institute's application technology principle
It is bright.It will be appreciated by those skilled in the art that invention scope involved in the application, and it is unlimited
In the technical scheme that the particular combination of above-mentioned technical characteristic forms, while it should also cover and not depart from
In the case of inventive concept, it is combined and shape by above-mentioned technical characteristic or its equivalent feature
Into other technical schemes.Such as features described above has with (but not limited to) disclosed herein
The technical scheme that the technical characteristic of similar functions is replaced mutually and formed.
Claims (12)
- A kind of 1. method for the data store organisation for optimizing key value database, it is characterised in that institute The method of stating includes：The keyword of predetermined number is read from database and each keyword to reading performs Analytical procedure is until all keywords in analytical database, the analytical procedure include：Point The keyword read is analysed to be segmented, to the participle perform detection step, the detection step Suddenly include：Last participle is deleted to obtain predicting prefix, whether identifies the prediction prefix The prefix in prefix list is hit, if hit, the prefix in the prefix list is ordered Middle number adds 1, if miss, the prediction prefix is added into the prefix list, and Identify whether the prediction prefix is single participle, if so, then to next keyword of reading Analytical procedure is performed, if it is not, then performing the detecting step to the participle in the prediction prefix；Obtain the quantity and memory space for the keyword for hitting the prefix in the prefix list；The quantity of the keyword of prefix in the hit prefix list and storage are empty Between, optimize the data store organisation of database.
- 2. according to the method for claim 1, it is characterised in that described to obtain described in hit The quantity and memory space of the keyword of prefix in prefix list include：To the prefix list In be hit number and re-execute analytical procedure less than keyword corresponding to the prefix of 2 times, obtain Prefix list after renewal；Obtain the key of the prefix in the prefix list after hitting the renewal The quantity and memory space of word；AndThe quantity of the keyword of the prefix in the hit prefix list and storage Space, optimizing the data store organisation of database includes：According to the prefix after the hit renewal The quantity and memory space of the keyword of prefix in list, optimize the data storage knot of database Structure.
- 3. according to the method for claim 1, it is characterised in that described to be read from database Take the keyword of predetermined number and each keyword to reading performs analytical procedure until having divided All keywords in analysis database include：The keyword of predetermined number is read from database simultaneously Analytical procedure is performed to each keyword of reading, performs and divides in the keyword to predetermined number At the end of analysing step, using prefix of the hit mark less than 2 times in the prefix list as independent Prefix is moved in individual Hash table up to all keywords in analytical database；The quantity and memory space for obtaining the keyword for hitting the prefix in the prefix list Including：Pair key corresponding with the individual prefix re-executes the analytical procedure to obtain more Prefix list after new, obtain the keyword of the prefix in the prefix list after hitting the renewal Quantity and memory space；AndThe quantity of the keyword of the prefix in the hit prefix list and storage Space, optimizing the data store organisation of database includes：According to the prefix after the hit renewal The quantity and memory space of the keyword of prefix in list, optimize the data storage knot of database Structure.
- 4. according to the method described in claim 1-3 any one, it is characterised in that the number Include example and/or memory image file according to storehouse.
- 5. according to the method described in claim 1-3 any one, it is characterised in that described point The keyword that analysis is read is included with obtaining participle：The keywords read according to following one or more analyses are to be segmented：It is separator, big Small letter changes and digital alphabet change.
- 6. according to the method described in claim 1-3 any one, it is characterised in that described According to the quantity and memory space of the keyword of the prefix in the hit prefix list, optimization The data store organisation of database includes：The quantity of the keyword of prefix in the hit prefix list and storage are empty Between, the option for optimizing data store organisation is presented to user；In response to receiving selection of the user to the option, optimize the data storage of the database Structure.
- A kind of 7. device for the data store organisation for optimizing key value database, it is characterised in that institute Stating device includes：Analytic unit, for reading the keyword of predetermined number from database and to the every of reading One keyword performs analytical procedure until all keywords in analytical database, described point Analysis step includes：The keyword read is analyzed to be segmented, to the participle perform detection step Suddenly, the detecting step includes：Last participle is deleted to obtain predicting prefix, identifies institute State and predict whether prefix hits the prefix in prefix list, if hit, by the prefix list In the number that is hit of prefix add 1, if miss, the prediction prefix is added to institute Prefix list is stated, and identifies whether the prediction prefix is single participle, if so, then to reading Next keyword perform analytical procedure, if it is not, then to it is described prediction prefix in participle hold The row detecting step；Acquiring unit, for obtaining the quantity for the keyword for hitting the prefix in the prefix list And memory space；Optimize unit, for according to the keyword for hitting the prefix in the prefix list Quantity and memory space, optimize the data store organisation of database.
- 8. device according to claim 7, it is characterised in that the acquiring unit enters one Walk and be used for：To being hit keyword corresponding to prefix of the number less than 2 times in the prefix list Analytical procedure is re-executed, the prefix list after being updated；Obtain after hitting the renewal The quantity and memory space of the keyword of prefix in prefix list；AndThe optimization unit is further used for：According in the prefix list after the hit renewal The quantity and memory space of the keyword of prefix, optimize the data store organisation of database.
- 9. device according to claim 7, it is characterised in that the analytic unit enters one Walk and be used for：The keyword of predetermined number is read from database and to each keyword of reading Analytical procedure is performed, at the end of analytical procedure is performed to the keyword of predetermined number, by described in Prefix of the hit mark less than 2 times is moved in individual Hash table as individual prefix in prefix list All keywords in analytical database；The acquiring unit is further used for：Pair key corresponding with the individual prefix is held again The row analytical procedure is with the prefix list after update, before obtaining after the hit renewal Sew the quantity and memory space of the keyword of the prefix in list；AndThe optimization unit is further used for：According in the prefix list after the hit renewal The quantity and memory space of the keyword of prefix, optimize the data store organisation of database.
- 10. according to the device described in claim 7-9 any one, it is characterised in that described The keyword that analytic unit is used to read predetermined number from the database includes：The analysis Unit is used for the keyword that predetermined number is read from example and/or memory image file.
- 11. according to the device described in claim 7-9 any one, it is characterised in that described The keyword that analytic unit is used to analyze reading is included with obtaining participle：The analytic unit is used for according to the keyword of following one or more analysis readings to obtain Participle：Separator, capital and small letter change and digital alphabet change.
- 12. according to the device described in claim 7-9 any one, it is characterised in that described Optimization module is further used for：The quantity of the keyword of prefix in the hit prefix list and storage are empty Between, the option for optimizing data store organisation is presented to user；In response to receiving selection of the user to the option, optimize the data storage of the database Structure.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610305828.7A CN107357794B (en) | 2016-05-10 | 2016-05-10 | Method and device for optimizing data storage structure of key value database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610305828.7A CN107357794B (en) | 2016-05-10 | 2016-05-10 | Method and device for optimizing data storage structure of key value database |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107357794A true CN107357794A (en) | 2017-11-17 |
| CN107357794B CN107357794B (en) | 2020-06-05 |
Family
ID=60271375
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610305828.7A Active CN107357794B (en) | 2016-05-10 | 2016-05-10 | Method and device for optimizing data storage structure of key value database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107357794B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110413546A (en) * | 2019-06-19 | 2019-11-05 | 平安科技（深圳）有限公司 | Date storage method, device and computer readable storage medium based on redis |
| WO2020000734A1 (en) * | 2018-06-28 | 2020-01-02 | 平安科技（深圳）有限公司 | Space management method and apparatus for storage object, and computer apparatus and storage medium |
| CN111125095A (en) * | 2019-11-26 | 2020-05-08 | 北京文渊佳科技有限公司 | Data prefix adding method and device, electronic equipment and medium |
| CN111339736A (en) * | 2020-02-18 | 2020-06-26 | 江苏满运软件科技有限公司 | Method for adding prefix name, configuration acquisition method and device and electronic equipment |
| CN112148554A (en) * | 2020-09-14 | 2020-12-29 | 北京金和网络股份有限公司 | Method and device for calculating occupation size of redis service data in real time |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101282313A (en) * | 2008-05-22 | 2008-10-08 | 北京航空航天大学 | Electronic mail system for electric conference accessory system |
| US20130138646A1 (en) * | 2010-04-27 | 2013-05-30 | Emin Gun Sirer | System and methods for mapping and searching objects in multidimensional space |
| CN105373541A (en) * | 2014-08-22 | 2016-03-02 | 博雅网络游戏开发（深圳）有限公司 | Processing method and system for data operation request of database |
| CN105491116A (en) * | 2015-11-26 | 2016-04-13 | 广州华多网络科技有限公司 | Cross-window data submitting method and system |
| US20160110292A1 (en) * | 2014-10-21 | 2016-04-21 | Samsung Electronics Co., Ltd. | Efficient key collision handling |
-
2016
- 2016-05-10 CN CN201610305828.7A patent/CN107357794B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101282313A (en) * | 2008-05-22 | 2008-10-08 | 北京航空航天大学 | Electronic mail system for electric conference accessory system |
| US20130138646A1 (en) * | 2010-04-27 | 2013-05-30 | Emin Gun Sirer | System and methods for mapping and searching objects in multidimensional space |
| CN105373541A (en) * | 2014-08-22 | 2016-03-02 | 博雅网络游戏开发（深圳）有限公司 | Processing method and system for data operation request of database |
| US20160110292A1 (en) * | 2014-10-21 | 2016-04-21 | Samsung Electronics Co., Ltd. | Efficient key collision handling |
| CN105491116A (en) * | 2015-11-26 | 2016-04-13 | 广州华多网络科技有限公司 | Cross-window data submitting method and system |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020000734A1 (en) * | 2018-06-28 | 2020-01-02 | 平安科技（深圳）有限公司 | Space management method and apparatus for storage object, and computer apparatus and storage medium |
| CN110413546A (en) * | 2019-06-19 | 2019-11-05 | 平安科技（深圳）有限公司 | Date storage method, device and computer readable storage medium based on redis |
| CN110413546B (en) * | 2019-06-19 | 2024-03-12 | 平安科技（深圳）有限公司 | Redis-based data storage method, device and computer readable storage medium |
| CN111125095A (en) * | 2019-11-26 | 2020-05-08 | 北京文渊佳科技有限公司 | Data prefix adding method and device, electronic equipment and medium |
| CN111125095B (en) * | 2019-11-26 | 2023-11-10 | 北京文渊佳科技有限公司 | Method, device, electronic equipment and medium for adding data prefix |
| CN111339736A (en) * | 2020-02-18 | 2020-06-26 | 江苏满运软件科技有限公司 | Method for adding prefix name, configuration acquisition method and device and electronic equipment |
| CN111339736B (en) * | 2020-02-18 | 2023-09-01 | 江苏满运软件科技有限公司 | Method for adding prefix name, configuration acquisition method, device and electronic equipment |
| CN112148554A (en) * | 2020-09-14 | 2020-12-29 | 北京金和网络股份有限公司 | Method and device for calculating occupation size of redis service data in real time |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107357794B (en) | 2020-06-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106446228B (en) | Method and device for collecting and analyzing WEB page data | |
| CN105183912B (en) | Abnormal log determines method and apparatus | |
| CN107357794A (en) | Optimize the method and apparatus of the data store organisation of key value database | |
| JP5492814B2 (en) | SEARCH DEVICE, SEARCH SYSTEM, METHOD, AND PROGRAM | |
| CN104077407B (en) | A kind of intelligent data search system and method | |
| CN106528894B (en) | The method and device of label information is set | |
| US20180107686A1 (en) | Search method and apparatus | |
| CN107908615A (en) | A kind of method and apparatus for obtaining search term corresponding goods classification | |
| CN107704202A (en) | A kind of method and apparatus of data fast reading and writing | |
| CN113157947A (en) | Knowledge graph construction method, tool, device and server | |
| CN107908616A (en) | The method and apparatus of anticipation trend word | |
| CN111400361A (en) | Data real-time storage method and device, computer equipment and storage medium | |
| CN105653724B (en) | A kind of monitoring method and device of page light exposure | |
| CN104580109B (en) | Generation clicks the method and device of identifying code | |
| CN106919593B (en) | Searching method and device | |
| CN107153697A (en) | Product search method and device in a kind of commodity transaction website | |
| CN103324641A (en) | Information record recommendation method and device | |
| CN107368484A (en) | Compression method and device, the acquisition methods and device of the static resource file of webpage | |
| CN107368407A (en) | Information processing method and device | |
| WO2018208412A1 (en) | Detection of caption elements in documents | |
| CN111062736A (en) | Model training and clue sequencing method, device and equipment | |
| CN108268488B (en) | Webpage main graph identification method and device | |
| US20130230248A1 (en) | Ensuring validity of the bookmark reference in a collaborative bookmarking system | |
| CN104240107A (en) | Community data screening system and method thereof | |
| CN106055688A (en) | Search result display method and device and mobile terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |