CN109033360A - A kind of data query method, apparatus, server and storage medium - Google Patents
A kind of data query method, apparatus, server and storage medium Download PDFInfo
- Publication number
- CN109033360A CN109033360A CN201810833389.6A CN201810833389A CN109033360A CN 109033360 A CN109033360 A CN 109033360A CN 201810833389 A CN201810833389 A CN 201810833389A CN 109033360 A CN109033360 A CN 109033360A
- Authority
- CN
- China
- Prior art keywords
- data
- business datum
- period
- index
- inquiry request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of data query method, apparatus, server and storage mediums, which comprises receives data inquiry request；Determine that the data type of the requested business datum of the data inquiry request, the data type include cold data and/or dsc data；Index data corresponding to type obtains the requested business datum of data inquiry request from the storage region of business datum based on the data；Wherein, index data corresponding to the cold data includes the grand index of cloth, and index data corresponding to the dsc data includes two subindex.It can effectively improve data query speed using technical solution provided in an embodiment of the present invention, save storage resource.
Description
Technical field
The present invention relates to technical field of information processing more particularly to a kind of data query method, apparatus, server and storage
Medium.
Background technique
With making rapid progress for Internet communication technology, internet obtains in the daily study of people, work and life
It is widely applied.The daily various affairs of people can be presented by network.Therefore, it is all being generated daily in internet
The data of magnanimity.
Currently, the mass data that Internet service system generates, carries out storage management often through a database, so as to
It is subsequent to need that acquisition is inquired when the data from database.Wherein, data storage processing frame HBase general as one
Frame is widely used, and HBase is a PostgreSQL database distributed, towards column, and the data in HBase are subregions
Management, different subregions need different partitioned server management, when user needs to inquire data, need first from data directory
Table determines which subregion includes the data；Then, it is determined that managing the address of the partitioned server of the subregion；Finally, being based on the ground
Location connects the partitioned server, finds the data.It is above-mentioned from data directory determine which subregion include the data procedures in by
The data information recorded in data directory is too many, generally requires to carry out subregion again to the data in data directory, accordingly
, each subregion has corresponding zone manager.Therefore also it needs to be determined that managing data directory subregion corresponding to the data
Zone manager, cause the existing data query process based on HBase complicated, seriously affect inquiry velocity, and occupy big
Measure memory space.Accordingly, it is desirable to provide more efficient or faster scheme.
Summary of the invention
The present invention provides a kind of data query method, apparatus, server and storage mediums, can effectively improve data and look into
Speed is ask, storage resource is saved.
In a first aspect, the present invention provides a kind of data query methods, which comprises
Receive data inquiry request；
Determine that the data type of the requested business datum of the data inquiry request, the data type include cold data
And/or dsc data；
Index data corresponding to type obtains the data query from the storage region of business datum based on the data
Request requested business datum；
Wherein, index data corresponding to the cold data includes the grand index of cloth, index number corresponding to the dsc data
According to including two subindex.
Second aspect provides a kind of data query device, and described device includes:
Data inquiry request receiving module, for receiving data inquiry request；
Data type determining module, for determining the data type of the requested business datum of the data inquiry request,
The data type includes cold data and/or dsc data；
Data acquisition module is inquired, for index data corresponding to type based on the data from the storage of business datum
Region obtains the requested business datum of data inquiry request；
Wherein, index data corresponding to the cold data includes the grand index of cloth, index number corresponding to the dsc data
According to including two subindex.
The third aspect provides a kind of data query server, and the server includes processor and memory, described to deposit
Be stored at least one instruction, at least one section of program, code set or instruction set in reservoir, at least one instruction, it is described extremely
Few one section of program, the code set or instruction set are loaded by the processor and are executed to realize data as described in relation to the first aspect
Querying method.
Fourth aspect provides a kind of computer readable storage medium, and at least one finger is stored in the storage medium
Enable, at least one section of program, code set or instruction set, at least one instruction, at least one section of program, the code set or
Instruction set is loaded by processor and is executed to realize data query method as described in relation to the first aspect.
Data query method, apparatus, server and storage medium provided by the invention, have the following technical effect that
The present invention can need the data type for the data inquired by distinguishing, and different index datas can be used from phase
The storage region answered obtains the data for needing to inquire；And the index data by the grand index of cloth as a large amount of cold datas, Ke Yi
While improving data query speed, the size of index file is reduced, storage resource is saved；To the hot number for usually needing to inquire
, as index data, data query speed can be preferably improved according to using two subindex.
Detailed description of the invention
It in order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology and advantage, below will be to implementation
Example or attached drawing needed to be used in the description of the prior art are briefly described, it should be apparent that, the accompanying drawings in the following description is only
It is only some embodiments of the present invention, for those of ordinary skill in the art, without creative efforts,
It can also be obtained according to these attached drawings other attached drawings.
Fig. 1 is a kind of schematic diagram of application environment provided in an embodiment of the present invention；
Fig. 2 is a kind of flow diagram of data query method provided in an embodiment of the present invention；
Fig. 3 is a kind of exemplary diagram at data query interface provided in an embodiment of the present invention；
Fig. 4 is one kind of the data type of the requested business datum of the determination provided by the invention data inquiry request
The flow diagram of embodiment；
Fig. 5 is the another of the data type of the requested business datum of the determination provided by the invention data inquiry request
The flow diagram of kind embodiment；
Fig. 6 is the another of the data type of the requested business datum of the determination provided by the invention data inquiry request
The flow diagram of kind embodiment；
Fig. 7 is the exemplary diagram at another data query interface provided in an embodiment of the present invention；
Fig. 8 is a kind of structural schematic diagram of data query device provided in an embodiment of the present invention；
Fig. 9 is the structural schematic diagram of another data query device provided in an embodiment of the present invention；
Figure 10 is the structural schematic diagram of another data query device provided in an embodiment of the present invention；
Figure 11 is a kind of structural schematic diagram of server provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art without making creative work it is obtained it is all its
His embodiment, shall fall within the protection scope of the present invention.
It should be noted that description and claims of this specification and term " first " in above-mentioned attached drawing, "
Two " etc. be to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should be understood that using in this way
Data be interchangeable under appropriate circumstances, so as to the embodiment of the present invention described herein can in addition to illustrating herein or
Sequence other than those of description is implemented.In addition, term " includes " and " having " and their any deformation, it is intended that cover
Cover it is non-exclusive include, for example, containing the process, method of a series of steps or units, system, product or server need not limit
In step or unit those of is clearly listed, but may include be not clearly listed or for these process, methods, produce
The other step or units of product or equipment inherently.
Referring to Fig. 1, Fig. 1 is a kind of schematic diagram of application environment provided in an embodiment of the present invention, as shown in Figure 1, this is answered
It may include server 100 and client 200 with environment.
In this specification embodiment, the server 100 can be used for collecting the business number of each business module of operation system
According to when by business datum storage to memory module 101, can establish the business datum of collection storage to memory module 101
First index data of business datum；And it can be by first index datastore to the memory module 101.
Further, in order to improve inquiry velocity, in this specification embodiment, partial service data can also be synchronized and is deposited
Solid state hard disk 102 is stored up, correspondingly, can establish the part when by partial service data sync storage to solid state hard disk 102
Second index data of business datum, and the second index datastore can will be gone back to solid state hard disk 102.
In this specification embodiment, the memory module 101 and solid state hard disk 102 can be located in the server 100,
It can also be located in the accessible equipment of other servers 100.
In practical applications, user is often higher than the query demand of the business datum of a period of time nearest in operation system
The query demand of time business datum remote, in this specification embodiment, the partial service data of storage to solid state hard disk can
To include the business datum of nearest a period of time, in this specification embodiment, the business datum of nearest a period of time can be heat
Data, correspondingly, the data in the business datum that server 100 is collected in addition to dsc data can be cold data.
In addition, it should be noted that, being stored with the storage of cold data in this specification embodiment to control carrying cost
Module 101 may include lower-cost mechanical hard disk (HDD conventional hard).
It is above-mentioned to be carried out in storing process by business datum, cold and hot data are stored to different storage regions, and is established
The index datas of cold and hot data, subsequent can be obtained according to different index datas from corresponding storage region need inquire
Data, and dsc data is stored to solid state hard disk, the speed of data query can be greatly promoted.
Specifically, the server 100 may include the electronic equipment with operation and network interaction function；It can also
Think and run in the electronic equipment, provides the software of support for data processing and network interaction.
In this specification embodiment, the client 200 can be used for providing a user data query interface, in response to
Family sends data inquiry request to server 100 in the data query triggering command at the data query interface；Server 100 can
With the data type (data class corresponding to the requested data of data inquiry request for being sent according to client 200
Type may include cold data and/or dsc data) determine corresponding index data, and the index data is based on from corresponding memory block
Domain obtains the business datum of the data inquiry request；The business datum that server 100 can be also used for will acquire is sent to client
End 200, so that business datum is showed user by client 200.
Specifically, the client 200 may include smart phone, desktop computer, tablet computer, laptop, number
The entity device of the types such as word assistant, intelligent wearable device, also may include the software run in entity device, such as empty
Quasi- machine etc..The operating system run in client in the embodiment of the present invention can include but is not limited to Android system, IOS system,
Linux, windows etc..
Further, the first index data of business datum can be in conjunction with the data of business datum in this specification embodiment
Operational requirements determine that in a specific embodiment, by taking the business datum is pipelined data as an example, pipelined data is generally not
It needs to modify and delete, only insertion operation.Correspondingly, the first index data described in this specification embodiment may include but
It is not limited to the grand index of cloth, the grand index of cloth may include: index file and data file.Specifically, the index file can be with
For recording the information of Bloom filter.The data file can be used for recording existing data and store information in source file
(for example, data storage location, size of data etc.).
The specific embodiment of the foundation of the grand index of cloth introduced below.
In this specification embodiment, it is assumed that Bloom filter has k layers, first according to each layer hash (Hash) function meter
Calculating business datum key, (key may include a certain characteristic in business datum here, such as business datum includes input
Date, birthday, class, name.Correspondingly, key can be inputting date, the birthday, class, in any business datum in name
Characteristic) hash value, then, then the corresponding mark position 1 of each layer；Finally, business datum in source file
Storage information is written in data file, and records flag bit into index file.
In practical applications, pipelined data often has that data volume is big, above-mentioned to establish Bu Longsuo for pipelined data
The size of index file can be also reduced, save storage resource while substantially increasing data query speed by drawing.
Further, in this specification embodiment, dsc data is stored in solid state hard disk, and two subindex are for random read-write
Solid state hard disk inquiry velocity it is very fast.Therefore, in this specification embodiment, second index data may include but unlimited
In two subindex.Two subindex include the information to have sorted according to index value.
In the specific embodiment that two subindex introduced below is established:
The source file for reading business datum, getting the key of business datum, (key may include in business datum here
A certain characteristic, such as business datum include inputting date, birthday, class, name.Correspondingly, key can be input day
Phase, birthday, class, the characteristic in any business datum in name) and storage information；The key and storage information are deposited
It stores up into a mapping table；Then, this mapping table sorts according to key, obtains two subindex of the business datum.
The embodiment of data query method provided by the invention introduced below, Fig. 2 are one kind provided in an embodiment of the present invention
The flow diagram of data query method, present description provides the method operating procedures as described in embodiment or flow chart, but
It may include more or less operating procedure based on routine or without creative labor.The step of enumerating in embodiment is suitable
Sequence is only one of numerous step execution sequence mode, does not represent and unique executes sequence.System or clothes in practice
Device product be engaged in when executing, can be executed according to embodiment or method shown in the drawings sequence or it is parallel execute it is (such as parallel
The environment of processor or multiple threads).It is specific as shown in Fig. 2, the method may include:
S201: data inquiry request is received.
In this specification embodiment, the client-side of data query system can provide data query interface for user's touching
Send out data inquiry request.Specifically, as shown in figure 3, Fig. 3 is a kind of example at data query interface provided in an embodiment of the present invention
Figure.As can be seen from Fig. 3, user can input the period for needing the data content inquired and inquiry.It is specific real at one
It applies in example, it is assumed that user needs inquiry business A from 19:00:00's on May 20,16:00:00 to 2018 years on the 20th May in 2018
Pipelined data.Correspondingly, incoming traffic A pipelined data, query time section in May, 2018 can be inputted at inquiry data content
16:00:00 to 2018 years on the 20th May 20 then 19:00:00 clicks inquiry button with trigger data inquiry request.
In addition, it should be noted that, above-mentioned query time section may include such as above-mentioned (May 20 in 2018 a period of time
Day on May 20th, 16:00:00 to 2018 19:00:00), also may include a time point, such as 16:00 on May 20th, 2018:
On May 20th, 00 to 2018 16:00:00.
S203: determine that the data type of the requested business datum of the data inquiry request, the data type include
Cold data and/or dsc data.
In this specification embodiment, after receiving data inquiry request, the data inquiry request institute can be determined
The data type of the business datum of request, as shown in figure 4, Fig. 4 is that the determination provided by the invention data inquiry request is asked
The flow diagram of a kind of embodiment of the data type for the business datum asked, specifically, may include:
S401: the period corresponding to the requested business datum of the data inquiry request is obtained.
S403: judge in the corresponding period whether to include the period preset in the nearest period.
In this specification embodiment, described to preset the nearest period may include the preset time period of current time forward.
Such as current time is toward the last week.
S405: when the judgment result is no, the data class of the requested business datum of the data inquiry request is determined
Type is cold data.
In a specific embodiment, it is assumed that the period corresponding to the requested business datum of data inquiry request
Are as follows: on May 20,16:00:00 to 2018 years on the 20th May in 2018 19:00:00, preset the nearest period are as follows: current time is forward
One week, and current time are as follows: on June 21st, 2018 12:00:00；Correspondingly, it can be the time that this, which presets the nearest period,
Section: on June 21,12:00:00 to 2018 years on the 14th June in 2018 12:00:00.Correspondingly, the requested industry of data inquiry request
It does not include presetting in May 20 16:00:00 to 2018 years on the 20th May 2018 period 19:00:00 corresponding to data of being engaged in
Period in nearest 12:00:00 on June 21 12:00:00 to 2018 years on the 14th June 2018 period, correspondingly, can be with
The data type for determining the requested business datum of the determination data inquiry request is cold data.
In further embodiments, as shown in figure 5, the method can also include:
S407: when the result judged is is, judge in the corresponding period whether to include described default nearest
The period other than period in period.
S409: when judge do not include the period preset in the nearest period in the corresponding period with
When the outer period, determine that the data type of the requested business datum of the data inquiry request is dsc data.
In specific embodiment, it is assumed that the period corresponding to the requested business datum of data inquiry request are as follows: 2018
June 20 on June 16:00:00 to 2018 years 20,19:00:00, preset the nearest period are as follows: current time is toward previous star
Phase, and current time are as follows: on June 21st, 2018 12:00:00；Correspondingly, this, which presets the nearest period, to be the period:
On June 21,12:00:00 to 2018 years on the 14th June in 2018 12:00:00.Correspondingly, the requested business of data inquiry request
It include default nearest in June 20 16:00:00 to 2018 years on the 20th June 2018 period 19:00:00 corresponding to data
Period in June 21 12:00:00 to 2018 years on the 14th June 2018 period 12:00:00, correspondingly, may determine that
It whether include institute in the corresponding period (19:00:00 on June 20,16:00:00 to 2018 years on the 20th June in 2018)
It states other than the period preset the nearest period in (12:00:00 on June 21,12:00:00 to 2018 years on the 14th June in 2018)
Period, correspondingly, not including here 2018 in 19:00:00 on June 20,16:00:00 to 2018 years on the 20th June in 2018
The period other than period in June 21 on June 12:00:00 to 2018 years 14,12:00:00；Correspondingly, can be true
The data type for determining the requested business datum of data inquiry request is dsc data.
In further embodiments, as shown in fig. 6, the method can also include:
S411: when judging in the corresponding period to include other than the period preset in the nearest period
Period when, determine that the data type of the requested business datum of the data inquiry request includes cold data and dsc data；
Wherein, the data type of the requested business datum corresponding to the period other than described is cold data,
The requested business number corresponding to the period in the default nearest period for including in the corresponding period
According to data type be dsc data.
In specific embodiment, it is assumed that the period corresponding to the requested business datum of data inquiry request are as follows: 2018
June 20 on May 16:00:00 to 2018 years 20,19:00:00, preset the nearest period are as follows: current time is toward previous star
Phase, and current time are as follows: on June 21st, 2018 12:00:00；Correspondingly, this, which presets the nearest period, to be the period:
On June 21,12:00:00 to 2018 years on the 14th June in 2018 12:00:00.Correspondingly, the requested business of data inquiry request
It include default nearest in June 20 16:00:00 to 2018 years on the 20th May 2018 period 19:00:00 corresponding to data
Period in June 21 12:00:00 to 2018 years on the 14th June 2018 period 12:00:00, correspondingly, may determine that
It whether include institute in the corresponding period (19:00:00 on June 20,16:00:00 to 2018 years on the 20th June in 2018)
It states other than the period preset the nearest period in (12:00:00 on June 21,12:00:00 to 2018 years on the 14th June in 2018)
Period, correspondingly, here in 19:00:00 on June 20,16:00:00 to 2018 years on the 20th May in 2018 include 2018 6
Other than period in month 12:00:00 to 2018 years on the 14th June 21 12:00:00 period (20 days 16 May in 2018:
On June 14th, 00:00 to 2018 12:00:00)；Correspondingly, the number of the requested business datum of data inquiry request can be determined
It include cold data and dsc data according to type, wherein (16:00:00 to 2018 years on the 20th 6 May in 2018 period other than described
Month 12:00:00 on the 14th) corresponding to the data type of requested business datum be cold data；The corresponding period
In include the default nearest period in period (on June 20,12:00:00 to 2018 years on the 14th June in 2018 19:00:
00) data type of the requested business datum corresponding to is dsc data.
S205: index data corresponding to type obtains the data from the storage region of business datum based on the data
The requested business datum of inquiry request.
Wherein, index data corresponding to the cold data includes the grand index of cloth, index number corresponding to the dsc data
According to including two subindex.
In this specification embodiment, the business datum of different types of data is stored in different storage regions, and is provided with
Different index datas.It, can be based on corresponding to data type after the data type for determining requested business datum
Index data obtains the business datum of the data inquiry request from the storage region of business datum.
In a specific embodiment, by taking the first index data is the grand index of cloth as an example, when the data inquiry request
When data type corresponding to requested data includes cold data, index data corresponding to the type based on the data
Obtaining the requested business datum of data inquiry request from the storage region of business datum may include:
The requested business datum of data inquiry request is obtained from default hard disk based on the grand index of the cloth.
In this specification embodiment, the default hard disk may include the memory module for being stored with cold data.
Specifically, introduced below obtain the data inquiry request institute from the storage region of business datum based on the grand index of cloth
The specific steps of the business datum of request:
The hash value that the key of business datum is calculated according to each layer of Bloom filter of hash function, according to each layer
Hash value check corresponding flag bit whether all for 1, if k layer (assuming that Bloom filter has k layers) are all 1, it may be determined that
Business datum exists；Then, business datum being found out from data file, information is stored in source file；Finally, based on storage letter
The source file for ceasing corresponding storage region obtains business datum.
In addition, if there is the not flag bit for 1 in k layers, it is considered that data are not present, the letter of inquiry failure can be returned to
Breath.
In another specific embodiment, by taking the second index data is two subindex as an example, when the data query is asked
When to seek data type corresponding to requested data include dsc data, index number corresponding to the type based on the data
Include according to the requested business datum of data inquiry request is obtained from the storage region of business datum:
The requested business datum of data inquiry request is obtained from the solid state hard disk based on two subindex.
It is based on two subindex specifically, introduced below from the solid state hard disk to obtain the data inquiry request requested
The specific steps of business datum:
All it is the mapping table to have sorted according to the key of business datum in two subindex, therefore, two points can be done with key and is looked into
It looks for, finds corresponding storage information, and give storage information and find business corresponding to key from the source file in solid state hard disk
Data.
In another specific embodiment, using the first index data as the grand index of cloth, the second index data is two points of ropes
It is cited as example, when the data type corresponding to the requested data of the data inquiry request includes dsc data and cold data, institute
It states index data corresponding to type based on the data and obtains the data inquiry request institute from the storage region of business datum
The business datum of request may include:
Cold number in the requested business datum of data inquiry request is obtained from default hard disk based on the grand index of the cloth
According to.
It is obtained in the requested business datum of data inquiry request based on two subindex from the solid state hard disk
Dsc data.
In another embodiment, the method can also include:
Business datum is obtained, by business datum storage to default hard disk；
The grand index storage of the cloth is arrived the default hard disk by the grand index of cloth for establishing the business datum.
In another embodiment, the method can also include:
By dsc data storage in the business datum to solid state hard disk；
Two subindex for establishing the dsc data, by two subindex storage to the solid state hard disk.
The technical solution provided by above this specification embodiment is as it can be seen that by being a large amount of cold in this specification embodiment
Data are established the grand subsequent query that is indexed for of cloth and are indexed, and can reduce index while substantially increasing data query speed
The size of file, saves storage resource.In addition, the dsc data that will often be required to inquiry is stored in solid state hard disk, and use
Index data of two subindex as dsc data makes full use of two subindex fast for the solid state hard disk inquiry velocity of random read-write
Advantage, can preferably improve data query speed；And cold and hot data combination actual demand is stored to different memory blocks
Domain can effectively control cost.
Below in conjunction with Fig. 7, another example at the data query interface of this specification embodiment offer is introduced:
Different index datas is used to cold and hot data in this specification embodiment, and stores and arrives different storage regions,
Solid state hard disk is arrived by storage for dsc data, and using two subindex by the way of the inquiry dsc data of solid state hard disk, it can
To realize inquiry velocity more faster than cold data.Therefore, in order to improve experience of the user in data query, this specification is implemented
In example, the prompt information for accelerating inquiry can be set at data query interface, specifically, as described in Figure 7, the acceleration inquiry
Prompt information can be the information of current fast period.
In addition, this specification embodiment can also data query interface provide accelerate inquiry triggering interface, only when
User triggers acceleration inquiry instruction, just can be to the dsc data that can accelerate using the number based on two subindex from solid state hard disk
It is investigated that asking；Otherwise, business datum directly can be obtained from default hard disk according to the grand index of cloth.
The embodiment of the invention also provides a kind of data query devices, as shown in figure 8, described device includes:
Data inquiry request receiving module 810 can be used for receiving data inquiry request；
Data type determining module 820 is determined for the number of the requested business datum of the data inquiry request
According to type, the data type includes cold data and/or dsc data；
Data acquisition module 830 is inquired, can be used for index data corresponding to type based on the data from business number
According to storage region obtain the requested business datum of data inquiry request；
Wherein, index data corresponding to the cold data includes the grand index of cloth, index number corresponding to the dsc data
According to including two subindex.
In another embodiment, the data type determining module 820 may include:
Period acquiring unit, when can be used for obtaining corresponding to the requested business datum of the data inquiry request
Between section；
Whether the first judging unit can be used for judging in the corresponding period including presetting in the nearest period
Period；
First data type determination unit can be used for when the result that first judging unit judges is no, determine
The data type of the requested business datum of data inquiry request is cold data.
In another embodiment, the data type determining module 820 can also include:
Second judgment unit can be used for when the result that first judging unit judges is is, judge described pair
Whether including the period other than the period preset in the nearest period in the period answered；
Second data type determination unit can be used for when the result that the second judgment unit judges is no, determine
The data type of the requested business datum of data inquiry request is dsc data.
In another embodiment, the data type determining module 820 can also include:
Third data type determination unit, when result for judging when the second judgment unit is is, determine described in
The data type of the requested business datum of data inquiry request includes cold data and dsc data；
Wherein, the data type of the requested business datum corresponding to the period other than described is cold data,
The requested business number corresponding to the period in the default nearest period for including in the corresponding period
According to data type be dsc data.
Apparatus and method embodiment in the Installation practice is based on similarly inventive concept.
The embodiment of the invention also provides another data query devices, as shown in figure 9, described device includes:
Data acquisition module 910 can be used for obtaining business datum.
First data memory module 920 can be used for storing the business datum to default hard disk.
First index establishes module 930, can be used for establishing the grand index of cloth of the business datum.
First index datastore module 940 can be used for the grand index storage of the cloth to the default hard disk.
Data inquiry request receiving module 950 can be used for receiving data inquiry request.
Data type determining module 960 is determined for the number of the requested business datum of the data inquiry request
According to type, the data type includes cold data and/or dsc data.
Data acquisition module 970 is inquired, can be used for index data corresponding to type based on the data from business number
According to storage region obtain the requested business datum of data inquiry request.
Wherein, index data corresponding to the cold data includes the grand index of cloth, index number corresponding to the dsc data
According to including two subindex.
Apparatus and method embodiment in the Installation practice is based on similarly inventive concept.
The embodiment of the invention also provides another data query devices, and as shown in Figure 10, described device includes:
Data acquisition module 1010 can be used for obtaining business datum.
First data memory module 1020 can be used for storing the business datum to default hard disk.
First index establishes module 1030, can be used for establishing the grand index of cloth of the business datum.
First index datastore module 1040 can be used for the grand index storage of the cloth to the default hard disk.
Second data memory module 1050 can be used for dsc data storage in the business datum to solid state hard disk；
Second index establishes module 1060, can be used for establishing two subindex of the dsc data；
Second index datastore module 1070 can be used for storing two subindex to the solid state hard disk.
Data inquiry request receiving module 1080 can be used for receiving data inquiry request.
Data type determining module 1090 is determined for the number of the requested business datum of the data inquiry request
According to type, the data type includes cold data and/or dsc data.
Data acquisition module 1100 is inquired, can be used for index data corresponding to type based on the data from business number
According to storage region obtain the requested business datum of data inquiry request.
Apparatus and method embodiment in the Installation practice is based on similarly inventive concept.
The embodiment of the invention provides a kind of data query servers, which includes processor and storage
Device is stored at least one instruction, at least one section of program, code set or instruction set in the memory, at least one instruction, should
At least one section of program, the code set or instruction set is loaded by the processor and is executed to realize as above method embodiment is provided
Data query method.
Memory can be used for storing software program and module, and processor is stored in the software program of memory by operation
And module, thereby executing various function application and data processing.Memory can mainly include storing program area and storage number
According to area, wherein storing program area can application program needed for storage program area, function etc.；Storage data area can store basis
The equipment uses created data etc..In addition, memory may include high-speed random access memory, can also include
Nonvolatile memory, for example, at least a disk memory, flush memory device or other volatile solid-state parts.Phase
Ying Di, memory can also include Memory Controller, to provide access of the processor to memory.
Embodiment of the method provided by the embodiment of the present invention can be in mobile terminal, terminal, server or class
As execute in arithmetic unit.For running on the server, Figure 11 is a kind of data query provided in an embodiment of the present invention
The hardware block diagram of the server of method.As shown in figure 11, which can generate ratio because configuration or performance are different
Biggish difference may include one or more central processing units (Central Processing Units, CPU) 1110
(processing unit that processor 1110 can include but is not limited to Micro-processor MCV or programmable logic device FPGA etc.) is used for
The memory 1130 of storing data, the storage medium 1120 of one or more storage application programs 1123 or data 1122
(such as one or more mass memory units).Wherein, memory 1130 and storage medium 1120 can be of short duration storage
Or persistent storage.The program for being stored in storage medium 1120 may include one or more modules, and each module can wrap
It includes to the series of instructions operation in server.Further, central processing unit 1110 can be set to and storage medium
1120 communications execute the series of instructions operation in storage medium 1120 on server 1100.Server 1100 can also wrap
One or more power supplys 1160 are included, one or more wired or wireless network interfaces 1150, one or more
Input/output interface 1140, and/or, one or more operating systems 1121, such as Windows ServerTM, Mac OS
XTM, UnixTM, LinuxTM, FreeBSDTM etc..
Input/output interface 1140 can be used for that data are received or sent via a network.Above-mentioned network is specifically real
Example may include the wireless network that the communication providers of server 1100 provide.In an example, input/output interface 1140 wraps
A network adapter (Network Interface Controller, NIC) is included, base station and other network equipments can be passed through
It is connected so as to be communicated with internet.In an example, input/output interface 1140 can be radio frequency (Radio
Frequency, RF) module, it is used to wirelessly be communicated with internet.
It will appreciated by the skilled person that structure shown in Figure 11 is only to illustrate, above-mentioned electronics is not filled
The structure set causes to limit.For example, server 1100 may also include more perhaps less component or tool than shown in Figure 11
There is the configuration different from shown in Figure 11.
The embodiments of the present invention also provide a kind of storage medium, the storage medium be may be disposed among server to protect
It deposits for realizing relevant at least one instruction of data query method a kind of in embodiment of the method, at least one section of program, code set
Or instruction set, this at least one instruction, at least one section of program, the code set or the instruction set loaded by the processor and executed with
Realize the data query method that above method embodiment provides.
Optionally, in the present embodiment, above-mentioned storage medium can be located in multiple network servers of computer network
At least one network server.Optionally, in the present embodiment, above-mentioned storage medium can include but is not limited to: USB flash disk, only
Read memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), movement
The various media that can store program code such as hard disk, magnetic or disk.
The embodiment of the data query method, apparatus, server or the storage medium that are provided by aforementioned present invention is as it can be seen that this hair
It is indexed in bright by establishing the grand subsequent query that is indexed for of cloth for a large amount of cold data, data query speed can substantially increased
While spending, the size of index file is reduced, storage resource is saved.In addition, the dsc data that will often be required to inquiry is stored in
In solid state hard disk, and the index data using two subindex as dsc data, make full use of two subindex for random read-write
The fast advantage of solid state hard disk inquiry velocity can preferably improve data query speed；And by cold and hot data combination actual demand
Different storage regions is stored, cost can be effectively controlled.
It should be understood that embodiments of the present invention sequencing is for illustration only, do not represent the advantages or disadvantages of the embodiments.
And above-mentioned this specification specific embodiment is described.Other embodiments are within the scope of the appended claims.One
In a little situations, the movement recorded in detail in the claims or step can be executed according to the sequence being different from embodiment and
Still desired result may be implemented.In addition, process depicted in the drawing not necessarily requires the particular order shown or company
Continuous sequence is just able to achieve desired result.In some embodiments, multitasking and parallel processing it is also possible or
It may be advantageous.
All the embodiments in this specification are described in a progressive manner, same and similar portion between each embodiment
Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for device and
For server example, since it is substantially similar to the method embodiment, so being described relatively simple, related place is referring to side
The part of method embodiment illustrates.
Those of ordinary skill in the art will appreciate that realizing that all or part of the steps of above-described embodiment can pass through hardware
It completes, relevant hardware can also be instructed to complete by program, the program can store in a kind of computer-readable
In storage medium, storage medium mentioned above can be read-only memory, disk or CD etc..
The foregoing is merely presently preferred embodiments of the present invention, is not intended to limit the invention, it is all in spirit of the invention and
Within principle, any modification, equivalent replacement, improvement and so on be should all be included in the protection scope of the present invention.
Claims (14)
1. a kind of data query method, which is characterized in that the described method includes:
Receive data inquiry request；
Determine the data type of the requested business datum of the data inquiry request, the data type include cold data and/
Or dsc data；
Index data corresponding to type obtains the data inquiry request from the storage region of business datum based on the data
Requested business datum；
Wherein, index data corresponding to the cold data includes the grand index of cloth, index data packet corresponding to the dsc data
Include two subindex.
2. the method according to claim 1, wherein the method also includes:
Business datum is obtained, by business datum storage to default hard disk；
The grand index storage of the cloth is arrived the default hard disk by the grand index of cloth for establishing the business datum.
3. according to the method described in claim 2, it is characterized in that, the method also includes:
By dsc data storage in the business datum to solid state hard disk；
Two subindex for establishing the dsc data, by two subindex storage to the solid state hard disk.
4. method according to any one of claims 1 to 3, which is characterized in that the determination data inquiry request is asked
The data type for the business datum asked includes:
Obtain the period corresponding to the requested business datum of the data inquiry request；
Judge in the corresponding period whether to include the period preset in the nearest period；
When the judgment result is no, the data type for determining the requested business datum of the data inquiry request is cold number
According to.
5. according to the method described in claim 4, it is characterized in that, the method also includes:
When the result judged is is, judge in the corresponding period whether to include described preset in the nearest period
Period other than period；
When judging not include the time other than the period preset in the nearest period in the corresponding period
Duan Shi determines that the data type of the requested business datum of the data inquiry request is dsc data.
6. according to the method described in claim 4, it is characterized in that, the method also includes:
When judging to include the period other than the period preset in the nearest period in the corresponding period
When, determine that the data type of the requested business datum of the data inquiry request includes cold data and dsc data；
Wherein, the data type of the requested business datum corresponding to the period other than described is cold data, described
The requested business datum corresponding to the period in the default nearest period for including in the corresponding period
Data type is dsc data.
7. a kind of data query device, which is characterized in that described device includes:
Data inquiry request receiving module, for receiving data inquiry request；
Data type determining module, it is described for determining the data type of the requested business datum of the data inquiry request
Data type includes cold data and/or dsc data；
Data acquisition module is inquired, for index data corresponding to type based on the data from the storage region of business datum
Obtain the requested business datum of the data inquiry request；
Wherein, index data corresponding to the cold data includes the grand index of cloth, index data packet corresponding to the dsc data
Include two subindex.
8. device according to claim 7, which is characterized in that described device further include:
Data acquisition module, for obtaining business datum；
First data memory module, for storing the business datum to default hard disk；
First index establishes module, the grand index of the cloth for establishing the business datum；
First index datastore module, for the default hard disk to be arrived in the grand index storage of the cloth.
9. device according to claim 8, which is characterized in that described device further include:
Second data memory module, for storing dsc data in the business datum to solid state hard disk；
Second index establishes module, for establishing two subindex of the dsc data；
Second index datastore module, for storing two subindex to the solid state hard disk.
10. according to any device of claim 7 to 9, which is characterized in that the data type determining module includes:
Period acquiring unit, for obtaining the period corresponding to the requested business datum of the data inquiry request；
First judging unit, for judging in the corresponding period whether to include the time preset in the nearest period
Section；
First data type determination unit determines the data when result for judging when first judging unit is no
The data type of the requested business datum of inquiry request is cold data.
11. device according to claim 10, which is characterized in that the data type determining module further include:
Second judgment unit when result for judging when first judging unit is is, judges the corresponding time
Whether including the period other than the period preset in the nearest period in section；
Second data type determination unit determines the data when result for judging when the second judgment unit is no
The data type of the requested business datum of inquiry request is dsc data.
12. device according to claim 11, which is characterized in that the data type determining module further include:
Third data type determination unit determines the data when result for judging when the second judgment unit is is
The data type of the requested business datum of inquiry request includes cold data and dsc data；
Wherein, the data type of the requested business datum corresponding to the period other than described is cold data, described
The requested business datum corresponding to the period in the default nearest period for including in the corresponding period
Data type is dsc data.
13. a kind of data query server, which is characterized in that the server includes processor and memory, the memory
In be stored at least one instruction, at least one section of program, code set or instruction set, at least one instruction, described at least one
Duan Chengxu, the code set or instruction set are loaded by the processor and are executed to realize as described in claim 1 to 6 is any
Data query method.
14. a kind of computer readable storage medium, which is characterized in that be stored at least one instruction, extremely in the storage medium
Few one section of program, code set or instruction set, at least one instruction, at least one section of program, the code set or the instruction
Collection is loaded by processor and is executed to realize the data query method as described in claim 1 to 6 is any.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810833389.6A CN109033360B (en) | 2018-07-26 | 2018-07-26 | Data query method, device, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810833389.6A CN109033360B (en) | 2018-07-26 | 2018-07-26 | Data query method, device, server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109033360A true CN109033360A (en) | 2018-12-18 |
| CN109033360B CN109033360B (en) | 2020-11-17 |
Family
ID=64646407
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810833389.6A Active CN109033360B (en) | 2018-07-26 | 2018-07-26 | Data query method, device, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109033360B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111475506A (en) * | 2020-03-30 | 2020-07-31 | 广州虎牙科技有限公司 | Data storage and query method, device, system, equipment and storage medium |
| CN112650453A (en) * | 2020-12-31 | 2021-04-13 | 北京千方科技股份有限公司 | Method and system for storing and inquiring traffic data |
| CN113163003A (en) * | 2021-04-09 | 2021-07-23 | 北京汇钧科技有限公司 | File transmission method, cloud server, client and fog node router |
| CN113190507A (en) * | 2021-05-14 | 2021-07-30 | 杭州海康威视数字技术股份有限公司 | Index information synchronization method and device and electronic equipment |
| CN113704346A (en) * | 2020-05-20 | 2021-11-26 | 杭州海康威视数字技术股份有限公司 | Hbase table cold and hot data conversion method and device and electronic equipment |
| CN114513373A (en) * | 2022-04-20 | 2022-05-17 | 北京掌趣无限科技有限公司 | Trusted data exchange method, device, system, electronic equipment and storage medium |
| CN114691677A (en) * | 2020-12-31 | 2022-07-01 | 深圳云天励飞技术股份有限公司 | Data searching method and device, electronic equipment and storage medium |
| CN115033603A (en) * | 2022-06-16 | 2022-09-09 | 支付宝(杭州)信息技术有限公司 | Data storage method, data query method, data storage device and data query device |
| CN117973812A (en) * | 2024-03-29 | 2024-05-03 | 北京卓导科技有限公司 | Enterprise informatization management platform and method based on big data |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150009307A1 (en) * | 2011-11-23 | 2015-01-08 | Samsung Electronics Co., Ltd. | Stereoscopic image display device |
| CN104850564A (en) * | 2014-02-18 | 2015-08-19 | 腾讯科技（深圳）有限公司 | Index search method and system of data file |
| CN105630847A (en) * | 2014-11-21 | 2016-06-01 | 深圳市腾讯计算机系统有限公司 | Data storage method as well as data query method, apparatus and system |
| CN108021585A (en) * | 2016-10-28 | 2018-05-11 | 腾讯科技（深圳）有限公司 | Distributed data storage method and device |
-
2018
- 2018-07-26 CN CN201810833389.6A patent/CN109033360B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150009307A1 (en) * | 2011-11-23 | 2015-01-08 | Samsung Electronics Co., Ltd. | Stereoscopic image display device |
| CN104850564A (en) * | 2014-02-18 | 2015-08-19 | 腾讯科技（深圳）有限公司 | Index search method and system of data file |
| CN105630847A (en) * | 2014-11-21 | 2016-06-01 | 深圳市腾讯计算机系统有限公司 | Data storage method as well as data query method, apparatus and system |
| CN108021585A (en) * | 2016-10-28 | 2018-05-11 | 腾讯科技（深圳）有限公司 | Distributed data storage method and device |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111475506B (en) * | 2020-03-30 | 2024-03-01 | 广州虎牙科技有限公司 | Method, device, system, equipment and storage medium for data storage and query |
| CN111475506A (en) * | 2020-03-30 | 2020-07-31 | 广州虎牙科技有限公司 | Data storage and query method, device, system, equipment and storage medium |
| CN113704346A (en) * | 2020-05-20 | 2021-11-26 | 杭州海康威视数字技术股份有限公司 | Hbase table cold and hot data conversion method and device and electronic equipment |
| CN113704346B (en) * | 2020-05-20 | 2024-06-04 | 杭州海康威视数字技术股份有限公司 | Hbase table cold-hot data conversion method and device and electronic equipment |
| CN114691677A (en) * | 2020-12-31 | 2022-07-01 | 深圳云天励飞技术股份有限公司 | Data searching method and device, electronic equipment and storage medium |
| CN112650453B (en) * | 2020-12-31 | 2024-05-14 | 北京千方科技股份有限公司 | Method and system for storing and inquiring traffic data |
| CN112650453A (en) * | 2020-12-31 | 2021-04-13 | 北京千方科技股份有限公司 | Method and system for storing and inquiring traffic data |
| CN113163003B (en) * | 2021-04-09 | 2023-05-30 | 北京汇钧科技有限公司 | File transmission method, cloud server, client and cloud node router |
| CN113163003A (en) * | 2021-04-09 | 2021-07-23 | 北京汇钧科技有限公司 | File transmission method, cloud server, client and fog node router |
| CN113190507A (en) * | 2021-05-14 | 2021-07-30 | 杭州海康威视数字技术股份有限公司 | Index information synchronization method and device and electronic equipment |
| CN114513373A (en) * | 2022-04-20 | 2022-05-17 | 北京掌趣无限科技有限公司 | Trusted data exchange method, device, system, electronic equipment and storage medium |
| CN115033603A (en) * | 2022-06-16 | 2022-09-09 | 支付宝(杭州)信息技术有限公司 | Data storage method, data query method, data storage device and data query device |
| CN117973812A (en) * | 2024-03-29 | 2024-05-03 | 北京卓导科技有限公司 | Enterprise informatization management platform and method based on big data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109033360B (en) | 2020-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109033360A (en) | A kind of data query method, apparatus, server and storage medium | |
| US10505964B2 (en) | Context aware threat protection | |
| CN109739815B (en) | File processing method, system, device, equipment and storage medium | |
| CN109299157B (en) | Data export method and device for distributed big single table | |
| CN107341033A (en) | A kind of data statistical approach, device, electronic equipment and storage medium | |
| CN109240607B (en) | File reading method and device | |
| CN107798135B (en) | Paging query method and device and electronic equipment | |
| JP6557361B2 (en) | Method and device for querying short-range objects | |
| CN104090899A (en) | Method and device for feeding back display content information | |
| US10417192B2 (en) | File classification in a distributed file system | |
| CN111831618A (en) | Data writing method, data reading method, device, equipment and storage medium | |
| CN110677492A (en) | Access request processing method and device, electronic equipment and storage medium | |
| CN107305570B (en) | Data retrieval method and system | |
| CN106657182B (en) | Cloud file processing method and device | |
| CN106953924B (en) | Processing method of shared information and shared client | |
| CN106202307A (en) | A kind of batch log preservation method and device | |
| CN112272137B (en) | Method, computing device and computer readable memory device for mass data management in a communication application via multiple mailboxes | |
| CN113407356A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN110909072B (en) | Data table establishment method, device and equipment | |
| CN109815351B (en) | Information query method and related product | |
| WO2017076027A1 (en) | Wallpaper processing method and apparatus | |
| EP3572951A1 (en) | A method for handling requests in a storage system and a storage node for a storage system | |
| CN104408056B (en) | Data processing method and device | |
| CN110928895A (en) | Data query method, data table establishing method, device and equipment | |
| CN105939388A (en) | Method for pushing business content and content controller |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |