CN107679158A - Data managing method, device, computer-readable medium and electronic equipment - Google Patents
Data managing method, device, computer-readable medium and electronic equipment Download PDFInfo
- Publication number
- CN107679158A CN107679158A CN201710896009.9A CN201710896009A CN107679158A CN 107679158 A CN107679158 A CN 107679158A CN 201710896009 A CN201710896009 A CN 201710896009A CN 107679158 A CN107679158 A CN 107679158A
- Authority
- CN
- China
- Prior art keywords
- data
- storage system
- subregion
- incremental
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2272—Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a kind of data managing method, device, computer-readable medium and electronic equipment.The data managing method includes：Obtain the incremental data of database；Data query index is generated according to the incremental data；Pre- multidomain treat-ment is carried out to data-storage system based on data query index, wherein, the data query index is associated with the pre- subregion scope of the data-storage system；The incremental data is stored to the pre- subregion of the data-storage system.Technical scheme can divide under the scene of table in database point storehouse, effectively improve efficiency data query, ensure that database has good performance.
Description
Technical field
The present invention relates to technical field of data processing, in particular to a kind of data managing method, device, computer
Computer-readable recording medium and electronic equipment.
Background technology
At present, the increasingly incremental data volume of many Large-Scale Interconnected net enterprises causes Query Efficiency to slow down, to understand
Certainly the problem of search efficiency, many enterprises divide the strategy of table using point storehouse, and the strategy is to carry out business burst fractionation to data, is looked into
Ask scene to be limited in the data hash of burst field into different storehouses and table, improve search efficiency to a certain extent.
Under dividing the strategy of table in point storehouse, due to dividing the inquiry data of table not know, therefore paging behaviour is being in correlation technique
All points of table data need to be collected when making and carry out paging activity again, often increase the inquiry operation of one page, its all page previous need to be obtained
The data set in face, causing paging activity, search efficiency is lower more rearward, the data of all points of tables need to be found to last page, seriously
It has impact on the search efficiency and database performance of database.
It should be noted that information is only used for strengthening the reason of the background to the present invention disclosed in above-mentioned background section
Solution, therefore can include not forming the information to prior art known to persons of ordinary skill in the art.
The content of the invention
It is an object of the invention to provide a kind of data managing method, device, computer-readable medium and electronic equipment, enter
And one or more problem caused by the limitation of correlation technique and defect is at least overcome to a certain extent.
Other characteristics and advantage of the present invention will be apparent from by following detailed description, or partially by the present invention
Practice and acquistion.
According to the first aspect of the invention, there is provided a kind of data managing method, including：Obtain the incremental number of database
According to；Data query index is generated according to the incremental data；Data-storage system is carried out based on data query index pre-
Multidomain treat-ment, wherein, the data query index is associated with the pre- subregion scope of the data-storage system；By the increment
In the range of data storage to the pre- subregion of the data-storage system.
In some embodiments of the invention, based on aforementioned schemes, in addition to：Receive data inquiry request；Based on described
Data query indexes the incidence relation between the pre- subregion scope, is determined to need what is retrieved according to the data inquiry request
Pre- subregion scope；In the range of the pre- subregion for needing to retrieve, the number for needing to inquire about is retrieved according to the data inquiry request
According to.
In some embodiments of the invention, based on aforementioned schemes, the step of obtaining the incremental data of database, including：
Obtain the Incremental Log of the database；The Incremental Log is parsed, to obtain the incremental data.
In some embodiments of the invention, based on aforementioned schemes, indexed based on the data query to data storage system
System is carried out the step of pre- multidomain treat-ment, including：The pre- subregion scope of the data-storage system is determined by random hash algorithm；
According to the pre- subregion scope of determination, carry out pre- multidomain treat-ment to the data-storage system, and by data query index with
The pre- subregion scope of the data-storage system is associated.
In some embodiments of the invention, based on aforementioned schemes, the incremental data is stored to the data storage
The step of in the range of the pre- subregion of system, including：The random hash algorithm formed based on data query index is by the increasing
Measure in the range of data storage to the pre- subregion.
In some embodiments of the invention, based on aforementioned schemes, the data-storage system is HBase, described pre- point
Area's scope is line unit scope.
In some embodiments of the invention, included based on aforementioned schemes, the incremental data：The database is carried out
The SQL statement of operation and associated metadata information.
According to the second aspect of the invention, there is provided a kind of data administrator, including：Acquiring unit, for obtaining number
According to the incremental data in storehouse；Generation unit, for generating data query index according to the incremental data；Processing unit, for base
Pre- multidomain treat-ment is carried out to data-storage system in data query index, wherein, the data query index and the number
It is associated according to the pre- subregion scope of storage system；Memory cell, for the incremental data to be stored to the data storage system
In the range of the pre- subregion of system.
According to the third aspect of the invention we, there is provided a kind of computer-readable medium, be stored thereon with computer program, institute
State the data managing method realized when program is executed by processor as described in above-mentioned first aspect.
According to the fourth aspect of the invention, there is provided a kind of electronic equipment, including：One or more processors；Storage dress
Put, for storing one or more programs, when one or more of programs are by one or more of computing devices, make
Obtain data managing method of one or more of processors realizations as described in above-mentioned first aspect.
In the technical scheme that some embodiments of the present invention are provided, by being indexed based on data query to data storage
System carries out pre- multidomain treat-ment, and data query index is associated with the pre- subregion scope of data-storage system, and by incremental number
According to storing to the pre- subregion of data-storage system so that when carrying out data query, data query can be based on and indexed
Data query scope is reduced with pre- subregion scope, and then carrys out quick search data in the range of the data query after diminution.Can
See, the technical scheme of the embodiment of the present invention can divide under the scene of table in database point storehouse, effectively improve efficiency data query, protect
Card database has good performance.
It should be appreciated that the general description and following detailed description of the above are only exemplary and explanatory, not
Can the limitation present invention.
Brief description of the drawings
Accompanying drawing herein is merged in specification and forms the part of this specification, shows the implementation for meeting the present invention
Example, and for explaining principle of the invention together with specification.It should be evident that drawings in the following description are only the present invention
Some embodiments, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis
These accompanying drawings obtain other accompanying drawings.In the accompanying drawings：
Fig. 1 diagrammatically illustrates the flow chart of data managing method according to first embodiment of the invention；
Fig. 2 diagrammatically illustrates the flow chart of the data managing method of second embodiment according to the present invention；
Fig. 3 diagrammatically illustrates the flow chart of the data managing method of the 3rd embodiment according to the present invention；
Fig. 4 diagrammatically illustrates the Organization Chart of data management system according to an embodiment of the invention；
Fig. 5 diagrammatically illustrates the block diagram of data administrator according to an embodiment of the invention.
Embodiment
Example embodiment is described more fully with referring now to accompanying drawing.However, example embodiment can be with a variety of shapes
Formula is implemented, and is not understood as limited to example set forth herein；On the contrary, these embodiments are provided so that the present invention will more
Fully and completely, and by the design of example embodiment comprehensively it is communicated to those skilled in the art.
In addition, described feature, structure or characteristic can be incorporated in one or more implementations in any suitable manner
In example.In the following description, there is provided many details fully understand so as to provide to embodiments of the invention.However,
It will be appreciated by persons skilled in the art that technical scheme can be put into practice without one or more in specific detail,
Or other methods, constituent element, device, step etc. can be used.In other cases, side known in being not shown in detail or describe
Method, device, realization are operated to avoid fuzzy each aspect of the present invention.
Block diagram shown in accompanying drawing is only functional entity, not necessarily must be corresponding with physically separate entity.
I.e., it is possible to realize these functional entitys using software form, or realized in one or more hardware modules or integrated circuit
These functional entitys, or these functional entitys are realized in heterogeneous networks and/or processor device and/or microcontroller device.
Flow chart shown in accompanying drawing is merely illustrative, it is not necessary to including all contents and operation/step,
It is not required to perform by described order.For example, some operation/steps can also decompose, and some operation/steps can close
And or partly merging, therefore the order actually performed is possible to be changed according to actual conditions.
Fig. 1 diagrammatically illustrates the flow chart of data managing method according to first embodiment of the invention.
Reference picture 1, data managing method according to first embodiment of the invention, including：
Step S10, obtain the incremental data of database.
In an embodiment of the present invention, when to database execution DML (Data Manipulation Language, data
Manipulation language) after operation, the incremental data of database can be obtained.Wherein, DML operations include the insertion of tables of data
(Insert) (delete) and renewal (update) operation etc., are deleted.In an embodiment of the present invention, described incremental data bag
Include：SQL (Structured Query Language, the SQL) sentences that are operated to the database and
Associated metadata information.
According to the exemplary embodiment of the present invention, step S10 is specifically included：Obtain the Incremental Log of the database；Solution
The Incremental Log is analysed, to obtain the incremental data.
Step S12, data query index is generated according to the incremental data.
In an embodiment of the present invention, the purpose of data query index is to be able to be quickly found out the number for needing to inquire about
According to.
Step S14, pre- multidomain treat-ment is carried out to data-storage system based on data query index, wherein, the number
It is associated with the pre- subregion scope of data-storage system according to search index.
According to the exemplary embodiment of the present invention, step S14 is specifically included：The data are determined by random hash algorithm
The pre- subregion scope of storage system；According to the pre- subregion scope of determination, pre- multidomain treat-ment is carried out to the data-storage system, and
The pre- subregion scope of data query index and the data-storage system is associated.
In this embodiment, by the way that the pre- subregion scope of data query index and data-storage system is associated, make
Obtain when receiving data inquiry request, can determine to need the pre- subregion scope retrieved according to data query index, and then
Data query scope can be reduced, to improve efficiency data query.
It should be noted that data-storage system can be database, for example can be a kind of HBase (distributed, faces
The PostgreSQL database of nematic).When data-storage system is HBase, above-mentioned pre- subregion scope is line unit (row key) model
Enclose.
Step S16, the incremental data is stored to the pre- subregion of the data-storage system.
According to the exemplary embodiment of the present invention, step S16 is specifically included：Based on the data query index form with
Machine hashing algorithm stores the incremental data to the pre- subregion.
In this embodiment, incremental data is stored to pre- by indexing the random hash algorithm formed based on data query
In the range of subregion, enabling be uniformly distributed to incremental data in the range of different pre- subregions, while ensure to store to difference
Data in the range of pre- subregion can be associated with data query index.
Based on the data managing method shown in Fig. 1, in an embodiment of the present invention, as shown in Fig. 2 also including：
Step S20, receive data inquiry request.
Step S22, based on the incidence relation between data query index and the pre- subregion scope, according to the number
Determine to need the pre- subregion scope retrieved according to inquiry request.
Step S24, in the range of the pre- subregion for needing to retrieve, need to inquire about according to data inquiry request retrieval
Data.
Below with least with application service layer, inquiry middleware, search server cluster, coprocessor and HBase collection
Exemplified by the system of group, the data management scheme of the embodiment of the present invention is described in detail with reference to Fig. 3 and Fig. 4：
Reference picture 3, and the system architecture with reference to shown in Fig. 4, according to the data management side of the 3rd embodiment of the present invention
Method, including：
Step S302, application service layer call database-driven to perform DML to database and operated, including tables of data
Insert, delete, update operation etc..
Step S304, the parsing of database journal increment.In an embodiment of the present invention, tactful fake data storehouse can be used
Slave dump agreements, read binary log simultaneously parse.
Such as can be by canal (increment of database binary log is subscribed to and consumption component) to mysql, oracle
Database carries out binary log (binary log) parsing.
Step 306, search index is imported search server cluster and safeguards secondary index by coprocessor.
In an embodiment of the present invention, when parsing incremental data by daily record analytical tool, (including sql and metadata are believed
Breath) after, search index can be obtained.Simultaneously as HBase's writes focus failure, it is necessary to do pre- multidomain treat-ment to HBase,
Than to carry out HBase clusters pre- multidomain treat-ment such as by random hash algorithm, pre- subregion rowkey definition rule is with looking into
It is associated to ask index, and then search index is imported into search server cluster by coprocessor and safeguards secondary index.It is therein
Secondary index is the index relative between search index and pre- subregion rowkey.
It should be noted that：It (is high flexible, High Availabitity, based on Apache that ElasticSearch, which can be selected, in search server
Lucene increase income search and analysis engine) or Solr (an independent enterprise-level search application server).
Step 308, incremental data is divided equally by the rowkey hashing algorithms that search index forms and inserts pre- subregion HBase
In cluster.It can either so ensure that incremental data is assigned in the range of different pre- subregions, and can enough ensures to pass through search index
Determine which pre- subregion is incremental data is stored in the range of.
Step 310, client initiates paging query request sql by inquiring about middleware.
Step 314, based on step 310, inquiry middleware is retrieved to search server cluster first, and receives search
The rowkeyList results that server cluster returns, the rowkey hashing algorithm rules for parsing rowKeyList narrow down to Hbase
The range of search of pre- subregion.
Step 316, Hbase clusters are entered in the range of the pre- subregions of HBase of diminution based on step 314, inquiry middleware
Row depth recursive query and amalgamation result collection, are back to application service layer.
In the embodiment shown in Fig. 3 and Fig. 4, under the background of table can be divided in point storehouse, a set of database journal solution is disposed
Analyse platform and carry out Incremental Log parsing, a set of search server cluster stores rowkey and index data, a set of Hbase clusters
Data storage file.During daily record increment is parsed, current delta data are parsed and in search service by coprocessor
Index relative recording is preserved in device, while the rowKey data by certain rule definition are preserved into Hbase clusters.Work as application
When inquiry request is initiated at end, inquiry middleware request condition in search server is inquired about and returns to rowKeyList, and then right
The pre- subregions of Hbase to reduce the scope carry out depth recursive query, obtain result set.
Fig. 5 diagrammatically illustrates the block diagram of data administrator according to an embodiment of the invention.
Reference picture 5, data administrator 500 according to an embodiment of the invention, including：Acquiring unit 502, generation unit
504th, processing unit 506 and memory cell 508.
Specifically, acquiring unit 502 is used for the incremental data for obtaining database；Generation unit 504 is used for according to the increasing
Measure data generation data query index；Processing unit 506 is used to carry out data-storage system based on data query index
Pre- multidomain treat-ment, wherein, the data query index is associated with the pre- subregion scope of the data-storage system；Memory cell
508 are used to store the incremental data to the pre- subregion of the data-storage system.
In some embodiments of the invention, also included based on aforementioned schemes, described data administrator 500：Receive
Unit (not shown in Fig. 5) and query unit (not shown in Fig. 5).Specifically, receiving unit is used to receive data inquiry request；
Query unit is used to, based on the incidence relation between data query index and the pre- subregion scope, be looked into according to the data
Ask request to determine to need the pre- subregion scope retrieved, and be used in the range of the pre- subregion for needing to retrieve, according to the number
The data for needing to inquire about according to inquiry request retrieval.
In some embodiments of the invention, it is configured to based on aforementioned schemes, acquiring unit 502：Obtain the database
Incremental Log；The Incremental Log is parsed, to obtain the incremental data.
In some embodiments of the invention, it is configured to based on aforementioned schemes, processing unit 506：Calculated by random hash
Method determines the pre- subregion scope of the data-storage system；According to the pre- subregion scope of determination, the data-storage system is entered
The pre- multidomain treat-ment of row, and the pre- subregion scope of data query index and the data-storage system is associated.
In some embodiments of the invention, it is configured to based on aforementioned schemes, memory cell 508：Looked into based on the data
The random hash algorithm that index is formed is ask to store the incremental data to the pre- subregion.
In some embodiments of the invention, based on aforementioned schemes, the data-storage system is HBase, described pre- point
Area's scope is line unit scope.
In some embodiments of the invention, included based on aforementioned schemes, the incremental data：The database is carried out
The SQL statement of operation and associated metadata information.
In addition, embodiments of the present invention also provide a kind of electronic equipment, can include：One or more processors；Deposit
Storage device, for storing one or more programs, when one or more of programs are by one or more of computing devices
When so that one or more of processors realize the data managing method described in the above embodiment of the present invention.
In the exemplary embodiment, embodiments of the invention additionally provide a kind of computer-readable medium, and the computer can
It can be included in the electronic equipment described in above-described embodiment to read medium；Can also be individualism, and without supplying
In the electronic equipment.Above computer computer-readable recording medium carries one or more program, when said one or multiple programs
When being performed by the electronic equipment so that the electronic equipment can realize the data management side as described in above-mentioned embodiment
Method.Such as, it is possible to achieve as shown in Figure 1：Step S10, obtain the incremental data of database；Step S12, according to the increasing
Measure data generation data query index；Step S14, data-storage system is carried out at pre- subregion based on data query index
Reason, wherein, the data query index is associated with the pre- subregion scope of the data-storage system；Step S16, by the increasing
Measure in the range of data storage to the pre- subregion of the data-storage system.And for example, can also realize as shown in Figures 2 and 3
Step.
It is situated between it should be noted that the computer-readable medium described in the embodiment of the present invention can be computer-readable signal
Matter or computer-readable recording medium either the two any combination.Computer-readable recording medium for example can be with
System, device or the device of --- but being not limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, or it is any more than
Combination.The more specifically example of computer-readable recording medium can include but is not limited to：With one or more wires
Electrical connection, portable computer diskette, hard disk, random access storage device (RAM), read-only storage (ROM), erasable type may be programmed
Read-only storage (EPROM or flash memory), optical fiber, portable compact disc read-only storage (CD-ROM), light storage device, magnetic are deposited
Memory device or above-mentioned any appropriate combination.In an embodiment of the present invention, computer-readable recording medium can be appointed
What is included or the tangible medium of storage program, the program can be commanded execution system, device either device use or and its
It is used in combination.And in an embodiment of the present invention, computer-readable signal media can be included in a base band or as load
The data-signal that a ripple part is propagated, wherein carrying computer-readable program code.The data-signal of this propagation can be with
Take various forms, including but not limited to electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable letter
Number medium can also be any computer-readable medium beyond computer-readable recording medium, and the computer-readable medium can be with
Send, propagate and either transmit for by the use of instruction execution system, device or device or program in connection.Meter
The program code included on calculation machine computer-readable recording medium can be transmitted with any appropriate medium, be included but is not limited to：Wirelessly, electric wire,
Optical cable, RF etc., or above-mentioned any appropriate combination.
It should be noted that although some modules or list of the equipment for action executing are referred in above-detailed
Member, but this division is not enforceable.In fact, according to the embodiment of the present invention, it is above-described two or more
Either the feature of unit and function can embody module in a module or unit.A conversely, above-described mould
Either the feature of unit and function can be further divided into being embodied by multiple modules or unit block.
Through the above description of the embodiments, those skilled in the art is it can be readily appreciated that example described herein is implemented
Mode can be realized by software, can also be realized by way of software combines necessary hardware.Therefore, according to the present invention
The technical scheme of embodiment can be embodied in the form of software product, the software product can be stored in one it is non-volatile
Property storage medium (can be CD-ROM, USB flash disk, mobile hard disk etc.) in or network on, including some instructions are to cause a calculating
Equipment (can be personal computer, server, touch control terminal or network equipment etc.) is performed according to embodiment of the present invention
Method.
Those skilled in the art will readily occur to the present invention its after considering specification and putting into practice invention disclosed herein
Its embodiment.The application be intended to the present invention any modification, purposes or adaptations, these modifications, purposes or
Person's adaptations follow the general principle of the present invention and including undocumented common knowledges in the art of the invention
Or conventional techniques.Description and embodiments are considered only as exemplary, and true scope and spirit of the invention are by following
Claim is pointed out.
It should be appreciated that the invention is not limited in the precision architecture for being described above and being shown in the drawings, and
And various modifications and changes can be being carried out without departing from the scope.The scope of the present invention is only limited by appended claim.
Claims (10)
- A kind of 1. data managing method, it is characterised in that including：Obtain the incremental data of database；Data query index is generated according to the incremental data；Pre- multidomain treat-ment is carried out to data-storage system based on data query index, wherein, the data query index with The pre- subregion scope of the data-storage system is associated；The incremental data is stored to the pre- subregion of the data-storage system.
- 2. data managing method according to claim 1, it is characterised in that also include：Receive data inquiry request；It is true according to the data inquiry request based on the incidence relation between data query index and the pre- subregion scope The pre- subregion scope retrieved is needed calmly；In the range of the pre- subregion for needing to retrieve, the data for needing to inquire about are retrieved according to the data inquiry request.
- 3. data managing method according to claim 1, it is characterised in that the step of obtaining the incremental data of database, Including：Obtain the Incremental Log of the database；The Incremental Log is parsed, to obtain the incremental data.
- 4. data managing method according to claim 1, it is characterised in that data are deposited based on data query index Storage system is carried out the step of pre- multidomain treat-ment, including：The pre- subregion scope of the data-storage system is determined by random hash algorithm；According to the pre- subregion scope of determination, pre- multidomain treat-ment is carried out to the data-storage system, and by the data query rope Draw and be associated with the pre- subregion scope of the data-storage system.
- 5. data managing method according to claim 1, it is characterised in that store the incremental data to the data The step of in the range of the pre- subregion of storage system, including：The random hash algorithm formed based on data query index is stored the incremental data to the pre- subregion scope It is interior.
- 6. data managing method according to any one of claim 1 to 5, it is characterised in that the data-storage system For HBase, the pre- subregion scope is line unit scope.
- 7. data managing method according to any one of claim 1 to 5, it is characterised in that the incremental data includes： The SQL statement and associated metadata information operated to the database.
- A kind of 8. data administrator, it is characterised in that including：Acquiring unit, for obtaining the incremental data of database；Generation unit, for generating data query index according to the incremental data；Processing unit, for carrying out pre- multidomain treat-ment to data-storage system based on data query index, wherein, the number It is associated with the pre- subregion scope of the data-storage system according to search index；Memory cell, for the incremental data to be stored to the pre- subregion of the data-storage system.
- 9. a kind of computer-readable medium, is stored thereon with computer program, it is characterised in that described program is executed by processor Data managing methods of the Shi Shixian as any one of claim 1 to 7.
- 10. a kind of electronic equipment, it is characterised in that including：One or more processors；Storage device, for storing one or more programs, when one or more of programs are by one or more of processing When device performs so that one or more of processors realize the data management side as any one of claim 1 to 7 Method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710896009.9A CN107679158A (en) | 2017-09-28 | 2017-09-28 | Data managing method, device, computer-readable medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710896009.9A CN107679158A (en) | 2017-09-28 | 2017-09-28 | Data managing method, device, computer-readable medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107679158A true CN107679158A (en) | 2018-02-09 |
Family
ID=61138218
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710896009.9A Pending CN107679158A (en) | 2017-09-28 | 2017-09-28 | Data managing method, device, computer-readable medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107679158A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110309169A (en) * | 2018-04-12 | 2019-10-08 | 腾讯科技（北京）有限公司 | Data query method, apparatus, medium and the electronic equipment of relation chain storage system |
| CN110674152A (en) * | 2019-09-24 | 2020-01-10 | 京东数字科技控股有限公司 | Data synchronization method and device, storage medium and electronic equipment |
| CN113312353A (en) * | 2021-06-10 | 2021-08-27 | 中国民航信息网络股份有限公司 | Storage method and system for tracking journal |
| CN113961580A (en) * | 2021-12-22 | 2022-01-21 | 联通智网科技股份有限公司 | Data query method, service system and electronic equipment |
| CN114943021A (en) * | 2022-07-20 | 2022-08-26 | 之江实验室 | TB-level incremental data screening method and device |
| CN113312353B (en) * | 2021-06-10 | 2024-06-04 | 中国民航信息网络股份有限公司 | Storage method and system for tracking belt log |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103617176A (en) * | 2013-11-04 | 2014-03-05 | 广东电子工业研究院有限公司 | Method for achieving automatic synchronization of multi-source heterogeneous data resources |
| CN105787118A (en) * | 2016-03-25 | 2016-07-20 | 武汉工程大学 | Design method and query method for HBase secondary index |
| CN106777258A (en) * | 2016-12-28 | 2017-05-31 | 银江股份有限公司 | The coding and compression method of Hbase line units in a kind of medical big data storage |
-
2017
- 2017-09-28 CN CN201710896009.9A patent/CN107679158A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103617176A (en) * | 2013-11-04 | 2014-03-05 | 广东电子工业研究院有限公司 | Method for achieving automatic synchronization of multi-source heterogeneous data resources |
| CN105787118A (en) * | 2016-03-25 | 2016-07-20 | 武汉工程大学 | Design method and query method for HBase secondary index |
| CN106777258A (en) * | 2016-12-28 | 2017-05-31 | 银江股份有限公司 | The coding and compression method of Hbase line units in a kind of medical big data storage |
Non-Patent Citations (2)
| Title |
|---|
| AINHOA AZQUETA-ALZ´UAZ等: "Massive Data Load on Distributed Database Systems over HBase", 《2017 17TH IEEE/ACM INTERNATIONAL SYMPOSIUM ON CLUSTER, CLOUD AND GRID COMPUTING》 * |
| 许俊东: "基于HBase的电信网管告警存储方式设计和实施", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110309169A (en) * | 2018-04-12 | 2019-10-08 | 腾讯科技（北京）有限公司 | Data query method, apparatus, medium and the electronic equipment of relation chain storage system |
| CN110674152A (en) * | 2019-09-24 | 2020-01-10 | 京东数字科技控股有限公司 | Data synchronization method and device, storage medium and electronic equipment |
| CN113312353A (en) * | 2021-06-10 | 2021-08-27 | 中国民航信息网络股份有限公司 | Storage method and system for tracking journal |
| CN113312353B (en) * | 2021-06-10 | 2024-06-04 | 中国民航信息网络股份有限公司 | Storage method and system for tracking belt log |
| CN113961580A (en) * | 2021-12-22 | 2022-01-21 | 联通智网科技股份有限公司 | Data query method, service system and electronic equipment |
| CN114943021A (en) * | 2022-07-20 | 2022-08-26 | 之江实验室 | TB-level incremental data screening method and device |
| US11789639B1 (en) | 2022-07-20 | 2023-10-17 | Zhejiang Lab | Method and apparatus for screening TB-scale incremental data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107679158A (en) | Data managing method, device, computer-readable medium and electronic equipment | |
| US9275422B2 (en) | Distributed k-core view materialization and maintenance for graphs | |
| US9680830B2 (en) | Evaluating security of data access statements | |
| CN105989076A (en) | Data statistical method and device | |
| US10929397B2 (en) | Forecasting query access plan obsolescence | |
| CN107704202B (en) | Method and device for quickly reading and writing data | |
| US9251227B2 (en) | Intelligently provisioning cloud information services | |
| US20130262434A1 (en) | Querying spatial events in a vehicle network, and optimized querier | |
| CN105556517A (en) | Smart search refinement | |
| CN111314063A (en) | Big data information management method, system and device based on Internet of things | |
| CN117033424A (en) | Query optimization method and device for slow SQL (structured query language) statement and computer equipment | |
| CN105095515A (en) | Bucket dividing method, device and equipment supporting fast query of Map-Reduce output result | |
| CN103365923A (en) | Method and device for assessing partition schemes of database | |
| CN103902614B (en) | A kind of data processing method, equipment and system | |
| CN113761390A (en) | Method and system for analyzing attribute intimacy | |
| US20130332465A1 (en) | Database management device and database management method | |
| US20150248467A1 (en) | Real-time calculation, storage, and retrieval of information change | |
| CN111563107A (en) | Information recommendation method and device, electronic equipment and storage medium | |
| CN107562842A (en) | The guard method of resource data and equipment | |
| CN113495891A (en) | Data processing method and device | |
| CN111290927A (en) | Data monitoring method and device | |
| CN110703119B (en) | Method and device for evaluating battery health status | |
| CN110851438A (en) | Database index optimization suggestion and verification method and device | |
| CN109063199B (en) | Resource filtering method and device, electronic equipment and computer readable medium | |
| CN117331959A (en) | Access model processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180209 |