CN111797095B - Index construction method and JSON data query method - Google Patents
Index construction method and JSON data query method Download PDFInfo
- Publication number
- CN111797095B CN111797095B CN202010521663.3A CN202010521663A CN111797095B CN 111797095 B CN111797095 B CN 111797095B CN 202010521663 A CN202010521663 A CN 202010521663A CN 111797095 B CN111797095 B CN 111797095B
- Authority
- CN
- China
- Prior art keywords
- target
- query
- index
- keyword
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000010276 construction Methods 0.000 title abstract description 60
- 238000012216 screening Methods 0.000 claims abstract description 22
- 238000012545 processing Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 8
- 238000004458 analytical method Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 8
- 230000009471 action Effects 0.000 description 3
- 238000013500 data storage Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 239000002131 composite material Substances 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001680 brushing effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013497 data interchange Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an index construction method and a JSON data query method. The JSON data query method comprises the following steps: acquiring a query request, wherein the query request comprises a query target and a query condition, and the query condition comprises a first target keyword and a first target value corresponding to the first target keyword; according to the first target keyword, the first target value and the query target, searching data from JSON data stored in a memory to obtain a first query result; screening a first target index corresponding to a first target value from index data associated with the first target keyword; and obtaining a second query result from the JSON data stored in the disk according to the first target index and the query target. According to the embodiment of the invention, the query precision and the query efficiency of JSON data can be improved.
Description
Technical Field
The application relates to the field of data processing, in particular to an index construction method and a JSON data query method.
Background
JS object numbered musical notation (JavaScript Object Notation, JSON) is a lightweight data interchange format that stores and represents data in a text format that is completely independent of the programming language. JSON data is increasingly being used by various service systems due to its ease of use. Meanwhile, as JSON data is rapidly increased, a strong demand for using JSON data as a basic data type is generated. For example, the elastosearch is taken as a distributed search server, is an analysis type database, naturally supports JSON data types, and can conveniently process the semi-structured data. MySQL is a relational database management system of open source codes, is also an analytic database, and also supports JSON data types.
At present, under the scene of retrieving JSON data in an analysis type database, some databases do not support the construction of indexes for the JSON data, and a user is required to manually create index fields, so that the query efficiency is reduced. And because the newly written JSON data of the user has a delay for a period of time, the latest JSON data cannot be queried, and the query precision is reduced.
Disclosure of Invention
The embodiment of the invention provides an index construction method and a JSON data query method, which can improve the query precision and the query efficiency of JSON data.
In a first aspect, an embodiment of the present invention provides an index construction method, including:
acquiring target JSON data;
analyzing the target JSON data to obtain first target data with an original data type, wherein the first target data comprises a first keyword and a first value corresponding to the first keyword;
Determining a target index construction mode based on a corresponding relation between a pre-established value type and an index construction mode and a value type of a first value;
And constructing a target index corresponding to the first value based on a target index construction mode, wherein the target index is associated with the first key word and is used for indicating the storage position of target JSON data.
In a second aspect, an embodiment of the present invention provides a JSON data query method, including:
Acquiring a query request, wherein the query request comprises a query target and a query condition, and the query condition comprises a first target keyword and a first target value corresponding to the first target keyword;
According to the first target keyword, the first target value and the query target, searching data from JSON data stored in a memory to obtain a first query result;
screening a first target index corresponding to a first target value from index data associated with the first target keyword;
And obtaining a second query result from the JSON data stored in the disk according to the first target index and the query target.
In a third aspect, an embodiment of the present invention provides an index building apparatus, including:
The target JSON data acquisition module is used for acquiring target JSON data;
the analysis module is used for analyzing the target JSON data to obtain first target data with an original data type, wherein the first target data comprises a first keyword and a first value corresponding to the first keyword;
The target index construction mode determining module is used for determining a target index construction mode based on the corresponding relation between the pre-established value type and the index construction mode and the value type of the first value;
the index construction module is used for constructing a target index corresponding to the first value based on a target index construction mode, the target index is associated with the first key word, and the target index is used for indicating the storage position of target JSON data.
In a fourth aspect, an embodiment of the present invention provides a JSON data query device, including:
the query request acquisition module is used for acquiring a query request, wherein the query request comprises a query target and a query condition, and the query condition comprises a first target keyword and a first target value corresponding to the first target keyword;
The query module is used for searching data from the JSON data stored in the memory according to the first target keyword, the first target value and the query target to obtain a first query result;
The screening module is used for screening a first target index corresponding to the first target value from index data associated with the first target keyword;
and the query module is also used for obtaining a second query result from the JSON data stored in the disk according to the first target index and the query target.
In a fifth aspect, embodiments of the present invention provide a computing device comprising: a processor and a memory storing computer program instructions;
the processor, when executing the computer program instructions, implements the method as provided in the first or second aspect above.
In a sixth aspect, embodiments of the present invention provide a computer storage medium having stored thereon computer program instructions which, when executed by a processor, implement a method as provided in the first or second aspects above.
According to the index construction method provided by the embodiment of the application, after the target JSON data is obtained, the data is analyzed to obtain the first target data with the original data type, the first target data comprises the first key word and the first value corresponding to the first key word, then the corresponding target construction index mode of the first value is determined based on the corresponding relation between the pre-established value type and the index construction mode, and finally the index of the first value is constructed according to the mode. Therefore, the application determines the index corresponding to each first value through the corresponding relation between the pre-established value type and the index construction mode, and automatically constructs the index for the first values with different value types without the need of user intervention to designate an index field, thereby greatly simplifying the operation time of the user and improving the query efficiency.
In addition, the JSON data query method provided by the application can query most of JSON data stored in the disk based on the index of the first value and the query request constructed by the index construction method, and can utilize the query request to filter and query a small part of JSON data stored in the memory, so that the query method can search most of data stored in the disk and can search the latest data cached in the memory, thereby realizing comprehensive search of the stored JSON data, reducing query delay and improving query precision.
Drawings
In order to more clearly illustrate the technical solution of the embodiments of the present invention, the drawings that are needed to be used in the embodiments of the present invention will be briefly described, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of an analytical database according to an embodiment of the present application;
FIG. 2 is a flowchart of an index building method according to an embodiment of the present application;
FIG. 3 is a flowchart of an index building method according to another embodiment of the present application;
fig. 4 is a schematic flow chart of a JSON data query method provided by an embodiment of the present application;
FIG. 5 is a schematic flow diagram of JSON data storage provided by an embodiment of the present application;
FIG. 6 is a schematic diagram of an index building device according to an embodiment of the present application;
Fig. 7 is a schematic structural diagram of a JSON data query device provided by an embodiment of the present application;
fig. 8 is a structural schematic diagram of an exemplary hardware architecture of a computing device according to an embodiment of the invention.
Detailed Description
Features and exemplary embodiments of various aspects of the present invention will be described in detail below, and in order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely configured to illustrate the invention and are not configured to limit the invention. It will be apparent to one skilled in the art that the present invention may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the invention by showing examples of the invention.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
For a better understanding of the present invention, the implementation of the present invention will be described in detail below by means of specific examples, which are not intended to limit the scope of the present invention.
The analysis type database is a database facing analysis application, and is different from the traditional database, can perform work of on-line statistics, on-line analysis of data, immediate inquiry and the like on the data to discover information data value, and is an important branch of database products.
For example, some analytics databases may support real-time high concurrency online analytics cloud computing services for massive data so that users can conduct immediate multidimensional analytics perspective and exploration for billions of data in the millisecond level. The analysis type database has the capability of calculating mass data and extremely fast response, so that a user can flexibly search data among transients, quickly find the data value, and can be directly embedded into a service system to provide analysis service for the terminal user.
Fig. 1 is a schematic diagram of an architecture of an analytical database according to an embodiment of the present application. As shown in fig. 1, the analytical database includes a front-end node (Controller) and a back-end node (Worker).
The front-end node is used for carrying out data interaction with a user (Client) end accessing the analysis type database and is responsible for request receiving and result returning, such as database definition language (Data Definition Language, DDL)/data manipulation language (Data Manipulation Language, DML), data writing, query and the like.
The background node is used for storing data and inquiring calculation. Wherein, a plurality of front-end nodes and a plurality of background nodes are both provided for high availability.
When a user needs to store JSON data into a background node, the user sends a data storage request to a front-end node, and the front-end node checks the JSON data to be stored. And under the condition that the front end node determines that the JSON data to be stored meets the requirements, the data to be stored is sent to the background node. The background node stores data to be stored. JSON data stored on the background node provides queries externally.
For example, elastiscsearch natively supports JSON data. For fields in JSON data, firstly, determining the type corresponding to the field, and supporting static appointed type mapping or dynamic evaluation type mapping. After determining the field type, an index is built based on Lucene for each field, and then a query is provided externally. Where Lucene is a full text search engine toolkit. Since Lucene only supports near real-time retrieval and not real-time retrieval, the background storage index depends on the elastic search of Lucene and cannot provide real-time retrieval capability. The newly written data of the user has a delay for a period of time, or the latest data can be queried only by manually calling a specific application program interface, so that the query precision is lower.
However, the original JSON data type does not support the construction of an index, and the query is carried out by directly filtering each record, extracting the corresponding field to carry out expression evaluation judgment, so that the query efficiency is extremely low under a large data volume. MySQL provides a way to indirectly build a specified field index to speed up the query performance of specified fields. The user needs to create a virtual column for the fields to be indexed and then build the index on the virtual column. Although the index can be built for the fields in a virtual column mode to accelerate query, the method requires the user to perceive the index fields on the service, the fields needing to be indexed are added one by one, frequent changes are required when the service changes, and query efficiency is reduced.
In order to support the processing requirement of a user on the JSON semi-structured data, the analysis type database reduces the difficulty of the user in processing the semi-structured data and needs to further provide the capability of quickly and accurately searching the JSON. Based on the technical problems, the embodiment of the application provides an index construction method and a JSON data query method, so as to improve query accuracy and query efficiency of JSON data stored in an analysis type database.
In order to realize the query of the JSON data, the JSON data needs to be indexed, and therefore, the index construction method provided by the application is firstly described in detail by combining the drawings and the specific embodiments.
Fig. 2 is a flowchart of an index construction method according to an embodiment of the present application. Referring to fig. 2, the method includes steps 210 to 240.
Step 210, obtaining target JSON data.
Step 220, parsing the target JSON data to obtain first target data with an original data type, where the first target data includes a first keyword and a first value corresponding to the first keyword.
Step 230, determining the target index construction mode based on the corresponding relation between the pre-established value type and the index construction mode and the value type of the first value.
Step 240, constructing a target index corresponding to the first value based on the target index construction mode, wherein the target index is associated with the first keyword, and the target index is used for indicating a storage position of target JSON data.
According to the index construction method provided by the embodiment of the application, after the target JSON data is obtained, the data is analyzed to obtain the first target data with the original data type, the first target data comprises the first key word and the first value corresponding to the first key word, then the corresponding target construction index mode of the first value is determined based on the corresponding relation between the pre-established value type and the index construction mode, and finally the index of the first value is constructed according to the mode. Therefore, the application determines the index corresponding to each first value through the corresponding relation between the pre-established value type and the index construction mode, and automatically constructs the index for the first values with different value types without the need of user intervention to designate an index field, thereby greatly simplifying the operation time of the user and improving the query efficiency.
The specific implementation of each of steps 210 through 240 is described below.
A specific implementation of step 210 will first be described.
In some embodiments of the present invention, the target JSON data is JSON data that needs to be stored in the disk in accordance with a preset legal format condition.
For example, referring to fig. 1, when new JSON data needs to be stored in the background node, the user first sends the new JSON data that needs to be stored to the front-end node. The front-end node performs validity check on JSON data to be stored.
As an example, the front-end node determines whether the data format of JSON data to be stored meets a preset legal format condition. The front-end node determines that the data format of the JSON data to be stored meets the preset legal format condition, and if the new JSON data to be stored is legal, the JSON data is sent to the background node. And the background node acquires JSON data to be stored, which accords with the legitimacy, namely target JSON data. Then, the background node automatically builds an index for the target JSON data. It should be noted that, the background node also needs to store the target JSON data to provide query presentation.
As one example, target JSON data may be stored in a data list.
A specific implementation of step 220 is described below.
In computer science, there is one type of data called raw data type. The original data type refers to the most basic type of the first bit in the data. The original data type may be: a number, character, boolean type, object, array, or null.
The data of the original data types can be mutually nested to form the data of the composite type. As one example, the target JSON data for the composite type is:
{"UserID":11,"Name":{"FirstName":"Truly","LastName":"Zhu"},"Email":"xxxxmail.com"}。
The Value (Value) corresponding to the key Name is an object, that is, the object is nested with the object.
When the target JSON data is analyzed, if the target JSON data is nested, the target JSON data is directly disassembled to the last layer.
As one example, the target JSON data in the above example will be parsed into 4 first target data: "UserID" 11, "name. First name" Truly, "name. LastName" Zhu "and" Email "xxxxmail. Com.
Each first target data comprises a first keyword and a first value corresponding to the first keyword.
For example, the first key of the first target data "UserID" 11 is UserID, and the first value corresponding to the first key UserID is 11.
A specific implementation of step 230 is described below.
In the embodiment of the application, since the first values corresponding to different first keywords are different, in order to improve the query performance, an appropriate index construction mode can be selected for each first value according to the value type of the first value.
In some embodiments, to improve query performance, a correspondence between value types and index building manners may be pre-established to determine an appropriate index building manner for the first values of different value types.
In some embodiments, the pre-established correspondence between data types and index building methods includes at least one of the following correspondence:
The corresponding relation between the numerical value type and the multidimensional point data index constructing method, the corresponding relation between the character string type and the inverted index constructing method and the corresponding relation between the Boolean type and the bitmap index constructing method.
As an example, if the value type of the first value is a numerical value type, a multidimensional point data index constructing method may be selected to construct a target index of the first value. For example, the multidimensional point data index building method may be a one-dimensional BKD index building method.
As an example, if the value type of the first value is a string type, an inverted index construction method may be selected to construct a target index of the first value. The inverted index is different from the traditional index mode of searching the attribute of the record through the identification of the data record, and the inverted index is used for searching the data record through the attribute. That is, the reverse index construction method is to search based on the value of the keyword as the keyword, thereby acquiring the record associated with the keyword. This results in faster query speed than conventional line-by-line (page) scanning approaches to obtain query records.
As an example, if the value type of the first value is boolean, a bitmap index construction method may be selected to construct the target index of the first value. Because the number of the Boolean type is only "false" and "true", a simple and space-saving bitmap index construction method is selected, and the query efficiency can be improved by adopting the bitmap index.
The bitmap index is a database index built based on bitmaps and is suitable for column queries with a large number of repeated values. As one example, a bitmap index may be considered a bit sequence that stores a large number of bits, and responds to a query request by bitwise manipulation on the bit sequences, with the number of bits in each bit sequence being consistent with the number of rows in the data table.
A specific implementation of step 240 is described below.
For example, the database list includes 3 target JSON data, which is specifically as follows:
{"UserID":11,"Name":{"FirstName":"Truly","LastName":"Zhu"},"Email":"xxxxmail.com"},"Poweruser":"true";
{"UserID":12,"Name":{"FirstName":"San","LastName":"Zhang"},"Email":"yyyymail.com","Poweruser":"true"};
{"UserID":13,"Name":{"FirstName":"Si","LastName":"Li"},"Email":"zzzz mail.com","Poweruser":"false"}.
Wherein the first target JSON data exists in a first row in the database list, the second target JSON data exists in a second row in the database list, and the third target JSON data exists in a third row in the database list. Namely, the line number of the first target JSON data is 1, the line number of the second target JSON data is 2, and the line number of the third target JSON data is 3. Each JSON data is a JSIN record.
For example, the storage location of the target JSON data may be characterized by the line number of the target JSON data.
As an example, for the value type of the first value corresponding to the first keyword UserID in each target JSON data is a numerical value type, so the target index of the first value corresponding to each first keyword UserID can be constructed by using a one-dimensional BKD index construction method. The target indexes of all the first values corresponding to the first keyword UserID can be characterized by using one index tree. The method comprises the steps of storing a first value corresponding to a userID and a line number of JSON data where the first value is located in nodes in an index tree.
For the first value corresponding to the first keyword Email in each target JSON data, the value type is a character string type, so that an index of the first value corresponding to the Email can be constructed by using an inverted index construction method.
As an example, the target index corresponding to the first value xxxxmail.com is line number 1, the target index corresponding to the first value yyymail.com is line number 2, and the target index corresponding to the first value zzzzmail.com is line number 3.
In order to save storage space and improve index searching efficiency, index information of all first values corresponding to the keywords Eamil can be stored in association with the keywords Eamil, so that target indexes of each first value corresponding to the keywords Eamil can be quickly searched according to the keywords Eamil.
For example, the value type of the first value corresponding to the first key Poweruser in each target JSON data is boolean type, so the target index of the first value corresponding to the first key Poweruser can be constructed using a bitmap index construction method.
For the above example, the index for the first value corresponding to the first key Poweruser may be {110}, where the number of bits in the index represents the row number corresponding to the first value, 1 represents the first value corresponding to Poweruser as true, and 0 represents the first value corresponding to Poweruser as false.
That is, {110} represents that the first value corresponding to Poweruser in the JSON data of the first and second lines is true, and the first value corresponding to Poweruser in the JSON data of the third line is false.
In order to save storage space and improve index searching efficiency, index information of all first values corresponding to a first keyword can be stored in association with the first keyword, so that target indexes of all first values corresponding to the first keyword can be quickly searched according to the first keyword.
In the embodiment of the application, the target index is constructed for each first value in the JSON data by selecting the proper index construction mode according to the value type of each first value, so that the query efficiency of the JSON data can be improved.
In other embodiments of the present application, in order to further improve the query efficiency, after step 220, the index building method provided by the embodiment of the present application further includes:
And taking the preset field as an index of the first keyword, wherein the preset field is used for indicating that the first keyword exists in the target JSON data.
In some embodiments, the user may need to first query whether a first keyword is included in JSON data stored in the database, i.e., whether the first keyword exists, and thus index the first keyword for querying the keyword null/not null query.
As one example, the following is directed to this JSON data record:
{"UserID":11,"Name":{"FirstName":"Truly","LastName":"Zhu"},"Email":"xxxxmail.com"}”，
the first key included in the record includes: userID, name.FirstName, name.LastName, email.
Thus, the preset field field_not_ nul may be used as an index to all first keywords (UserID, name.FirstName, name.LastName, email) in the record. Wherein the preset field field_not_ nul is used to characterize the first key that exists in the record.
That is, in addition to constructing an index for each first value, the first key present in the per-entry tag JSON data is additionally recorded as a value to be indexed, the index being in a special field to provide a null/non null query of the field.
Fig. 3 is a flowchart of an index building method according to another embodiment of the present application. As shown in fig. 3, the background node receives a legitimate JSON data record. After receiving the legal JSON data record, the original JSON data is stored. An index of the JSON data is then constructed. Firstly, JSON data is analyzed to obtain first target data with an original data type. The first target data includes a first key and a first value corresponding to the first key. And then determining a target index construction mode corresponding to the first value according to the value type of the first value in the first target data. Then, the target index of the first value is adaptively constructed according to a target index constructing mode. Then, the first keyword in the JSON data is used as a value, and a preset field is used as an index of the first keyword, so that a user inquires whether the keyword exists.
The index construction method provided by the application is introduced above, and a specific implementation manner of the JSON data query method provided by the embodiment of the application is introduced below by combining a specific embodiment with a drawing.
Fig. 4 is a flow chart of a JSON data query method provided by an embodiment of the present application. As shown in fig. 4, the method includes steps 410-440.
Step 410, obtaining a query request, wherein the query request comprises a query target and a query condition, and the query condition comprises a first target keyword and a first target value corresponding to the first target keyword;
Step 420, according to the first target keyword, the first target value and the query target, searching data from JSON data stored in the memory to obtain a first query result;
step 430, screening out a first target index corresponding to the first target value from index data associated with the first target keyword;
step 440, obtaining a second query result from JSON data stored in the disk according to the first target index and the query target.
In the embodiment of the application, in order to reduce the processing pressure of the background node, after the number of the JSON data in the memory reaches a certain threshold, the JSON data in the memory is flushed, i.e. stored in the disk. After the JSON data in the memory are stored in the disk, a target index is constructed for the first value corresponding to the first key in each JSON data according to the index construction method described above. For JSON data in the memory, i.e. JSON data without brushing, the query cannot be performed based on the index information.
It should be noted that, the order of querying the JSON data in the memory and querying the JSON data in the disk is not particularly limited, that is, step 420 may be performed before step 430 or after step 440. The execution sequence of step 420 and steps 430 and 440 is not particularly limited.
Fig. 5 is a schematic flow chart of JSON data storage according to an embodiment of the present application. As shown in fig. 5, the background node determines whether the JSON data record number in the memory reaches a preset number threshold. As one example, the threshold may be 8000. If the number of JSON data records in the memory reaches a preset number threshold, the JSON data in the memory is persisted to a disk, and then indexes of the JSON data are constructed. If the number of JSON data records in the memory does not reach the preset number threshold, the JSON data in the memory is not persisted to the disk, and the next judging period is waited to judge that the number of JSON data records in the memory reaches the preset number threshold.
In the embodiment of the application, most of JSON data stored in the disk can be queried based on the index of the first value and the query request constructed by the index construction method, and the query request can be utilized to filter and query a small part of JSON data stored in the memory, so that the query method can search most of data stored in the disk and can search the latest data cached in the memory, thereby realizing comprehensive search of the stored JSON data, reducing query delay and improving query precision.
The specific implementation of steps 410 and 420, respectively, is described in detail below.
A specific implementation of step 410 will first be described.
In some embodiments of the application, the query request may be a query statement: select from table where json _extract. Where the following contents of select are query targets and json_extract is followed by query conditions.
In some embodiments of the present application, the query criteria may generally include a first target keyword and a first target value corresponding to the first target keyword.
In other embodiments, the query request may also include a plurality of query terms having a logical relationship
In some embodiments of the present application, the query target may be the entire JSON data where the query condition is located, or may be data corresponding to a second target keyword preset in the entire JSON data where the query condition is located, or data matching a second target value corresponding to the second target keyword preset and the second target keyword in the entire JSON data where the query condition is located.
As one example, the query request may be ：select json_extract(col,'$.name.firstname')from table where json_extract(col,'$.Email')＝xxxxmail.com.
The first target keyword is Email, and the first target value is xxxxmail.
The query condition represents a query from each JSON data record with a value of xxxxmail.com corresponding to Email, and the query target is a value corresponding to name.
That is, the query request wants to query all data in the column of name first name key in each JSON data record of which Email is xxxfmai.
A specific implementation of step 420 is described below.
In step 420, according to the first target keyword, the first target value and the query target, a data search is performed from JSON data stored in the memory, so as to obtain a first query result.
In some embodiments of the present application, according to the first target keyword, the first target value, and the query target, performing data lookup from JSON data stored in the memory, to obtain a first query result may include: for each JSON data stored in memory, steps a and B are performed.
And step A, analyzing the JSON data to obtain second target data with the original data type, wherein the second target data comprises a second keyword and a value corresponding to the second keyword.
And B, searching a first query result matched with the query target from the JSON data under the condition that a second keyword matched with the first target keyword exists in the JSON data and a value corresponding to the second keyword matched with the first target keyword is matched with the first target value.
For JSON data stored in the memory, index information is not yet completely established, but in order to improve query accuracy, data search needs to be performed on JSON data stored in the memory.
The specific implementation of step a is similar to that of step 220, and will not be described here again.
In the step B, the first target keyword is matched with each second keyword analyzed from the JSON data, and whether the second keywords matched with the first target keyword exist in the JSON data is judged.
As an example, one JSON data stored in memory is:
{ "UserID":11, "Name": { "FirstName": "Truly", "LastName": "Zhu" }, "Email": "xxxxmail.com" }. After the JSON data is parsed, 4 second keywords can be obtained: userID, name.FirstName, name.LastName, email.
If the query request is ：select json_extract(col,'$.name.firstname')from table where json_extract(col,'$.Email')＝xxxxmail.com,, the first target keyword is Email. And matching the first target keyword with the 4 second keywords, so that a second keyword Email matched with the first target keyword can be obtained. The second keyword matching the first target keyword is the same second keyword as the first target keyword.
After determining that a second keyword matched with the first target keyword exists in the JSON data, judging whether a value corresponding to the second keyword is matched with the first target value. It should be noted that, whether the value corresponding to the second keyword matches the first target value is determined as whether the value corresponding to the second keyword is identical to the first target value.
Since the value corresponding to the second keyword Email is xxxfmai.com, it is determined that the value matches the first target value as xxxfmai.com. And then taking a value corresponding to a keyword name. First name in the JSON data as data in the first query result, namely 'Truly' is one data in the first query result.
And searching the data of each JSON data in the memory according to the process, so as to obtain a first query result.
In the embodiment of the application, the user can inquire the latest data by searching the data in the JSON data stored in the memory according to the first target keyword, the first target value and the inquiry target, thereby improving the inquiry precision.
It should be noted that, the execution order of the steps 430 and 440, and the steps a and B is not limited. That is, data searching from JSON data stored in the memory according to the query request and data searching from JSON data stored in the disk according to the query request can be processed in parallel.
A specific implementation of step 430 is described below.
In order to save storage space and improve index searching efficiency, the description of step 240 may store index information of all first values corresponding to a first keyword in association with the first keyword, so as to implement that, according to the first keyword, target indexes of all first values corresponding to the keyword may be quickly retrieved.
Thus, the first target key will be associated with index information for all values corresponding to the key in all JSON data stored in disk.
Referring to the above example, the database list includes 3 target JSON data, which is specifically as follows:
{"UserID":11,"Name":{"FirstName":"Truly","LastName":"Zhu"},"Email":"xxxxmail.com"},"Poweruser":"true";
{"UserID":12,"Name":{"FirstName":"San","LastName":"Zhang"},"Email":"yyyymail.com","Poweruser":"true"};
{"UserID":13,"Name":{"FirstName":"Si","LastName":"Li"},"Email":"zzzz mail.com","Poweruser":"false"}.
If the first target key is Email, the index data associated with Email includes an index corresponding to xxxxmail.com (i.e., line number 1), an index corresponding to yyyymal.com (i.e., line number 2), and an index corresponding to zzzzzmail.com (i.e., line number 3).
Since the first target value is xxxxmail.com, the index corresponding to xxxxmail.com, namely, the line number 1, namely, the first target index corresponding to the first target value can be screened out according to the first target value.
In step 440, for example, based on the first target index, the query request ：select json_extract(col,'$.name.firstname')from table where json_extract(col,'$.Email')＝xxxxmail.com. may find the JSON data of line 1, and then use the value Truly corresponding to the name.
In some embodiments of the application, the query goal includes a second goal keyword and a second target value corresponding to the second goal keyword. On this basis, step 440 includes step 4401 and step 4402. Step 4401, screening a second target index corresponding to a second target value from index data associated with the second target keyword; step 4402, querying a second query result corresponding to the query target from JSON data stored in the disk based on the first target index and the second target index.
In the embodiment of the present application, the specific implementation of step 4401 is similar to the specific implementation of step 430, and will not be described herein.
In step 4402, a second query result corresponding to the query target may be queried from JSON data stored in the disk according to the intersection of the first target index and the second target index.
As one example, the second target keyword is age, and the second target value is 12. Assuming that the second target indexes corresponding to the second target values are row numbers 1,2 and 3, in each row of the JSON data records from row 1 to row 3, a second target keyword age exists, and the second target value corresponding to the second target keyword is 12.
Assuming that the first target keyword is name, first name, the first target value is zhang, and the first target index corresponding to the first target value is row numbers 2,3,4,5, that is, in each row record of the 2nd row to 5 th row JSON data records, the first target keyword is name, first name, and the first target value corresponding to the first target keyword is zhang.
Then, the intersection of the first target index and the second target index, i.e., line numbers 2,3, is taken. And then taking the line 2 and line 3 JSON data as a second query result.
In some embodiments of the application, the query request includes a plurality of query terms having a logical relationship. On this basis, step 430 includes: for each first target keyword, screening out a first target index corresponding to a first target value from index data associated with the first target keyword; step 440 includes: based on the logic relation between each query condition, carrying out data processing on each first target index to obtain a third target index; and inquiring a second inquiry result corresponding to the inquiry target from the JSON data stored in the disk based on the third target index and the second target index.
In some embodiments of the present application, the query request includes a plurality of query conditions having a logical relationship, wherein for each query condition, the query condition includes a first target keyword and a first target value corresponding to the first target keyword.
As one example, the query request is a query from JSON data of (col, '$age')=12 and (col, '$sex')=man (col, '$name first name'). The query request includes two query conditions and the logical relationship between the two query conditions is and.
Then, in step 440, the first target index corresponding to the first target value 12 and the first target index corresponding to the first target value man are intersected according to the logical relationship of the two query conditions, so as to obtain a third target index.
And then, according to the intersection of the third target index and the second target index, inquiring a second inquiry result corresponding to the inquiry target from JSON data stored in the disk.
In other embodiments of the present application, the query request includes a plurality of query targets having a logical relationship; for each query objective, the query objective includes a second objective keyword and a second target value corresponding to the second objective keyword. On this basis, step 4401 includes: for each second target keyword, screening a second target index corresponding to a second target value from index data associated with the second target keyword; step 4402 includes: based on the logic relation between each query target, carrying out data processing on each second target index to obtain a fourth target index; and inquiring a second inquiry result corresponding to the inquiry target from the JSON data stored in the disk based on the fourth target index and the first target index.
As one example, the query request is a query from JSON data of (col, '$age')=12 (col, '$name. First name')=zhang or (col, '$sex')= woman. The query request includes two query targets and the logical relationship between the two query conditions is "or".
Then, in step 4402, the second target index corresponding to the second target value zhang and the second target index corresponding to the second target value woman are combined according to the logical relationship or of the two query targets, to obtain a fourth target index.
And then, according to the intersection of the fourth target index and the first target index, inquiring a second inquiry result corresponding to the inquiry target from JSON data stored in the disk.
In some embodiments of the application, the query request includes a plurality of query targets having a logical relationship; for each query target, the query target includes a second target keyword and a second target value corresponding to the second target keyword, and the query request includes a plurality of query conditions having a logical relationship.
Step 430 includes: for each first target keyword, a first target index corresponding to the first target value is screened out from index data associated with the first target keyword. .
Step 440 includes: based on the logic relation between each query condition, carrying out data processing on each first target index to obtain a third target index; for each second target keyword, screening a second target index corresponding to a second target value from index data associated with the second target keyword; based on the logic relation between each query target, carrying out data processing on each second target index to obtain a fourth target index; and inquiring a second inquiry result corresponding to the inquiry target from the JSON data stored in the disk based on the intersection of the third target index and the fourth target index.
In the embodiment of the application, the JSON data is parsed into the first target data with the original data type, and then a proper index mode is selected according to different value types, so that the index is automatically constructed for the first value in the first target data, the index field is not required to be specified by the user intervention, and the use of the user is greatly simplified. The use of different types of first values with the appropriate index type also improves the performance of the search. The method can achieve millisecond-level query delay and can meet the query requirement in full real time through the mode of most data index and minimum data filtering.
Fig. 6 is a schematic structural diagram of an index building device according to an embodiment of the present application. As shown in fig. 6, an index construction device 600 provided in an embodiment of the present application includes:
the target JSON data acquisition module 610 is configured to acquire target JSON data.
The parsing module 620 is configured to parse the target JSON data to obtain first target data having an original data type, where the first target data includes a first keyword and a first value corresponding to the first keyword.
The target index construction mode determining module 630 is configured to determine a target index construction mode based on a pre-established correspondence between a value type and an index construction mode, and a value type of the first value.
The index construction module 640 is configured to construct a target index corresponding to the first value based on a target index construction manner, where the target index is associated with the first key, and the target index is used to indicate a storage location of the target JSON data.
In the embodiment of the application, after target JSON data is acquired, the data is analyzed to obtain first target data with an original data type, the first target data comprises a first keyword and a first value corresponding to the first keyword, then a corresponding target construction index mode of the first value is determined based on a corresponding relation between a pre-established value type and an index construction mode, and finally an index of the first value is constructed according to the mode. Therefore, the application determines the index corresponding to each first value through the corresponding relation between the pre-established value type and the index construction mode, and automatically constructs the index for the first values with different value types without the need of user intervention to designate an index field, thereby greatly simplifying the operation time of the user and improving the query efficiency.
In some embodiments, the index building module 640 is further to:
And taking the preset field as an index of the first keyword, wherein the preset field is used for indicating that the first keyword exists in the target JSON data.
In some embodiments, the pre-established correspondence between data types and index building methods includes at least one of the following correspondence:
The corresponding relation between the numerical value type and the multidimensional point data index constructing method, the corresponding relation between the character string type and the inverted index constructing method and the corresponding relation between the Boolean type and the bitmap index constructing method.
Other details of the index construction device according to the embodiment of the present invention are similar to those of the index construction method according to the embodiment of the present invention described above in connection with fig. 2 to 3, and are not repeated here.
Fig. 7 is a schematic structural diagram of a JSON data query device provided by an embodiment of the present application. As shown in fig. 7, a JSON data query device 700 provided by an embodiment of the present application includes:
The query request acquiring module 710 is configured to acquire a query request, where the query request includes a query target and a query condition, and the query condition includes a first target keyword and a first target value corresponding to the first target keyword.
And a query module 720, configured to perform data search from JSON data stored in the memory according to the first target keyword, the first target value, and the query target, to obtain a first query result.
A screening module 730, configured to screen a first target index corresponding to the first target value from index data associated with the first target keyword;
The query module 720 is further configured to obtain a second query result from JSON data stored in the disk according to the first target index and the query target.
In the embodiment of the application, the index and the query request of the first value constructed based on the index construction method provided by the embodiment can be used for querying most of JSON data stored in the disk, and the query request can be used for filtering and querying a small part of JSON data stored in the memory, so that the query method can be used for searching most of data stored in the disk and searching the latest data cached in the memory, thereby realizing comprehensive searching of the stored JSON data, reducing query delay and improving query precision.
In some embodiments of the present invention, the query module 720 is configured to:
For each JSON data stored in memory, the following operations are performed:
analyzing the JSON data to obtain second target data with the original data type, wherein the second target data comprises a second keyword and a value corresponding to the second keyword;
and under the condition that a second keyword matched with the first target keyword exists in the JSON data and a value corresponding to the second keyword matched with the first target keyword is matched with the first target value, searching a first query result matched with the query target from the JSON data.
In some embodiments of the present invention, the query target includes a second target keyword and a second target value corresponding to the second target keyword; the query module 720 is specifically configured to:
screening a second target index corresponding to a second target value from index data associated with the second target keyword;
And inquiring a second inquiry result corresponding to the inquiry target from the JSON data stored in the disk based on the first target index and the second target index.
In some embodiments of the invention, the query request includes a plurality of query conditions having a logical relationship; the screening module 730 specifically is configured to:
For each first target keyword, screening out a first target index corresponding to a first target value from index data associated with the first target keyword;
wherein, the query module 720 is configured to:
based on the logic relation between each query condition, carrying out data processing on each first target index to obtain a third target index;
And inquiring a second inquiry result corresponding to the inquiry target from the JSON data stored in the disk based on the third target index and the second target index.
In some embodiments of the invention, the query request includes a plurality of query targets having a logical relationship; the query module 720 is specifically configured to:
for each second target keyword, screening a second target index corresponding to a second target value from index data associated with the second target keyword;
based on the logic relation between each query target, carrying out data processing on each second target index to obtain a fourth target index;
and inquiring a second inquiry result corresponding to the inquiry target from the JSON data stored in the disk based on the fourth target index and the first target index.
Other details of the JSON data query device according to the embodiment of the present invention are similar to those of the JSON data query method according to the embodiment of the present invention described above in connection with fig. 4 to 5, and are not described here again.
The index construction method, JSON data query method, index construction apparatus, and JSON data query apparatus described in connection with fig. 2-7 according to embodiments of the present invention may be implemented by computing device 800. Fig. 8 is a schematic diagram illustrating a hardware architecture 800 of a computing device according to an embodiment of the invention.
As shown in fig. 8, computing device 800 includes an input device 801, an input interface 802, a processor 803, a memory 804, an output interface 805, and an output device 806. The input interface 802, the processor 803, the memory 804, and the output interface 805 are connected to each other through a bus 810, and the input device 801 and the output device 806 are connected to the bus 810 through the input interface 802 and the output interface 805, respectively, and further connected to other components of the computing device 800.
Specifically, the input device 801 receives input information from the outside and transmits the input information to the processor 803 through the input interface 802; the processor 803 processes the input information based on computer executable instructions stored in the memory 804 to generate output information, temporarily or permanently stores the output information in the memory 804, and then transmits the output information to an output device 806 through an output interface 805; output device 806 outputs the output information to the outside of computing device 800 for use by a user.
The processor 803 may include: a central processing unit (Central Processing Unit, CPU), a network processor (Network Processing Unit, NPU), a tensor processor (Tensor Processing Unit, TPU), a field programmable gate array (Field Programmable GATE ARRAY, FPGA) chip, or an artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) chip, to name but a few.
That is, the computing device shown in FIG. 8 may also be implemented to include: a memory storing computer-executable instructions; and a processor, which can implement the index construction method or the JSON data query method provided in connection with the embodiment of the present invention when executing the computer executable instructions.
The embodiment of the invention also provides a computer storage medium, and the computer storage medium is stored with computer program instructions; when the computer program instruction is executed by the processor, the index construction method or the JSON data query method provided by the embodiment of the invention is realized.
The functional blocks shown in the above block diagrams may be implemented in hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the invention are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave. A "machine-readable medium" may include any medium that can store or transfer information. Examples of machine-readable media include electronic circuitry, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio Frequency (RF) links, and the like. The code segments may be downloaded via computer networks such as the internet, intranets, etc.
It should also be noted that the exemplary embodiments mentioned in this disclosure describe some methods or systems based on a series of steps or devices. The present invention is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, or may be performed in a different order from the order in the embodiments, or several steps may be performed simultaneously.
In the foregoing, only the specific embodiments of the present invention are described, and it will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the systems, modules and units described above may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein. It should be understood that the scope of the present invention is not limited thereto, and any equivalent modifications or substitutions can be easily made by those skilled in the art within the technical scope of the present invention, and they should be included in the scope of the present invention.
Claims (8)
1. A JSON data query method, wherein the method comprises:
acquiring a query request, wherein the query request comprises a query target and a query condition, and the query condition comprises a first target keyword and a first target value corresponding to the first target keyword;
According to the first target keyword, the first target value and the query target, performing data searching from JSON data stored in a memory to obtain a first query result;
screening a first target index corresponding to the first target value from index data associated with the first target keyword;
And obtaining a second query result from JSON data stored in a disk according to the first target index and the query target.
2. The method of claim 1, wherein the performing data lookup from JSON data stored in a memory according to the first target keyword, the first target value, and the query target to obtain a first query result includes:
For each JSON data stored in memory, the following operations are performed:
Analyzing the JSON data to obtain second target data with an original data type, wherein the second target data comprises a second keyword and a value corresponding to the second keyword;
And searching a first query result matched with the query target from the JSON data under the condition that a second keyword matched with the first target keyword exists in the JSON data and a value corresponding to the second keyword matched with the first target keyword is matched with the first target value.
3. The method of claim 1, wherein the query objective includes a second objective keyword and a second target value corresponding to the second objective keyword;
and searching data from JSON data stored in a disk according to the first target index and the query target to obtain a second query result, wherein the method comprises the following steps:
screening a second target index corresponding to the second target value from index data associated with the second target keyword;
And inquiring a second inquiry result corresponding to the inquiry target from JSON data stored in a disk based on the first target index and the second target index.
4. A method according to claim 3, wherein the query request includes a plurality of query conditions having a logical relationship;
the screening the first target index corresponding to the first target value from the index data associated with the first target keyword includes:
Screening a first target index corresponding to the first target value from index data associated with the first target key for each first target key;
The querying, based on the first target index and the second target index, a second query result corresponding to the query target from JSON data stored in a disk, includes:
based on the logic relation between the query conditions, carrying out data processing on each first target index to obtain a third target index;
And inquiring a second inquiry result corresponding to the inquiry target from JSON data stored in a disk based on the third target index and the second target index.
5. A method according to claim 3, wherein the query request includes a plurality of query targets having a logical relationship;
wherein the screening the second target index corresponding to the second target value from the index data associated with the second target keyword includes:
Screening a second target index corresponding to the second target value from index data associated with the second target key for each second target key;
The querying, based on the first target index and the second target index, a second query result corresponding to the query target from JSON data stored in a disk, includes:
Based on the logic relation between each query target, carrying out data processing on each second target index to obtain a fourth target index;
And inquiring a second inquiry result corresponding to the inquiry target from JSON data stored in a disk based on the fourth target index and the first target index.
6. A JSON data query device, wherein the device comprises:
The query request acquisition module is used for acquiring a query request, wherein the query request comprises a query target and a query condition, and the query condition comprises a first target keyword and a first target value corresponding to the first target keyword;
the query module is used for searching data from JSON data stored in a memory according to the first target keyword, the first target value and the query target to obtain a first query result;
the screening module is used for screening a first target index corresponding to the first target value from index data associated with the first target keyword;
the query module is further configured to obtain a second query result from JSON data stored in a disk according to the first target index and the query target.
7. A computing device, wherein the computing device comprises: a processor and a memory storing computer program instructions;
The processor, when executing the computer program instructions, implements the method of any of claims 1-5.
8. A computer storage medium having stored thereon computer program instructions which, when executed by a processor, implement the method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010521663.3A CN111797095B (en) | 2020-06-10 | 2020-06-10 | Index construction method and JSON data query method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010521663.3A CN111797095B (en) | 2020-06-10 | 2020-06-10 | Index construction method and JSON data query method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111797095A CN111797095A (en) | 2020-10-20 |
| CN111797095B true CN111797095B (en) | 2024-05-03 |
Family
ID=72804050
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010521663.3A Active CN111797095B (en) | 2020-06-10 | 2020-06-10 | Index construction method and JSON data query method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111797095B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463827B (en) * | 2020-11-16 | 2024-03-12 | 北京达佳互联信息技术有限公司 | Query method, query device, electronic equipment and storage medium |
| CN113792043A (en) * | 2021-08-24 | 2021-12-14 | 微梦创科网络科技（中国）有限公司 | Real-time data storage method and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104298690A (en) * | 2013-07-19 | 2015-01-21 | 国际商业机器公司 | Method and device for building index structure for relational database table and method and device for conducting inquiring |
| WO2017070188A1 (en) * | 2015-10-23 | 2017-04-27 | Oracle International Corporation | Efficient in-memory db query processing over any semi-structured data formats |
| CN107038207A (en) * | 2017-02-20 | 2017-08-11 | 阿里巴巴集团控股有限公司 | A kind of data query method, data processing method and device |
| CN107220072A (en) * | 2016-03-21 | 2017-09-29 | 阿里巴巴集团控股有限公司 | Configuration file method of calibration and device based on serialization format |
| CN110618983A (en) * | 2019-08-15 | 2019-12-27 | 复旦大学 | JSON document structure-based industrial big data multidimensional analysis and visualization method |
| CN110795395A (en) * | 2018-07-31 | 2020-02-14 | 阿里巴巴集团控股有限公司 | File deployment system and file deployment method |
-
2020
- 2020-06-10 CN CN202010521663.3A patent/CN111797095B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104298690A (en) * | 2013-07-19 | 2015-01-21 | 国际商业机器公司 | Method and device for building index structure for relational database table and method and device for conducting inquiring |
| WO2017070188A1 (en) * | 2015-10-23 | 2017-04-27 | Oracle International Corporation | Efficient in-memory db query processing over any semi-structured data formats |
| CN107220072A (en) * | 2016-03-21 | 2017-09-29 | 阿里巴巴集团控股有限公司 | Configuration file method of calibration and device based on serialization format |
| CN107038207A (en) * | 2017-02-20 | 2017-08-11 | 阿里巴巴集团控股有限公司 | A kind of data query method, data processing method and device |
| CN110795395A (en) * | 2018-07-31 | 2020-02-14 | 阿里巴巴集团控股有限公司 | File deployment system and file deployment method |
| CN110618983A (en) * | 2019-08-15 | 2019-12-27 | 复旦大学 | JSON document structure-based industrial big data multidimensional analysis and visualization method |
Non-Patent Citations (1)
| Title |
|---|
| 基于Hadoop的SQL查询引擎性能研究;吴黎兵;邱鑫;叶璐瑶;王晓栋;聂雷;;华中师范大学学报(自然科学版);20160415(02);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111797095A (en) | 2020-10-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102407510B1 (en) | Method, apparatus, device and medium for storing and querying data | |
| CN107729371B (en) | Data indexing and querying method, device, equipment and storage medium of block chain | |
| US7680821B2 (en) | Method and system for index sampled tablescan | |
| US20140310302A1 (en) | Storing and querying graph data in a key-value store | |
| CN104769586A (en) | Profiling data with location information | |
| US20050240570A1 (en) | Partial query caching | |
| KR101083563B1 (en) | Method and System for Managing Database | |
| CN110659282B (en) | Data route construction method, device, computer equipment and storage medium | |
| CN107783985B (en) | Distributed database query method, device and management system | |
| CN111797095B (en) | Index construction method and JSON data query method | |
| US20180129708A1 (en) | Query processing management in a database management system | |
| JP6198845B2 (en) | Active database query maintenance | |
| CN112269816B (en) | Government affair appointment correlation retrieval method | |
| CN112765282B (en) | Data online analysis processing method, device, equipment and storage medium | |
| CN109992603B (en) | Data searching method and device, electronic equipment and computer readable medium | |
| US20140019454A1 (en) | Systems and Methods for Caching Data Object Identifiers | |
| CN109815240B (en) | Method, apparatus, device and storage medium for managing index | |
| CN109597829B (en) | Middleware method for realizing searchable encryption relational database cache | |
| CN112579610A (en) | Multi-data source structure analysis method, system, terminal device and storage medium | |
| US20190034247A1 (en) | Creating alerts associated with a data storage system based on natural language requests | |
| KR101544560B1 (en) | An online analytical processing system for big data by caching the results and generating 2-level queries by SQL parsing | |
| US10380115B2 (en) | Cross column searching a relational database table | |
| CN113918605A (en) | Data query method, device, equipment and computer storage medium | |
| KR20230129555A (en) | Join JavaScript object notation (JSON) queries for cloud resources | |
| US10019483B2 (en) | Search system and search method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |