CN113407657A - Data query method, device, equipment and storage medium based on single-level database - Google Patents
Data query method, device, equipment and storage medium based on single-level database Download PDFInfo
- Publication number
- CN113407657A CN113407657A CN202110729441.5A CN202110729441A CN113407657A CN 113407657 A CN113407657 A CN 113407657A CN 202110729441 A CN202110729441 A CN 202110729441A CN 113407657 A CN113407657 A CN 113407657A
- Authority
- CN
- China
- Prior art keywords
- user information
- data query
- data
- level database
- identification field
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000004590 computer program Methods 0.000 claims description 12
- 238000011068 loading method Methods 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000004044 response Effects 0.000 claims description 4
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000007726 management method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 238000010276 construction Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000009938 salting Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
- G06F16/316—Indexing structures
- G06F16/319—Inverted lists
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the field of data query, and discloses a data query method based on a single-level database, which comprises the following steps: acquiring the identification field and the user information of each original data from the single-level database, and combining the identification field and the user information of each original data to form a plurality of user records; constructing an inverted index for each user record to obtain an inverted index library of the user records; responding to the data query request, querying user information of the data query request from the inverted index database to obtain target user information, and querying an identification field of the data query request in the single-level database according to the target user information to obtain a target identification field; and querying data in the single-level database according to the target identification field to obtain a data query result of the data query request. In addition, the invention also provides a data query device, electronic equipment and a storage medium based on the single-level database. The invention can improve the data query speed of the single-level database.
Description
Technical Field
The present invention relates to the field of data query, and in particular, to a method and an apparatus for querying data based on a single-level database, an electronic device, and a computer-readable storage medium.
Background
The single-level database refers to a database with a single-level index, which does not have multi-condition query and a second-level index mechanism, such as an Hbase database, wherein the Hbase database is a distributed dynamic mode database with a top and a bottom, and can support real-time reading and writing and random access of an ultra-large-scale data set.
At present, table data of a single-level database is dictionary-sorted according to RowKey, which can be understood as a first-level Index (Primary Index) of a data table in the single-level database, and because the single-level database does not have a second-level Index (Secondary Index) mechanism, when multi-condition data query is encountered, all fields which may be used as query conditions need to be spliced into the RowKey one by one, so that a plurality of query conditions need to be configured, limitation of the multi-condition data query is easily brought, and data query speed of the single-level database is influenced.
Disclosure of Invention
The invention provides a data query method and device based on a single-level database, electronic equipment and a computer readable storage medium, and mainly aims to improve the data query speed of the single-level database.
In order to achieve the above object, the present invention provides a data query method based on a single-level database, which includes:
acquiring an identification field and user information of each piece of original data from a single-level database, and combining the identification field and the user information of each piece of original data to form a plurality of user records;
constructing an inverted index for each user record to obtain an inverted index library of the user records;
responding to a data query request, querying user information of the data query request from the inverted index database to obtain target user information, and querying an identification field of the data query request in a single-level database according to the target user information to obtain a target identification field;
and querying data in the single-level database according to the target identification field to obtain a data query result of the data query request.
Optionally, the merging the identification field and the user information of each piece of original data to form a plurality of user records includes:
according to the information type of each piece of user information, carrying out field conversion on the user information to obtain a user information field of each piece of user information;
and combining each user information field and the corresponding identification field in the original data to generate a plurality of user records.
Optionally, the constructing an inverted index for each user record to obtain an inverted index library of the user record includes:
performing document indexing on each user record by using a distributed search engine to generate an index document;
establishing an inverted index for the identification field of the index document according to the user information in each user record to obtain the inverted index of each user record;
and summarizing the reverse indexes recorded by each user to generate a reverse index library.
Optionally, the querying, in response to a data query request, user information of the data query request from the inverted index library to obtain target user information includes:
and acquiring the user identification of the data query request, matching the user identification of the data query request with the user information in the inverted index database, and taking the successfully matched user information as the target user information.
Optionally, the querying, according to the target user information, an identification field of the data query request in the single-level database to obtain a target identification field includes:
inquiring the user record of the data inquiry request from the inverted index library according to the target user information;
and identifying the identification field of the data query request in the single-level database according to the user record of the data query request to obtain a target identification field.
Optionally, before acquiring the identification field and the user information of each piece of original data from the single-level database, the method further includes:
acquiring original data and corresponding user information, and carrying out format unification on the original data to obtain standard data;
creating a field identifier of the standard data in a single-level database, and hashing the created field identifier to obtain an identifier field of the standard data in the single-level database;
and storing the original data and the corresponding user information into the single-level database according to the identification field.
Optionally, the storing the original data and the corresponding user information into the single-level database according to the identification field includes:
establishing a storage directory of the original data and the corresponding user information in a storage medium in the single-level database according to the identification field;
and loading the original data and the corresponding user information into the single-level database according to the storage directory.
In order to solve the above problem, the present invention further provides a data query apparatus based on a single-level database, the apparatus comprising:
the user record generating module is used for acquiring the identification field and the user information of each original data from the single-level database, and combining the identification field and the user information of each original data to form a plurality of user records;
the reverse index building module is used for building a reverse index for each user record to obtain a reverse index library of the user records;
the identification field query module is used for responding to a data query request, querying user information of the data query request from the inverted index database to obtain target user information, and querying the identification field of the data query request in the single-level database according to the target user information to obtain a target identification field;
and the data query module is used for querying data in the single-level database according to the target identification field to obtain a data query result of the data query request.
In order to solve the above problem, the present invention also provides an electronic device, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to implement the single-level database-based data query method described above.
In order to solve the above problem, the present invention further provides a computer-readable storage medium, in which at least one computer program is stored, and the at least one computer program is executed by a processor in an electronic device to implement the single-level database-based data query method described above.
The method comprises the steps of obtaining an identification field (Rowkey) and user information of each original data from a single-level database, combining the identification field and the user information of each original data to form a plurality of user records, and ensuring the construction premise of a secondary index of a subsequent single-level database; secondly, the embodiment of the invention constructs the inverted index for each user record to obtain the inverted index database of the user record, and can support the query request of multi-condition data so as to ensure the second-round query of a single-level database and meet the real-time property of data query. Therefore, the data query method, the data query device, the electronic equipment and the computer-readable storage medium based on the single-level database can improve the data query speed of the single-level database.
Drawings
Fig. 1 is a schematic flow chart of a single-level database-based data query method according to an embodiment of the present invention;
FIG. 2 is a block diagram of a single-level database-based data query apparatus according to an embodiment of the present invention;
fig. 3 is a schematic internal structural diagram of an electronic device implementing a single-level database-based data query method according to an embodiment of the present invention;
the implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The embodiment of the application provides a data query method based on a single-level database. The executing body of the data query method based on the single-level database includes, but is not limited to, at least one of electronic devices, such as a server, a terminal, and the like, which can be configured to execute the method provided by the embodiments of the present application. In other words, the data query method based on the single-level database may be performed by software or hardware installed in the terminal device or the server device, and the software may be a block chain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like.
Fig. 1 is a schematic flow chart of a data query method based on a single-level database according to an embodiment of the present invention. In an embodiment of the present invention, the data query method based on a single-level database includes:
and S1, acquiring the identification field and the user information of each original data from the single-level database, and combining the identification field and the user information of each original data to form a plurality of user records.
In the embodiment of the present invention, the single-level database refers to a database with a single-level index that does not have multi-condition query and does not have a secondary index mechanism, such as an Hbase database, where the Hbase database is a distributed column-oriented open source database and can support storage of unstructured data. In an embodiment of the present invention, the raw data includes semi-structured data and unstructured data, such as text data and user data, the text data includes insurance product type and quantity, fund product type and quantity, stock product type and quantity, and the user data includes: name, gender, identity card, mobile phone number, and the like. Further, the identification field refers to a first-level index, also called Rowkey, of each original data in the single-level database, and it should be understood that, in the single-level database, the storage of the data is in a column-based form rather than a row-based form, so that a first-level index query of the single-level database can be implemented by the Rowkey, the user information refers to user identity information (such as identity card number, name, enterprise account, and the like) of each original data, and the user information can be used as a search condition for searching each original data.
In one embodiment of the present invention, before acquiring the identification field and the user information of each original datum from the single-level database, the method further includes: the method comprises the steps of obtaining original data and corresponding user information, carrying out format unification on the original data to obtain standard data, creating field identification of the standard data in a single-level database, carrying out hash processing on the created field identification to obtain identification fields of the standard data in the single-level database, and storing the original data and the corresponding user information into the single-level database according to the identification fields.
In an optional embodiment, the normalizing the format of the original data to obtain standard data includes: reading the original data by using a reading instruction in a Spark frame; judging whether the original data is legal or not through a detection instruction in the Spark frame; if the original data is illegal, receiving the rest original data; if the original data is legal, acquiring a data format of the original data through an acquisition instruction in the Spark frame, calling a conversion instruction in the Spark frame to convert the data format of the original data into a key-value pair format, and outputting the original data in the key-value pair format according to an output instruction in the Spark frame to obtain standard data.
Wherein the Spark framework refers to a fast and general-purpose computing engine designed specifically for large-scale data processing. The read instruction comprises RDD.saveAsNewAPIHadooPfile, the detect instruction comprises checkOutputSpecs, the get instruction comprises getRecordWriter, the convert instruction comprises getWriter, and the output instruction comprises HFileOutputFormat.
In an alternative embodiment, the key field of the standard data is used as the identification field of the standard data. For example, if the standard data is that the Chinese achievement of a certain Student is 98 points, the fields "Student", "Chinese" and "achievement" are selected as the identification fields of the standard data. Further, in order to make the created identification field unique, the embodiment of the present invention further includes converting the format of the identification field into MD5 hash format, where MD5 hash refers to returning field values in the form of 32-bit 16-ary small-word strings.
It should be understood that when data is imported into the single-level database, a hot spot phenomenon exists in which the data load of a few data tables in the single-level database is too large and the loads of other data tables are very small, so that the load of a host where the hot spot data table is located is too large, performance is reduced, and even the data table is unusable. Therefore, according to another optional embodiment of the present invention, the identification field is hashed, so as to solve a hot spot phenomenon occurring when data is imported into the single-level database. Optionally, the hash processing includes: inversion processing, salting processing, and Hash hashing processing.
In an optional embodiment, the storing the original data and the corresponding user information into the single-level database according to the identification field includes: and establishing a storage directory of the original data and the corresponding user information in a storage medium in the single-level database according to the identification field, and loading the original data and the corresponding user information into the single-level database by using a preset loading method according to the storage directory. Optionally, the storage medium includes an hdfs (hadoop Distributed File system) Distributed storage system, and the preset loading method includes a loadincementalhfiles method.
Further, the embodiment of the present invention obtains the identification field (Rowkey) and the user information of each original data from the single-level database through the query statement. Optionally, the query statement includes an SQL statement.
Further, the merging the identification field and the user information of each piece of the original data to form a plurality of user records includes: according to the information type of each piece of user information, carrying out field conversion on the user information to obtain a user information field of each piece of user information; and combining each user information field and the corresponding identification field in the original data to generate a plurality of user records. Based on the generation of the user record, the basic information of the user and a primary index of data storage of the user in the single-level database can be inquired.
S2, constructing an inverted index for each user record to obtain an inverted index library of the user records.
In the embodiment of the invention, a reverse index is constructed for each user record through a distributed search engine, the distributed search engine comprises an elastic search, the construction of the reverse index based on the user record can realize the secondary index of the identification field in the user record in the single-level database, namely, the identification field which is pre-established in the single-level database is used as the primary index, the reverse index of the identification field is generated by the distributed search engine to be used as the secondary index, so as to support the subsequent multi-condition data query, and therefore, the single-level database can support the second-round query of the multi-condition data.
In an optional embodiment of the present invention, the constructing an inverted index for each user record by using a distributed search engine to obtain an inverted index library of the user records includes: and performing document indexing on each user record by using the distributed search engine to generate an index document, establishing a reverse index for the identification field of the index document according to the user information in each user record to obtain the reverse index of each user record, summarizing the reverse indexes of each user record, and generating a reverse index library.
The document index is a process of constructing a dictionary from the user records, and is used for integrating each user record, the inverted index is a process of constructing user information as an index directory, and constructing an identification field as index content, so as to form a corresponding relationship between the user information and the identification field, and the corresponding relationship includes: one-to-one correspondence, one-to-many correspondence, many-to-one correspondence, and many-to-many correspondence.
S3, responding to the data query request, querying the user information of the data query request from the inverted index database to obtain target user information, and querying the identification field of the data query request in the single-level database according to the target user information to obtain the target identification field.
In an embodiment of the present invention, the data query request includes at least two data query fields, for example, for a data query request of "querying sales data of insurance and insurance of user a", the data query fields include: in order to guarantee the second-round query of the data query request, the invention obtains the target user information by querying the user information of the data query request from the inverted index database, and obtains the identification field of the data query request in the single-level database according to the target user information, thereby realizing the multi-condition data query.
In an optional embodiment of the present invention, the querying, in response to a data query request, user information of the data query request from the inverted index library to obtain target user information includes: and acquiring the user identification of the data query request, matching the user identification of the data query request with the user information in the inverted index database, and taking the successfully matched user information as the target user information.
Further, in another optional embodiment of the present invention, the user identifier is matched with the user information in the inverted index library in the following manner;
wherein, X represents the field of the user identification, Y represents the field of the user information, m represents the number of the fields of the user identification, and n represents the number of the fields of the user information.
In an optional embodiment of the present invention, the querying, according to the target user information, an identification field of the data query request in the single-level database to obtain a target identification field includes: and inquiring the user record of the data inquiry request from the inverted index database according to the target user information, and identifying the identification field of the data inquiry request in the single-level database according to the user record to obtain a target identification field.
S4, inquiring data in the single-level database according to the target identification field to obtain a data inquiry result of the data inquiry request.
The embodiment of the invention takes the target identification field as the index of the single-level database, and inquires data in the single-level database so as to realize the data inquiry of the data inquiry request and obtain the data inquiry result.
In summary, in the embodiment of the present invention, an identification field (Rowkey) and user information of each piece of original data are obtained from a single-level database, and the identification field and the user information of each piece of original data are merged to form a plurality of user records, so as to ensure that a second-level index of a subsequent single-level database is constructed on the premise; secondly, the embodiment of the invention constructs the inverted index for each user record to obtain the inverted index database of the user record, and can support the query request of multi-condition data so as to ensure the second-round query of a single-level database and meet the real-time property of data query. Therefore, the data query method based on the single-level database can improve the data query speed of the single-level database.
Fig. 2 is a block diagram of a data query apparatus based on a single-level database according to an embodiment of the present invention.
The single-level database-based data query device 100 of the present invention can be installed in an electronic device. According to the implemented functions, the data query device based on the single-level database may include a user record generation module 101, an inverted index construction module 102, an identified bullet query module 103, and a data module 104. A module according to the present invention, which may also be referred to as a unit, refers to a series of computer program segments that can be executed by a processor of an electronic device and that can perform a fixed function, and that are stored in a memory of the electronic device.
In the present embodiment, the functions regarding the respective modules/units are as follows:
the user record generating module 101 is configured to obtain an identifier field and user information of each piece of original data from a single-level database, and merge the identifier field and the user information of each piece of original data to form a plurality of user records;
the inverted index constructing module 102 is configured to construct an inverted index for each user record to obtain an inverted index library of the user records;
the identification field query module 103 is configured to, in response to a data query request, query user information of the data query request from the inverted index database to obtain target user information, and query an identification field of the data query request in the single-level database according to the target user information to obtain a target identification field;
the data query module 104 is configured to query data from the single-level database according to the target identification field, so as to obtain a data query result of the data query request.
In detail, when the modules in the single-level database-based data query apparatus 100 according to the embodiment of the present invention are used, the same technical means as the single-level database-based data query method described in fig. 1 above are adopted, and the same technical effects can be produced, which is not described herein again.
Fig. 3 is a schematic structural diagram of an electronic device implementing a single-level database-based data query method according to an embodiment of the present invention.
The electronic device 1 may comprise a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may further comprise a computer program, such as a single level database based data query program, stored in the memory 11 and executable on the processor 10.
In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same function or different functions, and includes one or more Central Processing Units (CPUs), a microprocessor, a digital Processing chip, a graphics processor, a combination of various control chips, and the like. The processor 10 is a Control Unit (Control Unit) of the electronic device 1, connects various components of the electronic device 1 by using various interfaces and lines, and executes various functions and processes data of the electronic device 1 by running or executing programs or modules (for example, executing a data query program based on a single-level database, etc.) stored in the memory 11 and calling data stored in the memory 11.
The memory 11 includes at least one type of readable storage medium including flash memory, removable hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disks, optical disks, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device 1, such as a removable hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the electronic device 1. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 may be used not only to store application software installed in the electronic device 1 and various types of data, such as codes of a data query program based on a single-level database, etc., but also to temporarily store data that has been output or is to be output.
The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is arranged to enable connection communication between the memory 11 and at least one processor 10 or the like.
The communication interface 13 is used for communication between the electronic device 1 and other devices, and includes a network interface and a user interface. Optionally, the network interface may include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), which are generally used for establishing a communication connection between the electronic device 1 and other electronic devices 1. The user interface may be a Display (Display), an input unit such as a Keyboard (Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic device 1 and for displaying a visualized user interface, among other things.
Fig. 3 shows only the electronic device 1 with components, and it will be understood by those skilled in the art that the structure shown in fig. 3 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than those shown, or some components may be combined, or a different arrangement of components.
For example, although not shown, the electronic device 1 may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so as to implement functions of charge management, discharge management, power consumption management, and the like through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device 1 may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
The single-level database-based data query program stored in the memory 11 of the electronic device 1 is a combination of a plurality of computer programs, and when running in the processor 10, can realize:
acquiring an identification field and user information of each piece of original data from a single-level database, and combining the identification field and the user information of each piece of original data to form a plurality of user records;
constructing an inverted index for each user record to obtain an inverted index library of the user records;
responding to a data query request, querying user information of the data query request from the inverted index database to obtain target user information, and querying an identification field of the data query request in a single-level database according to the target user information to obtain a target identification field;
and querying data in the single-level database according to the target identification field to obtain a data query result of the data query request.
Specifically, the processor 10 may refer to the description of the relevant steps in the embodiment corresponding to fig. 1 for a specific implementation method of the computer program, which is not described herein again.
Further, the integrated modules/units of the electronic device 1, if implemented in the form of software functional units and sold or used as separate products, may be stored in a non-volatile computer-readable storage medium. The computer readable storage medium may be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
The present invention also provides a computer-readable storage medium, storing a computer program which, when executed by a processor of an electronic device 1, may implement:
acquiring an identification field and user information of each piece of original data from a single-level database, and combining the identification field and the user information of each piece of original data to form a plurality of user records;
constructing an inverted index for each user record to obtain an inverted index library of the user records;
responding to a data query request, querying user information of the data query request from the inverted index database to obtain target user information, and querying an identification field of the data query request in a single-level database according to the target user information to obtain a target identification field;
and querying data in the single-level database according to the target identification field to obtain a data query result of the data query request.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A data query method based on a single-level database is characterized by comprising the following steps:
acquiring an identification field and user information of each piece of original data from a single-level database, and combining the identification field and the user information of each piece of original data to form a plurality of user records;
constructing an inverted index for each user record to obtain an inverted index library of the user records;
responding to a data query request, querying user information of the data query request from the inverted index database to obtain target user information, and querying an identification field of the data query request in a single-level database according to the target user information to obtain a target identification field;
and querying data in the single-level database according to the target identification field to obtain a data query result of the data query request.
2. The single-level database-based data query method of claim 1, wherein the merging the identification field and the user information of each of the original data to form a plurality of user records comprises:
according to the information type of each piece of user information, carrying out field conversion on the user information to obtain a user information field of each piece of user information;
and combining each user information field and the corresponding identification field in the original data to generate a plurality of user records.
3. The single-level database-based data query method of claim 1, wherein the constructing an inverted index for each of the user records to obtain an inverted index library of the user records comprises:
performing document indexing on each user record by using a distributed search engine to generate an index document;
establishing an inverted index for the identification field of the index document according to the user information in each user record to obtain the inverted index of each user record;
and summarizing the reverse indexes recorded by each user to generate a reverse index library.
4. The single-level database-based data query method of claim 1, wherein the querying the user information of the data query request from the inverted index database in response to the data query request to obtain the target user information comprises:
and acquiring the user identification of the data query request, matching the user identification of the data query request with the user information in the inverted index database, and taking the successfully matched user information as the target user information.
5. The single-level database-based data query method according to any one of claims 1 to 4, wherein the querying the identification field of the data query request in the single-level database according to the target user information to obtain a target identification field comprises:
inquiring the user record of the data inquiry request from the inverted index library according to the target user information;
and identifying the identification field of the data query request in the single-level database according to the user record of the data query request to obtain a target identification field.
6. The single-level database-based data query method of claim 1, wherein before obtaining the identification field and the user information of each original data from the single-level database, the method further comprises:
acquiring original data and corresponding user information, and carrying out format unification on the original data to obtain standard data;
creating a field identifier of the standard data in a single-level database, and hashing the created field identifier to obtain an identifier field of the standard data in the single-level database;
and storing the original data and the corresponding user information into the single-level database according to the identification field.
7. The single-level database-based data query method of claim 6, wherein the storing the raw data and the corresponding user information into the single-level database according to the identification field comprises:
establishing a storage directory of the original data and the corresponding user information in a storage medium in the single-level database according to the identification field;
and loading the original data and the corresponding user information into the single-level database according to the storage directory.
8. A data query apparatus based on a single-level database, the apparatus comprising:
the user record generating module is used for acquiring the identification field and the user information of each original data from the single-level database, and combining the identification field and the user information of each original data to form a plurality of user records;
the reverse index building module is used for building a reverse index for each user record to obtain a reverse index library of the user records;
the identification field query module is used for responding to a data query request, querying user information of the data query request from the inverted index database to obtain target user information, and querying the identification field of the data query request in the single-level database according to the target user information to obtain a target identification field;
and the data query module is used for querying data in the single-level database according to the target identification field to obtain a data query result of the data query request.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the single level database based data query method of any one of claims 1 to 7.
10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, implements the single-level database-based data query method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110729441.5A CN113407657A (en) | 2021-06-29 | 2021-06-29 | Data query method, device, equipment and storage medium based on single-level database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110729441.5A CN113407657A (en) | 2021-06-29 | 2021-06-29 | Data query method, device, equipment and storage medium based on single-level database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113407657A true CN113407657A (en) | 2021-09-17 |
Family
ID=77680150
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110729441.5A Withdrawn CN113407657A (en) | 2021-06-29 | 2021-06-29 | Data query method, device, equipment and storage medium based on single-level database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113407657A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117009453A (en) * | 2023-10-07 | 2023-11-07 | 杭州雅拓信息技术有限公司 | Method and system for inquiring customer group list of customers in real time through digital marketing |
-
2021
- 2021-06-29 CN CN202110729441.5A patent/CN113407657A/en not_active Withdrawn
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117009453A (en) * | 2023-10-07 | 2023-11-07 | 杭州雅拓信息技术有限公司 | Method and system for inquiring customer group list of customers in real time through digital marketing |
| CN117009453B (en) * | 2023-10-07 | 2023-12-26 | 杭州雅拓信息技术有限公司 | Method and system for inquiring customer group list of customers in real time through digital marketing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111651453B (en) | User history behavior query method and device, electronic equipment and storage medium | |
| CN111258966A (en) | Data deduplication method, device, equipment and storage medium | |
| WO2022105135A1 (en) | Information verification method and apparatus, and electronic device and storage medium | |
| CN114610747A (en) | Data query method, device, equipment and storage medium | |
| CN114185895A (en) | Data import and export method and device, electronic equipment and storage medium | |
| CN115543198A (en) | Method and device for lake entering of unstructured data, electronic equipment and storage medium | |
| CN114979120A (en) | Data uploading method, device, equipment and storage medium | |
| CN112580079A (en) | Authority configuration method and device, electronic equipment and readable storage medium | |
| CN114138784A (en) | Information tracing method and device based on storage library, electronic equipment and medium | |
| CN112464619A (en) | Big data processing method, device and equipment and computer readable storage medium | |
| CN114398346A (en) | Data migration method, device, equipment and storage medium | |
| CN113434542B (en) | Data relationship identification method and device, electronic equipment and storage medium | |
| CN112506931B (en) | Data query method, device, electronic equipment and storage medium | |
| CN114637811A (en) | Data table entity relation graph generation method, device, equipment and storage medium | |
| CN112328656B (en) | Service query method, device, equipment and storage medium based on middle platform architecture | |
| CN113407657A (en) | Data query method, device, equipment and storage medium based on single-level database | |
| CN115617773A (en) | Data migration method, device and system | |
| CN113468175A (en) | Data compression method and device, electronic equipment and storage medium | |
| CN113722533A (en) | Information pushing method and device, electronic equipment and readable storage medium | |
| CN111538768A (en) | Data query method and device based on N-element model, electronic equipment and medium | |
| CN115114297A (en) | Data lightweight storage and search method and device, electronic equipment and storage medium | |
| CN114840388A (en) | Data monitoring method and device, electronic equipment and storage medium | |
| CN111651509B (en) | Hbase database-based data importing method and device, electronic equipment and medium | |
| CN113590856A (en) | Label query method and device, electronic equipment and readable storage medium | |
| CN114003661A (en) | Offline data entry method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210917 |