CN117425887A

CN117425887A - Data processing method, device, electronic equipment and computer readable storage medium

Info

Publication number: CN117425887A
Application number: CN202280001296.1A
Authority: CN
Inventors: 贺礼雄
Original assignee: BOE Technology Group Co Ltd
Current assignee: BOE Technology Group Co Ltd
Priority date: 2022-05-18
Filing date: 2022-05-18
Publication date: 2024-01-19
Also published as: WO2023220973A1

Abstract

A data processing method, apparatus, electronic device and computer readable storage medium applied to a database. The database includes a database table including a plurality of first fields and a plurality of records. The data processing method applied to the database comprises the following steps: creating a second field in the database table, wherein the second field is used for recording the combination information of the selected field of each of the plurality of records, and the selected field is at least part of the plurality of first fields (S10); an index is created for the data query based on the second field (S20). The method can improve the query efficiency, can improve the response speed of interfaces and interfaces in big data application, and improves the usability of the system.

Description

Data processing method, device, electronic equipment and computer readable storage medium

Technical Field

Embodiments of the present disclosure relate to a data processing method, apparatus, electronic device, and computer-readable storage medium.

Background

The current society is a society with high development speed, developed technology, information circulation, more and more intimate communication among people, more and more convenient life, and big data is the product of the high-technology age. In the society of today, the application of big data is increasingly showing its advantages. For example, by analyzing the big data, it is possible to judge the behavior of the consumer, predict the sales of the product, and the like. The database is a repository for data. The storage space is large, millions, tens of millions and even hundreds of millions of data can be stored, and the storage space can be used for storing and managing big data.

Disclosure of Invention

At least one embodiment of the present disclosure provides a data processing method applied to a database, the database including a database table including a plurality of first fields and a plurality of records, the method including: creating a second field in the database table, wherein the second field is used for recording the combination information of the selected field of each of the plurality of records, and the selected field is at least one part of the plurality of first fields; an index is created for the data query based on the second field.

For example, in the data processing method provided in an embodiment of the present disclosure, the method further includes: in response to receiving a data query request for at least a portion of the selection fields, data queries are performed according to the index and query results are returned.

For example, in a data processing method provided in an embodiment of the present disclosure, the index includes an inverted index.

For example, in a data processing method provided in an embodiment of the present disclosure, creating the second field in the database table includes: in response to the amount of data in the database table being greater than the preset threshold, a second field is created in the database table.

For example, in a data processing method provided in an embodiment of the present disclosure, a selection field includes a field capable of performing a fuzzy query among a plurality of first fields.

For example, in a data processing method provided in an embodiment of the present disclosure, a selection field includes at least two fields of a plurality of first fields, and creating a second field in a database table includes: aiming at each record in the database table, splitting the information of each field in at least two fields into a character string array according to a character string splitting rule; and determining, for each record in the database table, a value of a second field of each record from the array of strings of at least two fields.

For example, in a data processing method provided in an embodiment of the present disclosure, a data query request includes a query condition of a fuzzy query, the query condition includes field information of at least some of the selected fields, and in response to receiving the data query request for at least some of the selected fields, performing the data query according to an index and returning a query result, including: responding to a received data query request for at least part of the selected fields, and splitting the field information of each of the at least part of the fields into character strings according to a character string splitting rule to obtain a character string array to be queried of each of the at least part of the fields; and according to the index, inquiring the character string array to be inquired of each of at least part of the fields to obtain an inquiring result.

For example, in a data processing method provided in an embodiment of the present disclosure, an index includes an inverted index, and according to the index, a query is performed on an array of character strings to be queried of each of at least some fields to obtain a query result, including: inquiring the inverted index to obtain a character string array inquiry result corresponding to each character string array to be inquired of at least part of fields, wherein the character string array inquiry result comprises address information of records containing the character string array to be inquired; determining intersection of recorded address information of the character string array to be queried corresponding to the at least partial fields; and generating the query result according to the address information in the intersection. And querying records corresponding to the address information in the intersection by using a fuzzy query operator to remove records inconsistent with query conditions in the character string array query results, so as to obtain fuzzy query results.

For example, in a data processing method provided in an embodiment of the present disclosure, field information of at least some fields includes first field information of a first selected field and second field information of a second selected field, and in response to receiving a data query request for at least some of the selected fields, splitting the field information of each of the at least some fields into strings according to a string splitting rule to obtain a string to be queried of each of the at least some fields, including: responding to a received data query request for at least part of the selected fields, carrying out character string splitting on the first field information to obtain a first character string array to be queried, and carrying out character string splitting on the second field information to obtain a second character string array to be queried; querying the inverted index to obtain a string array query result corresponding to the string array to be queried of each of the at least partial fields, including: querying the inverted index to obtain a first query result of the first character string array to be queried; querying the inverted index to obtain a second query result of the second character string array to be queried; generating the query result according to the address information in the intersection, including: and querying records corresponding to the address information in the intersection of the first query result and the second query result by using a fuzzy query operator, so as to eliminate records which do not meet the query condition in the records corresponding to the address information in the intersection, thereby obtaining the query result.

For example, in the data processing method provided in an embodiment of the present disclosure, the query condition further includes page number information, and query the inverted index to obtain a string array query result corresponding to the string array to be queried of each of at least some fields, including: and inquiring the inverted index according to the page number information to obtain a plurality of character string array inquiring results corresponding to the character string arrays to be inquired of each of at least part of fields.

For example, in the data processing method provided in an embodiment of the present disclosure, the method further includes: and responding to the paging query request of the total data in the database table, and performing paging query on the database table to obtain a paging query result.

For example, in a data processing method provided in an embodiment of the present disclosure, in response to receiving a paging query request for a total amount of data in the database table, performing a paging query on the database table to obtain a paging query result, including: responding to a paging query request for the total data in the database table, and acquiring the total number of the plurality of records; determining demarcation points of sequential query and reverse query according to the total number, wherein the demarcation points divide the plurality of records into a first part and a second part; and sequentially inquiring the first part and reversely inquiring the second part to generate a paging inquiry result.

For example, in a data processing method provided in an embodiment of the present disclosure, in response to receiving the paging query request for the full amount of data in the database table, obtaining the total number of records includes: and in response to receiving the paging query request for the full amount of data in the database table, querying a statistics table of a database, and obtaining the total number of the plurality of records, wherein the statistics table of the database is obtained by periodically counting the records of the database table in the database by a database system.

For example, in the data processing method provided in an embodiment of the present disclosure, the character string splitting rule includes: every two adjacent two characters are split as a group.

At least one embodiment of the present disclosure provides a data processing apparatus applied to a database, the database including a database table including a plurality of first fields and a plurality of records, the apparatus comprising: a field creating unit configured to create a second field in the database table, the second field being used for recording combination information of a selected field of each of the plurality of records, the selected field being at least a part of the plurality of first fields; and an index creation unit configured to create an index for data query based on the second field.

At least one embodiment of the present disclosure provides an electronic device comprising a processor; a memory including one or more computer program modules; one or more computer program modules are stored in the memory and configured to be executed by the processor, the one or more computer program modules comprising instructions for implementing the data processing methods provided by any of the embodiments of the present disclosure.

At least one embodiment of the present disclosure provides a computer-readable storage medium having computer-readable instructions stored non-transitory thereon, which when executed by a processor, may implement the data processing method provided by any of the embodiments of the present disclosure.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly described below, and it is apparent that the drawings in the following description relate only to some embodiments of the present disclosure, not to limit the present disclosure.

FIG. 1 illustrates a flow chart of a data processing method provided by at least one embodiment of the present disclosure;

FIG. 2A illustrates a method flow diagram of step S10 of FIG. 1 provided by at least one embodiment of the present disclosure;

FIG. 2B illustrates another data processing method flow diagram provided by at least one embodiment of the present disclosure;

FIGS. 2C-2E are schematic diagrams illustrating a method of updating a value of a second field provided by at least one embodiment of the present disclosure;

FIG. 3A illustrates a method flow diagram of step S30 of FIG. 1 provided by at least one embodiment of the present disclosure;

FIG. 3B illustrates a schematic diagram of an interactive interface provided by at least one embodiment of the present disclosure;

FIG. 3C schematically illustrates a method flow diagram of step S32 of FIG. 3A provided by at least one embodiment of the present disclosure;

FIG. 3D schematically illustrates another method flow diagram of step S30 of FIG. 1 provided by at least one embodiment of the present disclosure;

FIG. 3E schematically illustrates another method flow diagram of step S30 of FIG. 1 provided by at least one embodiment of the present disclosure;

FIG. 3F schematically illustrates a result diagram of a fuzzy query provided by at least one embodiment of the present disclosure;

FIG. 4 illustrates a flow chart of a method of outputting full data of a database table provided by at least one embodiment of the present disclosure;

FIG. 5A illustrates a flow diagram of a method for paging a full volume of data provided by at least one embodiment of the present disclosure;

FIG. 5B illustrates a flowchart of another method for paging a full volume of data provided by at least one embodiment of the present disclosure;

FIG. 6 illustrates a flow chart of another method of data processing performed by a database system provided in accordance with at least one embodiment of the present disclosure;

FIG. 7 illustrates a schematic block diagram of a data processing apparatus provided by at least one embodiment of the present disclosure;

FIG. 8 illustrates a schematic block diagram of an electronic device provided by at least one embodiment of the present disclosure;

FIG. 9 illustrates a schematic block diagram of another electronic device provided by at least one embodiment of the present disclosure; and

fig. 10 illustrates a schematic diagram of a computer-readable storage medium provided by at least one embodiment of the present disclosure.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present disclosure. It will be apparent that the described embodiments are some, but not all, of the embodiments of the present disclosure. All other embodiments, which can be made by one of ordinary skill in the art without the need for inventive faculty, are within the scope of the present disclosure, based on the described embodiments of the present disclosure.

Unless defined otherwise, technical or scientific terms used in this disclosure should be given the ordinary meaning as understood by one of ordinary skill in the art to which this disclosure belongs. The terms "first," "second," and the like, as used in this disclosure, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. Likewise, the terms "a," "an," or "the" and similar terms do not denote a limitation of quantity, but rather denote the presence of at least one. The word "comprising" or "comprises", and the like, means that elements or items preceding the word are included in the element or item listed after the word and equivalents thereof, but does not exclude other elements or items. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", etc. are used merely to indicate relative positional relationships, which may also be changed when the absolute position of the object to be described is changed.

There is a wide range of information interconnections for meta-universe or smart cities. For example, various information platforms involve massive amounts of personnel data, which are often stored in databases that need to be quickly searched when needed. The database system is typically a Browser/Server (B/S) architecture with separated front-end and back-end. The front end acquires data and displays the data by calling the query parameters transmitted by the rear end interface, so that the front end can only pay attention to the displayed effect and the response rate of the rear end interface. If the back-end data increases sharply, the response time of the front-end in making the interface call is too long.

At least one embodiment of the present disclosure provides a data processing method applied to a database, the database including a database table including a plurality of first fields and a plurality of records, the method including: creating a second field in the database table, wherein the second field is used for recording the combination information of the selected field of each of the plurality of records, and the selected field is at least part of the plurality of first fields; an index is created for the data query based on the second field. The data processing method can improve the efficiency of data query, can improve the response speed of interfaces and interfaces in big data application, and improves the usability of the system. The data processing method provided by at least one embodiment of the present disclosure may be applied to a database corresponding to a scene such as a metauniverse or a smart city, so as to facilitate rapid query of massive personal data designed in the metauniverse or the smart city.

Fig. 1 illustrates a flow chart of a data processing method provided by at least one embodiment of the present disclosure.

As shown in fig. 1, the method may include steps S10 to S20.

Step S10: creating a second field in the database table, wherein the second field is used for recording the combination information of the selected fields of each of the plurality of records, and the selected fields are at least part of the plurality of first fields.

Step S20: an index is created for the data query based on the second field.

In this embodiment, the database comprises a database table comprising a plurality of first fields and a plurality of records. Each of the plurality of records includes a field value corresponding to the plurality of first fields.

According to the data processing method, the combination information of the plurality of selection fields is recorded through the created second field, so that the plurality of selection fields can be queried according to the index of the second field, and the efficiency of querying the plurality of selection fields is improved. In addition, the disclosed embodiments add a field to the database table and update it, reducing modifications to existing system code as much as possible without affecting the normal business logic of the existing system. The invention improves the rate of fuzzy queries by spatial time shifting, i.e., by redundant preservation of data (redundant preservation of data by the second field). Because redundancy is kept on the data, the utilization rate of the disk where the database is located is greatly increased.

For step S10, the selection field includes a field in which a fuzzy query can be made among the plurality of first fields. Fuzzy query is a query way to find with partial information of data. For example, the plurality of first fields in the database table include: name, account number, level, department, mailbox, type, etc., if the database provides the function of fuzzy query for two fields of "account number" and "mailbox", the selection field may include the fields "account number" and "mailbox". For example, when the "account" field is queried in a fuzzy manner, the "abc" may be input in the "account" input box, so as to query a record in the database, where the field value corresponding to the "account" field includes the "abc".

A second field, different from the plurality of first fields, is created in the database table, and the value of the second field of each record may be determined based on the combination information of the selected fields. For example, the combined information may be a concatenation of values for selected fields of each record. For example, the selection field includes two fields, namely "account" and "mailbox", and if the value of the field "account" of one record is "hlx" and the value of the field "mailbox" is "abc", the value of the second field of the record may be "hlx, abc". For example, the combination information may include a plurality of selection fields and a value corresponding to each selection field. For example, the selection field includes two fields, namely "account" and "mailbox", and if the value of the field "account" of one record is "hlx" and the value of the field "mailbox" is "abc", the value of the second field of the record may be "name: hlx email: abc).

In some embodiments of the present disclosure, the second field may be, for example, a field in jsonb format. For example, the database system in which the database resides is in the jsonb format supported by the cockreach db or PostgreSQL, cockroachDB or PostgreSQL. For example, one example of a structured query language (Structured Query Language, SQL) executed to create fields in jsonb format is as follows:

alter table sys_user add column username_email jsonb；

in the above example, "sys_user" is an example of a name of a table, "username_email" is an example of a name of a second field, "alter table" and "add column" are both standard operators in SQL, "jsonb" is a format of the second field.

In some embodiments of the present disclosure, the second field is created in the database table, for example, in response to an operation by the database manager.

In some embodiments of the present disclosure, the second field is created in the database table, for example, in response to the amount of data in the database table being greater than a preset threshold.

For example, when the amount of data in the database table reaches the billion level, a second field is automatically created in the database table according to the selection field set by the database manager. When the data volume is larger than the preset threshold value, the embodiment automatically creates the second field and queries the data according to the index of the second field, so that the query efficiency of querying the database table with large data volume is improved, and the self-adaption capability of the data processing method is improved.

In some embodiments of the present disclosure, the second field may not need to be created when the amount of data in the database table is small (e.g., not on the order of billions). The second field may not be required to be created when the response speed requirement can be met, for example, by obscuring the first field in the database table with an operator (e.g., like) of the obscured query. The second field is automatically created when the amount of data in the database table is large (e.g., on the order of billions).

For step S20, an inverted index is created, for example, based on the second field, so that a data query is made through the inverted index. The data query efficiency can be further improved by utilizing the inverted index to perform data query. When the fuzzy query is carried out on a large number of character string text data, the query result is rapidly obtained by utilizing the database characteristics of the inverted index, so that the time-consuming operation of full-table scanning is avoided, and the response speed of the system is improved.

In other embodiments of the present disclosure, a forward index may also be created based on the second field, and the present disclosure does not limit the type of index.

As shown in fig. 1, the data processing method may further include step S30 on the basis of including steps S10 and S20.

Step S30: in response to receiving a data query request for at least a portion of the selection fields, data queries are performed according to the index and query results are returned.

In some embodiments of the present disclosure, for example, a database system in which the database resides provides an interactive interface that provides a control for a user to enter query parameters through which the user can enter a data query request into a back-end interface. And the rear end of the database system queries the index to obtain a query result according to the data query request, and displays the query result on the interactive interface.

For example, the database system may be a cockreach db or PostgreSQL, etc., may be a structured relational database or other type of database, as embodiments of the present disclosure are not limited in this regard.

Fig. 2A illustrates a method flow diagram of step S10 of fig. 1 provided by at least one embodiment of the present disclosure.

As shown in fig. 2A, step S10 may include step S11 and step S12.

Step S11: and splitting the information of each field in at least two fields into a character string array according to a character string splitting rule aiming at each record in the database table.

Step S12: for each record in the database table, determining the value of the second field of each record according to the character string array of at least two fields.

For step S11, the character string splitting rule may include splitting every two adjacent two characters as a group (hereinafter referred to as "dichotomy"), for example.

For example, the selection field includes two fields, namely an "account number" and a "mailbox", the value of the "account number" of a record in the database table is "admin", the value of the "mailbox" of the record is siteam@be.com, and the string arrays obtained by splitting the "admin" and the "siteam@be.com" according to the dichotomy are [ "ad", "dm", "mi", "in" ] and [ "si", "it", "te", "ea", … "," om "], respectively.

For step S12, the value of the second field of each record may include the string array obtained by splitting each field information in the selected field of the record. For example, the value of the second field of each record may include a combination of the array of strings split by each field information of the selected field of the record. For example, in the example of step S11 above, the value of the second field of the record is { "email_array": "[ si", "it", "te", "ea", … "," om "]," username_array ": [" ad "," dm "," mi "," in "] }. "email_array" represents the string Array of the field "mailbox" and "username_array" represents the string Array of the field "account".

In some embodiments of the present disclosure, for example, the splitting of the dichotomy may be implemented using java code, and then the following example SQL statement is executed to write the value of the second field obtained by the java code into each record. In some embodiments of the present disclosure, the value of the second field derived by the java code is written to the second field of the database table, for example, using the database middleware myblocks-plus. In embodiments of the present disclosure, for example, database operations are performed using the backend technology framework Mybatis-plus, and it is easier for developers to start.

For example, the selection fields are "mailbox" and "account" and then the SQL statement of the following example may be executed to obtain the value of the second field of the record described above.

update sys_user set username_email＝‘{“email_Array”:[“si”,“it”,“te”,“ea”,…,“om”],“username_Array”:[“ad”,“dm”,“mi”,“in”]}’where id＝x。

In the above example, "update", "set" and "where" are standard operators of SQL, "sys_user" is an example of the name of the database table, "username_email" is an example of the name of the second field, and id= "x" indicates that the address information of the record is x. "email_array": [ "si", "it", "te", "ea", … "," om "]" username_array ": [" ad "," dm "," mi "," in "] is the value of the second field of the record resulting from execution of the java code.

The java code for realizing the dichotomy splitting can be designed by a person skilled in the art.

In the embodiment shown in fig. 2A, the value of the second field is determined according to the character string array obtained by splitting the binary method, so that the user can perform fuzzy query by inputting any two adjacent characters, the full fuzzy query is realized, and the query efficiency is improved.

FIG. 2B illustrates a flow chart of another data processing method provided by at least one embodiment of the present disclosure.

As shown in fig. 2B, in this embodiment, step S10 may include steps S101 to S105.

Step S101: a second field in jsonb format is created. For example, a second field for recording the combined information of the selection fields "mailbox" and "account" of each of the plurality of records is created from the example after table sys_ user add column username _email jsonb in the description of step S10 in fig. 1 above.

Step S102: an inverted index is created for the second field. For example, an inverted index of the second field "username_email" in the database table "sys_user" may be created in accordance with the following SQL statement.

creat index“idx_sys_user”on sys_user using gin(username_email)。

Where "create index", "on" and "using" are SQL operators.

Step S103: and acquiring a selection field capable of performing fuzzy query in the database table. For example, the selection field capable of fuzzy query may be preset by the database manager.

Step S104: for each record in the database table, splitting the field information of each field in the selected fields into a character string array by a dichotomy.

Step S105: for each record, updating the value of the second field according to the string array of each field.

The inverted index may be an index table in which each item includes an attribute value and address information of each record having the attribute value, thereby determining an address of the record according to the attribute value. In the embodiment of the present disclosure, the attribute value may be a plurality of strings in each string array obtained by a dichotomy, and the inverted index records a correspondence between each string and address information of a record containing the string, so that the record containing the string may be found according to the string.

Splitting the text field value of the character string in a two-word segmentation mode, storing the split text field value into a jsonb format field with an inverted index, and carrying out smooth expansion on the premise of not affecting the normal operation of the existing table, so that the stability of the system is fully ensured.

In this embodiment, in response to obtaining a data query request, the inverted index is queried to obtain a query result for the data query request. For example, if the data query request is to query the record containing the character "st" in the "mailbox", the inverted index is queried to obtain the addresses of all records containing "st" in the "mailbox" field of the database table, so that the record list containing "st" in the "mailbox" field of the database table is returned to the interactive interface according to the record addresses. The text field value of the character string is split in a dichotomy way, the data is updated to the field of the jsonb format again by combining the database characteristic of inverted index, and fuzzy query on a large amount of data is realized by a space time-changing mode.

Fig. 2C-2E are schematic diagrams illustrating a method for updating a value of a second field according to at least one embodiment of the present disclosure.

Fig. 2C is a record in the database table, where the value of the field "account" is admin and the value of the field "mailbox" is siteam@be.com.

Fig. 2D is a value of a second field username_email obtained by executing Java code. FIG. 2E is a diagram of the execution of an SQL operator to update the values of the second field in the database table.

Fig. 3A illustrates a method flow diagram of step S30 of fig. 1 provided by at least one embodiment of the present disclosure. The method flowchart of step S30 shown in fig. 3A may be applied to the embodiment shown in fig. 2A or fig. 2B, for example.

As shown in fig. 3A, step S30 may include step S31 and step S32. In this embodiment, the data query request includes a query condition that ambiguously queries, the query condition including field information that selects at least some of the fields.

Step S31: and responding to the received data query request for at least part of the selected fields, and carrying out character string splitting on the field information of each of the at least part of the fields according to a character string splitting rule to obtain a character string array to be queried of each of the at least part of the fields.

Step S32: and according to the index, inquiring the character string array to be inquired of each of at least part of fields to obtain an inquiring result.

For example, the query condition of the fuzzy query is that the value of the field "account" and the value of "mailbox" both contain a string of "demo". For another example, the query condition of the ambiguous query is that the value of the field "account" contains a demo string. For another example, the query condition of the ambiguous query is that the value of the field "account" contains a string of "demo" and the value of the field "mailbox" contains a string of "keep". The query condition of the fuzzy query is generated according to the query requirement of the user, the user can input the query requirement through the interactive interface, and the query condition is not limited by the present disclosure.

In some embodiments of the present disclosure, for example, an interactive interface provides an operational control for a user to input query parameters, in which the user can input the query parameters, the interactive interface generates a data query request according to the query parameters entered by the user in the operational control, and the data query request is entered into a back-end interface. The query conditions in the data query request are determined according to the query parameters input by the user.

FIG. 3B illustrates a schematic diagram of an interactive interface provided by at least one embodiment of the present disclosure.

In the example of fig. 3B, the selection fields may be, for example, "account number", "name", "cell phone number", and "mailbox", which the user may utilize the interactive interface 100 to fuzzify the query.

As shown in fig. 3B, the interactive interface provides an account number input control 1, a name input control 2, a mobile phone number input control 3, a mailbox input control 4, and a search control 5. The account number input control 1, the name input control 2, the mobile phone number input control 3 and the mailbox input control 4 are in one-to-one correspondence with the fields of 'account number', 'name', 'mobile phone number' and 'mailbox', and are respectively used for receiving query parameters of each field. The user may enter query parameters at any one or more of the 5 controls.

For example, after the user inputs the query parameter "demo" about the account in the account input control 1 and the query parameter "keep" about the mailbox in the mailbox input control 4, the search control 5 is clicked, and a data query request is generated in response to the above operation of the user, where the query condition of the data query request is that the value of the field "account" contains a string of "demo" and the value of the field "mailbox" contains a string of "keep".

For another example, when the user inputs the query parameter "demo" about the account in the account input control 1 and inputs the query parameter "demo" about the mailbox in the mailbox input control 4, the query condition of the data query request is that the value of the field "account" and the value of the mailbox both contain the character string of "demo".

For step S31, for example, the query condition of the data query request is that the value of the field "account" includes a character string of "demo", and the value of the field "mailbox" includes a character string of "keep", that is, the field information of the field "account" is "demo", and the field information of the field "mailbox" is "keep", then the character strings are split for "demo" and "keep" respectively, so as to obtain a plurality of character string arrays to be queried.

For example, the character string splitting is performed on the "demo" and the "keep" according to the dichotomy, so that the obtained character string array to be queried of the "demo" is [ de, em, mo ], "keep" is [ ke, ee, ep ].

For step S32, for example, according to the index, the to-be-queried string array [ de, em, mo ] of "demo" is queried to obtain the value of "account" and the records of "de", "em" and "mo" are included at the same time, and according to the index, the to-be-queried string array [ ke, ee, ep ] of "keep" is queried to obtain the value of "mailbox" and the records of "ke", "ee" and "ep" are included at the same time.

In some embodiments of the present disclosure, for example, the index is an inverted index, step S32 may be performed as follows in the method illustrated in fig. 3C.

Fig. 3C schematically illustrates a method flow diagram of step S32 of fig. 3B provided by at least one embodiment of the present disclosure.

As shown in fig. 3C, step S32 may include steps S321 to S323.

Step S321: and querying the inverted index to obtain a character string array query result corresponding to the character string array to be queried of each of at least part of fields, wherein the character string array query result comprises address information of records containing the character string array to be queried.

Step S322: and determining intersection of address information of records of the character string array to be queried corresponding to at least part of the fields.

Step S323: and generating a query result according to the address information in the intersection.

For step S321, for example, the value of the query inverted index obtaining field "account" contains the address information of the to-be-queried string array [ de, em, mo ] and the value of "mailbox" contains the address information of the to-be-queried string array [ ke, ee, ep ]. For example, the address information is a sequence number recorded in a database table.

For example, the record of the field "account number" obtained by querying the inverted index contains the to-be-queried string array [ de, em, mo ] ] as the records with the sequence numbers of 5, 7 and 8 in the database table, and the record of the "mailbox" contains the to-be-queried string array [ ke, ee, ep ] as the records with the sequence numbers of 7, 8 and 12 in the data packet.

In some embodiments of the present disclosure, for example, each string (e.g., "de", "em" and "mo") in the string array is queried separately, resulting in a record that contains each string in the string array at the same time, which is a record that contains the string array to be queried (e.g., [ de, em, mo ]).

In some embodiments of the present disclosure, before step S321, it may be performed: invoking a class of a first type in the persistence layer framework to establish a communication connection with the database; and calling a class of a second type to create a data encapsulation object, wherein the data encapsulation object is used for encapsulating the data returned by the database table.

For example, the class of the first type may be the PageFactoy class in the back-end technical framework Mybatis-plus. The second type of class (e.g., the PageUtil class of FIG. 3E) may be written by a developer, and functions to create a Java object that holds the paging result, i.e., encapsulates the data returned from the database table after a query operation based on the query conditions into the object, and finally responds to the interface via the interface. A third type of class may be written by a developer and may implement an interface to Mybatis-plus. After the creation of the class of the first type and the class of the second type is completed, the query of the inverted index is performed, for example, step S321 is performed.

Step S321 may include: and calling a class of a third type to be used for inquiring the inverted index to obtain a character string array inquiring result corresponding to the character string array to be inquired of each of at least part of fields. A third type of class (e.g., the UserMapper class in fig. 3E) may create a Java object and perform a first database query based on the created Java object and the set of strings to be queried generated by the user-entered query conditions.

For step S322, for example, in the example of step S321, the intersection of the address information of the record of the string array to be queried [ de, em, mo ] and the address information of the record of the string array to be queried [ ke, ee, ep ] is the record of serial numbers 7 and 8.

For step S323, for example, records corresponding to the address information in the intersection may be queried using the fuzzy query operator, so as to reject records that do not satisfy the query condition from the records corresponding to the address information in the intersection, and obtain the query result.

For example, records with sequence numbers 7 and 8 are queried again using a fuzzy query operator (e.g., like) to cull records that do not satisfy the query condition among records with sequence numbers 7 and 8.

For example, the values of the fields "account" and "mailbox" of records with serial numbers 7 and 8 are queried again using a fuzzy query operator (e.g., like) to reject records of records with serial numbers 7 and 8 that are inconsistent with the query conditions. For example, the values of the field "account number" of the records with serial numbers 7 and 8 are "kedemo" and "demo", respectively, and the fuzzy query operation of the fuzzy query operator like is used to obtain the query result including the record with serial number 7, and the record with serial number 8 is removed, because the value "demo" of the field "account number" of the record with serial number 8 obtained by the fuzzy query of the fuzzy query operation operator like is inconsistent with the "demo" in the query condition, and therefore the record with serial number 8 is removed.

Under the condition that records containing the character string array to be queried are obtained by using the inverted index, the method utilizes the fuzzy query operator to conduct screening check on the records containing the character string array to be queried again, and ensures the accuracy of fuzzy query on the basis of improving the efficiency of fuzzy query.

In some embodiments of the present disclosure, the field information of at least some of the fields includes first field information of the first selection field and second field information of the second selection field, and the step S31 may include: responding to a data query request for at least part of the selected fields, carrying out character string splitting on the first field information to obtain a first character string array to be queried, and carrying out character string splitting on the second field information to obtain a second character string array to be queried. Step S321 may include: querying the inverted index to obtain a first query result of the first character string array to be queried; and querying the inverted index to obtain a second query result of the second character string array to be queried. Step S323 may include: and querying records corresponding to the address information in the intersection of the first query result and the second query result by using the fuzzy query operator so as to reject records which do not meet the query condition in the records corresponding to the address information in the intersection to obtain the query result.

For example, in some embodiments of the present disclosure, the first field information and the second field information may be the same. For example, if the query condition of the fuzzy query is that the value of the field "account number" and the value of the "mailbox" both contain the character string of "demo", that is, if the first condition of the first selection field "account number" is that the value of the field "account number" contains the character string of "demo", and the second condition of the second selection field "mailbox" is that the value of the field "mailbox" contains the character string of "demo", the first field information and the second field information are both "demo". For example, in this example, step S31 may be that the first string array to be queried obtained by splitting the "demo" string is [ de, em, mo ] and the second string array to be queried is [ de, em, mo ]. In this example, for example, step S322 may obtain, for the query inverted index, a first query result of the first string array [ de, em, mo ] to be queried, where the first query result is, for example, records with serial numbers of 5, 7, and 8; the query inverted index results in a second query result of the second string array to be queried (e.g. also [ de, em, mo ]), for example records with sequence numbers 5, 8 and 10.

In this example, the intersection of the first query result and the second query result is records with sequence numbers 5 and 8, and the fuzzy query operator may be used to perform fuzzy query again on the records with sequence numbers 5 and 8 to obtain the query result. For example, if the query results obtained by using the fuzzy query operator are that the records with sequence numbers 5 and 8 satisfy the query condition, the back-end interface may provide the records with sequence numbers 5 and 8 to the interactive interface in this example, and display specific information of the records with sequence numbers 5 and 8 on the interactive interface.

For example, in some embodiments of the present disclosure, the first field information and the second dictionary information may be different. For example, the query condition of the fuzzy query is that the value of the field "account" contains a character string of "demo", and the value of the "mailbox" contains a character string of "keep", that is, the first condition of the first selection field "account" is that the value of the field "account" contains a character string of "demo", the second condition of the second selection field "mailbox" is that the value of the field "mailbox" contains a character string of "keep", the first field information is "demo" and the second field information is "keep". For example, in this example, step S31 may be that the first string array to be queried obtained by splitting the "demo" into strings is [ de, em, mo ], and the second string array to be queried obtained by splitting the "keep" into strings is [ ke, ee, ep ]. In this example, for example, step S32 may obtain, for the query inverted index, a first query result of the first string array [ de, em, mo ] to be queried, where the first query result is, for example, records with serial numbers of 5, 7, and 8; the inverted index is queried to obtain a second query result of the second string array [ ke, ee, ep ] to be queried, wherein the second query result is a record with the serial numbers of 7, 8 and 12. In this example, the intersection of the first query result and the second query result is the record of sequence numbers 7 and 8. For example, if the query result obtained by using the fuzzy query operator is that the record with the number 7 satisfies the query condition, the back-end interface may provide the record with the number 7 to the interactive interface in this example, and display specific information of the record with the number 7 on the interactive interface.

In some embodiments of the present disclosure, the query condition may further include page number information, and step S32 may include: and inquiring the inverted index according to the page number information to obtain a character string array inquiring result corresponding to the character string array to be inquired of each of at least part of the fields. The method realizes the paging inquiry of the fuzzy inquiry and is convenient for users to use. For example, the page number information may be a page sequence number after paging of data in the database.

For example, the user may input page information in the interactive interface, so that the back-end interface queries the to-be-queried character string in the plurality of records included in the corresponding page according to the page information, thereby obtaining a character string array query result including the to-be-queried character string array in the page.

For example, the plurality of records included in the page are queried according to the method described in fig. 3C, so as to obtain a string array query result including the string array to be queried in the page.

Fig. 3D schematically illustrates another method flowchart of step S30 of fig. 1 provided by at least one embodiment of the present disclosure.

In the example of FIG. 3D, the data query request includes query conditions for a fuzzy query. The query condition includes field information for selecting at least some of the fields.

As shown in fig. 3D, the method may include steps S301 to S303.

Step S301: and splitting the field information in the query condition into a character string array to be queried through a dichotomy.

In some embodiments of the present disclosure, the field information includes field information for each of a plurality of selection fields. And splitting each field information into character string arrays to be queried. This step S301 is similar to step S31 described above with respect to fig. 3A.

Step S302: and according to the character string array to be queried, querying the inverted index to obtain the address information of the record containing the character string array to be queried.

Under the condition that the number of the character string arrays to be queried is multiple, intersection is calculated on address information corresponding to the plurality of character string arrays to be queried.

This step S302 is similar to step S321 and step S322 described above with respect to fig. 3C.

Step S303: and carrying out fuzzy query on the intersection set by using the fuzzy query operator to remove records which are not matched with the query conditions in the intersection set, thereby obtaining a query result.

This step S303 is similar to step S323 described above with respect to fig. 3C.

Fig. 3E schematically illustrates another method flow diagram of step S30 of fig. 1 provided by at least one embodiment of the present disclosure.

As shown in fig. 3E, the method may include steps S3.11 to S3.22.

Step S3.11: for example, the user performs an input operation in the interactive interface to input the query parameters. The query parameters may include, for example, query conditions including field information that at least partially selects a field. The interactive interface generates a data query request according to the query parameters and provides the data query request to the back end.

Step S3.12: the back-end interface (e.g., written in Java code) parses the data query request to obtain field information of the data query request. The field information is, for example, "demo".

Step S3.13: and splitting the field information by using a dichotomy to obtain the character string array to be queried. For example, the "demo" is split by a dichotomy to obtain the character string array [ de, em, mo ] to be queried.

Step S3.12 and step S3.13 are to split a field information to obtain a character string array to be queried. Step S3.14 and step S3.15 are to split another field information to obtain another character string array to be queried. Step S3.16 and step S3.17 are to split the information of the other field to obtain another character string array to be queried. Regarding steps S3.14, S3.15, S3.16 and S3.17, please refer to steps S3.12 and S3.13, similarly to steps S3.12 and S3.13.

Step S3.18: the backend interface invokes the class PageFactory in the backend technology framework Mybatis-plus embedded in Java code to establish a connection with the database.

Step S3.19: the PageUtil class is called to create a paged Java class object for receiving returned result data after the query is completed. PageUtil class can be written by developer, and is used for creating a Java object for saving paging result, namely, after query operation is performed according to query condition, the data returned from database table is packaged into the object, and finally, the data is responded to interface through interface.

Step S3.20: and calling a UserMapper class to perform a first database query according to the Java object created by the UserMapper class and the character string array to be queried generated by the query condition input by the user, and acquiring the total number of all results under the query condition input by the user. UserMapper class may be written by a developer, for example, and may implement an interface to Mybatis-plus.

Step S3.21: and according to the page number input by the user (default value if the user does not input) and the array generated by the query condition, inquiring the database table through the Java object created by the UserMapper class and returning the result data of the corresponding page number required by the user.

The only difference between step S3.20 and step S3.21 is that S3.20 has no page number when executing the query, and the result of the query is a data set that all meet the conditions.

Step S3.22: and generating a query result according to the step S3.20 and the step S3.21. The query result contains current page data, page number of the current page, total page number, total bar number, and the like. In step S3.22, the total number may be displayed, the total number of pages calculated, etc. based on the total number of data sets obtained in step S3.20.

In some embodiments of the present disclosure, the data processing method may further include, in addition to step S10 to step S30: and responding to the paging query request for the total data in the database table, and performing paging query on the database table to obtain a paging query result.

Full data refers to all data in the database table.

For example, a paging query request may be a request to query for data located on page i of a database table, data on page i ahead, or all of the data in the database table, etc.

For example, a user may input the number of pages to be queried through the interactive interface, thereby generating a paging query request according to the number of pages input by the user, the back-end interface queries the database table in response to receiving the paging query request from the interactive interface, and provides the paging query result to the interactive interface.

FIG. 3F schematically illustrates a result diagram of a fuzzy query provided by at least one embodiment of the present disclosure.

The schematic diagram shown in fig. 3F is shown, for example, on a page of a browser.

As shown in fig. 3F, the query condition input by the user is that the field "account" and the field "mailbox" both contain "demo" strings.

For example, on the browser presentation page shown in fig. 3F, a plurality of records each including a "demo" character string are listed in sequence for the field "account" and the field "mailbox". As shown in fig. 3F, 1000000 fields "account" and "mailbox" are queried in the database table, each containing a record of "demo" character string, and are shown in a format of 1000 pages.

As shown in fig. 3F, the "demo" string may be presented in a browser presentation page in a highlighted form.

FIG. 4 illustrates a flow chart of a method of outputting full data of a database table provided by at least one embodiment of the present disclosure.

As shown in fig. 4, the method may include steps S401 to S403.

Step S401: in response to receiving a page query request for a full amount of data in a database table, a total number of records is obtained.

Step S402: based on the total number, demarcation points for the sequential query and the reverse query are determined, the demarcation points dividing the plurality of records into a first portion and a second portion.

Step S403: and sequentially inquiring the first part and reversely inquiring the second part to generate a paging inquiry result.

The method divides the full data into two parts, performs reverse order query on the second part arranged at the back and performs sequential query on the first part arranged at the front, thereby improving the query efficiency.

For step S401, in some embodiments of the present disclosure, in response to receiving a paging query request for a full amount of data, a statistics table of a database is queried, and a total number of the plurality of records is obtained. The statistics of the database are obtained by periodically counting records of the database tables in the database by the database system. The embodiment directly queries the statistical table to obtain the total number, so that the efficiency of obtaining the total number of records in the database table can be improved.

For example, by executing a SQL query against a statistics table of a database system, the total number of records in the database table to be queried is obtained. The statistics table is a built-in table in the database system, and the data of all tables in the database can be counted at regular time according to a certain trigger rule (insert or delete operation) and a built-in task, so that the SQL can be used for inquiring the statistics result of a certain large number of data tables.

For example, using show statistics for table sys _user to obtain the total number, sys_user is an example of the name of the database table, show statistics fot table is SQL operator.

In other embodiments of the present disclosure, the total number may also be obtained by SQL direct statistics database tables.

For step S402, for example, the demarcation point is a record with a sequence number equal to the total number/2 (i.e., one-half of the total number). If there is a remainder for the total number/2, the demarcation point may be determined rounded or rounded down, or rounded up.

In some embodiments of the present disclosure, the first portion is arranged before the second portion in the database table.

For step S403, for example, the first portion is queried sequentially, and the second portion is queried in reverse order to generate the paging query result.

For example, when the database queries the full data, each piece of ordered data has a pseudo column when the database executes the query, and the data to be acquired is calculated according to the pseudo column, so that the larger the number of pages is, the slower the query is, and the query rate can be improved in a halving way. For example: 102,837,243 data are ordered according to the primary key by 10 pages, namely 10,283,725 pages are all ordered, the inquiry can be ordered according to the sequence number when the front 5,141,862 pages are inquired, and the inquiry can be ordered according to the sequence number when the back 5,141,861 pages are inquired.

In other embodiments of the present disclosure, the front-end provided interactive interface may provide operational controls for sequential and reverse queries, through which a user may select either a sequential query (also referred to as a positive-sequence query, i.e., a query in the order of records stored in a database table) or a reverse-sequence query. In this embodiment, the method may further include obtaining a query pattern entered by the user, the query pattern indicating a sequential query or a reverse query on the database table. Responding to receiving a paging query request for the total data in the database table, and performing paging query on the database table to obtain a paging query result, wherein the paging query result comprises the following steps: and according to the query mode, carrying out paging query on the database table to obtain a paging query result.

For example, if the user selects a sequential query, the database table is queried sequentially, and if the user selects a reverse query, the database table is queried in reverse order.

When unconditional inquiry is carried out, a user can select positive-order inquiry or reverse-order inquiry according to the arrangement positions of records to be inquired in a database table, so that the inquiry efficiency is improved. For example, if the record to be queried by the user is located in the first half of the database table, then a positive-order query may be selected, and if the record to be queried by the user is located in the second half of the database table, then a negative-order query may be selected.

FIG. 5A illustrates a flow diagram of a method for paging a full volume of data provided in accordance with at least one embodiment of the present disclosure.

As shown in fig. 5A, the method may include steps S501 to S508.

Step S501: a page query request for a full-volume data page query is obtained.

Step S502: SQL is performed to obtain the total number of records in the database table.

Step S503: the total number of pages was calculated from the total number. Total number of pages = total number/number of pages +1.

Step S504: it is determined whether the number of query pages in the paging query request is less than the total number of pages/2.

If the number of query pages in the paging query request is less than or equal to the total number of pages/2, step S505 is executed; if the number of query pages in the paging query request is greater than the total number of pages/2, step S506 is performed.

Step S505: and executing the paging query SQL according to the number of pages and the number of each page.

Step S506: and calculating the demarcation point according to the number of pages and the number of each page. In this example, demarcation point = total number-number of pages x number of pages per page.

Step S507: for records located after the demarcation point, the paged query SQL is executed in reverse order.

Step S508: and outputting the query result.

FIG. 5B illustrates a flowchart of another method for paging a full volume of data provided in accordance with at least one embodiment of the present disclosure.

As shown in fig. 5B, the method may include steps S5.11 to S5.18.

Step S5.11: for example, the user performs an input operation in the interactive interface to input the query parameters. In this embodiment, the query parameter may be the number of query pages or the number of query pages. The interactive interface generates a data query request according to the query parameters and provides the data query request to the back end.

In some embodiments of the present disclosure, the user may also query the full volume data in the database table by default to return the result data to the interactive interface without entering the query parameters, but clicking directly on, for example, search control 5 in FIG. 3B. For example, the result data may be a default page number of page 1, a default number of display bars per page of 10.

Step S5.12: the back-end interface (e.g., written in Java code) invokes the PageFactoy class in the back-end technical framework Mybatis-plus embedded in Java code to establish a connection with the database.

Step S5.13: the PageUtil class is called to create a paged Java class object for receiving returned result data after the query is completed. The description of the PageUtil class is as described in step S3.20 above.

Step S5.14: the UserMapper class is called to query the total data amount in the database table for displaying the total number, calculating the total number of pages in the later steps. Similar to step S502 and step S503 in fig. 5A.

Step S5.15: firstly judging whether a user needs to inquire the data of the last page, and if the user needs to inquire the data of the last page, calling a UserMapper class to inquire the full data in a database table.

Step S5.16: judging whether the query page number in the paging query request is smaller than the total page number/2, and if so, calling a UserMapper class to directly perform query operation according to the sequence. If the number of query pages in the paging query request is greater than or equal to the total number of pages/2, step S5.17 is performed.

Step S5.17: the UserMapper class is called to query the database table in reverse order.

Step S3.22: the query result is generated according to steps S5.15 to S5.17, and may include current page data, page number of the current page, total page number, total number of bars, etc.

FIG. 6 illustrates a flow chart of a method of data processing performed by another database system provided by at least one embodiment of the present disclosure.

As shown in fig. 6, the database system includes a front end (e.g., browser) 610 and a back end 620, the back end 620 including a database, the database including a database table 621. The backend 620 is, for example, a server.

Front end 610 provides an interactive interface through which query results of a user query are presented to a user and input from the user is received. For example, the interactive interface provides ambiguous query functionality for account, name, and mailbox fields. The interactive interface is, for example, a page displayed by a browser.

As shown in fig. 6, when the total number of records of a certain database table in the database system is greater than a preset threshold, the back-end 620 is triggered to perform an asynchronous data update operation. The operations performed by the back end 620 to asynchronously update data are independent of the front end 610.

The asynchronous update data operation performed by the back end 620 is the operation described above with respect to fig. 1. For example, a second field (e.g., a "field) is created in the database table, and an inverted Index (field) is created from the second field. The operation of the asynchronously updated data is similar to the embodiment of FIG. 1, please refer to the description above with respect to FIG. 1.

In some embodiments of the present disclosure, the asynchronous update data operation performed by the back end 620 may include, in addition to operations similar to those of FIG. 1: in response to writing the newly added data into the database table through the back-end interface, the back-end 620 performs the addition logic. The adding logic writes corresponding newly added data to the second field asynchronously.

The back end 620 receives the data query request and, in response to the data query request, queries the database table for query results and provides the query results to the front end 610 for viewing by the user.

In some embodiments of the present disclosure, the operations of asynchronously updating data performed by the back-end and receiving a data query request and responding to the data query request are asynchronous operations. By executing in an asynchronous mode, the influence on the normal service operation of the system can be reduced as much as possible.

As shown in fig. 6, in some embodiments of the present disclosure, backend 620 provides both unconditional query functions and conditional query functions.

Unconditional queries include, for example, queries for the full amount of data in database tables, paged queries for the full amount of data, and the like. For example, the steps of fig. 4, 5A or 5B above are performed, please refer to the associated description of fig. 4, 5A or 5B. The conditional query function may be, for example, a fuzzy query. The data query request of the fuzzy query comprises query conditions, and the query conditions comprise field information of at least part of the selected fields. For example, performing the steps of fig. 3A and 3E above, please refer to the associated description of fig. 3A and 3E.

As shown in fig. 6, in response to receiving a data query request, it is determined whether a query condition including field information exists. If the query condition comprising field information exists, conditional query is executed; if the query condition including the field information does not exist, an unconditional query is performed.

Fig. 7 illustrates a schematic block diagram of a data processing apparatus 700 provided in at least one embodiment of the present disclosure.

For example, as shown in fig. 7, the data processing apparatus 700 includes a field creation unit 710 and an index creation unit 720. The data processing apparatus 700 is applied to a database comprising a database table comprising a plurality of first fields and a plurality of records.

The field creating unit 710 is configured to create a second field in the database table, where the second field is used to record the combination information of the selected field of each of the plurality of records, and the selected field is at least a part of the plurality of first fields.

The field creation unit 710 may perform step S10 described in fig. 1, for example.

The index creation unit 720 is configured to create an index based on the second field for data query.

The index creation unit 720 may perform, for example, step S20 described in fig. 1.

For example, the field creation unit 710 and the index creation unit 720 may be hardware, software, firmware, and any feasible combination thereof. For example, the field creation unit 710 and the index creation unit 720 may be dedicated or general-purpose circuits, chips, devices, or the like, or may be a combination of a processor and a memory. With respect to the specific implementation forms of the respective units described above, the embodiments of the present disclosure are not limited thereto.

It should be noted that, in the embodiment of the present disclosure, each unit of the data processing apparatus 700 corresponds to each step of the foregoing data processing method, and the specific function of the data processing apparatus 700 may refer to the related description of the data processing method, which is not repeated herein. The components and structures of data processing apparatus 700 shown in fig. 7 are exemplary only and not limiting, and data processing apparatus 700 may include other components and structures as desired.

At least one embodiment of the present disclosure also provides an electronic device comprising a processor and a memory including one or more computer program modules. One or more computer program modules are stored in the memory and configured to be executed by the processor, the one or more computer program modules comprising instructions for implementing the data processing methods described above. The electronic equipment can improve the query efficiency, can improve the response speed of the interface and the interface in big data application, and improves the usability of the system.

Fig. 8 is a schematic block diagram of an electronic device provided by some embodiments of the present disclosure. As shown in fig. 8, the electronic device 800 includes a processor 810 and a memory 820. Memory 820 is used to store non-transitory computer-readable instructions (e.g., one or more computer program modules). The processor 810 is configured to execute non-transitory computer readable instructions that, when executed by the processor 810, may perform one or more of the steps of the data processing method described above. The memory 820 and the processor 810 may be interconnected by a bus system and/or other forms of connection mechanisms (not shown).

For example, processor 810 may be a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), or other form of processing unit having data processing capabilities and/or program execution capabilities. For example, the Central Processing Unit (CPU) may be an X86 or ARM architecture, or the like. The processor 810 may be a general purpose processor or a special purpose processor that may control other components in the electronic device 800 to perform the desired functions.

For example, memory 820 may comprise any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory can include, for example, random Access Memory (RAM) and/or cache memory (cache) and the like. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, and the like. One or more computer program modules may be stored on the computer readable storage medium and executed by the processor 810 to implement the various functions of the electronic device 800. Various applications and various data, as well as various data used and/or generated by the applications, etc., may also be stored in the computer readable storage medium.

It should be noted that, in the embodiments of the present disclosure, specific functions and technical effects of the electronic device 800 may refer to the description of the data processing method hereinabove, and are not repeated herein.

Fig. 9 is a schematic block diagram of another electronic device provided by some embodiments of the present disclosure. The electronic device 900 is suitable for use, for example, to implement the data processing methods provided by the embodiments of the present disclosure. The electronic device 900 may be a terminal device, a server, etc. It should be noted that the electronic device 900 illustrated in fig. 9 is merely an example, and does not impose any limitation on the functionality and scope of use of the embodiments of the present disclosure.

As shown in fig. 9, the electronic device 900 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 910 that may perform various suitable actions and processes in accordance with programs stored in a Read Only Memory (ROM) 920 or loaded from a storage 980 into a Random Access Memory (RAM) 930. In the RAM930, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 910, the ROM 920, and the RAM930 are connected to each other by a bus 940. An input/output (I/O) interface 950 is also connected to bus 940.

In general, the following devices may be connected to the I/O interface 950: input devices 960 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, and the like; an output device 970 including, for example, a Liquid Crystal Display (LCD), speaker, vibrator, etc.; a storage device 980 including, for example, magnetic tape, hard disk, etc.; communication device 990. Communication device 990 may allow electronic device 900 to communicate wirelessly or by wire with other electronic devices to exchange data. While fig. 9 shows an electronic device 900 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided, and that electronic device 900 may alternatively be implemented or provided with more or fewer means.

For example, according to embodiments of the present disclosure, the above-described data processing method may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program, carried on a non-transitory computer readable medium, the computer program comprising program code for performing the above-described data processing method. In such embodiments, the computer program may be downloaded and installed from a network via communication device 990, or from storage device 980, or from ROM 920. The functions defined in the data processing method provided in the embodiment of the present disclosure may be implemented when the computer program is executed by the processing apparatus 910.

At least one embodiment of the present disclosure also provides a computer-readable storage medium having computer-readable instructions stored non-transitory thereon, which when executed by a processor, implement the above-described data processing method. By utilizing the computer readable storage medium, the query efficiency can be improved, the response speed of the interface and the interface can be improved in big data application, and the usability of the system is improved.

Fig. 10 is a schematic diagram of a storage medium according to some embodiments of the present disclosure. As shown in fig. 10, the storage medium 1000 non-transitory stores computer readable instructions 1010. For example, computer readable instructions 1010, when executed by a processor, may perform one or more steps in accordance with the data processing methods described above.

For example, the storage medium 1000 may be applied to the electronic device 800 described above. For example, the storage medium 1000 may be the memory 820 in the electronic device 800 shown in fig. 8. For example, the relevant description of the storage medium 1000 may refer to a corresponding description of the memory 820 in the electronic device 800 shown in fig. 8, which is not repeated here.

The following points need to be described:

(1) The drawings of the embodiments of the present disclosure relate only to the structures to which the embodiments of the present disclosure relate, and reference may be made to the general design for other structures.

(2) The embodiments of the present disclosure and features in the embodiments may be combined with each other to arrive at a new embodiment without conflict.

The foregoing is merely specific embodiments of the disclosure, but the scope of the disclosure is not limited thereto, and the scope of the disclosure should be determined by the claims.

Claims

A data processing method applied to a database, wherein the database comprises a database table comprising a plurality of first fields and a plurality of records, the method comprising:

creating a second field in the database table, wherein the second field is used for recording the combination information of the selected field of each of the plurality of records, and the selected field is at least part of the plurality of first fields;

an index is created for use in a data query based on the second field.
The method of claim 1, further comprising:

and responding to the received data query request for at least part of the selected fields, carrying out data query according to the index and returning a query result.
The method of claim 1 or 2, wherein the index comprises an inverted index.
A method according to any one of claims 1 to 3, wherein creating the second field in the database table comprises:

And creating the second field in the database table in response to the amount of data in the database table being greater than a preset threshold.
The method of any of claims 1-4, wherein the selection field comprises a field of the first plurality of fields in which a fuzzy query may be made.
The method of claim 2, wherein the selection field comprises at least two fields of the plurality of first fields,

creating the second field in the database table, comprising:

aiming at each record in the database table, splitting the information of each field in the at least two fields into a character string array according to a character string splitting rule; and

for each record in the database table, determining the value of the second field of each record according to the character string array of the at least two fields.
The method of claim 6, wherein the data query request includes a query condition of a fuzzy query, the query condition including field information of at least some of the selected fields,

responding to the data query request received for at least part of the selection fields, performing data query according to the index and returning the query result, and comprising the following steps:

Responding to the received data query request for at least part of the selected fields, and carrying out character string splitting on field information of each of the at least part of fields according to the character string splitting rule to obtain a character string array to be queried of each of the at least part of fields; and

and according to the index, inquiring the character string array to be inquired of each of the at least partial fields to obtain the inquiring result.
The method of claim 7, wherein the index comprises an inverted index,

according to the index, querying the character string array to be queried of each of the at least partial fields to obtain the query result, including:

querying the inverted index to obtain a character string array query result corresponding to each character string array to be queried of the at least partial fields, wherein the character string array query result comprises address information of records containing the character string array to be queried;

determining intersection of recorded address information of the character string array to be queried corresponding to the at least partial fields; and

and generating the query result according to the address information in the intersection.
The method of claim 8, wherein the field information of the at least partial field includes first field information of a first selected field and second field information of a second selected field,

responding to the data query request of at least part of the selected fields, carrying out character string splitting on the field information of each of the at least part of fields according to the character string splitting rule to obtain character strings to be queried of each of the at least part of fields, wherein the method comprises the following steps:

responding to the received data query request for at least part of the selected fields, carrying out character string splitting on the first field information to obtain a first character string array to be queried, and carrying out character string splitting on the second field information to obtain a second character string array to be queried;

querying the inverted index to obtain a string array query result corresponding to the string array to be queried of each of the at least partial fields, including:

querying the inverted index to obtain a first query result of the first character string array to be queried;

querying the inverted index to obtain a second query result of the second character string array to be queried;

Generating the query result according to the address information in the intersection, including:

and querying records corresponding to the address information in the intersection of the first query result and the second query result by using a fuzzy query operator, so as to eliminate records which do not meet the query condition in the records corresponding to the address information in the intersection, thereby obtaining the query result.
The method of claim 8, wherein the query conditions further include page number information,

querying the inverted index to obtain a string array query result corresponding to the string array to be queried of each of the at least partial fields, including:

and inquiring the inverted index according to the page number information to obtain a character string array inquiry result corresponding to the character string array to be inquired of each of the at least partial fields.
The method of claim 8, further comprising:

invoking a class of a first type in a persistence layer framework to establish a communication connection with the database;

invoking a class of a second type to create a data encapsulation object, wherein the data encapsulation object is used for encapsulating data returned by the database table;

querying the inverted index to obtain a character string array query result corresponding to the character string array to be queried of each of the at least partial fields:

And calling a class of a third type to query the inverted index to obtain a character string array query result corresponding to the character string array to be queried of each of the at least partial fields.
The method of any one of claims 1-11, further comprising:

and responding to receiving a paging query request for the total data in the database table, and performing paging query on the database table to obtain a paging query result.
The method of claim 12, wherein in response to receiving a paging query request for the full amount of data in the database table, paging the database table to obtain the paging query result comprises:

acquiring a total number of the plurality of records in response to receiving the paging query request for the total amount of data in the database table;

determining demarcation points for sequential and reverse queries according to the total number, wherein the demarcation points divide the plurality of records into a first portion and a second portion; and

and sequentially inquiring the first part and reversely inquiring the second part to generate the paging inquiry result.
The method of claim 12, further comprising:

Acquiring a query mode input by a user, wherein the query mode indicates that sequential query or reverse query is performed on the database table;

responding to receiving a paging query request for the total data in the database table, and performing paging query on the database table to obtain a paging query result, wherein the paging query result comprises the following steps:

and according to the query mode, carrying out paging query on the database table to obtain a paging query result.
The method of claim 13, wherein obtaining the total number of the plurality of records in response to receiving the page query request for the full amount of data in the database table comprises:

in response to receiving the page query request for the full amount of data, querying a statistics table of the database, obtaining a total number of the plurality of records,

the statistical table of the database is obtained by periodically counting records of the database table in the database by a database system.
The method of any of claims 6-11, wherein the string splitting rule comprises: every two adjacent two characters are split as a group.
A data processing apparatus for use with a database, wherein the database comprises a database table comprising a plurality of first fields and a plurality of records, the apparatus comprising:

A field creating unit configured to create a second field in the database table, where the second field is used to record combination information of a selection field of each of the plurality of records, and the selection field is at least a part of fields in the plurality of first fields;

and an index creation unit configured to create an index for data query based on the second field.
An electronic device, comprising:

a processor;

a memory comprising one or more computer program instructions;

wherein the one or more computer program instructions are stored in the memory and when executed by the processor implement the data processing method of any of claims 1-16.
A computer readable storage medium having computer readable instructions stored non-transitory, wherein the computer readable instructions, when executed by a processor, implement the data processing method of any one of claims 1-16.