CN113901055A

CN113901055A - Data retrieval processing method and device and electronic equipment

Info

Publication number: CN113901055A
Application number: CN202111242322.3A
Authority: CN
Inventors: 杨磊; 王殿志
Original assignee: Lenovo Beijing Ltd
Current assignee: Lenovo Beijing Ltd
Priority date: 2021-10-25
Filing date: 2021-10-25
Publication date: 2022-01-07

Abstract

The application discloses a data retrieval processing method, a data retrieval processing device and electronic equipment, wherein a pre-polymerization view is constructed in advance and stored in a pre-polymerization cache region, the pre-polymerization view comprises a retrieval directory set and first retrieval results respectively corresponding to the retrieval directories, and the retrieval directories further comprise: index information containing form attributes with index features; on the basis, after the data retrieval request for the application form is obtained, the target attribute carried in the obtained request is matched with the form attribute in the retrieval directory in the pre-polymerization cache region, and finally a second retrieval result of the data retrieval request is generated based on the matching result. According to the scheme, the retrieval of the data source of the form data can be avoided under the condition that the requested form data exists in the pre-polymerization view, and therefore the form data can be automatically and quickly retrieved according to the form attributes.

Description

Data retrieval processing method and device and electronic equipment

Technical Field

The present application belongs to the technical field of data retrieval, and in particular, to a data retrieval processing method and apparatus, and an electronic device.

Background

The Low-Code Development Platform (LCDP) is a Development Platform that can generate an application program quickly without encoding (0 Code) or with a small amount of encoding, and is powerful in that it allows an end user to develop his own application program using an easily understandable visualization tool, rather than a conventional way of writing Code.

The applicant researches and discovers that when the low-code platform is applied to searching large-scale data (for example, searching hundreds of millions of data), the performance is extremely poor, and the user experience is greatly influenced.

Disclosure of Invention

Therefore, the application discloses the following technical scheme:

a method of retrieval processing of data, the method comprising:

acquiring a data retrieval request for an application form; the data retrieval request comprises a target attribute used as a retrieval basis;

matching the target attribute with the form attribute in the retrieval directory in the pre-polymerization cache region to obtain a matching result; the pre-polymerization cache region is stored with a pre-constructed pre-polymerization view, the pre-polymerization view includes a pre-generated search directory set and first search results corresponding to each search directory in the search directory set, and the search directories include: index information containing form attributes with index features;

and generating a second retrieval result of the data retrieval request based on the matching result.

Optionally, the generating a second search result of the data search request based on the matching result includes:

if the matching is successful, generating a second retrieval result of the data retrieval request based on a first retrieval result corresponding to a retrieval directory to which the form attribute matched with the target attribute belongs in the pre-polymerization cache region;

and if the matching fails, retrieving the data source of the form data of the corresponding form according to the target attribute so as to generate a second retrieval result of the data retrieval request.

Optionally, when the application form is released, the pre-polymerization view is constructed and stored in the pre-polymerization cache region;

the attributes of the form in the search directory are: when the application form is issued, the attribute with the index feature of the application form is identified, and the attribute with the index feature is classified to obtain the attribute of the corresponding category form;

the first retrieval result corresponding to the retrieval catalog comprises: the method comprises the steps of retrieving data sources and aggregating retrieval results of all the retrievals based on different form attributes in the same retrieval directory and different retrieval paths corresponding to the different form attributes in the data sources of form data of application forms; retrieving index information in a directory includes: and the data retrieval paths respectively correspond to the form attributes with the index features in the same category and the form attributes with the index features.

Optionally, the generating a second search result of the data search request based on a first search result corresponding to a search directory to which the form attribute matched with the target attribute belongs in the pre-polymerization cache region includes:

and taking the first retrieval result as the second retrieval result, or retrieving the first retrieval result according to the target attribute to generate the second retrieval result of the data retrieval request.

Optionally, the first search result corresponding to the search directory included in the pre-polymerization view is: when the application form is issued, retrieving a data source of form data of the application form to obtain a retrieval result matched with a form attribute with index features of the application form and a target data range corresponding to the form attribute with index features;

the target data range is a form data range with static requirements on the form attributes with the index characteristics when the application form is issued;

the matching of the target attribute with the form attribute in the retrieval directory in the pre-polymerization cache region comprises:

and matching the target attribute and the current data range corresponding to the target attribute with the form attribute in the retrieval directory in the pre-polymerization cache region and the target data range corresponding to the form attribute respectively.

Optionally, the retrieving the first retrieval result corresponding to the directory includes: the result obtained by searching the single data file in the data source;

the pre-polymerization view is constructed by the following steps:

determining form attributes which have index characteristics and can be used for performing combined retrieval on a plurality of data files in the data source in the form attributes of the application form to obtain the attributes of the combined retrieval form;

based on the determined attributes of the combined retrieval form, performing combined retrieval on a plurality of data files of the data source according to the characteristics of a data definition language and/or a data manipulation language of the data source of the application form to obtain a combined retrieval result;

adding a joint retrieval directory into the pre-polymerization view, and recording a corresponding joint retrieval result for the joint retrieval directory in the pre-polymerization view;

wherein the federated retrieval directory comprises: index information containing the attributes of the joint search form.

Optionally, the method further includes:

deleting the expired data in the pre-polymerization cache region through a preset data expiration strategy;

and/or updating data from a data source of the form data of the application form to the pre-aggregation cache through a predetermined data update policy.

Optionally, the form attribute of the application form includes a service data parameter bound to each form element of the application form;

the binding of the table elements and the corresponding service data parameters is realized by setting the service data parameters as the interface parameters of the dynamic data interfaces and configuring the corresponding dynamic data interfaces for the table elements.

An apparatus for processing retrieval of data, the apparatus comprising:

the acquisition module is used for acquiring a data retrieval request of the application form; the data retrieval request comprises a target attribute used as a retrieval basis;

the matching module is used for matching the target attribute with the form attribute in the retrieval directory in the pre-polymerization cache region; the pre-polymerization cache region is stored with a pre-constructed pre-polymerization view, the pre-polymerization view includes a pre-generated search directory set and first search results corresponding to each search directory in the search directory set, and the search directories include: index information containing form attributes with index features;

and the generating module is used for generating a second retrieval result of the data retrieval request based on the matching result.

An electronic device, comprising:

a memory for storing a set of computer instructions;

a processor for implementing the data retrieval processing method according to any one of the above by executing the instruction set stored in the memory.

As can be seen from the above solutions, the data retrieval processing method, apparatus and electronic device disclosed in the present application construct a pre-polymerization view in advance, and store the pre-polymerization view in a pre-polymerization cache region, where the pre-polymerization view includes a retrieval directory set and first retrieval results corresponding to the retrieval directories, and the retrieval directories further include: index information containing form attributes with index features; on the basis, after the data retrieval request for the application form is obtained, the target attribute carried in the obtained request is matched with the form attribute in the retrieval directory in the pre-polymerization cache region, and finally a second retrieval result of the data retrieval request is generated based on the matching result. According to the scheme, the pre-polymerization view is constructed in advance, and the pre-polymerization view stored in the pre-polymerization cache region is used as a retrieval basis, so that the retrieval of the data source of the form data can be avoided under the condition that the requested form data exists in the pre-polymerization view, the form data can be retrieved automatically and quickly according to the form attribute, and the retrieval performance of the low-code platform in the process of retrieving large-scale data is improved effectively.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

FIG. 1 is a logic diagram of a process for implementing fast retrieval of form data based on form attributes provided herein;

FIG. 2 is a flow diagram of an implementation of optimizing a pre-polymerization view through multi-table joint search as provided herein;

FIG. 3 is a schematic flow chart of a data retrieval processing method provided by the present application;

FIG. 4 is a schematic flow chart of another data retrieval method provided in the present application;

fig. 5 is a schematic structural diagram of a data retrieval processing device provided in the present application;

fig. 6 is a schematic structural diagram of an electronic device provided in the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The application discloses a data retrieval processing method, a data retrieval processing device and electronic equipment, wherein the method and the device can be applied to a server which is used for providing form data retrieval service in a server side, and the problem of extremely poor performance when a low-code platform is applied to retrieving large-scale data (such as retrieving hundreds of millions of data) is solved by automatically realizing the rapid retrieval of form data according to the form attributes of an application form.

Optionally, in the embodiment of the application, the application form is an application form constructed based on a low-code development platform, and the low-code development platform can reduce the development code amount and the development difficulty of a developer by providing a visual programming component and a software full-life-cycle management capability.

The application form is used for realizing the interactive transmission of information between the user side and the server side, the information collection from the user side, such as user retrieval requests, user orders and the like, can be realized through the application form, and then the collected information is processed by the server side and then fed back to the user side. Application forms include, but are not limited to, basic form elements of the type text field, button, check box, drop-down box, list/drop-down menu, and more complex form objects built based on the basic form elements, such as employee information list components with page flipping, query functions, etc. built based on input boxes, drop-down boxes.

The retrieval processing method, the retrieval processing device, and the electronic device disclosed in the embodiments of the present application support automatic and fast retrieval of form data according to form attributes of an application form by pre-constructing a pre-polymerization view and storing the pre-polymerization view in a pre-polymerization cache region, and first describe an implementation process of constructing a pre-polymerization view and storing the pre-polymerization view in a pre-polymerization cache region.

The pre-polymerization view is constructed and stored in the pre-polymerization buffer, and the implementation process can be executed by a pre-constructed pre-polymerization engine, and referring to fig. 1, the process specifically includes:

11) a user finishes interactive design of an application form on a low-code development platform, binds a dynamic data Interface (API), and determines attribute configuration of form elements, the API and the like through the process.

Specifically, a user can perform layout and associated design of form elements on a page through operation modes such as dragging, stacking and combining on a low-code development platform, perform attribute configuration on the designed form elements, and configure a dynamic data interface and interface attributes which need to be associated.

For example, the attributes of length, width, font, language (chinese/english) allowed to be input are configured for a certain table element, and the dynamic data interface (such as dynamic data list, dynamic search box, data perspective, etc.) to be interfaced, and the attribute information of address, parameter, etc. of the dynamic data interface to be interfaced are configured.

Meanwhile, the business data parameters (such as staff basic information, department information of each organization department of a company and the like) are set as interface parameters of the dynamic data interface, the corresponding dynamic data interface is configured for the form element, the form element and the corresponding business data parameters are bound, and the business data parameters (such as the names, the sexes, the jobs and the like of the staff) bound through the interface parameters are used as form attributes.

That is, the user performs interactive design of the application form on the low-code development platform, and binds the form attributes determined by the dynamic data interface, including not only the basic attributes of the form elements, such as length, width, font, language allowed to be input, etc., but also the business data parameters bound by each form element of the application form, such as employee gender, organization department to which the form belongs, job, etc.

12) And selecting a proper database engine by the user according to the self requirement, and executing the release of the application to which the application form belongs based on the selected database engine.

Specifically, after a user finishes the design of the low-code platform application based on the interactive design and the dynamic data interface binding of the application form, database engines such as Clickhouse, Hive, Spark, PostgreSQL, MySQL and the like can be selected according to the self requirements, and the release processing of the low-code application is executed through the selected database engines.

13) In the low-code application release process, the system executes a scanning and screening process for the low-code application form attribute configuration, and creates an index feature model based on the scanning and screening results by using a classification algorithm.

In the process of releasing the low-code platform application, the server-side system scans the properties of the released application form of the low-code platform application, such as the length, the width, the font, the language allowed to be input, the sex of the employee corresponding to the bound business data, the department to which the employee belongs, the job and other properties of the form element of the scanned application form. The scanned form attribute information is filtered, and attributes with index features are filtered, where the attributes with index features refer to attributes that can be used for performing data retrieval on form data (i.e., form-related business data), such as employee gender, department of the form, job, and the like.

Then, the attributes with the index features are further classified according to the association between the attributes based on a predetermined classification algorithm, and the classification algorithm adopted can be, but is not limited to, linear regression and other classification algorithms, for example, when querying the "staff with programming skills" based on the attribute of "programming skill", querying the "staff with programming skills" based on the combined attribute of "programming skill" + "gender male", the two need to query the same data model (e.g., the same data table in the table database), and according to the association, the attribute of "programming skill" and the combined attribute of "programming skill" + "gender male" are classified into one class based on the classification algorithm, and the like.

And creating an index feature model based on the classification result, the created index feature model comprising: and the form attributes with the index features in the same category respectively correspond to data retrieval paths in a data source of the form data. The data source of the form data may be, for example, a database of the form data, a data retrieval path of the form attribute corresponding to the data source of the form data may include retrieval path information of a data table, a data file, and the like corresponding to the form data indicated by the form attribute (e.g., employee data indicated by a "gender" attribute) in the form database.

In this embodiment, each category of data retrieval paths corresponding to each form attribute having an index feature and each form attribute having an index feature are referred to as a retrieval directory, and each retrieval directory forms a retrieval directory set. That is, in the embodiment, the index feature model is essentially a search directory set.

14) And creating a pre-polymerization view based on the index characteristic model of the low-code platform application and the selected database engine and storing the view into a pre-polymerization cache region.

Each type of indexed feature-equipped attribute in the index feature model (i.e., attribute in each search directory) corresponds to a set of data search paths.

When the low-code platform application is released, the embodiment searches the form data source such as the form database by using the data search implementation process corresponding to the selected database engine based on different form attributes in the same search directory and different search paths corresponding to the different form attributes in the form data source of the application form,

and aggregating the form attributes and the search results corresponding to each search by using a predetermined classification algorithm (such as a decision tree) to create a pre-aggregation view, wherein the created pre-aggregation view comprises a search directory set and first search results corresponding to each search directory in the search directory set, and the attributes, the search paths and the corresponding first search results in each search directory with search characteristics are organized into a view form aggregated based on the predetermined classification algorithm (such as the decision tree).

Or, in another aspect, the pre-polymerization view includes a pre-generated search list set and first search results corresponding to each search list in the search list set, and the search list in the search list set includes: index information containing form attributes with index features; the attributes of the form in the retrieval directory are specifically as follows: when the application form is issued, the attribute with the index feature of the application form is identified, and the attribute with the index feature is classified to obtain the attribute of the corresponding category form; the first retrieval result corresponding to the retrieval catalog comprises: based on different form attributes in the same retrieval directory and different retrieval paths corresponding to the different form attributes in the data source of the form data of the application form, retrieving the data source and aggregating the retrieval results of the retrieval (for example, the retrieval result of the employee with programming skills and the retrieval result of the employee with programming skills are combined into one retrieval result, namely, the retrieval result corresponding to the employee with programming skills is combined to save cache); retrieving index information in a directory includes: and the data retrieval paths respectively correspond to the form attributes with the index features in the same category and the form attributes with the index features.

After the pre-polymerization view is constructed, the pre-polymerization view is stored in a pre-polymerization cache region to be used in form data retrieval, and the process of form data retrieval is accelerated.

Further, in this embodiment of the application, preferably, the first search result corresponding to the search directory included in the pre-polymerization view is: and when the application form is issued, retrieving the data source of the form data of the application form to obtain a retrieval result matched with the form attribute with the index characteristic of the application form and the target data range corresponding to the form attribute with the index characteristic. The target data range is: and when the form is issued, the form data range with the static requirement of the form attribute with the index characteristic is applied. The search result matched with the form data range with the static requirement of the form attribute with the index characteristic when the application form is issued is only stored in the pre-polymerization cache region, so that the data volume of the data stored in the pre-polymerization cache region is reduced as much as possible on the premise of accelerating the process of searching the form data.

The following are exemplified:

assuming that when the form data of the "employee data" is queried according to the form attribute of the "employee gender", page turning is performed, and the upper limit of each page is 100 pieces of employee data, only the employee data in the range of 1-100 pieces of data that can be determined through analysis when the form is issued is stored in the cache region (that is, in an initial state when a low-code application is issued, the data range required by the application form in a static state is 1-100, and the data range required by a user when turning pages is further dynamically converted into 101-200), and the other page data is not cached.

In the pre-polymerization view constructed through the processing procedures of the above steps 11) -14), the first search result corresponding to the search directory is specifically a result obtained by searching a single data file (e.g., a single data table) in the data source of the application form (e.g., the form database of the application form).

In this embodiment of the present application, optionally, the following processing may also be included:

15) the optimized pre-polymerization view is jointly retrieved based on multiple data files (e.g., multiple data tables) corresponding to a single data source.

When the application is released, the pre-polymerization engine can further perform multi-table combined retrieval through the corresponding table form Data source, and optimize the pre-polymerization view by combining the application table form Data source such as the DDL (Data Definition Language) and DML (Data management Language) characteristics of the database.

In particular, referring to fig. 2, the process of optimizing the prepolymerization profile can be further realized as:

step 201, determining form attributes which have index characteristics and can be used for performing combined retrieval on a plurality of data files in a data source of an application form in the form attributes of the application form to obtain the attributes of the combined retrieval form;

for example, when searching and querying employee information according to the attribute of "organization/department" to which an employee belongs, it is necessary to first query which employees (for example, job numbers of employees under the organization) are under a certain department according to the "organization/department-employee" data table, and then further query the basic information data table of the employees to obtain detailed information of the employees under the organization, such as names, ages, family addresses, jobs, and the like.

Step 202, based on the determined attributes of the joint retrieval form, performing joint retrieval on a plurality of data files of the data source of the application form according to the characteristics of the data definition language and/or the data manipulation language of the data source of the application form to obtain a joint retrieval result;

specifically, a DML statement composed of related fields of organization and employee and used for performing joint retrieval on the organization information table and the basic employee information table may be created according to the DDL and DML characteristics of the data source of the application form, for example, a DML statement composed of related fields of org and user and used for performing joint retrieval on the organization information table and the basic employee information table may be created according to the DML characteristics of the PostgreSQL of the database: "select a. org _ id, a.org _ name, u.userid, u.username from org a left join user u where a. org _ id? And performing joint retrieval on a plurality of data tables of the corresponding form database based on the constructed DML statement.

And 203, adding a joint retrieval directory into the pre-polymerization view, and recording a corresponding joint retrieval result for the joint retrieval directory in the pre-polymerization view.

Then, aiming at the joint retrieval result, establishing a corresponding joint retrieval catalog, wherein the established joint retrieval catalog comprises the following steps: index information containing the attributes of the joint search form. For example, for the above example, an index of an org _ id field and the like are established, and the established joint search directory and the corresponding joint search result are stored in the pre-aggregation buffer in a correlated manner, so as to optimize the pre-aggregation view in the pre-aggregation buffer, so that the pre-aggregation view includes not only the search result and the search directory thereof corresponding to a single data file in the table single data source when the application is issued, but also the joint search result and the joint search directory thereof corresponding to multiple data files.

Referring to the flowchart of the data retrieval processing method provided in fig. 3 based on the pre-polymerization view stored in the pre-polymerization buffer, the data retrieval processing method disclosed in the embodiment of the present application at least includes the following steps:

301, acquiring a data retrieval request of an application form; the obtained data retrieval request comprises a target attribute used as a retrieval basis.

In the embodiment of the application, the form data is quickly retrieved according to the form attributes.

After the application instance is issued and started by the low-code platform application, referring to fig. 1 in combination, a user may initiate a retrieval/query request for form data by operating an application form, where the initiated data retrieval request includes a target attribute used as a retrieval basis, and the included target attribute may specifically be a related service data parameter of the application form to be queried.

For example, the user inputs the target attributes "employee gender" and "organization/department to which the employee belongs" as the search basis to initiate the search of employee information, and the service end correspondingly receives the data search request containing "employee gender" and "organization/department to which the employee belongs" from the user end, and uses "employee gender" and "organization/department to which the employee belongs" as the search basis.

And step 302, matching the target attribute carried in the data retrieval request with the form attribute in the retrieval directory in the pre-polymerization cache region to obtain a matching result.

The pre-polymerization cache region caches a pre-constructed pre-polymerization view, and according to the description of the pre-polymerization view, the pre-polymerization view includes a pre-generated search directory set and first search results respectively corresponding to each search directory in the search directory set, and each search directory in the search directory set includes: index information containing form attributes having index features. Specifically, the pre-polymerization view includes a single data table search result obtained by performing single data table query on single data and a joint search result obtained by performing joint query on a plurality of data tables when the low-code platform application is issued, and also includes a search directory corresponding to each query result.

After obtaining the data retrieval request submitted by the user side, the embodiment matches the target attribute (such as the sex of the employee, the organization department to which the employee belongs, and the like) which is carried in the data retrieval request and is used as the retrieval basis with the form attribute in the retrieval directory in the pre-aggregation cache region.

Further, in order to solve the problem that the first retrieval result corresponding to the retrieval directory in the pre-aggregation view is a retrieval result matched with the form attribute with the index feature and the form data range with the static requirement of the form, which is obtained by retrieving the form data source of the application form when the application form is issued, the target attribute carried in the data retrieval request and the current data range with the corresponding requirement of the target attribute are respectively matched with the form attribute in the retrieval directory in the pre-aggregation cache region and the data range (the data range with the static requirement when the application form is issued) corresponding to the form attribute and the form attribute.

Step 303, generating a second search result of the data search request based on the matching result.

And on the basis of executing the matching operation, generating a second retrieval result of the data retrieval request further based on the matching result.

Specifically, if the matching is successful, that is, a form attribute consistent with the target attribute exists in the search directory set of the pre-polymerization view, or a form attribute consistent with the target attribute exists and a data range corresponding to the form attribute in the pre-polymerization cache region is consistent with a current data range required by the target attribute, a second search result of the data search request is generated directly based on a first search result corresponding to the search directory to which the form attribute matched with the target attribute belongs in the pre-polymerization cache region.

Wherein, based on the first search result in the pre-polymerization cache region, a second search result of the data search request is generated, further comprising: and taking the first retrieval result as a second retrieval result of the data retrieval request, or carrying out secondary retrieval on the first retrieval result in the pre-polymerization cache region according to the target attribute in the data retrieval request, and taking the secondary retrieval result as the second retrieval result of the data retrieval request.

For example, in the case where the search result of the "employee with programming skills" and the search result of the "employee with programming skills" are merged into one search result (that is, the search result corresponding to the "employee with programming skills") in the pre-aggregation buffer, if the target attribute in the data search request is the "programming skill", the search result (first search result) obtained by merging may be directly used as the second search result of the data search request, and if the target attribute in the data search request includes the "programming skill" and the "male gender", the search result obtained by merging in the pre-aggregation buffer may be subjected to a secondary search using the target attributes of the "programming skill" and the "male gender", and the second search result of the data search request may be obtained by the secondary search.

Under the condition of successful matching, a second retrieval result of the data retrieval request can be obtained directly based on the pre-polymerization view in the pre-polymerization cache region, and a form data source of the application form, such as a form database, does not need to be retrieved, so that the data retrieval process of the application form is accelerated.

Otherwise, if the matching fails, the form data source (such as the form database) of the form is searched according to the target attribute carried in the data search request, and a second search result of the data search request is generated through the data source for searching the form data.

As can be seen from the above solution, in the method of this embodiment, a pre-polymerization view is pre-constructed and stored in the pre-polymerization cache region, the pre-polymerization view includes a search directory set and first search results respectively corresponding to the search directories, and the search directories further include: index information containing form attributes with index features; on the basis, after the data retrieval request for the application form is obtained, the target attribute carried in the obtained request is matched with the form attribute in the retrieval directory in the pre-polymerization cache region, and finally a second retrieval result of the data retrieval request is generated based on the matching result. According to the scheme, the pre-polymerization view is constructed in advance, and the pre-polymerization view stored in the pre-polymerization cache region is used as a retrieval basis, so that the retrieval of the data source of the form data can be avoided under the condition that the requested form data exists in the pre-polymerization view, the form data can be retrieved automatically and quickly according to the form attribute, and the retrieval performance of the low-code platform in the process of retrieving large-scale data is improved effectively.

In an embodiment, referring to a flowchart of a data retrieval processing method provided in fig. 4, the data retrieval processing method disclosed in the embodiment of the present application may further include the following processing:

step 304, deleting the expired data in the pre-polymerization cache region through a preset data expiration policy; and/or updating data from a data source of the form data of the application form to the pre-aggregation cache through a predetermined data update policy.

Optionally, the data expiration policy may be set as: periodically detecting whether a deletion event occurs to form data corresponding to data stored in a pre-polymerization cache region in a data source of the form data or not based on a preset time length, and if the deletion event occurs (causing corresponding data stored in the pre-polymerization cache region to be overdue), synchronously deleting overdue data corresponding to the deleted data in the pre-polymerization cache region;

the data update policy can be set as: periodically detecting whether a new event occurs to the form data corresponding to the data stored in the pre-polymerization cache region in the data source of the form data based on the preset time length, and if the new event occurs, synchronously updating the new data to the pre-polymerization cache region.

But not limited to, a callback function may be registered in advance with the data source end of the form data, and the data expiration policy or the data update policy may be set to execute the corresponding expired data deletion operation or data addition operation in the pre-aggregation cache region when a callback notification triggered by the data source end of the form data based on a data deletion or addition event is received.

On the basis, through a set corresponding strategy, deleting the expired data in the pre-polymerization cache region or updating the data from the form data source of the application form to the pre-polymerization cache region. For example, when a certain employee leaves, the employee data in the form database is deleted, which correspondingly causes the employee data cached in the pre-polymerization cache region to be overdue, and for this situation, the employee information in the pre-polymerization cache region may be deleted according to the data overdue policy; meanwhile, if the data of the newly-enrolled staff is added to the form database due to the fact that the newly-enrolled staff performs the job, the data information of the newly-enrolled staff can be synchronously added to the pre-polymerization cache region according to a data updating strategy.

According to the embodiment, the relevant outdated data in the pre-polymerization cache region is deleted or added with data according to the above strategy, so that the consistency between the pre-polymerization cache region and the data in the application form data source can be maintained, the validity of the data in the pre-polymerization cache region is ensured, and effective support is provided for accelerating the retrieval process of the form data.

Corresponding to the above method, the embodiment of the present application further discloses a data retrieval processing apparatus, where the apparatus has a structure as shown in fig. 5, and the apparatus at least includes:

an obtaining module 501, configured to obtain a data retrieval request for an application form; the data retrieval request comprises a target attribute used as a retrieval basis;

a matching module 502, configured to match the target attribute with a form attribute in a search directory in the pre-polymerization cache region; the pre-polymerization cache region is stored with a pre-constructed pre-polymerization view, the pre-polymerization view includes a pre-generated search directory set and first search results corresponding to each search directory in the search directory set, and the search directories include: index information containing form attributes with index features;

a generating module 503, configured to generate a second search result of the data search request based on the matching result.

In an embodiment, the generating module 503 is specifically configured to:

In one embodiment, when the application form is released, the pre-polymerization view is constructed and stored in a pre-polymerization cache region;

In an embodiment, when generating the second search result of the data search request based on the first search result corresponding to the search directory to which the form attribute matching the target attribute belongs in the pre-aggregation cache, the generating module 503 is specifically configured to:

In one embodiment, the first search result corresponding to the search directory included in the pre-polymerization view is: when the application form is issued, retrieving a data source of form data of the application form to obtain a retrieval result matched with a form attribute with index features of the application form and a target data range corresponding to the form attribute with index features;

the matching module 502 is specifically configured to: and matching the target attribute and the current data range corresponding to the target attribute with the form attribute in the retrieval directory in the pre-polymerization cache region and the target data range corresponding to the form attribute respectively.

In one embodiment, retrieving the first retrieval result corresponding to the directory includes: the result obtained by searching the single data file in the data source;

the pre-polymerization view is constructed by the following steps:

In one embodiment, the apparatus further comprises:

an update module to: deleting the expired data in the pre-polymerization cache region through a preset data expiration strategy; and/or updating data from a data source of the form data of the application form to the pre-aggregation cache through a predetermined data update policy.

In one embodiment, the form attribute of the application form includes a service data parameter bound to each form element of the application form;

The data retrieval processing device disclosed in the embodiments of the present application is relatively simple in description because it corresponds to the data retrieval processing method disclosed in the above embodiments of the method, and for the relevant similarities, please refer to the description of the above corresponding method embodiments, and details are not described here.

The embodiment of the present application further discloses an electronic device, where the electronic device may be, but is not limited to, a server in a server for providing a form data retrieval service, and a composition structure of the electronic device is shown in fig. 6, and specifically includes:

a memory 601 for storing a set of computer instructions;

the set of computer instructions may be embodied in the form of a computer program.

Processor 602 for implementing a method for retrieval processing of data as disclosed in any of the above method embodiments by executing a set of computer instructions.

The processor 602 may be a Central Processing Unit (CPU), an application-specific integrated circuit (ASIC), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device.

Besides, the electronic device may further include a communication interface, a communication bus, and the like. The memory, the processor and the communication interface communicate with each other via a communication bus.

The communication interface is used for communication between the electronic device and other devices. The communication bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like, and may be divided into an address bus, a data bus, a control bus, and the like.

In summary, the data retrieval processing method, apparatus and electronic device disclosed in the embodiments of the present application have at least the following technical advantages compared with the conventional technology:

21) under the same data scale, the query burden is automatically reduced by using smaller storage space consumption, the data retrieval process of the application form is accelerated, the retrieval efficiency is improved, and better user experience can be obtained;

22) the pre-polymerization creation is realized through a classification algorithm (such as a decision tree), and meanwhile, the pre-polymerization creation process is closely combined with a database engine (such as Clickhouse, Hive, Spark, Postgresql, MySQL and the like), so that the expansion capability of the existing database is fully utilized.

It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other.

For convenience of description, the above system or apparatus is described as being divided into various modules or units by function, respectively. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.

From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present application may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments of the present application.

Finally, it is further noted that, herein, relational terms such as first, second, third, fourth, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. A method of retrieval processing of data, the method comprising:

2. The method of claim 1, the generating a second search result of the data search request based on the matching result, comprising:

3. The method of claim 2, wherein the pre-polymerization view is built and stored in the pre-polymerization cache upon application form publication;

4. The method of claim 3, wherein generating the second search result of the data search request based on the first search result corresponding to the search directory to which the form attribute matching the target attribute belongs in the pre-polymerization cache comprises:

5. The method of claim 3, the pre-polymerization view comprising a first search result corresponding to a search directory as: when the application form is issued, retrieving a data source of form data of the application form to obtain a retrieval result matched with a form attribute with index features of the application form and a target data range corresponding to the form attribute with index features;

6. The method of claim 3, retrieving the first retrieval result corresponding to the catalog comprising: the result obtained by searching the single data file in the data source;

the pre-polymerization view is constructed by the following steps:

7. The method of claim 1, further comprising:

8. The method of claim 1, wherein the form attributes of the application form comprise business data parameters bound for each form element of the application form;

9. An apparatus for processing retrieval of data, the apparatus comprising:

10. An electronic device, comprising:

a memory for storing a set of computer instructions;

a processor for implementing a method of retrieving data as claimed in any one of claims 1 to 8 by executing a set of instructions stored in a memory.