CN115130043B - Database-based data processing method, device, equipment and storage medium - Google Patents
Database-based data processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN115130043B CN115130043B CN202211050745.XA CN202211050745A CN115130043B CN 115130043 B CN115130043 B CN 115130043B CN 202211050745 A CN202211050745 A CN 202211050745A CN 115130043 B CN115130043 B CN 115130043B
- Authority
- CN
- China
- Prior art keywords
- database
- target
- node
- sub
- result corresponding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
Abstract
The invention relates to the field of data processing, and discloses a database-based data processing method, device, equipment and storage medium, which are used for improving the accuracy of data mining. The method comprises the following steps: reading and analyzing the initial interface to obtain a target node, and searching the target node to obtain a target child node; performing sub-node grouping on the target sub-nodes according to the number of the databases to obtain a plurality of first sub-node sets; carrying out node sequence arrangement on the plurality of first child node sets to obtain child node arrangement results, and respectively distributing the plurality of first child node sets to a plurality of databases according to the child node arrangement results to obtain second child node sets; performing database operation on the second child node set to obtain a target operation result, and performing assignment on the target operation result to obtain an assignment result; and performing data fusion on the assignment result to obtain target data, and transcoding and outputting the target data.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a database-based data processing method, apparatus, device, and storage medium.
Background
Currently, no well-established definition or concept has been formed for big data. Big data is defined as big data (BigData), the main core of which is on big words, which means that in the process of data processing based on database, the big data is beyond the category of common data processing based on database, and the work of data processing based on database is difficult to be carried out by applying modern general software.
Therefore, how to perform data mining on the database becomes a hot research currently, but the accuracy of data mining is low in some existing data mining schemes.
Disclosure of Invention
The invention provides a database-based data processing method, a database-based data processing device, a database-based data processing equipment and a database-based data processing storage medium, which are used for improving the accuracy of data mining.
The first aspect of the present invention provides a database-based data processing method, including: receiving a target processing request to be processed, and analyzing the target processing request to obtain an initial interface corresponding to the target processing request; reading and analyzing the initial interface to obtain a target node, and performing node search on the target node to obtain a target sub-node corresponding to the target node; acquiring the number of preset databases corresponding to a plurality of databases, and performing sub-node grouping on the target sub-nodes according to the number of the databases to obtain a plurality of first sub-node sets; carrying out node sequence arrangement on the plurality of first sub-node sets to obtain sub-node arrangement results, and respectively allocating the plurality of first sub-node sets to the plurality of databases according to the sub-node arrangement results to obtain second sub-node sets corresponding to each database; performing database operation on the second sub-node set corresponding to each database to obtain a target operation result corresponding to each database, and assigning values to the target operation result corresponding to each database to obtain an assignment result corresponding to each database; and forwarding the assignment result corresponding to each database, performing data fusion on the assignment result corresponding to each database to obtain target data, and transcoding and outputting the target data.
Optionally, in a first implementation manner of the first aspect of the present invention, the reading and analyzing the initial interface to obtain a target node, and performing node search on the target node to obtain a target child node corresponding to the target node, includes: calling a preset data reading function to read the initial interface, and performing data analysis on the initial interface to obtain a target node; and calling a preset node search algorithm, and performing node search extraction on the target node to obtain a target child node corresponding to the target node.
Optionally, in a second implementation manner of the first aspect of the present invention, the obtaining a preset number of databases corresponding to a plurality of databases, and performing child node grouping on the target child node according to the number of databases to obtain a plurality of first child node sets includes: determining a plurality of databases based on preset database parameters, and calculating the number of the databases corresponding to the plurality of databases; determining the sub-node grouping number of the target sub-node according to the database number; and performing sub-node grouping on the target sub-nodes according to the sub-node grouping number to obtain a plurality of first sub-node sets.
Optionally, in a third implementation manner of the first aspect of the present invention, the performing node sequence arrangement on the plurality of first child node sets to obtain child node arrangement results, and respectively allocating the plurality of first child node sets to the plurality of databases according to the child node arrangement results to obtain a second child node set corresponding to each database includes: respectively acquiring variable data of each first sub-node set, and determining the target weight of each first sub-node set according to the variable data; carrying out node sequence arrangement on the plurality of first child node sets according to the target weight of each first child node set to obtain a child node arrangement result; and respectively distributing the plurality of first sub-node sets to the plurality of databases according to a preset distribution strategy and the sub-node arrangement result to obtain a second sub-node set corresponding to each database.
Optionally, in a fourth implementation manner of the first aspect of the present invention, the performing database operation on the second child node set corresponding to each database to obtain a target operation result corresponding to each database, and assigning values to the target operation result corresponding to each database to obtain an assignment result corresponding to each database includes: performing database operation on the second child node set corresponding to each database to obtain a target operation result corresponding to each database; and calling a preset assignment function, and assigning the target operation result corresponding to each database to obtain the assignment result corresponding to each database.
Optionally, in a fifth implementation manner of the first aspect of the present invention, the forwarding the assignment result corresponding to each database, performing data fusion on the assignment result corresponding to each database to obtain target data, and transcoding and outputting the target data includes: forwarding the assignment result corresponding to each database, and performing data fusion on the assignment result corresponding to each database to obtain target data; judging whether the target data meets a preset output condition or not; if yes, transcoding and outputting the target data; and if not, repeatedly calculating the assignment result corresponding to each database.
Optionally, in a sixth implementation manner of the first aspect of the present invention, the database-based data processing method further includes: obtaining a plurality of training data and a training model, the plurality of training data comprising: a plurality of target clause sets; model training is carried out on the training model based on the training data to obtain a trained target model; and carrying out sub-node grouping on the target sub-nodes based on the target model to obtain a plurality of first sub-node sets.
A second aspect of the present invention provides a database-based data processing apparatus, comprising: the receiving module is used for receiving a target processing request to be processed and analyzing the target processing request to obtain an initial interface corresponding to the target processing request; the analysis module is used for reading and analyzing the initial interface to obtain a target node, and searching the target node to obtain a target sub-node corresponding to the target node; the grouping module is used for acquiring the number of databases corresponding to a plurality of preset databases, and performing sub-node grouping on the target sub-nodes according to the number of the databases to obtain a plurality of first sub-node sets; the distribution module is used for carrying out node sequence arrangement on the plurality of first sub-node sets to obtain sub-node arrangement results, and distributing the plurality of first sub-node sets to the plurality of databases respectively according to the sub-node arrangement results to obtain second sub-node sets corresponding to each database; the assignment module is used for carrying out database operation on the second sub-node set corresponding to each database to obtain a target operation result corresponding to each database, and assigning values to the target operation result corresponding to each database to obtain an assignment result corresponding to each database; and the fusion module is used for forwarding the assignment result corresponding to each database, performing data fusion on the assignment result corresponding to each database to obtain target data, and transcoding and outputting the target data.
Optionally, in a first implementation manner of the second aspect of the present invention, the parsing module is specifically configured to: calling a preset data reading function to read the initial interface, and performing data analysis on the initial interface to obtain a target node; and calling a preset node search algorithm, and performing node search extraction on the target node to obtain a target child node corresponding to the target node.
Optionally, in a second implementation manner of the second aspect of the present invention, the grouping module is specifically configured to: determining a plurality of databases based on preset database parameters, and calculating the number of the databases corresponding to the plurality of databases; determining the sub-node grouping number of the target sub-node according to the database number; and carrying out sub-node grouping on the target sub-nodes according to the sub-node grouping number to obtain a plurality of first sub-node sets.
Optionally, in a third implementation manner of the second aspect of the present invention, the allocation module is specifically configured to: respectively acquiring variable data of each first sub-node set, and determining the target weight of each first sub-node set according to the variable data; carrying out node sequence arrangement on the plurality of first child node sets according to the target weight of each first child node set to obtain child node arrangement results; and respectively distributing the plurality of first sub-node sets to the plurality of databases according to a preset distribution strategy and the sub-node arrangement result to obtain a second sub-node set corresponding to each database.
Optionally, in a fourth implementation manner of the second aspect of the present invention, the assignment module is specifically configured to: performing database operation on the second child node set corresponding to each database to obtain a target operation result corresponding to each database; and calling a preset assignment function, and assigning the target operation result corresponding to each database to obtain the assignment result corresponding to each database.
Optionally, in a fifth implementation manner of the second aspect of the present invention, the fusion module is specifically configured to: forwarding the assignment result corresponding to each database, and performing data fusion on the assignment result corresponding to each database to obtain target data; judging whether the target data meet preset output conditions or not; if yes, transcoding and outputting the target data; and if not, repeatedly calculating the assignment result corresponding to each database.
Optionally, in a sixth implementation manner of the second aspect of the present invention, the database-based data processing apparatus further includes: a training module to obtain a plurality of training data and a training model, the plurality of training data comprising: a plurality of target clause sets; performing model training on the training model based on the plurality of training data to obtain a trained target model; and performing sub-node grouping on the target sub-nodes based on the target model to obtain a plurality of first sub-node sets.
A third aspect of the present invention provides a database-based data processing apparatus, comprising: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to cause the database-based data processing apparatus to perform the database-based data processing method described above.
A fourth aspect of the present invention provides a computer-readable storage medium having stored therein instructions, which, when run on a computer, cause the computer to execute the above-described database-based data processing method.
In the technical scheme provided by the invention, a plurality of first child node sets are obtained by acquiring the number of databases corresponding to a plurality of preset databases, and performing child node grouping on target child nodes according to the number of the databases, node sequence arrangement is performed on the plurality of first child node sets to obtain child node arrangement results, and the plurality of first child node sets are respectively distributed to the plurality of databases according to the child node arrangement results to obtain a second child node set corresponding to each database; and performing data fusion on the assignment results corresponding to each database to obtain target data, thereby improving the accuracy of data mining.
Drawings
FIG. 1 is a diagram of an embodiment of a database-based data processing method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of another embodiment of a database-based data processing method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an embodiment of a database-based data processing apparatus according to the embodiment of the present invention;
FIG. 4 is a schematic diagram of another embodiment of a database-based data processing apparatus according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an embodiment of a database-based data processing device in the embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a database-based data processing method, a database-based data processing device, a database-based data processing equipment and a database-based storage medium, which are used for improving the accuracy of data mining. The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Moreover, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
For convenience of understanding, a detailed flow of an embodiment of the present invention is described below, and referring to fig. 1, an embodiment of a database-based data processing method in an embodiment of the present invention includes:
101. receiving a target processing request to be processed, and analyzing the target processing request to obtain an initial interface corresponding to the target processing request;
it is to be understood that the execution subject of the present invention may be a data processing apparatus based on a database, and may also be a terminal or a server, which is not limited herein. The embodiment of the present invention is described by taking a server as an execution subject.
Specifically, the server receives a target processing request to be processed, analyzes the target processing request, and obtains a corresponding data identifier, and then the server obtains an initial interface corresponding to the target processing request from a preset database according to the data identifier.
102. Reading and analyzing the initial interface to obtain a target node, and searching the target node to obtain a target sub-node corresponding to the target node;
specifically, the server extracts a plurality of operator macros or mathematical expressions from a preset database, then verifies the equivalence of the mathematical expression of the target node, and if the mathematical expression is equivalent to the mathematical expression corresponding to the preset template data, the server obtains a target child node corresponding to the target node.
103. Acquiring the number of databases corresponding to a plurality of preset databases, and performing sub-node grouping on target sub-nodes according to the number of the databases to obtain a plurality of first sub-node sets;
specifically, the server obtains the number of databases corresponding to the multiple databases, further determines the number of sub-node groups according to the number of databases corresponding to the multiple databases, and finally performs sub-node grouping according to the number of sub-node groups to obtain multiple first sub-node sets.
104. Carrying out node sequence arrangement on the plurality of first child node sets to obtain child node arrangement results, and respectively distributing the plurality of first child node sets to the plurality of databases according to the child node arrangement results to obtain second child node sets corresponding to each database;
specifically, the server obtains variable data of each first sub-node set respectively, determines a target weight of each first sub-node set according to the variable data, performs node sequence arrangement on the plurality of first sub-node sets according to the target weight of each first sub-node set to obtain a sub-node arrangement result, and finally, the server distributes the plurality of first sub-node sets to the plurality of databases according to the sub-node arrangement result to obtain a second clause corresponding to each database.
105. Performing database operation on the second sub-node set corresponding to each database to obtain a target operation result corresponding to each database, and assigning values to the target operation result corresponding to each database to obtain an assignment result corresponding to each database;
specifically, database operation is performed on the second child node set corresponding to each database to obtain a target operation result corresponding to each database; calling a preset assignment function, assigning the target operation result corresponding to each database to obtain the assignment result corresponding to each database, wherein the assignment function maps the MLN structure with the target field to establish the association between the two fields, and then optimizing the structure obtained by mapping to adapt to the index of the target field, so as to assign the target operation result corresponding to each database.
106. And forwarding the assignment result corresponding to each database, performing data fusion on the assignment result corresponding to each database to obtain target data, and transcoding and outputting the target data.
Specifically, a preset random assignment restart algorithm is used for carrying out close node search on assignment results corresponding to each database to obtain a node assignment network table, then a server takes a target node as a center and carries out secondary search on the node assignment network table by combining similarity on behavior pattern vectors among different nodes to obtain the target node assignment network table, data fusion is carried out on the assignment results corresponding to each database to obtain target data, and transcoding and outputting are carried out on the target data.
In the embodiment of the invention, a plurality of first child node sets are obtained by obtaining the number of databases corresponding to a plurality of preset databases and performing child node grouping on target child nodes according to the number of the databases, a node sequence arrangement is performed on the plurality of first child node sets to obtain a child node arrangement result, and the plurality of first child node sets are respectively distributed to the plurality of databases according to the child node arrangement result to obtain a second child node set corresponding to each database; and performing data fusion on the assignment results corresponding to each database to obtain target data, thereby improving the accuracy of data mining.
Referring to fig. 2, another embodiment of the database-based data processing method according to the embodiment of the present invention includes:
201. receiving a target processing request to be processed, and analyzing the target processing request to obtain an initial interface corresponding to the target processing request;
specifically, in this embodiment, the specific implementation of step 201 is similar to that of step 101, and is not described herein again.
202. Reading and analyzing the initial interface to obtain a target node, and searching the target node to obtain a target sub-node corresponding to the target node;
specifically, a preset data reading function is called to read an initial interface, and data analysis is performed on the initial interface to obtain a target node; and calling a preset node search algorithm, and searching and extracting nodes of the target node to obtain a target sub-node corresponding to the target node.
The method comprises the steps that a server creates a CNF code, the CNF code is solved through a preset SAT solver, a preset data reading function is called to read an initial interface, data analysis is conducted on the initial interface, and a target node is obtained; invoking a preset node search algorithm, in an exemplary embodiment, since the data in the initial interface has been normalized to the same structural pattern, solving the CNF encoding of the conjunctive normal form is very simple for the SAT solver, it does not get stuck in an exponential complexity process as it is usually, and the solution can be completed in a few seconds, e.g., two equivalent logical data with the same structure, which the SAT solver can solve in polynomial time. For example, the SAT solver traverses two equivalent data and generates CNF clauses in topological order. Since the two equivalent data are exactly in the same implementation form, the SAT solver merges each pair of CNF clauses with the same inputs and subfunctions. The verification result can be obtained by traversing once in polynomial time, and finally the server obtains a target child node corresponding to the target node.
203. Acquiring the number of databases corresponding to a plurality of preset databases, and performing sub-node grouping on target sub-nodes according to the number of the databases to obtain a plurality of first sub-node sets;
specifically, a plurality of databases are determined based on preset database parameters, and the number of databases corresponding to the plurality of databases is calculated; determining the sub-node grouping number of the target sub-node according to the database number; and carrying out sub-node grouping on the target sub-nodes according to the sub-node grouping number to obtain a plurality of first sub-node sets.
The method comprises the steps that after a server screens a plurality of databases in the determination process, a corresponding CNF formula is constructed to serve as a permanent clause group, wherein the server dynamically selects a segmentation set, the equivalence of each pair of databases is verified, if signals in the segmentation set have dependency relations, namely a certain signal is a fan-out signal of other signals, the signal is deleted from the segmentation set, in addition, signals with a large number of fan-ins are selected as far as possible to enter the segmentation set, the server further realizes an increment satisfiability algorithm by grouping clauses of the CNF formula, and after a pair of signals are verified to be equivalent, sub-node grouping is carried out on target sub-nodes according to the sub-node grouping number, and a plurality of first sub-node sets are obtained.
204. Respectively acquiring variable data of each first sub-node set, and determining the target weight of each first sub-node set according to the variable data;
205. carrying out node sequence arrangement on the plurality of first child node sets according to the target weight of each first child node set to obtain child node arrangement results;
206. respectively distributing the plurality of first sub-node sets to a plurality of databases according to a preset distribution strategy and the arrangement result of the sub-nodes to obtain a second sub-node set corresponding to each database;
specifically, a weight value corresponding to each first sub-node set in the current process and the use times of each first sub-node set in the use record of the first sub-node set are obtained, and the weight optimization value of each first sub-node set is calculated by using the use times and the weight value; and according to the weight optimization value, the total number of the first child node sets and identification information of each first child node set, calculating an arrangement position value of each first child node set, arranging each first child node set according to the arrangement position value to obtain a first child node set sequence, and then respectively allocating the plurality of first child node sets to the plurality of databases by the server according to a preset distribution strategy and a child node arrangement result to obtain a second child node set corresponding to each database.
207. Performing database operation on the second sub-node set corresponding to each database to obtain a target operation result corresponding to each database, and assigning values to the target operation result corresponding to each database to obtain an assignment result corresponding to each database;
specifically, database operation is performed on the second child node set corresponding to each database to obtain a target operation result corresponding to each database; and calling a preset assignment function, and assigning the target operation result corresponding to each database to obtain the assignment result corresponding to each database.
Wherein, the encoding operation algorithm outputs code words according to the syntax element values. The encoding algorithm includes a look-up table, a suffix generator, and a combiner. The lookup table outputs a second string according to the syntax element value. The suffix generator performs exponential golomb binary encoding to generate a string according to the syntax element value. When the syntax element value is less than or equal to a threshold value, the meta-string is output as the codeword. When the syntax element value is larger than a critical value, the combiner combines the second binary character string and the second binary character string into the code word, and then the server obtains a target operation result corresponding to each database; and calling a preset assignment function, and assigning the target operation result corresponding to each database to obtain the assignment result corresponding to each database.
208. And forwarding the assignment result corresponding to each database, performing data fusion on the assignment result corresponding to each database to obtain target data, and transcoding and outputting the target data.
Specifically, the assignment result corresponding to each database is forwarded, and data fusion is performed on the assignment result corresponding to each database to obtain target data; judging whether the target data meet a preset output condition or not; if yes, transcoding and outputting the target data; and if not, repeatedly calculating the assignment result corresponding to each database.
The method comprises the steps that assignment results corresponding to each database of a server are forwarded, data fusion is carried out on the assignment results corresponding to each database, it is required to be explained that a preset system database stores received production data, basic data are provided for calculation assignment and verification comparison, a fixed assignment calculation model or a variable assignment calculation model is provided through correlation analysis of assignment and assigned values, assignment results are obtained, and whether target data meet preset output conditions or not is judged for the assigned target data; if yes, transcoding and outputting the target data; and if not, repeatedly calculating the assignment result corresponding to each database.
Optionally, a plurality of training data and training models are obtained, where the plurality of training data includes: a plurality of target clause sets; performing model training on the training model based on the plurality of training data to obtain a trained target model; and carrying out sub-node grouping on the target sub-nodes based on the target model to obtain a plurality of first sub-node sets.
In the embodiment of the invention, a plurality of first child node sets are obtained by obtaining the number of databases corresponding to a plurality of preset databases and performing child node grouping on target child nodes according to the number of the databases, a node sequence arrangement is performed on the plurality of first child node sets to obtain a child node arrangement result, and the plurality of first child node sets are respectively distributed to the plurality of databases according to the child node arrangement result to obtain a second child node set corresponding to each database; and performing data fusion on the assignment results corresponding to each database to obtain target data, thereby improving the accuracy of data mining.
With reference to fig. 3, the above describes a data processing method based on a database in an embodiment of the present invention, and the following describes a data processing apparatus based on a database in an embodiment of the present invention, where an embodiment of the data processing apparatus based on a database in an embodiment of the present invention includes:
a receiving module 301, configured to receive a target processing request to be processed, and analyze the target processing request to obtain an initial interface corresponding to the target processing request;
the analysis module 302 is configured to read and analyze the initial interface to obtain a target node, and perform node search on the target node to obtain a target child node corresponding to the target node;
the grouping module 303 is configured to obtain the number of databases corresponding to a plurality of preset databases, and perform child node grouping on the target child node according to the number of databases to obtain a plurality of first child node sets;
the allocating module 304 is configured to perform node sequence arrangement on the plurality of first child node sets to obtain child node arrangement results, and allocate the plurality of first child node sets to the plurality of databases according to the child node arrangement results to obtain a second child node set corresponding to each database;
an assignment module 305, configured to perform database operation on the second child node set corresponding to each database to obtain a target operation result corresponding to each database, and assign a value to the target operation result corresponding to each database to obtain an assignment result corresponding to each database;
and the fusion module 306 is configured to forward the assignment result corresponding to each database, perform data fusion on the assignment result corresponding to each database to obtain target data, and perform transcoding and output on the target data.
In the embodiment of the invention, a plurality of first child node sets are obtained by acquiring the number of databases corresponding to a plurality of preset databases, performing child node grouping on target child nodes according to the number of the databases, performing node sequence arrangement on the plurality of first child node sets to obtain child node arrangement results, and distributing the plurality of first child node sets to the plurality of databases respectively according to the child node arrangement results to obtain a second child node set corresponding to each database; and performing data fusion on the assignment results corresponding to each database to obtain target data, thereby improving the accuracy of data mining.
Referring to fig. 4, another embodiment of the database-based data processing apparatus according to the embodiment of the present invention includes:
a receiving module 301, configured to receive a target processing request to be processed, and analyze the target processing request to obtain an initial interface corresponding to the target processing request;
an analysis module 302, configured to read and analyze the initial interface to obtain a target node, and perform node search on the target node to obtain a target child node corresponding to the target node;
a grouping module 303, configured to obtain the number of databases corresponding to a plurality of preset databases, and perform child node grouping on the target child node according to the number of databases to obtain a plurality of first child node sets;
the allocating module 304 is configured to perform node sequence arrangement on the plurality of first child node sets to obtain child node arrangement results, and allocate the plurality of first child node sets to the plurality of databases according to the child node arrangement results to obtain a second child node set corresponding to each database;
an assignment module 305, configured to perform database operation on the second child node set corresponding to each database to obtain a target operation result corresponding to each database, and assign a value to the target operation result corresponding to each database to obtain an assignment result corresponding to each database;
and the fusion module 306 is configured to forward the assignment result corresponding to each database, perform data fusion on the assignment result corresponding to each database to obtain target data, and transcode and output the target data.
Optionally, the parsing module 302 is specifically configured to: calling a preset data reading function to read the initial interface, and performing data analysis on the initial interface to obtain a target node; and calling a preset node search algorithm, and performing node search extraction on the target node to obtain a target child node corresponding to the target node.
Optionally, the grouping module 303 is specifically configured to: determining a plurality of databases based on preset database parameters, and calculating the number of the databases corresponding to the plurality of databases; determining the sub-node grouping number of the target sub-node according to the database number; and performing sub-node grouping on the target sub-nodes according to the sub-node grouping number to obtain a plurality of first sub-node sets.
Optionally, the allocating module 304 is specifically configured to: respectively acquiring variable data of each first sub-node set, and determining the target weight of each first sub-node set according to the variable data; carrying out node sequence arrangement on the plurality of first child node sets according to the target weight of each first child node set to obtain a child node arrangement result; and respectively distributing the plurality of first sub-node sets to the plurality of databases according to a preset distribution strategy and the sub-node arrangement result to obtain a second sub-node set corresponding to each database.
Optionally, the assignment module 305 is specifically configured to: performing database operation on the second child node set corresponding to each database to obtain a target operation result corresponding to each database; and calling a preset assignment function, and assigning the target operation result corresponding to each database to obtain the assignment result corresponding to each database.
Optionally, the fusion module 306 is specifically configured to: forwarding the assignment result corresponding to each database, and performing data fusion on the assignment result corresponding to each database to obtain target data; judging whether the target data meets a preset output condition or not; if yes, transcoding and outputting the target data; and if not, repeatedly calculating the assignment result corresponding to each database.
Optionally, the database-based data processing apparatus further includes:
a training module 307, configured to obtain a plurality of training data and a training model, where the plurality of training data includes: a plurality of target clause sets; performing model training on the training model based on the plurality of training data to obtain a trained target model; and carrying out sub-node grouping on the target sub-nodes based on the target model to obtain a plurality of first sub-node sets.
In the embodiment of the invention, a plurality of first child node sets are obtained by obtaining the number of databases corresponding to a plurality of preset databases and performing child node grouping on target child nodes according to the number of the databases, a node sequence arrangement is performed on the plurality of first child node sets to obtain a child node arrangement result, and the plurality of first child node sets are respectively distributed to the plurality of databases according to the child node arrangement result to obtain a second child node set corresponding to each database; and performing data fusion on the assignment results corresponding to each database to obtain target data, thereby improving the accuracy of data mining.
Fig. 3 and fig. 4 above describe the database-based data processing apparatus in the embodiment of the present invention in detail from the perspective of the modular functional entity, and the database-based data processing apparatus in the embodiment of the present invention is described in detail below from the perspective of hardware processing.
Fig. 5 is a schematic structural diagram of a database-based data processing apparatus 500 according to an embodiment of the present invention, which may include one or more processors (CPUs) 510 (e.g., one or more processors) and a memory 520, and one or more storage media 530 (e.g., one or more mass storage devices) for storing applications 533 or data 532, where the database-based data processing apparatus 500 may generate relatively large differences due to different configurations or performances. Memory 520 and storage media 530 may be, among other things, transient or persistent storage. The program stored on the storage medium 530 may include one or more modules (not shown), each of which may include a series of instruction operations for the database-based data processing apparatus 500. Still further, the processor 510 may be configured to communicate with the storage medium 530 to execute a series of instruction operations in the storage medium 530 on the database-based data processing apparatus 500.
The database-based data processing apparatus 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input-output interfaces 560, and/or one or more operating systems 531, such as Windows Server, mac OS X, unix, linux, freeBSD, and the like. Those skilled in the art will appreciate that the database-based data processing apparatus configuration shown in fig. 5 does not constitute a limitation of database-based data processing apparatus and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
The present invention also provides a database-based data processing apparatus, which includes a memory and a processor, wherein the memory stores computer-readable instructions, and the computer-readable instructions, when executed by the processor, cause the processor to execute the steps of the database-based data processing method in the above embodiments.
The present invention also provides a computer readable storage medium, which may be a non-volatile computer readable storage medium, and which may also be a volatile computer readable storage medium, having stored therein instructions, which, when executed on a computer, cause the computer to perform the steps of the database-based data processing method.
Further, the computer-readable storage medium may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to the use of the blockchain node, and the like.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A database-based data processing method, comprising:
receiving a target processing request to be processed, and analyzing the target processing request to obtain an initial interface corresponding to the target processing request;
reading and analyzing the initial interface to obtain a target node, and performing node search on the target node to obtain a target sub-node corresponding to the target node;
acquiring the number of databases corresponding to a plurality of preset databases, and performing sub-node grouping on the target sub-nodes according to the number of the databases to obtain a plurality of first sub-node sets;
carrying out node sequence arrangement on the plurality of first sub-node sets to obtain sub-node arrangement results, and respectively allocating the plurality of first sub-node sets to the plurality of databases according to the sub-node arrangement results to obtain second sub-node sets corresponding to each database;
performing database operation on the second child node set corresponding to each database to obtain a target operation result corresponding to each database, and performing assignment on the target operation result corresponding to each database to obtain an assignment result corresponding to each database; specifically, database operation is performed on the second child node set corresponding to each database to obtain a target operation result corresponding to each database; calling a preset assignment function, assigning a target operation result corresponding to each database to obtain an assignment result corresponding to each database, wherein a coding operation algorithm outputs a codeword according to a syntax element value, the coding operation algorithm comprises a lookup table, a suffix generator and a combiner, the lookup table outputs a second binary character string according to the syntax element value, the suffix generator performs exponential Golomb binary coding according to the syntax element value to generate a character string, when the syntax element value is smaller than or equal to a critical value, the meta character string is output to be the codeword, when the syntax element value is larger than the critical value, the combiner combines the second binary character string and the second binary character string to be the codeword to obtain a target operation result corresponding to each database;
and forwarding the assignment result corresponding to each database, performing data fusion on the assignment result corresponding to each database to obtain target data, and transcoding and outputting the target data.
2. The database-based data processing method according to claim 1, wherein the reading and analyzing the initial interface to obtain a target node, and performing node search on the target node to obtain a target child node corresponding to the target node, comprises:
calling a preset data reading function to read the initial interface, and performing data analysis on the initial interface to obtain a target node;
and calling a preset node search algorithm, and searching and extracting nodes of the target node to obtain a target sub-node corresponding to the target node.
3. The database-based data processing method according to claim 1, wherein the obtaining of the preset number of databases corresponding to the plurality of databases and the performing of the sub-node grouping on the target sub-node according to the number of databases to obtain a plurality of first sub-node sets comprises:
determining a plurality of databases based on preset database parameters, and calculating the number of the databases corresponding to the plurality of databases;
determining the sub-node grouping number of the target sub-node according to the database number;
and carrying out sub-node grouping on the target sub-nodes according to the sub-node grouping number to obtain a plurality of first sub-node sets.
4. The database-based data processing method according to claim 1, wherein the performing node sequence arrangement on the plurality of first child node sets to obtain child node arrangement results, and respectively allocating the plurality of first child node sets to the plurality of databases according to the child node arrangement results to obtain a second child node set corresponding to each database includes:
respectively acquiring variable data of each first sub-node set, and determining the target weight of each first sub-node set according to the variable data;
carrying out node sequence arrangement on the plurality of first child node sets according to the target weight of each first child node set to obtain a child node arrangement result;
and respectively distributing the plurality of first sub-node sets to the plurality of databases according to a preset distribution strategy and the sub-node arrangement result to obtain a second sub-node set corresponding to each database.
5. The database-based data processing method of claim 1, wherein the performing database operations on the second child node set corresponding to each database to obtain a target operation result corresponding to each database, and assigning values to the target operation result corresponding to each database to obtain an assignment result corresponding to each database comprises:
performing database operation on the second child node set corresponding to each database to obtain a target operation result corresponding to each database;
and calling a preset assignment function, and assigning the target operation result corresponding to each database to obtain the assignment result corresponding to each database.
6. The database-based data processing method of claim 1, wherein the forwarding assignment results corresponding to each database, performing data fusion on the assignment results corresponding to each database to obtain target data, and performing transcoding and outputting on the target data, comprises:
forwarding the assignment result corresponding to each database, and performing data fusion on the assignment result corresponding to each database to obtain target data;
judging whether the target data meets a preset output condition or not;
if yes, transcoding and outputting the target data;
and if not, repeatedly calculating the assignment result corresponding to each database.
7. The database-based data processing method according to any of claims 1-6, wherein the database-based data processing method further comprises:
obtaining a plurality of training data and a training model, the plurality of training data comprising: a plurality of target clause sets;
performing model training on the training model based on the plurality of training data to obtain a trained target model;
and carrying out sub-node grouping on the target sub-nodes based on the target model to obtain a plurality of first sub-node sets.
8. A database-based data processing apparatus, characterized in that the database-based data processing apparatus comprises:
the receiving module is used for receiving a target processing request to be processed and analyzing the target processing request to obtain an initial interface corresponding to the target processing request;
the analysis module is used for reading and analyzing the initial interface to obtain a target node, and searching the target node to obtain a target sub-node corresponding to the target node;
the grouping module is used for acquiring the number of databases corresponding to a plurality of preset databases, and performing sub-node grouping on the target sub-nodes according to the number of the databases to obtain a plurality of first sub-node sets;
the distribution module is used for carrying out node sequence arrangement on the plurality of first sub-node sets to obtain sub-node arrangement results, and distributing the plurality of first sub-node sets to the plurality of databases respectively according to the sub-node arrangement results to obtain second sub-node sets corresponding to each database;
the assignment module is used for carrying out database operation on the second sub-node set corresponding to each database to obtain a target operation result corresponding to each database, and assigning values to the target operation result corresponding to each database to obtain an assignment result corresponding to each database; specifically, database operation is performed on the second child node set corresponding to each database to obtain a target operation result corresponding to each database; calling a preset assignment function, assigning a target operation result corresponding to each database to obtain an assignment result corresponding to each database, wherein a coding operation algorithm outputs a codeword according to a syntax element value, the coding operation algorithm comprises a lookup table, a suffix generator and a combiner, the lookup table outputs a second binary character string according to the syntax element value, the suffix generator performs exponential Golomb binary coding according to the syntax element value to generate a character string, when the syntax element value is smaller than or equal to a critical value, the meta character string is output to be the codeword, when the syntax element value is larger than the critical value, the combiner combines the second binary character string and the second binary character string to be the codeword to obtain a target operation result corresponding to each database;
and the fusion module is used for forwarding the assignment result corresponding to each database, performing data fusion on the assignment result corresponding to each database to obtain target data, and transcoding and outputting the target data.
9. A database-based data processing apparatus, characterized in that the database-based data processing apparatus comprises: a memory and at least one processor, the memory having instructions stored therein;
the at least one processor invokes the instructions in the memory to cause the database-based data processing apparatus to perform the database-based data processing method of any of claims 1-7.
10. A computer-readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, implement the database-based data processing method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211050745.XA CN115130043B (en) | 2022-08-30 | 2022-08-30 | Database-based data processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211050745.XA CN115130043B (en) | 2022-08-30 | 2022-08-30 | Database-based data processing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115130043A CN115130043A (en) | 2022-09-30 |
| CN115130043B true CN115130043B (en) | 2022-11-25 |
Family
ID=83387894
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211050745.XA Active CN115130043B (en) | 2022-08-30 | 2022-08-30 | Database-based data processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115130043B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115329151B (en) * | 2022-10-17 | 2023-03-14 | 北方健康医疗大数据科技有限公司 | Graph database optimization method and device, electronic equipment and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017031082A1 (en) * | 2015-08-14 | 2017-02-23 | California Institute Of Technology | Algebraic query language (aql) database management system |
| US10846286B2 (en) * | 2018-07-20 | 2020-11-24 | Dan Benanav | Automatic object inference in a database system |
| DE102018008923A1 (en) * | 2018-10-09 | 2020-05-20 | makmad.org e. V. | Process for creating an efficient, logically complete, ontological level in the extended relational database concept |
| CN113010547B (en) * | 2021-05-06 | 2023-04-07 | 电子科技大学 | Database query optimization method and system based on graph neural network |
-
2022
- 2022-08-30 CN CN202211050745.XA patent/CN115130043B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN115130043A (en) | 2022-09-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110609759B (en) | Fault root cause analysis method and device | |
| US7809758B2 (en) | Database and method of generating same | |
| US8271523B2 (en) | Coordination server, data allocating method, and computer program product | |
| CN110866029B (en) | sql statement construction method, device, server and readable storage medium | |
| CN101751333A (en) | Method, computer program and computer system for assisting in analyzing program | |
| CN105550225A (en) | Index construction method and query method and apparatus | |
| CN113420537B (en) | Method, device, equipment and storage medium for processing electronic form data | |
| CN115130043B (en) | Database-based data processing method, device, equipment and storage medium | |
| CN113495902A (en) | Data processing method and data standard management system | |
| CN111045670B (en) | Method and device for identifying multiplexing relationship between binary code and source code | |
| US20100005203A1 (en) | Method of Merging and Incremantal Construction of Minimal Finite State Machines | |
| US20060117252A1 (en) | Systems and methods for document analysis | |
| CN114218266A (en) | Data query method and device, electronic equipment and storage medium | |
| CN112529543A (en) | Method, device and equipment for verifying mutual exclusion relationship of workflow and storage medium | |
| CN115147020B (en) | Decoration data processing method, device, equipment and storage medium | |
| CN114328525A (en) | Data processing method and device | |
| CN104636474A (en) | Method and equipment for establishment of audio fingerprint database and method and equipment for retrieval of audio fingerprints | |
| CN114995719A (en) | List rendering method, device, equipment and storage medium | |
| JP6123372B2 (en) | Information processing system, name identification method and program | |
| CN113342647A (en) | Test data generation method and device | |
| CN112287005A (en) | Data processing method, device, server and medium | |
| CN111400320B (en) | Method and device for generating information | |
| CN117389908B (en) | Dependency analysis method, system and medium for interface automation test case | |
| CN113076721B (en) | Coding length control method and device based on XPath | |
| CN111258271B (en) | Cutting graph generation method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |