CN118113789A - Data analysis method and device, electronic equipment and storage medium - Google Patents
Data analysis method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN118113789A CN118113789A CN202410006866.7A CN202410006866A CN118113789A CN 118113789 A CN118113789 A CN 118113789A CN 202410006866 A CN202410006866 A CN 202410006866A CN 118113789 A CN118113789 A CN 118113789A
- Authority
- CN
- China
- Prior art keywords
- data
- container
- database
- analysis
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000007405 data analysis Methods 0.000 title claims abstract description 44
- 238000003860 storage Methods 0.000 title claims abstract description 27
- 238000004458 analytical method Methods 0.000 claims abstract description 72
- 238000007781 pre-processing Methods 0.000 claims abstract description 13
- 238000012544 monitoring process Methods 0.000 claims abstract description 8
- 238000012549 training Methods 0.000 claims description 54
- 230000004044 response Effects 0.000 claims description 40
- 238000004590 computer program Methods 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 8
- 238000010276 construction Methods 0.000 claims description 5
- 238000002372 labelling Methods 0.000 claims description 5
- 230000006870 function Effects 0.000 description 18
- 238000012545 processing Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 14
- 230000005540 biological transmission Effects 0.000 description 4
- 238000013507 mapping Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000013145 classification model Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/256—Integrating or interfacing systems involving database management systems in federated or virtual databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44521—Dynamic linking or loading; Link editing at or after load time, e.g. Java class loading
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present disclosure relates to a data analysis method and apparatus, an electronic device, and a storage medium, where the method is applied to a database, and the database is a relational database, and includes: constructing a first container in a database server for managing the database, wherein the first container is used for finishing data preprocessing, the first container contains data to be analyzed, and the data to be analyzed corresponds to a first table in the database; monitoring the first table, and synchronizing the changed data to the first container under the condition that the data in the first table is changed; and carrying out first analysis on the data to be analyzed based on the first container to obtain a first analysis result. Therefore, when the first container performs data analysis, the changed data can be automatically preprocessed, and the changed data can be timely subjected to data analysis, so that the timeliness of the data analysis is improved. Moreover, the relational database has a semantic analysis function.
Description
Technical Field
The present disclosure relates to the field of data processing, and in particular, to a data analysis method and apparatus, an electronic device, and a storage medium.
Background
The data analysis may extract useful information in the mass data. Thus, data analysis may assist users in detailed research and summarization of data and conclusions may be drawn therefrom.
In data analysis, data to be analyzed needs to be obtained first, and the data to be analyzed needs to be preprocessed, for example: the table data is cleaned, or deduplicated, or converted into vector data, which requires a professional to do so separately. The preprocessed data is then analyzed.
However, in a scenario where data frequently fluctuates and timeliness of processing results is pursued, such a data processing method cannot meet the scenario needs.
Disclosure of Invention
In view of this, the present disclosure proposes a data analysis scheme.
According to an aspect of the present disclosure, there is provided a data analysis method applied to a database, the database being a relational database, including: constructing a first container in a database server for managing the database, wherein the first container is used for finishing data preprocessing, and the first container contains data to be analyzed, and the data to be analyzed corresponds to a first table in the database; monitoring the first table, and synchronizing the changed data to the first container under the condition that the data in the first table is changed; and carrying out first analysis on the data to be analyzed based on the first container to obtain a first analysis result.
In one possible implementation, the building a first container includes: acquiring a preset initial container, wherein the initial container comprises at least one scene table for storing data to be processed; loading a first model into the initial container in response to a first selection operation of the first model; determining first target data in the first table in response to a first input operation of a selection parameter, and storing the first target data in the scene table; and labeling at least one column of types in the scene table in response to a second input operation of the semantic parameters, and obtaining at least one type label.
In one possible implementation, the method further includes: and updating the first analysis result and the type label to the first table.
In one possible implementation manner, the first model is a trained initial model, the first container may complete model training, and the training process of the initial model includes: loading the initial model into the first container in response to a second selection operation of the initial model; loading a training table and a truth table in the database into the first container in response to a third selection operation of the training table and the truth table, the training table including training data, the truth table including truths corresponding to the training data; and training the initial model by using the training table and the truth table to obtain the first model.
In one possible implementation, the method further includes: determining display parameters based on the type label and the first analysis result, wherein the display parameters are used for representing display positions and display styles; displaying the data of the scene table according to the display parameters; the selection parameters are updated in response to a first altering operation of the selection parameters.
In one possible implementation, the method further includes: in response to a third input operation on the column identity, the data type, establishing a second table of blanks in a transaction container within a database server managing the database, the second table comprising at least: an input form, an output form, the transaction container comprising an external interface; generating an image file of the selected first container in response to a fourth selection operation of any first container, and loading the image file into the transaction container; determining second target data in a database based on the external interface, and storing the second target data in the input form; and carrying out second analysis on the data in the input form based on the image file to obtain a second analysis result, storing the second analysis result into the output form, and displaying the second analysis result.
In one possible implementation, the method further includes: and updating a third table preset in the database by using the output table, wherein the third table is not used for storing the second target data.
According to another aspect of the present disclosure, there is provided a data analysis apparatus applied to a database, the database being a relational database, including:
a first container construction unit, configured to construct a first container in a database server that manages the database, where the first container is used to complete data preprocessing, and the first container contains data to be analyzed, where the data to be analyzed corresponds to a first table in the database;
A data synchronization unit, configured to monitor the first table, and synchronize the changed data to the first container when the data in the first table changes;
and the first analysis result determining unit is used for carrying out first analysis on the data to be analyzed based on the first container to obtain a first analysis result.
In one possible implementation, the first container construction unit is configured to:
Acquiring a preset initial container, wherein the initial container comprises at least one scene table for storing data to be processed;
loading a first model into the initial container in response to a first selection operation of the first model;
Determining first target data in the first table in response to a first input operation of a selection parameter, and storing the first target data in the scene table;
and labeling at least one column of types in the scene table in response to a second input operation of the semantic parameters, and obtaining at least one type label.
In one possible implementation, the apparatus further includes:
and the first table updating unit is used for updating the first analysis result and the type label to the first table.
In one possible implementation manner, the first model is a trained initial model, the first container may complete model training, and the training process of the initial model includes:
Loading the initial model into the first container in response to a second selection operation of the initial model;
Loading a training table and a truth table in the database into the first container in response to a third selection operation of the training table and the truth table, the training table including training data, the truth table including truths corresponding to the training data;
And training the initial model by using the training table and the truth table to obtain the first model.
In one possible implementation, the apparatus further includes:
A display parameter determining unit, configured to determine a display parameter based on the type label and the first analysis result, where the display parameter is used to characterize a display position and a display style;
the display unit is used for displaying the data of the scene table according to the display parameters;
a selection parameter updating unit for updating the selection parameter in response to a first change operation to the selection parameter.
In one possible implementation, the apparatus is further configured to:
In response to a third input operation on the column identity, the data type, establishing a second table of blanks in a transaction container within a database server managing the database, the second table comprising at least: an input form, an output form, the transaction container comprising an external interface;
Generating an image file of the selected first container in response to a fourth selection operation of any first container, and loading the image file into the transaction container;
Determining second target data in a database based on the external interface, and storing the second target data in the input form;
Based on the image file, carrying out second analysis on the data in the input form to obtain a second analysis result,
Storing the second analysis result in the output table, and displaying the second analysis result.
In one possible implementation, the apparatus further includes:
And a third table updating unit for updating a third table preset in the database, which is not a table for storing the second target data, using the output table.
According to another aspect of the present disclosure, there is provided an electronic device including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing the instructions stored by the memory.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer program instructions, wherein the computer program instructions, when executed by a processor, implement the above-described method.
According to another aspect of the present disclosure, there is provided a computer program product comprising a computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when run in a processor of an electronic device, performs the above method.
In the embodiment of the disclosure, since the first container is built in the database server, the first table is easy to monitor, and in the case that the first table changes data, the changed data can be automatically synchronized to the first container. And because the first container can preprocess the data, the first container can automatically preprocess the changed data while carrying out data analysis, and the changed data can be timely subjected to data analysis, so that the timeliness of the data analysis is improved. Moreover, the relational database has a semantic analysis function. The operation is simple and convenient, and the user without data preprocessing skills or database development skills can also perform data analysis, so that the labor cost is saved.
Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features and aspects of the present disclosure and together with the description, serve to explain the principles of the disclosure.
Fig. 1 is a flow chart of a data analysis method according to an embodiment of the disclosure.
Fig. 2 is a schematic structural diagram of a data analysis device according to an embodiment of the disclosure.
Fig. 3 is a schematic structural diagram of an electronic device for data analysis according to an embodiment of the present disclosure.
Detailed Description
Various exemplary embodiments, features and aspects of the disclosure will be described in detail below with reference to the drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Although various aspects of the embodiments are illustrated in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
In addition, numerous specific details are set forth in the following detailed description in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order not to obscure the present disclosure.
Fig. 1 is a flow chart of a data analysis method according to an embodiment of the disclosure. As shown in fig. 1, the method includes:
S11, constructing a first container in a database server for managing the database, wherein the first container is used for finishing data preprocessing, the first container contains data to be analyzed, and the data to be analyzed corresponds to a first table in the database.
The database server may receive client requests, which may include requests to manipulate data of the database, as well as requests to analyze, requests to build containers, and the like. The database server may process client requests and provide services and resources. The database may be used to organize and store data.
The first container may be built within a database server in embodiments of the present disclosure. The database server may comprise at least one first container. The first container may perform data preprocessing. The data preprocessing here may include one or more of the following: cleaning, data vector conversion, normalization, etc. The data after data preprocessing can meet the requirement of data analysis. The first container may correspond to at least one table in the database, and for convenience of description below, the table corresponding to the first container is named as a first table. The first table may contain data to be analyzed. The data to be analyzed may be copied from the first table into the first container.
And S12, monitoring the first table, and synchronizing the changed data to the first container when the data in the first table is changed.
The first table contains at least one record, and each record may contain at least one data. The data of the first table may be editable data. Thus, one or more records may be added to the first table, or one or more records may be deleted, or the first one or more data in the first table may be modified (i.e., records in the first table may be modified). In the disclosed embodiments, a first table may be monitored, and in the event that the first table is populated with records, a newly populated record may be added to the first container. In the case where the first table is deleted a record, the record may be deleted from the first container. In the case where a record in the first table is modified, the modified record may be used to replace the corresponding record in the first container or to add the modified record to the first container.
Because the first container is built in the database server, the first table is easy to monitor, and the monitoring result can be responded in time. Moreover, as the first container corresponds to the first table, the function of monitoring the first table can be realized only by using the first container for a user, additional setting is not needed, the operation is simpler and more convenient, and the user experience is improved.
And S13, based on the first container, performing first analysis on the data to be analyzed to obtain a first analysis result.
The first container may contain data to be analyzed. The first analysis may include: data classification, and/or data ordering, etc. The first analysis result may include: and classifying the data to obtain a classification result and/or sorting the data to obtain a sorting result.
Illustratively, the e-commerce platform allows merchants to modify commodity records at any time, so the commodity records of the database may be deleted, added, or modified at any time. Then, in the scene that the e-commerce platform classifies the sold commodities according to sales volume and sets the display position of the commodities on the homepage according to classification results, record changes can be responded in time in the classification process, and the accuracy and timeliness of subsequent analysis are improved. For example: under the condition that the electronic commerce platform classifies the on-sale commodities according to sales, the commercial tenant A adds a commodity a, so that a commodity record is added into the database; when this newly added item record is monitored, it may be automatically added to the first container. Thus, after the sorting operation, the commodity a can also be displayed on the homepage. For another example: under the condition that the e-commerce platform classifies the on-sale commodities according to sales, a merchant B puts off a shelf a commodity B because the commodity source is not in time for supplying the commodity B, so that a commodity record is deleted from a database; when it is detected that the item record is deleted, the item record may also be deleted from the first container. Thus, after the sorting operation, the commodity b is not displayed on the homepage. The above is merely an example, and embodiments of the present disclosure are not limited to application scenarios.
In the embodiment of the disclosure, since the first container is built in the database server, the first table is easy to monitor, and in the case that the first table changes data, the changed data can be automatically synchronized to the first container. And because the first container can preprocess the data, the first container can automatically preprocess the changed data while carrying out data analysis, and the changed data can be timely subjected to data analysis, so that the timeliness of the data analysis is improved. Moreover, the relational database has a semantic analysis function. The operation is simple and convenient, and the user without data preprocessing skills or database development skills can also perform data analysis, so that the labor cost is saved.
In one possible implementation, the building a first container includes: acquiring a preset initial container, wherein the initial container comprises at least one scene table for storing data to be processed; loading a first model into the initial container in response to a first selection operation of the first model; determining first target data in the first table in response to a first input operation of a selection parameter, and storing the first target data in the scene table; and labeling at least one column of types in the scene table in response to a second input operation of the semantic parameters, and obtaining at least one type label.
The initial container may be a container preset in the database server. The initial container may implement a data preprocessing function. In addition, the initial container can realize the function of monitoring the database table; after the selection parameters are entered, the first form is specified, at which point the initial container's monitoring function for the first form may be activated. Wherein, the selection parameters may include: an identification of the first table, a row identification and a column identification of the first target data in the first table. The selection parameters may characterize which table in the database the data to be analyzed is stored in, and which region in the table is stored.
One or more scene tables may be contained in the first container, and the scene tables may contain data to be analyzed. At least one column of data may be included in the scene table. Through a second input operation, the type of column in the scene table may be annotated. The semantic parameters characterize the type of a single column as a primary key column, or a numerical column, or an enumeration column, or an foreign key column.
Through a second input operation, each scene table in the first container can contain a column type, so that the model can conveniently conduct data analysis. Moreover, in some scenarios, the input data for the model may be from multiple scenario tables. Then, the first container may establish a first mapping relationship between the foreign key column of one scene table and the main key column of the other scene table, and correspond the records of the two scene tables according to the first mapping relationship. For example: the scene table A and the scene table B both contain commodity identification columns, the commodity identification columns of the scene table A can be marked as main key columns, and the commodity identification columns of the scene table B can be marked as external key columns. Then, the first container may establish a first mapping relationship according to the commodity identification column. The first mapping relationship may correspond records of the same commodity identifier in the scene table a and the scene table B. Therefore, in the embodiments of the present disclosure, since the type of column of the scene table can be identified, the accuracy of data analysis can be improved.
The first model may be a model for data analysis and the first model specific class may be related to the data analysis to be performed. For example: the first model may be a classification model, requiring data classification. For another example: the data ordering is required and the first model may be an ordering model.
After the first container is built, the first container may include a first model and a scene table. The scene table may contain data to be analyzed. The first container also records the type of at least one column in the scene table.
In the embodiment of the present disclosure, the user can construct the first container through a simple input operation and a selection operation. In addition, the type label is semantic information, the first container can also obtain the semantic information of the data in the scene table, and the accuracy of data analysis is improved.
In one possible implementation, the method further includes: and updating the first analysis result and the type label to the first table.
In the disclosed embodiments, the database may be a relational database.
As previously described, the first analysis result may be a data classification result or may be a data sorting result, which is a kind of semantic information. And the type label is also semantic information. That is, through the data analysis of the first container, semantic information of the data to be analyzed can be obtained.
In the embodiment of the disclosure, the relational database can have the function of semantic annotation of data. Moreover, the semantic information of the data to be analyzed can be updated into the database from the first container, so that the data in the database has the semantic information, and the subsequent use is convenient.
In one possible implementation manner, the first model is a trained initial model, the first container may complete model training, and the training process of the initial model includes: loading the initial model into the first container in response to a second selection operation of the initial model; loading a training table and a truth table in the database into the first container in response to a third selection operation of the training table and the truth table, the training table including training data, the truth table including truths corresponding to the training data; and training the initial model by using the training table and the truth table to obtain the first model.
In the embodiment of the disclosure, the initial container and the first container also have model training functions. That is, the parameters of the initial model may be adjusted by training the model, and training may be stopped when the training preset condition is satisfied.
The initial model may be a classification model, or a ranking model. In order to improve the adaptation degree of the data analysis operation and the scene, the initial model can be trained to obtain a first model.
A single training table may be suitable for one or more scenarios, and the data in the training table may be modified as needed for the scenario. A single training table may correspond to one truth table, where records in the training table may correspond one-to-one to records in the truth table.
In the embodiment of the disclosure, a common user can perform simple operation to train the initial model to obtain the first model applicable to the current scene, so that the accuracy of data analysis is improved, and the model training function of the relational database is also realized. Moreover, because the operation is simple, the operation of artificial intelligence professionals is not needed, the labor cost is saved, and the user experience is improved.
In one possible implementation, the method further includes: determining display parameters based on the type label and the first analysis result, wherein the display parameters are used for representing display positions and display styles; displaying the data of the scene table according to the display parameters; the selection parameters are updated in response to a first altering operation of the selection parameters.
The data in the scene table may include data to be analyzed and a first analysis result.
In the embodiment of the disclosure, the display style of the first analysis result may be determined according to the type label. The display style may include: scatter plots, pie charts, bar charts, line charts, and the like. For example: the first container performs a first analysis on the array of values, and a scatter plot may be used to present the first analysis result. For another example: the first container is for performing a first analysis on the enumeration column, and a line graph may be used to present a first analysis result.
In the embodiment of the disclosure, the display position may be determined according to the first analysis result. In the scenario that the e-commerce platform classifies the sold goods according to sales volume and sets the display position of the goods on the homepage according to classification results (first analysis results), the first analysis results may be sales volume grades of the goods, for example, three grades are classified: a first level, a second level, and a third level. The first level is characterized by more sales, the second level is characterized by more sales, and the third level is characterized by less sales. The homepage is divided into an upper area, a middle area and a lower area. Thus, after the first analysis result is obtained, the commodity labeled with the first level may be set in the upper region, the commodity labeled with the second level may be set in the middle region, and the commodity labeled with the third level may be set in the lower region. The above is merely an example, and the embodiments of the present disclosure do not limit the first analysis result and the display position.
The method in the embodiment of the disclosure can visualize the first analysis result, and the user can select the first target data from the first table again according to the display screen so as to update the data in the scene table. Because the data analysis method in the embodiment of the application is simple and convenient to operate, users without artificial intelligence expertise can obtain satisfactory analysis results through several attempts. The labor cost is saved.
In one possible implementation, the method further includes: in response to a third input operation on the column identity, the data type, establishing a second table of blanks in a transaction container within a database server managing the database, the second table comprising at least: an input form, an output form, the transaction container comprising an external interface; generating an image file of the selected first container in response to a fourth selection operation of any first container, and loading the image file into the transaction container; determining second target data in a database based on the external interface, and storing the second target data in the input form; and carrying out second analysis on the data in the input form based on the image file to obtain a second analysis result, storing the second analysis result into the output form, and displaying the second analysis result.
The transactional container may be a container that has a sequence of database operations built in. The transaction container is atomic and isolated in performing a sequence of database operations. The transaction container contains an external interface. The user may designate data in the database as data to be processed using the external result, or may designate data of other data sources as data to be processed.
The user may determine, through a third input operation, a column identification and a data type of the second table to establish the second table in the transaction container. The second table may include: input form, output form. The input form is used for storing data to be processed, and the output form is used for storing a processing result, namely a second analysis result. The second table may also include an intermediate table for storing intermediate results of the processing. The data entered into the form may be specified by the user from the database via an external interface. The user can also determine the first container through a fourth selection operation, and load the image file of the first container into the transaction container, so that the transaction container can have a data analysis function. Because the transaction container only needs to load the mirror image file of the first container, the transaction container occupies less resources, and is easy to realize parallel execution of a plurality of transaction containers, thereby improving the data analysis efficiency or meeting the requirement of simultaneous data analysis of multiple users.
In addition, the transaction container has isolation, so that the risk of modifying database data when an external user performs data analysis can be reduced, and the safety of the data is improved.
And, the transaction container can visually display the second analysis result, and the external user can adjust the input data of the second analysis according to the display result. The transaction container may have the advantage of the first container, since the second analysis may be performed by the image file, which is generated using the selected first container. The method can protect the data of the database and enable the external user who does not know the expertise to perform data analysis.
In one possible implementation, the method further includes: and updating a third table preset in the database by using the output table, wherein the third table is used for storing second target data.
As previously described the transaction container may be available to external users and in order to protect the data of the database, the second analysis result may be updated to the third table instead of back to the second table. After the third form is internally reviewed, it can be updated to the second form.
Illustratively, the merchant C sorts complaints of each commodity sold on the e-commerce platform by itself, determines data to be processed from the second table of the database by using the transaction container, and analyzes the data to obtain a sorting result. The transaction container updates the ordering result to the third table instead of the second table. Because the ordering result may be incorrect, if updated directly back to the second table, the database may store unreliable data, affecting data reliability. After the e-commerce platform examines the sorting result, if the sorting result is correct, the sorting result is updated to the second table, and if the sorting result is correct, the merchant C is notified, and the sorting result is not accepted.
Fig. 2 is a schematic structural diagram of a data analysis device according to an embodiment of the disclosure. As shown in fig. 2, the apparatus 200 includes:
the method is applied to a database, wherein the database is a relational database and comprises the following steps:
A first container construction unit 201, configured to construct a first container in a database server that manages the database, where the first container is used to complete data preprocessing, and the first container contains data to be analyzed, and the data to be analyzed corresponds to a first table in the database;
a data synchronization unit 202, configured to monitor the first table, and in case of a change in data in the first table, synchronize the changed data to the first container;
and a first analysis result determining unit 203, configured to perform a first analysis on the data to be analyzed based on the first container, to obtain a first analysis result.
In a possible implementation manner, the first container construction unit 201 is configured to:
Acquiring a preset initial container, wherein the initial container comprises at least one scene table for storing data to be processed;
loading a first model into the initial container in response to a first selection operation of the first model;
Determining first target data in the first table in response to a first input operation of a selection parameter, and storing the first target data in the scene table;
and labeling at least one column of types in the scene table in response to a second input operation of the semantic parameters, and obtaining at least one type label.
In one possible implementation, the apparatus 200 further includes:
and the first table updating unit is used for updating the first analysis result and the type label to the first table.
In one possible implementation manner, the first model is a trained initial model, the first container may complete model training, and the training process of the initial model includes:
Loading the initial model into the first container in response to a second selection operation of the initial model;
Loading a training table and a truth table in the database into the first container in response to a third selection operation of the training table and the truth table, the training table including training data, the truth table including truths corresponding to the training data;
And training the initial model by using the training table and the truth table to obtain the first model.
In one possible implementation, the apparatus 200 further includes:
A display parameter determining unit, configured to determine a display parameter based on the type label and the first analysis result, where the display parameter is used to characterize a display position and a display style;
the display unit is used for displaying the data of the scene table according to the display parameters;
a selection parameter updating unit for updating the selection parameter in response to a first change operation to the selection parameter.
In one possible implementation, the apparatus 200 is further configured to:
In response to a third input operation on the column identity, the data type, establishing a second table of blanks in a transaction container within a database server managing the database, the second table comprising at least: an input form, an output form, the transaction container comprising an external interface;
Generating an image file of the selected first container in response to a fourth selection operation of any first container, and loading the image file into the transaction container;
Determining second target data in a database based on the external interface, and storing the second target data in the input form;
Based on the image file, carrying out second analysis on the data in the input form to obtain a second analysis result,
Storing the second analysis result in the output table, and displaying the second analysis result.
In one possible implementation, the apparatus 200 further includes:
And a third table updating unit for updating a third table preset in the database, which is not a table for storing the second target data, using the output table.
In some embodiments, functions or modules included in an apparatus provided by the embodiments of the present disclosure may be used to perform a method described in the foregoing method embodiments, and specific implementations thereof may refer to descriptions of the foregoing method embodiments, which are not repeated herein for brevity.
The disclosed embodiments also provide a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method. The computer readable storage medium may be a volatile or nonvolatile computer readable storage medium.
The embodiment of the disclosure also provides an electronic device, which comprises: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing the instructions stored by the memory.
Embodiments of the present disclosure also provide a computer program product comprising computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when run in a processor of an electronic device, performs the above method.
Fig. 3 is a schematic structural diagram of an electronic device for data analysis according to an embodiment of the present disclosure. For example, electronic device 1900 may be provided as a server or terminal device. Referring to FIG. 3, electronic device 1900 includes a processing component 1922 that further includes one or more processors and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by processing component 1922. The application programs stored in memory 1932 may include one or more modules each corresponding to a set of instructions. Further, processing component 1922 is configured to execute instructions to perform the methods described above.
The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output interface 1958 (I/O interface). The electronic device 1900 may operate based on an operating system stored in memory 1932, such as Windows Server TM,Mac OS XTM,UnixTM,LinuxTM,FreeBSDTM or the like.
In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 1932, including computer program instructions executable by processing component 1922 of electronic device 1900 to perform the methods described above.
The present disclosure may be a system, method, and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for causing a processor to implement aspects of the present disclosure.
The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: portable computer disks, hard disks, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static Random Access Memory (SRAM), portable compact disk read-only memory (CD-ROM), digital Versatile Disks (DVD), memory sticks, floppy disks, mechanical coding devices, punch cards or in-groove structures such as punch cards or grooves having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, are not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses through fiber optic cables), or electrical signals transmitted through wires.
The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.
The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as SMALLTALK, C ++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present disclosure are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), with state information of computer readable program instructions, which can execute the computer readable program instructions.
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the technical improvements in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (10)
1. A data analysis method, characterized by being applied to a database, the database being a relational database, comprising:
Constructing a first container in a database server for managing the database, wherein the first container is used for finishing data preprocessing, and the first container contains data to be analyzed, and the data to be analyzed corresponds to a first table in the database;
Monitoring the first table, and synchronizing the changed data to the first container under the condition that the data in the first table is changed;
And carrying out first analysis on the data to be analyzed based on the first container to obtain a first analysis result.
2. The method of claim 1, wherein the constructing a first container comprises:
Acquiring a preset initial container, wherein the initial container comprises at least one scene table for storing data to be processed;
loading a first model into the initial container in response to a first selection operation of the first model;
Determining first target data in the first table in response to a first input operation of a selection parameter, and storing the first target data in the scene table;
and labeling at least one column of types in the scene table in response to a second input operation of the semantic parameters, and obtaining at least one type label.
3. The method according to claim 2, wherein the method further comprises:
And updating the first analysis result and the type label to the first table.
4. The method of claim 2, wherein the first model is a trained initial model, the first container is capable of completing model training, and the training process of the initial model comprises:
Loading the initial model into the first container in response to a second selection operation of the initial model;
Loading a training table and a truth table in the database into the first container in response to a third selection operation of the training table and the truth table, the training table including training data, the truth table including truths corresponding to the training data;
And training the initial model by using the training table and the truth table to obtain the first model.
5. The method according to claim 2, wherein the method further comprises:
determining display parameters based on the type label and the first analysis result, wherein the display parameters are used for representing display positions and display styles;
displaying the data of the scene table according to the display parameters;
the selection parameters are updated in response to a first altering operation of the selection parameters.
6. The method according to claim 1, wherein the method further comprises:
In response to a third input operation on the column identity, the data type, establishing a second table of blanks in a transaction container within a database server managing the database, the second table comprising at least: an input form, an output form, the transaction container comprising an external interface;
Generating an image file of the selected first container in response to a fourth selection operation of any first container, and loading the image file into the transaction container;
Determining second target data in a database based on the external interface, and storing the second target data in the input form;
Based on the image file, carrying out second analysis on the data in the input form to obtain a second analysis result,
Storing the second analysis result in the output table, and displaying the second analysis result.
7. The method of claim 6, wherein the method further comprises:
And updating a third table preset in the database by using the output table, wherein the third table is not used for storing the second target data.
8. A data analysis device, characterized by being applied to a database, the database being a relational database, comprising:
a first container construction unit, configured to construct a first container in a database server that manages the database, where the first container is used to complete data preprocessing, and the first container contains data to be analyzed, where the data to be analyzed corresponds to a first table in the database;
A data synchronization unit, configured to monitor the first table, and synchronize the changed data to the first container when the data in the first table changes;
and the first analysis result determining unit is used for carrying out first analysis on the data to be analyzed based on the first container to obtain a first analysis result.
9. An electronic device, comprising:
A processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to implement the method of any one of claims 1 to 7 when executing the instructions stored by the memory.
10. A non-transitory computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410006866.7A CN118113789A (en) | 2024-01-02 | 2024-01-02 | Data analysis method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410006866.7A CN118113789A (en) | 2024-01-02 | 2024-01-02 | Data analysis method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118113789A true CN118113789A (en) | 2024-05-31 |
Family
ID=91215325
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410006866.7A Pending CN118113789A (en) | 2024-01-02 | 2024-01-02 | Data analysis method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118113789A (en) |
-
2024
- 2024-01-02 CN CN202410006866.7A patent/CN118113789A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11403006B2 (en) | Configurable machine learning systems through graphical user interfaces | |
| Claes et al. | Merging event logs for process mining: A rule based merging method and rule suggestion algorithm | |
| US20170308595A1 (en) | Learning from historical logs and recommending database operations on a data-asset in an etl tool | |
| US11966873B2 (en) | Data distillery for signal detection | |
| CN105573966A (en) | Adaptive Modification of Content Presented in Electronic Forms | |
| CN113168645A (en) | Knowledge search system | |
| CN114936301B (en) | Intelligent household building material data management method, device, equipment and storage medium | |
| CN110688844A (en) | Text labeling method and device | |
| CN110674117A (en) | Data modeling method and device, computer readable medium and electronic equipment | |
| JP6963062B1 (en) | Information processing device and information processing method | |
| CN113112336A (en) | Commodity information processing method and device and computer equipment | |
| CN118093801A (en) | Information interaction method and device based on large language model and electronic equipment | |
| JP6732084B1 (en) | Computer program, transmission method and transmission device | |
| JP6308339B1 (en) | Clustering system, method and program, and recommendation system | |
| US20170308838A1 (en) | Service Design Assistance System and Service Design Assistance Method | |
| US20150379529A1 (en) | System, service and methodology of data collection, processing, analysis and effective delivery of dashboard snapshot presentations of organized customer proprietary information on an individualized secure basis to multiple customers | |
| CN118113789A (en) | Data analysis method and device, electronic equipment and storage medium | |
| US11928658B2 (en) | System and method for monitoring and updating content for an e-commerce platform | |
| CN113762992B (en) | Method and device for processing data | |
| CN113722577B (en) | Feedback information processing method, device, equipment and storage medium | |
| KR101700838B1 (en) | Method and system for managing business project step by step | |
| CN114445190A (en) | Commodity recommendation method and device | |
| JPH09282307A (en) | Commodity sales trend analysis method and system therefor | |
| KR20220053398A (en) | Service System for Connecting and Recommending Clothing Products by Using AI | |
| US20200167353A1 (en) | Automated Data Source Join Proposals Based on Process Relations |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |