CN112560939A - Model verification method and device and computer equipment - Google Patents
Model verification method and device and computer equipment Download PDFInfo
- Publication number
- CN112560939A CN112560939A CN202011463493.4A CN202011463493A CN112560939A CN 112560939 A CN112560939 A CN 112560939A CN 202011463493 A CN202011463493 A CN 202011463493A CN 112560939 A CN112560939 A CN 112560939A
- Authority
- CN
- China
- Prior art keywords
- data
- model
- data input
- verification
- calculation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Debugging And Monitoring (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a model verification method, a model verification device and computer equipment, wherein the method comprises the following steps: acquiring training process data of a target model for executing model training based on baseline data; splitting a data input process and a calculation process from the training process data; receiving a model validation task with respect to the target model; scheduling the data input process and the calculation process to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a corresponding second verification result; and aggregating the first verification result and the second verification result to obtain a model verification result. The present application also provides a computer-readable storage medium. The method and the device can improve the flexibility of the model verification process and improve the model verification efficiency.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a model verification method, an apparatus, and a computer device.
Background
With the development of internet technology, more and more users choose to browse, select or purchase the required goods on the internet. With the increase of the number and the variety of the commodities, users often need to spend a great deal of time to find the commodities needed by the users. In order to solve the problem, each e-commerce platform adopts various forms of recommendation technologies to recommend commodities to users to different degrees. In order to achieve the purpose of recommending various useful information to a user in time and avoiding recommending useless information as much as possible, user characteristic data of the user is usually constructed according to user information; and then inputting the user characteristic data of the user into the initial click rate estimation model, thereby training a click rate estimation model capable of estimating the click probability of different users to the recommended data.
Generally, for models such as click rate estimation models, model verification is required after the models are trained, and in the existing model verification method, different test data are generally sequentially input into a model to be verified, and then the verification result of each model verification is sequentially recorded. However, this model verification approach is inflexible and inefficient.
Disclosure of Invention
The application provides a model verification method, a model verification device and computer equipment, which can solve the problems of poor flexibility and low efficiency of the model verification mode.
First, to achieve the above object, the present application provides a model verification method, including:
acquiring training process data of a target model for executing model training based on baseline data; splitting a data input process and a calculation process from the training process data; receiving a model validation task with respect to the target model; scheduling the data input process and the calculation process to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a corresponding second verification result; and aggregating the first verification result and the second verification result to obtain a model verification result.
In one example, the splitting of the data input process and the calculation process from the training process data comprises: searching and splitting the data input process from the training process data according to the field identification of the start and the end of the data flow operation; and searching and splitting the calculation process from the training process data according to the function body structure of the calculation function.
In one example, the splitting the data input process and the calculation process from the training process data further comprises: inquiring the time sequence relation and/or the dependency relation of the data input process and the calculation process in the training process data; and acquiring initial input data corresponding to the data input process and initial computing logic corresponding to the computing process.
In one example, said scheduling said data input process and said computing process to cooperatively perform said model validation task comprises: acquiring correction input data corresponding to the data input process and correction calculation logic corresponding to the calculation process in the model verification task; and replacing the initial input data with the corrected input data, replacing the initial calculation logic with the corrected calculation logic, and executing the data input process and the calculation process according to the time sequence relation and/or the dependency relation.
In one example, the performing the data input process and the calculation process according to the timing relationship includes: acquiring the time watermark of the data input process and the time watermark of the calculation process; and sequentially triggering and executing the data input process and the calculation process according to the sequence of the time watermarks.
In one example, said performing said data entry process and said calculation process according to said dependencies comprises: acquiring triggering items in the dependency relationship; and in the process of executing the data input process/the calculation process, if the trigger is detected, starting to execute the calculation process/the data input process.
In one example, the data input process includes an input process of real-time training data and/or an input process of off-line training data.
In addition, to achieve the above object, the present application also provides a model verification apparatus, including:
the acquisition module is used for acquiring training process data of the target model for executing model training based on the baseline data; the splitting module is used for splitting a data input process and a calculation process from the training process data; a receiving module for receiving a model verification task with respect to the target model; the verification module is used for scheduling the data input process and the calculation process to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a corresponding second verification result; and the aggregation module is used for aggregating the first verification result and the second verification result to obtain a model verification result.
Further, the present application also proposes a computer device, which includes a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and the computer program implements the steps of the model verification method as described above when executed by the processor.
Further, to achieve the above object, the present application also provides a computer-readable storage medium storing a computer program, which is executable by at least one processor to cause the at least one processor to perform the steps of the model verification method as described above.
Compared with the prior art, the model verification method, the model verification device, the computer equipment and the computer readable storage medium can acquire training process data of the target model for executing model training based on the baseline data; splitting a data input process and a calculation process from the training process data; receiving a model validation task with respect to the target model; scheduling the data input process and the calculation process to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a corresponding second verification result; and aggregating the first verification result and the second verification result to obtain a model verification result. The training process data of the model is divided into a data input process and a calculation process, and then the data input process and the calculation process are cooperated to execute model verification, so that the flexibility of the model verification process is improved, and the model verification efficiency is also improved.
Drawings
FIG. 1 is a schematic diagram of an application environment according to an embodiment of the present application;
FIG. 2 is a schematic flow chart diagram illustrating one embodiment of a model verification method of the present application;
FIG. 3 is a flowchart illustrating the effect of performing model validation in coordination with multiple data input processes and/or computational processes based on timing relationships and dependencies in accordance with an illustrative example of the present invention;
FIG. 4 is a block diagram of a program module of an embodiment of the model verification apparatus of the present application;
FIG. 5 is a diagram of an alternative hardware architecture of the computer device of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the descriptions in this application referring to "first", "second", etc. are for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present application.
Fig. 1 is a schematic diagram of an application environment according to an embodiment of the present application. Referring to fig. 1, the computer device 1 is connected to a verification device 1, a verification device 2, and a server 20, and the server 20 stores training process data for performing model training based on baseline data for a template model. The computer device 1 is able to acquire training process data of a target model performing model training based on baseline data through the server 20; splitting a data input process and a calculation process from the training process data; then the data input process and the calculation process are respectively sent to a verification device 1 and a verification device 2; when the computer device 1 receives a model verification task with respect to the target model; the data input process and the calculation process on the verification device 1 and the verification device 2 can be scheduled to cooperatively execute the model verification task; and finally, receiving a first verification result and a second verification result fed back by the verification device 1 and the verification device 2, and aggregating the first verification result and the second verification result to obtain a model verification result.
In this embodiment, the server 20 may be used as a mobile phone, a tablet, a portable device, a PC, or other data service platforms, such as a video service platform, an online shopping platform, and the like; the verification device 1 and the verification device 2 can be used as a mobile phone, a tablet, a portable device, a PC and the like; the computer device 1 can be used as a mobile phone, a tablet, a portable device, a PC, a server or the like. Of course, in other embodiments, the computer device 1 may be combined with the server 20 into the same electronic device, or the computer device 1 may be attached to the server 20 as a separate functional module to implement the function of model verification.
Example one
Fig. 2 is a schematic flowchart of an embodiment of a model verification method according to the present application. It is to be understood that the flow charts in the embodiments of the present method are not intended to limit the order in which the steps are performed. The following description is made by way of example with the computer apparatus 1 as the execution subject.
As shown in FIG. 2, the model verification method may include steps S200-S208.
Step S200, training process data of the target model for executing model training based on the baseline data are obtained.
Specifically, the computer device 1 is connected to a server, and can acquire training process data of a target model executing model training based on baseline data from the server. The server stores baseline data for training the target model, wherein the baseline data can be understood as mainstream data or reference data for training out the target model. Therefore, the target model is relatively stable in comprehensive performance after being trained through the baseline data, and the effect and the energy consumption of the model are relatively excellent. Therefore, the computer device 1 mainly verifies the target model trained based on the baseline data. In this embodiment, after the computer device 1 is connected to the server, training process data of the target model for performing model training based on the baseline data may be obtained first.
Step S202, splitting a data input process and a calculation process from the training process data.
Specifically, the training process data includes the trend of all input data and the operation process of the operation data of the target model in the training process, so that the computer device 1 can identify the training process data and then split the data input process and the calculation process; the data input process comprises input data and flow information of the input data in a model training process, namely the trend of the input data; the calculation process comprises calculation logic and a calculation process of the calculation logic on the target data.
In a specific embodiment, the splitting of the data input process and the calculation process from the training process data by the computer device 1 includes: searching and splitting the data input process from the training process data according to the field identification of the start and the end of the data flow operation; and searching and splitting the calculation process from the training process data according to the function body structure of the calculation function. Wherein the data input process comprises a real-time training data input process and/or an off-line training data input process. Specifically, the computer device 1 sets corresponding identification modes in advance for both the data input process and the calculation process, for example, the complete trend of the input data of the data input process, that is, the stream information, can be identified by identifying the fields for the start and the end of the data stream job; the range covered by the computation logic, i.e. the computation process, can also be identified from the training process data by the functional body structure of the computation function.
Of course, in another specific embodiment, the splitting of the data input process and the calculation process from the training process data by the computer device 1 further includes: inquiring the time sequence relation and/or the dependency relation of the data input process and the calculation process in the training process data; and acquiring initial input data corresponding to the data input process and initial computing logic corresponding to the computing process. Specifically, in the process of splitting the data input process and the calculation process from the training process data, the computer device 1 may query the training process data for the time sequence, that is, the time sequence relationship, of the data input process and the calculation process; triggers, i.e. dependencies, between the data input process and the calculation process can also be queried.
Step S204, receiving a model verification task about the target model.
Step S206, scheduling the data input process and the calculation process to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a corresponding second verification result.
After splitting the data input process and the calculation process from the training process data, the computer device 1 defines the data input process and the calculation process as "task nodes"; therefore, the two "task nodes" can also be sent to different verification terminals, so that the model verification work can be distributed to different verification terminals, such as the verification device 1 and the verification device 2. Of course, in this embodiment, in the process of sending the "task node" to the verification device 1 or the verification device 2, all the environment data required for the entire model verification of the target model needs to be sent; each verification device executes the verification task of the response part mainly according to the corresponding task node.
Thus, after receiving a model verification task for the target model, the computer device 1 may then schedule the data input process and the computing process to cooperatively execute the model verification task to obtain corresponding first and second verification results. In a specific embodiment, the scheduling, by the computer device 1, the data input process and the computing process to cooperatively execute the model verification task includes: acquiring correction input data corresponding to the data input process and correction calculation logic corresponding to the calculation process in the model verification task; and replacing the initial input data with the corrected input data, replacing the initial calculation logic with the corrected calculation logic, and executing the data input process and the calculation process according to the time sequence relation and/or the dependency relation.
In an illustrative example, the computer device 1 executing the data input process and the calculation process according to the timing relationship includes: acquiring the time watermark of the data input process and the time watermark of the calculation process; and sequentially triggering and executing the data input process and the calculation process according to the sequence of the time watermarks. In another illustrative example, the computer device 1 executing the data input process and the calculation process according to the dependency relationship includes: acquiring triggering items in the dependency relationship; and in the process of executing the data input process/the calculation process, if the trigger is detected, starting to execute the calculation process/the data input process.
Specifically, the computer device 1 schedules the data input process and the calculation process to execute the verification task, mainly replaces the initial input data with the correction input data in the verification task through the data input process, and replaces the initial calculation logic with the correction calculation logic in the verification task through the calculation process, thereby realizing the verification of the target model according to the correction input data and/or the correction calculation logic. Finally, the computer device 1 obtains a first verification result and a second verification result on the verification task, which are fed back by the data input process and the calculation process.
Step S208, aggregating the first verification result and the second verification result to obtain a model verification result.
As shown in fig. 3, fig. 3 is a flowchart illustrating the effect of performing model verification by cooperating multiple data input processes and/or computing processes according to timing relationships and dependency relationships in an exemplary embodiment of the present invention. In this embodiment, the task node 1 is used for data input, and the corresponding process of the task node 1 includes three steps of data input, data statistics, and data output; the task node 2 is used for feature calculation, and the corresponding process of the task node 2 comprises four steps of data input, data stream consumption, data statistics and data output. The computer device 1 sets a global watermark collector in the task node 1, so that the detailed conditions including time sequence of the task node 1 executing the model verification task can be collected; when the time watermark collected by the global watermark collector of the task node 1 reaches a preset time node, such as an hour or a day, the task node 1 feeds back "completed data input" to a notification service; the notification service triggers the task node 2 to start executing the model verification task; similarly, the task node 2 is also provided with a global watermark collector, so that the detailed conditions of the task node 1 executing the model verification task, including the time sequence, can be collected; when the time watermark collected by the global watermark collector of the task node 1 reaches a preset time node, such as an hour or a day, the task node 2 feeds back "feature calculation completed" to a notification service. Finally, the notification service informs the AI of the scheduling, thereby initiating an offline job of Spark. Namely, the scheduling task node n executes the corresponding model verification task.
In summary, the model verification method provided by the embodiment can obtain training process data of the target model for executing model training based on the baseline data; splitting a data input process and a calculation process from the training process data; receiving a model validation task with respect to the target model; scheduling the data input process and the calculation process to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a corresponding second verification result; and aggregating the first verification result and the second verification result to obtain a model verification result. The training process data of the model is divided into a data input process and a calculation process, and then the data input process and the calculation process are cooperated to execute model verification, so that the flexibility of the model verification process is improved, and the model verification efficiency is also improved.
Example two
Fig. 4 schematically shows a block diagram of a model verification apparatus according to the second embodiment of the present application, which may be divided into one or more program modules, the one or more program modules being stored in a storage medium and executed by one or more processors to complete the embodiments of the present application. The program modules referred to in the embodiments of the present application refer to a series of computer program instruction segments that can perform specific functions, and the following description will specifically describe the functions of the program modules in the embodiments.
As shown in fig. 4, the model verification apparatus 400 may include an obtaining module 410, a splitting module 420, a receiving module 430, a verifying module 440, and an aggregating module 450, wherein:
an obtaining module 410 is configured to obtain training process data of the target model for performing model training based on the baseline data.
A splitting module 420, configured to split a data input process and a calculation process from the training process data. Wherein the data input process comprises a real-time training data input process and/or an off-line training data input process.
A receiving module 430, configured to receive a model verification task regarding the target model.
The verification module 440 is configured to schedule the data input process and the calculation process to cooperatively execute the model verification task, so as to obtain a corresponding first verification result and a corresponding second verification result.
An aggregating module 450, configured to aggregate the first verification result and the second verification result to obtain a model verification result.
In an exemplary embodiment, the splitting module 420 is further configured to: searching and splitting the data input process from the training process data according to the field identification of the start and the end of the data flow operation; and searching and splitting the calculation process from the training process data according to the function body structure of the calculation function. And querying a time sequence relation and/or a dependency relation of the data input process and the calculation process in the training process data; and acquiring initial input data corresponding to the data input process and initial computing logic corresponding to the computing process.
In an exemplary embodiment, the verification module 440 is further configured to obtain modified input data corresponding to the data input process and modified computation logic corresponding to the computation process in the model verification task; and replacing the initial input data with the corrected input data, replacing the initial calculation logic with the corrected calculation logic, and executing the data input process and the calculation process according to the time sequence relation and/or the dependency relation. Acquiring the time watermark of the data input process and the time watermark of the calculation process; and sequentially triggering and executing the data input process and the calculation process according to the sequence of the time watermarks. And acquiring a trigger item in the dependency relationship; and in the process of executing the data input process/the calculation process, if the trigger is detected, starting to execute the calculation process/the data input process.
EXAMPLE III
Fig. 5 schematically shows a hardware architecture diagram of a computer device 1 suitable for implementing the model verification method according to the third embodiment of the present application. In the present embodiment, the computer device 1 is a device capable of automatically performing numerical calculation and/or information processing in accordance with a command set or stored in advance. For example, the server may be a rack server, a blade server, a tower server or a rack server (including an independent server or a server cluster composed of a plurality of servers) with a gateway function. As shown in fig. 5, the computer device 1 includes at least, but is not limited to: memory 510, processor 520, and network interface 530 may be communicatively linked to each other by a system bus. Wherein:
the memory 510 includes at least one type of computer-readable storage medium including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage 510 may be an internal storage module of the computer device 1, such as a hard disk or a memory of the computer device 1. In other embodiments, the memory 510 may also be an external storage device of the computer device 1, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the computer device 1. Of course, the memory 510 may also comprise both an internal memory module of the computer device 1 and an external memory device thereof. In this embodiment, the memory 510 is generally used for storing an operating system installed in the computer device 1 and various types of application software, such as program codes of the model verification method. In addition, the memory 510 may also be used to temporarily store various types of data that have been output or are to be output.
Processor 520 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 520 is generally used for controlling the overall operation of the computer device 1, such as performing control and processing related to data interaction or communication with the computer device 1. In this embodiment, processor 520 is configured to execute program codes stored in memory 510 or process data.
Network interface 530 may include a wireless network interface or a wired network interface, and network interface 530 is typically used to establish communication links between computer device 1 and other computer devices. For example, the network interface 530 is used to connect the computer apparatus 1 with an external terminal through a network, establish a data transmission channel and a communication link between the computer apparatus 1 and the external terminal, and the like. The network may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth (Bluetooth), or Wi-Fi.
It should be noted that FIG. 5 only shows a computer device having components 510 and 530, but it should be understood that not all of the shown components are required and that more or fewer components may be implemented instead.
In this embodiment, the program code of the model verification method stored in the memory 510 may also be divided into one or more program modules and executed by one or more processors (in this embodiment, the processor 520) to implement the embodiments of the present application.
Example four
The present embodiments also provide a computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing the steps of:
acquiring training process data of a target model for executing model training based on baseline data; splitting a data input process and a calculation process from the training process data; receiving a model validation task with respect to the target model; scheduling the data input process and the calculation process to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a corresponding second verification result; and aggregating the first verification result and the second verification result to obtain a model verification result.
In this embodiment, the computer-readable storage medium includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the computer readable storage medium may be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. In other embodiments, the computer readable storage medium may be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the computer device. Of course, the computer-readable storage medium may also include both internal and external storage devices of the computer device. In this embodiment, the computer-readable storage medium is generally used for storing an operating system and various types of application software installed in the computer device, for example, the program code of the model verification method in the embodiment, and the like. Further, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.
It will be apparent to those skilled in the art that the modules or steps of the embodiments of the present application described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different from that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all modifications that can be made by the use of the equivalent structures or equivalent processes in the specification and drawings of the present application or that can be directly or indirectly applied to other related technologies are also included in the scope of the present application.
Claims (10)
1. A method of model validation, the method comprising:
acquiring training process data of a target model for executing model training based on baseline data;
splitting a data input process and a calculation process from the training process data;
receiving a model validation task with respect to the target model;
scheduling the data input process and the calculation process to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a corresponding second verification result;
and aggregating the first verification result and the second verification result to obtain a model verification result.
2. The model validation method of claim 1, wherein the splitting of the data input process and the computation process from the training process data comprises:
searching and splitting the data input process from the training process data according to the field identification of the start and the end of the data flow operation; and
and searching and splitting the calculation process from the training process data according to the function body structure of the calculation function.
3. The model validation method of claim 1, wherein the splitting of the data input process and the computation process from the training process data further comprises:
inquiring the time sequence relation and/or the dependency relation of the data input process and the calculation process in the training process data; and
and acquiring initial input data corresponding to the data input process and initial computing logic corresponding to the computing process.
4. The model validation method of claim 3, wherein said scheduling the data input process and the computational process to cooperatively perform the model validation task comprises:
acquiring correction input data corresponding to the data input process and correction calculation logic corresponding to the calculation process in the model verification task;
and replacing the initial input data with the corrected input data, replacing the initial calculation logic with the corrected calculation logic, and executing the data input process and the calculation process according to the time sequence relation and/or the dependency relation.
5. The model validation method of claim 4, wherein said performing said data entry process and said calculation process according to said timing relationship comprises:
acquiring the time watermark of the data input process and the time watermark of the calculation process;
and sequentially triggering and executing the data input process and the calculation process according to the sequence of the time watermarks.
6. The model validation method of claim 4, wherein said performing the data entry process and the computation process according to the dependencies comprises:
acquiring triggering items in the dependency relationship;
and in the process of executing the data input process/the calculation process, if the trigger is detected, starting to execute the calculation process/the data input process.
7. The model validation method of any of claims 1-6, wherein the data input process comprises an input process of real-time training data and/or an input process of offline training data.
8. A model validation apparatus, the apparatus comprising:
the acquisition module is used for acquiring training process data of the target model for executing model training based on the baseline data;
the splitting module is used for splitting a data input process and a calculation process from the training process data;
a receiving module for receiving a model verification task with respect to the target model;
the verification module is used for scheduling the data input process and the calculation process to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a corresponding second verification result;
and the aggregation module is used for aggregating the first verification result and the second verification result to obtain a model verification result.
9. A computer arrangement, characterized in that the computer arrangement comprises a memory, a processor, the memory having stored thereon a computer program being executable on the processor, the computer program, when executed by the processor, implementing the steps of the model verification method as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which is executable by at least one processor to cause the at least one processor to perform the steps of the model verification method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011463493.4A CN112560939B (en) | 2020-12-11 | 2020-12-11 | Model verification method and device and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011463493.4A CN112560939B (en) | 2020-12-11 | 2020-12-11 | Model verification method and device and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112560939A true CN112560939A (en) | 2021-03-26 |
| CN112560939B CN112560939B (en) | 2023-05-23 |
Family
ID=75062877
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011463493.4A Active CN112560939B (en) | 2020-12-11 | 2020-12-11 | Model verification method and device and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112560939B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114861912A (en) * | 2022-07-06 | 2022-08-05 | 武汉山川软件有限公司 | Data verification method and device based on big data |
| WO2023028996A1 (en) * | 2021-09-03 | 2023-03-09 | Huawei Technologies Co., Ltd. | Methods and devices for ensuring the reproducibility of software systems |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0974902A1 (en) * | 1998-07-21 | 2000-01-26 | Bull S.A. | Method of detecting errors on a serial link of an integrated circuit and device to realise this method |
| CN101183428A (en) * | 2007-12-18 | 2008-05-21 | 北京中星微电子有限公司 | Image detection method and apparatus |
| CN101938333A (en) * | 2010-09-21 | 2011-01-05 | 山东大学 | Gray code pi/ M-MPSK modulating soft bit information calculation method |
| CN102566506A (en) * | 2011-12-23 | 2012-07-11 | 东南大学 | Online coordination control and Petri net verification method combined with water cutting process |
| CN106227507A (en) * | 2016-07-11 | 2016-12-14 | 姚颂 | Calculating system and controller thereof |
| EP3301597A1 (en) * | 2016-10-03 | 2018-04-04 | Institute of Science and Technology Austria | Method for computationally designing a re-usable flexible mold |
| CN111695035A (en) * | 2020-06-10 | 2020-09-22 | 广东数果科技有限公司 | Recommendation system and multi-algorithm fusion recommendation processing flow |
| CN111966289A (en) * | 2020-08-13 | 2020-11-20 | 上海哔哩哔哩科技有限公司 | Partition optimization method and system based on Kafka cluster |
| CN112052868A (en) * | 2020-06-15 | 2020-12-08 | 上海集成电路研发中心有限公司 | Model training method, image similarity measuring method, terminal and storage medium |
-
2020
- 2020-12-11 CN CN202011463493.4A patent/CN112560939B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0974902A1 (en) * | 1998-07-21 | 2000-01-26 | Bull S.A. | Method of detecting errors on a serial link of an integrated circuit and device to realise this method |
| CN101183428A (en) * | 2007-12-18 | 2008-05-21 | 北京中星微电子有限公司 | Image detection method and apparatus |
| CN101938333A (en) * | 2010-09-21 | 2011-01-05 | 山东大学 | Gray code pi/ M-MPSK modulating soft bit information calculation method |
| CN102566506A (en) * | 2011-12-23 | 2012-07-11 | 东南大学 | Online coordination control and Petri net verification method combined with water cutting process |
| CN106227507A (en) * | 2016-07-11 | 2016-12-14 | 姚颂 | Calculating system and controller thereof |
| EP3301597A1 (en) * | 2016-10-03 | 2018-04-04 | Institute of Science and Technology Austria | Method for computationally designing a re-usable flexible mold |
| CN111695035A (en) * | 2020-06-10 | 2020-09-22 | 广东数果科技有限公司 | Recommendation system and multi-algorithm fusion recommendation processing flow |
| CN112052868A (en) * | 2020-06-15 | 2020-12-08 | 上海集成电路研发中心有限公司 | Model training method, image similarity measuring method, terminal and storage medium |
| CN111966289A (en) * | 2020-08-13 | 2020-11-20 | 上海哔哩哔哩科技有限公司 | Partition optimization method and system based on Kafka cluster |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023028996A1 (en) * | 2021-09-03 | 2023-03-09 | Huawei Technologies Co., Ltd. | Methods and devices for ensuring the reproducibility of software systems |
| CN114861912A (en) * | 2022-07-06 | 2022-08-05 | 武汉山川软件有限公司 | Data verification method and device based on big data |
| CN114861912B (en) * | 2022-07-06 | 2022-09-16 | 武汉山川软件有限公司 | Data verification method and device based on big data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112560939B (en) | 2023-05-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107844634B (en) | Modeling method of multivariate general model platform, electronic equipment and computer readable storage medium | |
| CN113220657B (en) | Data processing method and device and computer equipment | |
| CN112560939B (en) | Model verification method and device and computer equipment | |
| CN112115039B (en) | Test case generation method, device and equipment | |
| CN110599354A (en) | Online reconciliation method, system, computer device and computer-readable storage medium | |
| CN111177121A (en) | Order data feedback method and device, computer equipment and storage medium | |
| CN110855477A (en) | Link log monitoring method and device, computer equipment and storage medium | |
| CN109542962B (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN112613938A (en) | Model training method and device and computer equipment | |
| CN109446515A (en) | Group information analysis method, electronic device and computer readable storage medium | |
| CN111580948A (en) | Task scheduling method and device and computer equipment | |
| CN112286664A (en) | Task scheduling method and device, computer equipment and readable storage medium | |
| CN111651454A (en) | Data processing method and device and computer equipment | |
| CN111459629A (en) | Azkaban-based project operation method and device and terminal equipment | |
| CN108509440A (en) | A kind of data processing method and device | |
| CN112396480B (en) | Order business data processing method, system, computer equipment and storage medium | |
| CN112560938A (en) | Model training method and device and computer equipment | |
| CN113093702B (en) | Fault data prediction method and device, electronic equipment and storage medium | |
| CN115185514A (en) | Zero code modeling processing method and device and related equipment | |
| CN113392131A (en) | Data processing method and device and computer equipment | |
| CN108509513B (en) | Platform client data reflow method, electronic device, equipment and storage medium | |
| CN110674214A (en) | Big data synchronization method and device, computer equipment and storage medium | |
| CN109582680A (en) | Method for processing business, electronic device and readable storage medium storing program for executing based on new product development | |
| CN113836411B (en) | Data processing method and device and computer equipment | |
| CN112532750B (en) | Big data push processing method and system and cloud platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |