CN112560939B - Model verification method and device and computer equipment - Google Patents
Model verification method and device and computer equipment Download PDFInfo
- Publication number
- CN112560939B CN112560939B CN202011463493.4A CN202011463493A CN112560939B CN 112560939 B CN112560939 B CN 112560939B CN 202011463493 A CN202011463493 A CN 202011463493A CN 112560939 B CN112560939 B CN 112560939B
- Authority
- CN
- China
- Prior art keywords
- data
- model
- data input
- calculation
- verification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Debugging And Monitoring (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a model verification method, a device and computer equipment, wherein the method comprises the following steps: acquiring training process data of a target model for performing model training based on baseline data; splitting a data input process and a calculation process from the training process data; receiving a model verification task regarding the target model; scheduling the data input process and the calculation process to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a corresponding second verification result; and aggregating the first verification result and the second verification result to obtain a model verification result. The present application also provides a computer-readable storage medium. The method and the device can improve the flexibility of the model verification process and also improve the model verification efficiency.
Description
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to a method and apparatus for model verification, and a computer device.
Background
With the development of internet technology, more and more users choose to browse, pick or purchase goods that are needed on the internet. With the increase of the number and variety of commodities, users often spend a great deal of time to find the commodity they need. In order to solve the problem, various electronic commerce platforms adopt various types of recommendation technologies to recommend commodities to users to different degrees. In order to timely recommend various useful information to a user and avoid recommending useless information as much as possible, user characteristic data of the user are generally constructed according to the user information; and then inputting the user characteristic data of the user into an initial click rate estimation model, so as to train a click rate estimation model capable of estimating the click probabilities of different users on the recommended data.
Generally, for models such as click rate estimation models, model verification is required after the models are trained, and in the existing model verification method, different test data are sequentially input into the model to be verified, and then verification results of each model verification are sequentially recorded. However, this model verification approach lacks flexibility and is inefficient.
Disclosure of Invention
The application provides a model verification method, a model verification device and computer equipment, which can solve the problems of poor flexibility and low efficiency of the model verification mode.
First, to achieve the above object, the present application provides a model verification method, the method including:
acquiring training process data of a target model for performing model training based on baseline data; splitting a data input process and a calculation process from the training process data; receiving a model verification task regarding the target model; scheduling the data input process and the calculation process to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a corresponding second verification result; and aggregating the first verification result and the second verification result to obtain a model verification result.
In one example, the splitting of the data input process and the computing process from the training process data includes: searching and splitting the data input process from the training process data according to the field identifiers of the start and end of the data flow operation; and searching and splitting the calculation process from the training process data according to the function body structure of the calculation function.
In one example, the splitting the data input process and the computing process from the training process data further comprises: inquiring a time sequence relation and/or a dependency relation of the data input process and the calculation process in the training process data; and acquiring initial input data corresponding to the data input process and initial calculation logic corresponding to the calculation process.
In one example, the scheduling the data input process and the computing process to cooperatively perform the model validation task includes: obtaining correction input data corresponding to the data input process in the model verification task and correction calculation logic corresponding to the calculation process; replacing the initial input data with the corrected input data, replacing the initial calculation logic with the corrected calculation logic, and executing the data input process and the calculation process according to the time sequence relation and/or the dependency relation.
In one example, the performing the data input process and the computing process according to the timing relationship includes: acquiring a time watermark of the data input process and a time watermark of the calculation process; and triggering and executing the data input process and the calculation process in sequence according to the sequence of the time watermarks.
In one example, the performing the data input process and the computing process according to the dependency relationship includes: acquiring trigger matters in the dependency relationship; in the course of executing the data input process/the calculation process, if the trigger event is detected, the execution of the calculation process/the data input process is started.
In one example, the data input process includes a real-time training data input process and/or an offline training data input process.
In addition, to achieve the above object, the present application further provides a model verification apparatus, including:
the acquisition module is used for acquiring training process data of a target model for performing model training based on the baseline data; the splitting module is used for splitting a data input process and a calculation process from the training process data; a receiving module for receiving a model verification task with respect to the target model; the verification module is used for scheduling the data input process and the calculation process to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a corresponding second verification result; and the aggregation module is used for aggregating the first verification result and the second verification result to obtain a model verification result.
Further, the application also proposes a computer device comprising a memory, a processor, said memory having stored thereon a computer program executable on said processor, said computer program implementing the steps of the model verification method as described above when executed by said processor.
Further, to achieve the above object, the present application also provides a computer-readable storage medium storing a computer program executable by at least one processor to cause the at least one processor to perform the steps of the model verification method as described above.
Compared with the prior art, the model verification method, the device, the computer equipment and the computer readable storage medium can acquire training process data of a target model for executing model training based on baseline data; splitting a data input process and a calculation process from the training process data; receiving a model verification task regarding the target model; scheduling the data input process and the calculation process to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a corresponding second verification result; and aggregating the first verification result and the second verification result to obtain a model verification result. The training process data of the model are split into the data input process and the calculation process, and then the data input process and the calculation process are cooperated to perform model verification, so that the flexibility of the model verification process is improved, and the model verification efficiency is also improved.
Drawings
FIG. 1 is a schematic view of an application environment according to an embodiment of the present application;
FIG. 2 is a flow chart of an embodiment of a method for model verification of the present application;
FIG. 3 is a flowchart illustrating the effect of implementing a plurality of data input processes and/or computing processes to cooperatively perform model verification according to a timing relationship and a dependency relationship in an exemplary embodiment of the present invention;
FIG. 4 is a schematic diagram of a program module of an embodiment of the model verification device of the present application;
fig. 5 is a schematic diagram of an alternative hardware architecture of the computer device of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
It should be noted that the description herein of "first," "second," etc. is for descriptive purposes only and is not to be construed as indicating or implying a relative importance or implying an indication of the number of technical features being indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions of the embodiments may be combined with each other, but it is necessary to base that the technical solutions can be realized by those skilled in the art, and when the technical solutions are contradictory or cannot be realized, the combination of the technical solutions should be regarded as not exist and not within the protection scope of the present application.
FIG. 1 is a schematic view of an application environment according to an embodiment of the present application. Referring to fig. 1, the computer device 1 is connected to the verification device 1, the verification device 2, and the server 20 stores training process data for performing model training by using a template model based on baseline data. The computer device 1 is capable of acquiring training process data for performing model training of a target model based on baseline data through the server 20; splitting a data input process and a calculation process from the training process data; then the data input process and the calculation process are respectively sent to the verification device 1 and the verification device 2; when the computer device 1 receives a model verification task regarding the target model; the data input process and the calculation process on the verification device 1 and the verification device 2 may be scheduled to cooperatively perform the model verification task; and finally, receiving a first verification result and a second verification result fed back by the verification device 1 and the verification device 2, and aggregating the first verification result and the second verification result to obtain a model verification result.
In this embodiment, the server 20 may be used as a mobile phone, a tablet, a portable device, a PC or other data service platform, such as a video service platform, an online shopping platform, etc.; the verification device 1 and the verification device 2 can be used as a mobile phone, a tablet, a portable device, a PC (personal computer) and the like; the computer device 1 may be a mobile phone, tablet, portable device, PC, server, etc. Of course, in other embodiments, the computer device 1 may be combined with the server 20 into the same electronic device, or the computer device 1 may be attached to the server 20 as a separate functional module to implement the model verification function.
Example 1
FIG. 2 is a flow chart of an embodiment of a method for model verification of the present application. It will be appreciated that the flow charts in the method embodiments are not intended to limit the order in which the steps are performed. An exemplary description will be made below with the computer apparatus 1 as an execution subject.
As shown in fig. 2, the model verification method may include steps S200 to S208.
Step S200, obtaining training process data for performing model training of the target model based on the baseline data.
Specifically, the computer device 1 is connected to a server, and can acquire training process data of a target model for performing model training based on baseline data from the server. The server stores baseline data for training the target model, wherein the baseline data can be understood as main stream data or reference data for training the target model. Therefore, the comprehensive performance of the target model after being trained through the baseline data is stable, and the effect and the energy consumption of the model are excellent. Thus, the computer device 1 is mainly validating the target model trained based on baseline data. In this embodiment, after the computer device 1 is connected to the server, training process data for performing model training based on baseline data by the target model may be acquired first.
Step S202, splitting a data input process and a calculation process from the training process data.
Specifically, the training process data includes the trend of all input data and the operation process of operation data of the target model in the trained process, so that the computer device 1 can identify the training process data and then split the data input process and the calculation process; the data input process comprises input data and flow information of the input data in the model training process, namely trend of the input data; the calculation process includes calculation logic and calculation process of the calculation logic for the target data.
In a specific embodiment, the splitting the data input process and the calculation process by the computer device 1 from the training process data includes: searching and splitting the data input process from the training process data according to the field identifiers of the start and end of the data flow operation; and searching and splitting the calculation process from the training process data according to the function body structure of the calculation function. The data input process comprises a real-time training data input process and/or an offline training data input process. Specifically, the computer device 1 is preset with corresponding identification modes for both the data input process and the calculation process, for example, the complete trend of the input data of the data input process, that is, the stream information, can be identified through the field identification for the beginning and the ending of the data stream operation; the range covered by the calculation logic, i.e. the calculation process, can also be identified from the training process data by the function body structure of the calculation function.
Of course, in another specific embodiment, the splitting the data input process and the calculation process from the training process data by the computer device 1 further includes: inquiring a time sequence relation and/or a dependency relation of the data input process and the calculation process in the training process data; and acquiring initial input data corresponding to the data input process and initial calculation logic corresponding to the calculation process. Specifically, in the process of separating the data input process and the calculation process from the training process data, the computer device 1 may query the time sequence, that is, the time sequence relationship, of the data input process and the calculation process in the training process data; trigger matters, namely dependency relations, between the data input process and the calculation process can also be queried.
Step S204, receiving a model verification task about the target model.
Step S206, scheduling the data input process and the calculation process to cooperatively execute the model verification task, so as to obtain a corresponding first verification result and a corresponding second verification result.
After splitting the data input process and the calculation process from the training process data, the computer device 1 defines the data input process and the calculation process as "task nodes"; thus, the two "task nodes" may also be sent to different authentication terminals, thereby enabling to distribute model authentication work to different authentication terminals, such as authentication device 1 and authentication device 2. Of course, in the present embodiment, the process in which the computer device 1 transmits the "task node" to the verification device 1 or the verification device 2 requires that all the environmental data required for the entire model verification of the target model be transmitted in the past; each authentication device, however, performs the authentication task of the response section primarily in accordance with the corresponding "task node".
Thus, after receiving a model verification task with respect to the target model, the computer device 1 may then schedule the data input process and the computing process to cooperatively perform the model verification task to obtain corresponding first and second verification results. In a specific embodiment, the computer device 1 scheduling the data input process and the computing process to cooperatively perform the model verification task includes: obtaining correction input data corresponding to the data input process in the model verification task and correction calculation logic corresponding to the calculation process; replacing the initial input data with the corrected input data, replacing the initial calculation logic with the corrected calculation logic, and executing the data input process and the calculation process according to the time sequence relation and/or the dependency relation.
In an illustrative example, the computer device 1 performing the data input process and the calculation process according to the timing relationship includes: acquiring a time watermark of the data input process and a time watermark of the calculation process; and triggering and executing the data input process and the calculation process in sequence according to the sequence of the time watermarks. In another illustrative example, the computer device 1 performing the data input process and the calculation process according to the dependency relationship includes: acquiring trigger matters in the dependency relationship; in the course of executing the data input process/the calculation process, if the trigger event is detected, the execution of the calculation process/the data input process is started.
Specifically, the computer device 1 schedules the data input process and the calculation process to execute the verification task, mainly by replacing the initial input data with the correction input data in the verification task through the data input process, and replacing the initial calculation logic with the correction calculation logic in the verification task through the calculation process, so as to realize verification of the target model according to the correction input data and/or the correction calculation logic. Finally, the computer device 1 obtains a first verification result and a second verification result about the verification task, which are fed back by the data input process and the calculation process.
Step S208, the first verification result and the second verification result are aggregated to obtain a model verification result.
As shown in FIG. 3, FIG. 3 is a flow chart illustrating the effect of implementing a plurality of data input processes and/or computing processes in coordination with performing model verification according to a time sequence relationship and a dependency relationship in an exemplary embodiment of the present invention. In this embodiment, the task node 1 is used for data input, and the process corresponding to the task node 1 includes three steps of data input, data statistics and data output; the task node 2 is used for calculating characteristics, and the corresponding process of the task node 2 comprises four steps of data input, data stream consumption, data statistics and data output. The computer device 1 sets a global watermark collector in the task node 1, so that details of executing a model verification task by the task node 1 can be collected, including time sequence; when the time watermark collected by the global watermark collector of the task node 1 reaches a preset time node, such as an hour or a day, the task node 1 feeds back "completed data input" to the notification service; the notification service triggers the task node 2 to start executing the model verification task; similarly, the task node 2 is also provided with a global watermark collector, so that details of executing the model verification task by the task node 1 can be collected, including time sequence; when the time watermark collected by the global watermark collector of the task node 1 reaches a preset time node, such as an hour or a day, the task node 2 feeds back "completed feature calculation" to the notification service. Finally, the notification service informs the AI of the scheduling, thereby initiating a Spark offline job. I.e. the scheduling task node n performs the corresponding model verification task.
In summary, the model verification method provided by the embodiment can obtain training process data of performing model training on the target model based on the baseline data; splitting a data input process and a calculation process from the training process data; receiving a model verification task regarding the target model; scheduling the data input process and the calculation process to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a corresponding second verification result; and aggregating the first verification result and the second verification result to obtain a model verification result. The training process data of the model are split into the data input process and the calculation process, and then the data input process and the calculation process are cooperated to perform model verification, so that the flexibility of the model verification process is improved, and the model verification efficiency is also improved.
Example two
Fig. 4 schematically shows a block diagram of a model verification device according to a second embodiment of the present application, which may be divided into one or more program modules, which are stored in a storage medium and executed by one or more processors to complete the embodiments of the present application. Program modules in the embodiments of the present application refer to a series of computer program instruction segments capable of implementing specific functions, and the following description specifically describes the functions of each program module in the embodiment.
As shown in fig. 4, the model verification apparatus 400 may include an acquisition module 410, a splitting module 420, a receiving module 430, a verification module 440, and an aggregation module 450, wherein:
an acquisition module 410 is configured to acquire training process data for performing model training of the target model based on the baseline data.
A splitting module 420, configured to split the data input process and the calculation process from the training process data. The data input process comprises a real-time training data input process and/or an offline training data input process.
A receiving module 430 for receiving model verification tasks with respect to the target model.
And the verification module 440 is configured to schedule the data input process and the calculation process to cooperatively execute the model verification task, so as to obtain a corresponding first verification result and a corresponding second verification result.
And the aggregation module 450 is configured to aggregate the first verification result and the second verification result to obtain a model verification result.
In an exemplary embodiment, the splitting module 420 is further configured to: searching and splitting the data input process from the training process data according to the field identifiers of the start and end of the data flow operation; and searching and splitting the calculation process from the training process data according to the function body structure of the calculation function. Inquiring the time sequence relation and/or the dependency relation of the data input process and the calculation process in the training process data; and acquiring initial input data corresponding to the data input process and initial calculation logic corresponding to the calculation process.
In an exemplary embodiment, the verification module 440 is further configured to obtain correction input data corresponding to the data input process and correction calculation logic corresponding to the calculation process in the model verification task; replacing the initial input data with the corrected input data, replacing the initial calculation logic with the corrected calculation logic, and executing the data input process and the calculation process according to the time sequence relation and/or the dependency relation. Acquiring a time watermark of the data input process and a time watermark of the calculation process; and triggering and executing the data input process and the calculation process in sequence according to the sequence of the time watermarks. Acquiring trigger matters in the dependency relationship; in the course of executing the data input process/the calculation process, if the trigger event is detected, the execution of the calculation process/the data input process is started.
Example III
Fig. 5 schematically shows a hardware architecture diagram of a computer device 1 adapted to implement a model verification method according to a third embodiment of the present application. In the present embodiment, the computer apparatus 1 is an apparatus capable of automatically performing numerical calculation and/or information processing in accordance with an instruction set or stored in advance. For example, the server may be a rack server, a blade server, a tower server, or a rack server (including a stand-alone server or a server cluster formed by a plurality of servers) with a gateway function, or the like. As shown in fig. 5, the computer device 1 includes at least, but is not limited to: the memory 510, processor 520, and network interface 530 may be communicatively linked to each other by a system bus. Wherein:
the memory 510 includes at least one type of computer-readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 510 may be an internal storage module of the computer device 1, such as a hard disk or memory of the computer device 1. In other embodiments, the memory 510 may also be an external storage device of the computer device 1, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the computer device 1. Of course, the memory 510 may also include both internal memory modules of the computer device 1 and external memory devices. In this embodiment, the memory 510 is typically used to store an operating system and various types of application software installed on the computer device 1, such as program codes of a model verification method, and the like. In addition, the memory 510 may also be used to temporarily store various types of data that have been output or are to be output.
Processor 520 may be a central processing unit (Central Processing Unit, simply CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 520 is generally used to control the overall operation of the computer device 1, such as performing control and processing related to data interaction or communication with the computer device 1, and the like. In this embodiment, the processor 520 is configured to execute program codes or process data stored in the memory 510.
The network interface 530 may comprise a wireless network interface or a wired network interface, which network interface 530 is typically used to establish a communication link between the computer device 1 and other computer devices. For example, the network interface 530 is used to connect the computer device 1 to an external terminal through a network, establish a data transmission channel and a communication link between the computer device 1 and the external terminal, and the like. The network may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a global system for mobile communications (Global System of Mobile communication, abbreviated as GSM), wideband code division multiple access (Wideband Code Division Multiple Access, abbreviated as WCDMA), a 4G network, a 5G network, bluetooth (Bluetooth), wi-Fi, etc.
It should be noted that fig. 5 only shows a computer device having components 510-530, but it should be understood that not all of the illustrated components are required to be implemented, and that more or fewer components may be implemented instead.
In this embodiment, the program code of the model verification method stored in the memory 510 may be further divided into one or more program modules and executed by one or more processors (the processor 520 in this embodiment) to complete the embodiments of the present application.
Example IV
The present embodiment also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
acquiring training process data of a target model for performing model training based on baseline data; splitting a data input process and a calculation process from the training process data; receiving a model verification task regarding the target model; scheduling the data input process and the calculation process to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a corresponding second verification result; and aggregating the first verification result and the second verification result to obtain a model verification result.
In this embodiment, the computer-readable storage medium includes a flash memory, a hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the computer readable storage medium may be an internal storage unit of a computer device, such as a hard disk or a memory of the computer device. In other embodiments, the computer readable storage medium may also be an external storage device of a computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card), etc. that are provided on the computer device. Of course, the computer-readable storage medium may also include both internal storage units of a computer device and external storage devices. In this embodiment, the computer-readable storage medium is typically used to store an operating system and various types of application software installed on a computer device, such as program code of the model verification method in the embodiment, and the like. Furthermore, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.
It will be apparent to those skilled in the art that the modules or steps of the embodiments of the application described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may alternatively be implemented in program code executable by computing devices, so that they may be stored in a storage device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than what is shown or described, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps of them may be fabricated into a single integrated circuit module. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.
The foregoing is only the preferred embodiments of the present application, and is not intended to limit the scope of the embodiments of the present application, and all equivalent structures or equivalent processes using the descriptions of the embodiments of the present application and the contents of the drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the embodiments of the present application.
Claims (10)
1. A method of model verification, the method comprising:
acquiring training process data of a target model for performing model training based on baseline data;
splitting a data input process and a calculation process from the training process data, wherein the data input process comprises input data and stream information of the input data in a model training process, and the calculation process comprises calculation logic and a calculation process of the calculation logic on target data;
receiving a model verification task regarding the target model;
scheduling the data input process and the calculation process to different verification terminals to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a second verification result;
and aggregating the first verification result and the second verification result to obtain a model verification result.
2. The model verification method of claim 1, wherein said splitting data input process and calculation process from said training process data comprises:
searching and splitting the data input process from the training process data according to the field identifiers of the start and end of the data flow operation; and
and searching and separating the calculation process from the training process data according to the function body structure of the calculation function.
3. The model verification method of claim 1, wherein the splitting of the data input process and the calculation process from the training process data further comprises:
inquiring a time sequence relation and/or a dependency relation of the data input process and the calculation process in the training process data; and
and acquiring initial input data corresponding to the data input process and initial calculation logic corresponding to the calculation process.
4. The model verification method of claim 3, wherein said scheduling said data input process and said computing process to cooperatively perform said model verification task comprises:
obtaining correction input data corresponding to the data input process in the model verification task and correction calculation logic corresponding to the calculation process;
replacing the initial input data with the corrected input data, replacing the initial calculation logic with the corrected calculation logic, and executing the data input process and the calculation process according to the time sequence relation and/or the dependency relation.
5. The model verification method as claimed in claim 4, wherein said performing said data input process and said calculation process according to said timing relationship comprises:
acquiring a time watermark of the data input process and a time watermark of the calculation process;
and triggering and executing the data input process and the calculation process in sequence according to the sequence of the time watermarks.
6. The model verification method as claimed in claim 4, wherein said performing said data input process and said calculation process according to said dependency relationship comprises:
acquiring trigger matters in the dependency relationship;
in the course of executing the data input process/the calculation process, if the trigger event is detected, the execution of the calculation process/the data input process is started.
7. The model verification method according to any one of claims 1 to 6, wherein the data input process includes a real-time training data input process and/or an offline training data input process.
8. A model verification apparatus, the apparatus comprising:
the acquisition module is used for acquiring training process data of a target model for performing model training based on the baseline data;
the splitting module is used for splitting a data input process and a calculation process from the training process data, the data input process comprises input data and stream information of the input data in a model training process, and the calculation process comprises calculation logic and a calculation process of the calculation logic on target data;
a receiving module for receiving a model verification task with respect to the target model;
the verification module is used for scheduling the data input process and the calculation process to different verification terminals to cooperatively execute the model verification task so as to obtain a corresponding first verification result and a corresponding second verification result;
and the aggregation module is used for aggregating the first verification result and the second verification result to obtain a model verification result.
9. A computer device, characterized in that it comprises a memory, a processor, on which a computer program is stored which can be run on the processor, the computer program, when being executed by the processor, implementing the steps of the model verification method according to any one of claims 1-7.
10. A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program executable by at least one processor to cause the at least one processor to perform the steps of the model verification method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011463493.4A CN112560939B (en) | 2020-12-11 | 2020-12-11 | Model verification method and device and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011463493.4A CN112560939B (en) | 2020-12-11 | 2020-12-11 | Model verification method and device and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112560939A CN112560939A (en) | 2021-03-26 |
| CN112560939B true CN112560939B (en) | 2023-05-23 |
Family
ID=75062877
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011463493.4A Active CN112560939B (en) | 2020-12-11 | 2020-12-11 | Model verification method and device and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112560939B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023028996A1 (en) * | 2021-09-03 | 2023-03-09 | Huawei Technologies Co., Ltd. | Methods and devices for ensuring the reproducibility of software systems |
| CN114861912B (en) * | 2022-07-06 | 2022-09-16 | 武汉山川软件有限公司 | Data verification method and device based on big data |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0974902A1 (en) * | 1998-07-21 | 2000-01-26 | Bull S.A. | Method of detecting errors on a serial link of an integrated circuit and device to realise this method |
| CN101183428A (en) * | 2007-12-18 | 2008-05-21 | 北京中星微电子有限公司 | Image detection method and apparatus |
| CN101938333A (en) * | 2010-09-21 | 2011-01-05 | 山东大学 | Gray code pi/ M-MPSK modulating soft bit information calculation method |
| CN102566506A (en) * | 2011-12-23 | 2012-07-11 | 东南大学 | Online coordination control and Petri net verification method combined with water cutting process |
| CN106227507A (en) * | 2016-07-11 | 2016-12-14 | 姚颂 | Calculating system and controller thereof |
| EP3301597A1 (en) * | 2016-10-03 | 2018-04-04 | Institute of Science and Technology Austria | Method for computationally designing a re-usable flexible mold |
| CN111695035A (en) * | 2020-06-10 | 2020-09-22 | 广东数果科技有限公司 | Recommendation system and multi-algorithm fusion recommendation processing flow |
| CN111966289A (en) * | 2020-08-13 | 2020-11-20 | 上海哔哩哔哩科技有限公司 | Partition optimization method and system based on Kafka cluster |
| CN112052868A (en) * | 2020-06-15 | 2020-12-08 | 上海集成电路研发中心有限公司 | Model training method, image similarity measuring method, terminal and storage medium |
-
2020
- 2020-12-11 CN CN202011463493.4A patent/CN112560939B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0974902A1 (en) * | 1998-07-21 | 2000-01-26 | Bull S.A. | Method of detecting errors on a serial link of an integrated circuit and device to realise this method |
| CN101183428A (en) * | 2007-12-18 | 2008-05-21 | 北京中星微电子有限公司 | Image detection method and apparatus |
| CN101938333A (en) * | 2010-09-21 | 2011-01-05 | 山东大学 | Gray code pi/ M-MPSK modulating soft bit information calculation method |
| CN102566506A (en) * | 2011-12-23 | 2012-07-11 | 东南大学 | Online coordination control and Petri net verification method combined with water cutting process |
| CN106227507A (en) * | 2016-07-11 | 2016-12-14 | 姚颂 | Calculating system and controller thereof |
| EP3301597A1 (en) * | 2016-10-03 | 2018-04-04 | Institute of Science and Technology Austria | Method for computationally designing a re-usable flexible mold |
| CN111695035A (en) * | 2020-06-10 | 2020-09-22 | 广东数果科技有限公司 | Recommendation system and multi-algorithm fusion recommendation processing flow |
| CN112052868A (en) * | 2020-06-15 | 2020-12-08 | 上海集成电路研发中心有限公司 | Model training method, image similarity measuring method, terminal and storage medium |
| CN111966289A (en) * | 2020-08-13 | 2020-11-20 | 上海哔哩哔哩科技有限公司 | Partition optimization method and system based on Kafka cluster |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112560939A (en) | 2021-03-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109474578B (en) | Message checking method, device, computer equipment and storage medium | |
| CN107689982B (en) | Multi-data source data synchronization method, application server and computer readable storage medium | |
| CN112560939B (en) | Model verification method and device and computer equipment | |
| CN113220657B (en) | Data processing method and device and computer equipment | |
| CN110599354B (en) | Online checking method, online checking system, computer device and computer readable storage medium | |
| CN108449313B (en) | Electronic device, Internet service system risk early warning method and storage medium | |
| CN112613938A (en) | Model training method and device and computer equipment | |
| CN107133233B (en) | Processing method and device for configuration data query | |
| CN111143165A (en) | Monitoring method and device | |
| CN111177121A (en) | Order data feedback method and device, computer equipment and storage medium | |
| CN112181835A (en) | Automatic testing method and device, computer equipment and storage medium | |
| CN110807050B (en) | Performance analysis method, device, computer equipment and storage medium | |
| CN115145708A (en) | System simulation task calling method and device, computer equipment and storage medium | |
| CN112560938B (en) | Model training method and device and computer equipment | |
| CN113093702B (en) | Fault data prediction method and device, electronic equipment and storage medium | |
| WO2019169696A1 (en) | Platform client data backflow method, electronic apparatus, device, and storage medium | |
| CN113836411B (en) | Data processing method and device and computer equipment | |
| CN112308172B (en) | Identification method and device and electronic equipment | |
| CN117891433A (en) | Order flow management system, operation method thereof and related medium | |
| CN105808787B (en) | A kind of data processing method and electronic equipment | |
| CN114663073A (en) | Abnormal node discovery method and related equipment | |
| CN116244365A (en) | Method and device for data aggregation, electronic equipment and storage medium | |
| CN116594742A (en) | Task scheduling method, device, computer equipment and storage medium | |
| CN116384696A (en) | Pipeline task operation method, device and system and computer equipment | |
| CN113781237A (en) | Product purchase order consumption method based on distributed artificial intelligence system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |