CN115309638A - Method and device for assisting model optimization - Google Patents
Method and device for assisting model optimization Download PDFInfo
- Publication number
- CN115309638A CN115309638A CN202210922178.6A CN202210922178A CN115309638A CN 115309638 A CN115309638 A CN 115309638A CN 202210922178 A CN202210922178 A CN 202210922178A CN 115309638 A CN115309638 A CN 115309638A
- Authority
- CN
- China
- Prior art keywords
- model
- optimized
- interface
- parameters
- abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/362—Software debugging
- G06F11/366—Software debugging using diagnostics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3692—Test management for test results analysis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The invention discloses a method and a device for assisting model optimization, wherein the method comprises the following steps: monitoring each interface in real time, and determining the actual application scene of the model to be optimized when the interface calls the model to be optimized; determining and recording abnormal parameters in the actual application scene; and generating a monitoring report corresponding to the interface, wherein the monitoring report comprises abnormal parameters recorded in a certain time. By using the scheme of the invention, the optimization of the model can be rapidly and effectively promoted, and the performance and effect of the model are further improved.
Description
Technical Field
The invention relates to the technical field of model optimization, in particular to a method and a device for assisting model optimization.
Background
At present, with continuous progress and improvement of core elements such as technical theory, computing power, application scenes and the like of artificial intelligence, the artificial intelligence enters a rapid development stage. The basis of artificial intelligence is machine learning and big data, wherein the machine learning is a computer algorithm, namely a model, for predicting data such as images and sounds through multilayer nonlinear feature learning and hierarchical feature extraction.
The training of the model is usually obtained by training data with labels in a specific field, and some models may not be ideal in practical application due to problems of training samples, fields and the like, and cannot adapt to the requirements of various application environments and tasks. Therefore, the model which is used online needs to be further optimized subsequently to improve the performance of the model and enable the model to better adapt to different application scenarios.
Currently, optimization for online models is mainly to collect more training sets and testing sets to cover more business scenarios. For example, for specific services, problem feedback of users and other information, a scene to be optimized is screened out through research and development after being summarized, and then model optimization of relevant scenes is performed. The optimization method has certain hysteresis and influences user experience; and the description of part of problems is not clear, the investigation is difficult, and the like, and the problems can influence the optimization effect of the model.
Disclosure of Invention
The embodiment of the invention provides a method and a device for assisting model optimization, which are used for rapidly and effectively promoting the model optimization so as to improve the performance and the effect of the model.
In one aspect, an embodiment of the present invention provides a method for assisting model optimization, where the method includes:
monitoring each interface in real time, and determining the actual application scene of the model to be optimized when the interface calls the model to be optimized;
determining and recording abnormal parameters in the actual application scene;
and generating a monitoring report corresponding to the interface, wherein the monitoring report comprises abnormal parameters recorded in a certain time.
Optionally, the method further comprises: recording historical information of calling the model to be optimized by each interface;
the determining of the abnormal parameter in the actual application scenario includes: and calling the historical data of the model to be optimized according to the interface to determine abnormal parameters in the actual application scene.
Optionally, the recording historical information of the model to be optimized called by each interface includes: when the interface calls the model to be optimized each time, recording the times of calling the model to be optimized by each interface, and determining whether the actual application scene of the model to be optimized is consistent with the applicable scene of the model to be optimized; and recording the occurrence times of the abnormal scene and the abnormal parameters.
Optionally, the determining whether the actual application scenario of the model to be optimized is consistent with the applicable scenario of the model includes: acquiring transmission parameters when the interface calls the model to be optimized, wherein the transmission parameters comprise: input parameters, and/or output parameters; and determining whether the actual application scene of the model to be optimized is consistent with the applicable scene of the model to be optimized according to the transmission parameters.
Optionally, the determining, according to the transmission parameter, whether an actual application scenario of the model to be optimized is consistent with an application scenario of the model to be optimized includes: determining whether the actual application scene of the model to be optimized is consistent with the applicable scene of the model to be optimized according to any one or more of the following characteristics of the transmission parameters: parameter type, parameter quantity, parameter value range and whether the parameter is empty or not.
Optionally, the invoking, by the interface, the historical data of the model to be optimized includes: and the interface recently calls the calling amount of the model to be optimized, the calling ratio of the abnormal scene and the abnormal parameters.
Optionally, the determining, according to the historical data of the model to be optimized called by the interface, an abnormal parameter in the actual application scenario includes: calling historical data of the model to be optimized according to the interface to determine alarm threshold values of all parameters; and determining abnormal parameters in the actual application scene according to the alarm threshold values of the parameters.
Optionally, the method further comprises: generating training samples according to abnormal parameters in the monitoring reports corresponding to the interfaces; and optimizing the model to be optimized by utilizing the training sample.
Optionally, the generating a training sample according to the abnormal parameter in the monitoring report corresponding to each interface includes: determining the priority of different application scenes according to the monitoring reports corresponding to each interface; and generating a training sample according to the abnormal parameters in the application scenes with higher priority in the monitoring reports corresponding to the interfaces.
In another aspect, an embodiment of the present invention further provides a device for assisting model optimization, where the device includes:
the monitoring module is used for monitoring each interface in real time, and determining the actual application scene of the model to be optimized when the interface calls the model to be optimized;
the judging module is used for determining abnormal parameters in the actual application scene;
the recording module is used for recording the abnormal parameters;
and the report generating module is used for generating a monitoring report corresponding to the interface, and the monitoring report comprises abnormal parameters recorded in a certain time.
Optionally, the recording module is further configured to record historical information of the model to be optimized called by each interface; the judging module is specifically configured to determine an abnormal parameter in the actual application scene according to the historical data of the model to be optimized called by the interface.
Optionally, the apparatus further comprises: the training sample generation module is used for generating training samples according to the abnormal parameters in the monitoring reports corresponding to the interfaces; and the optimization processing module is used for optimizing the model to be optimized by utilizing the training sample.
Optionally, the apparatus further comprises: the priority determining module is used for determining the priorities of different application scenes according to the monitoring reports corresponding to the interfaces; the training sample generation module is specifically configured to generate a training sample according to the abnormal parameter in the application scenario with a higher priority in the monitoring report corresponding to each interface.
According to the method and the device for assisting model optimization, provided by the embodiment of the invention, each interface of the model to be optimized is monitored and called, data monitoring is carried out on the online model to be optimized, abnormal parameters in an actual application scene are screened out in real time and recorded, and a monitoring report corresponding to the corresponding interface is generated according to the abnormal parameters recorded in a certain time, so that the monitoring report can be timely fed back to related research and development, research and development personnel are assisted in quickly positioning problems, and the model is promoted to be optimized. Because the collection, the record of the abnormal parameters and the generation of the monitoring report are automatically completed through the real-time monitoring of the interface, the timeliness and the integrity of the data are better, the problems of the model to be optimized in the practical application can be clearer and more effective, the problems can be effectively and quickly fed back, the continuous optimization of the model is promoted, and the performance and the effect of the model are improved.
Furthermore, the scheme of the invention can also determine the priority of different application scenes according to the monitoring report corresponding to each interface, so that the model can be optimized preferentially aiming at the problems in the application scenes with high priority, and the actual application requirements on the model can be better met.
Drawings
FIG. 1 is a flow chart of a method of assisting model optimization according to an embodiment of the present invention;
FIG. 2 is another flow chart of a method of assisting model optimization according to an embodiment of the present invention;
FIG. 3 is another flow chart of a method of assisting model optimization according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of an apparatus for assisting model optimization according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of another structure of an apparatus for assisting model optimization according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of another structure of an apparatus for assisting model optimization according to an embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.
In the prior art, data acquisition in a training set required for model optimization mostly depends on information such as problem business and user feedback, and therefore a series of problems are caused. In view of these problems, embodiments of the present invention provide a method and an apparatus for assisting model optimization, where each interface of a model to be optimized is monitored and called, data monitoring is performed on the online model to be optimized, abnormal parameters in an actual application scene are screened out in real time and recorded, and a monitoring report corresponding to a corresponding interface is generated according to the recorded abnormal parameters within a certain time, so that the monitoring report can be timely fed back to related research and development, thereby assisting research and development personnel in quickly locating problems and promoting optimization of the model.
Fig. 1 is a flowchart of a method for assisting model optimization according to an embodiment of the present invention, including the following steps:
And step 102, determining and recording abnormal parameters in the actual application scene.
Specifically, the abnormal parameter in the actual application scene may be determined according to the historical data of the model to be optimized called by the interface. That is to say, when each interface calls the model to be optimized, relevant information is recorded, for example, the number of times that each interface calls the model to be optimized is recorded, and whether the actual application scenario of the model to be optimized is consistent with the applicable scenario of the model to be optimized is determined; and recording the occurrence times of the abnormal scene and the abnormal parameters. Of course, other data may be available according to actual needs, and the embodiment of the present invention is not limited. The recorded information is history information of calling the model to be optimized for the interface for the follow-up calling of the interface to the model to be optimized.
The abnormal parameters may include abnormal parameters in a normal scene (for example, abnormal parameters occurring due to problems of the model itself in the normal scene), and abnormal parameters in an abnormal scene (for example, abnormal parameters occurring due to problems limited by applicable scenes of the model itself). And if the actual application scene of the model to be optimized is consistent with the applicable scene of the model to be optimized, determining the actual application scene as a normal scene, otherwise, determining the actual application scene as an abnormal scene.
It should be noted that, when determining whether the actual application scenario of the model to be optimized is consistent with the applicable scenario thereof, the determination may be performed according to transmission parameters of an interface invoking the model, where the transmission parameters include input parameters and/or output parameters. Specifically, the determination may be made based on one or more characteristics of the parameters, which may include, but are not limited to: the number and the type of the parameters, whether the parameters are empty, the numerical range of the parameters and the like are judged, whether the characteristics of the transfer parameters of the calling interface are consistent with the characteristics required by the applicable scene of the model is judged, and if the characteristics are consistent, the actual application scene of the model is determined to be consistent with the applicable scene; otherwise, the actual application scene is an abnormal scene. Of course, the determination method may be different for different models, and the embodiment of the present invention is not limited thereto. Moreover, according to different service requirements of each model, some inspection logics of a service calling party can be further integrated for judgment, and the judgment is not limited in the same way.
In determining the abnormal parameter, the abnormal parameter may be determined based on an output result of the model, i.e., the output parameter. For example, for an OCR (optical character recognition) model, a service calling the model is a user credit score, that is, the OCR model is used to identify the user credit score, the service requires that an identification result, that is, an output parameter is non-null and a digital type, if the output parameter of the OCR model for the credit score is chinese, english, or null character, etc., it can be determined that the model identification is wrong, and the corresponding output parameter is an abnormal parameter.
Of course, the determination of the abnormal parameter is only illustrated for the OCR model, and for other models, there may be more determination manners, which need to be determined according to the characteristics of each specific model, and the embodiment of the present invention is not limited thereto.
In the embodiment of the present invention, the historical data of the interface calling the model to be optimized may include, but is not limited to, the following data: and the interface recently calls the calling amount of the model to be optimized, the calling ratio of the abnormal scene and the abnormal parameters.
The recent period refers to a latest period of time, and the specific length of the period of time may be set according to application needs, such as a week, a month, and the like, which is not limited herein.
The number of times that the interface calls the model to be optimized, which is recorded in the near term, can be used as the calling amount of the model to be optimized; the calling proportion of the abnormal scene can be calculated according to statistical data, namely the ratio of the number of times of the abnormal scene when the interface calls the model to be optimized in the near term to the total number of times of calling the model to be optimized.
The historical data can be used as reference information for determining abnormal parameters when the interface calls the model to be optimized subsequently. Specifically, the alarm threshold of each parameter may be determined according to the historical data of the model to be optimized called by the interface; and determining abnormal parameters in the actual application scene according to the alarm threshold values of the parameters. For example, if 10 abnormalities of a certain parameter obtained by referring to historical data occur within 5 minutes, the parameter belongs to a normal condition, if 10 abnormalities are found, the abnormal condition is found, and the alarm threshold corresponding to the parameter is 10; and if the number of the abnormal conditions of the parameter is more than 10 within 5 minutes, recording error data of the parameter.
It should be noted that, corresponding to different interfaces, the actual application scenarios for calling the model to be optimized may be different, and therefore the recorded historical information may also be different. Therefore, the determination of the alarm threshold needs to be calculated corresponding to each interface.
In addition, when the alarm threshold is determined, the proportion of abnormal parameters in the historical data can also be comprehensively considered, for example, the passing rate of a certain scene required by a service is 99%, and in the historical data, the abnormal data in the scene should be less than or equal to 1%; if some services do not specifically specify the scene, the complaint amount, the call amount and the abnormal amount in the historical data can be counted, for example, the total call amount 10000, the abnormal amount 200 and the related complaint amount 10 of the past month of the scene have an abnormal proportion of 2%, because of complaints, the abnormal amount is not satisfactory, the alarm threshold can be set to be less than 2%.
In addition, according to different storage modes of online data, the abnormal parameters can be monitored in different modes, for example, when the storage mode of a database is adopted, the abnormal parameters can be monitored by sql statements; when the log storage mode is adopted, the abnormal parameters can be monitored by a python script. Of course, two monitoring methods available for common data storage modes are listed here, and the embodiment of the present invention does not limit the data storage method, as long as data monitoring can be conveniently achieved.
103, generating a monitoring report corresponding to the interface, wherein the monitoring report comprises abnormal parameters recorded within a certain time.
The monitoring report can adopt html format, and the monitoring start time, the monitoring end time and the like corresponding to the monitoring report can be marked. Of course, the monitoring report can adopt other formats as long as the relevant monitoring data can be clearly recorded.
In addition to the above abnormal parameters, the monitoring report may include, but is not limited to, any one or more of the following information: userID (user identification), request time, input parameters, output parameters, etc.
Further, the monitoring report may also be stored in a local or remote database to facilitate viewing or recall when needed.
According to the method for assisting model optimization provided by the embodiment of the invention, each interface of the model to be optimized is monitored and called, data monitoring is carried out on the online model to be optimized, abnormal parameters in an actual application scene are screened out in real time and recorded, and a monitoring report corresponding to the corresponding interface is generated according to the recorded abnormal parameters within a certain time, so that the monitoring report can be timely fed back to related research and development, a research and development worker is assisted in rapidly positioning problems, and the model is promoted to be optimized. Because the collection, the record of the abnormal parameters and the generation of the monitoring report are automatically completed through the real-time monitoring of the interface, the timeliness and the integrity of the data are better, the problems of the model to be optimized in the practical application can be clearer and more effective, the problems can be effectively and quickly fed back, the continuous optimization of the model is promoted, and the performance and the effect of the model are improved.
Further, in another embodiment of the method for assisting model optimization of the present invention, the monitoring report may be further utilized to optimize the model to be optimized.
Fig. 2 is another flow chart of a method for assisting model optimization according to an embodiment of the present invention.
Step 201 to step 203 are the same as step 101 to step 103 in fig. 1, and are not described herein again. In addition, the embodiment shown in fig. 2 further comprises the following steps:
and step 204, generating training samples according to the abnormal parameters in the monitoring reports corresponding to the interfaces.
It should be noted that the specific information of the training sample may be determined according to information included in the data sample required by the training of the model to be optimized, and the embodiment of the present invention is not limited thereto.
And step 205, optimizing the model to be optimized by using the training sample.
When the model to be optimized is optimally trained, the model to be optimized may be optimally trained by using the sample data generated according to the abnormal parameters in the monitoring report alone, or the sample data may be added to the previous sample set, and the model to be optimized is optimally trained together with the previous sample data, which is not limited in the embodiment of the present invention.
Furthermore, the scheme of the invention can respectively monitor a plurality of different interfaces to obtain the monitoring reports corresponding to the interfaces, thereby obtaining the monitoring results of the model to be optimized in a plurality of different application scenes. Correspondingly, in another embodiment of the model optimization assisting method, different application scenes can be distinguished, and monitoring reports of application scenes with higher priorities can be fed back in time, so that the model can be optimized quickly, the model can have better performance in the application scenes with the high priorities, and the use experience of a user on the model is improved.
Fig. 3 is another flow chart of a method for assisting model optimization according to an embodiment of the present invention, which includes the following steps:
And step 304, determining the priority of different application scenes according to the monitoring reports corresponding to the interfaces.
For example, the priorities of different application scenes can be determined according to the occupation ratios of the application scenes, and the priority is higher when the occupation ratio is larger; for another example, the priorities of different application scenarios may be determined according to the proportion of the abnormal parameters in the application scenarios, and the higher the proportion is, the higher the priority is. Of course, multiple factors may also be considered to determine the priority of each application scenario, which is not limited in this embodiment of the present invention.
And step 306, optimizing the model to be optimized by using the training sample.
Similarly, when the model to be optimized is optimally trained, the model to be optimized may be optimally trained by using sample data generated according to the abnormal parameters in the monitoring report alone, or the sample data may be added to the previous sample set, and the model to be optimized is optimally trained together with the previous sample data, which is not limited in the embodiment of the present invention.
It should be noted that the optimization cycle of the model may also be determined according to actual needs and monitoring results of each interface, and may also be dynamically adjusted. For example, if the recent model to be optimized has more errors in a high-priority scene, the optimization cycle is shortened; the optimization cycle can be extended if there are more errors in only some less applicable scenarios and few errors in some commonly used scenarios. The determination may be determined according to the number, importance, and the like of the specific model and the application scenario thereof, and the embodiment of the present invention is not limited.
Therefore, by using the method for assisting model optimization provided by the embodiment of the invention, the acquisition, recording and feedback of abnormal parameters of the model in the actual application scene can be automatically completed in real time, and the priority of different application scenes can be determined according to the monitoring reports corresponding to each interface, so that the model can be optimized preferentially for the problems in the application scenes with high priority, and the actual application requirements on the model can be better met.
Correspondingly, the embodiment of the invention also provides a device for assisting model optimization, and as shown in fig. 4, the device is a schematic structural diagram of the device.
In this embodiment, the apparatus 400 for assisting model optimization includes the following modules:
the monitoring module 401 is configured to monitor each interface in real time, and when the interface calls a model to be optimized, determine an actual application scene in which the model to be optimized is called currently;
a determining module 402, configured to determine an abnormal parameter in the actual application scenario;
a recording module 403, configured to record the abnormal parameter;
a report generating module 404, configured to generate a monitoring report corresponding to the interface, where the monitoring report includes abnormal parameters recorded within a certain time.
The recording module 403 is further configured to record history information of the model to be optimized called by each interface, and the content of the history information and the determination and recording modes of some data may refer to the description in the foregoing embodiment of the method of the present invention, which is not described herein again.
Accordingly, the determining module 402 may determine the abnormal parameter in the actual application scenario according to the historical data of the model to be optimized called by the interface.
By utilizing the device for assisting model optimization provided by the embodiment of the invention, the calling condition of each interface to the model to be optimized can be monitored in real time, the monitoring report corresponding to each interface is generated, and the monitoring report can be timely fed back to related research and development, so that research and development personnel are assisted to quickly locate problems, and the model is promoted to be optimized. Because the collection, the record of the abnormal parameters and the generation of the monitoring report are automatically completed through the real-time monitoring of the interface, the timeliness and the integrity of the data are better, the problems of the model to be optimized in the practical application can be clearer and more effective, the problems can be effectively and quickly fed back, the continuous optimization of the model is promoted, and the performance and the effect of the model are improved.
Fig. 5 is a schematic structural diagram of an apparatus for assisting model optimization according to an embodiment of the present invention.
Unlike the embodiment shown in fig. 4, in this embodiment, the apparatus 400 for assisting model optimization may further include: a training sample generation module 501 and an optimization processing module 502. Wherein:
the training sample generating module 501 is configured to generate training samples according to the abnormal parameters in the monitoring reports corresponding to each interface;
the optimization processing module 502 is configured to optimize the model to be optimized by using the training samples.
It should be noted that, when performing optimization training on the model to be optimized, sample data generated according to abnormal parameters in a monitoring report may be used alone to perform optimization training on the model to be optimized, or the sample data may be added to a previous sample set to perform optimization training on the model to be optimized together with the previous sample data, which is not limited in the embodiment of the present invention.
Fig. 6 is a schematic structural diagram of an apparatus for assisting model optimization according to an embodiment of the present invention.
Unlike the embodiment shown in fig. 5, in this embodiment, the apparatus 400 for assisting model optimization may further include: a priority determining module 503, configured to determine priorities of different application scenarios according to the monitoring reports corresponding to the interfaces.
Accordingly, in this embodiment, the training sample generating module 501 may generate a training sample according to the abnormal parameter in the application scenario with higher priority in the monitoring report corresponding to each interface.
The device for assisting model optimization provided by the embodiment of the invention not only can automatically complete the acquisition, recording and feedback of abnormal parameters of the model in an actual application scene in real time, but also can determine the priorities of different application scenes according to the monitoring reports corresponding to each interface, so that the model can be optimized preferentially for problems in the application scenes with high priorities, and the actual application requirements on the model can be better met.
In a specific implementation, each module/unit included in each apparatus and product described in the foregoing embodiments may be a software module/unit, may also be a hardware module/unit, or may also be a part of a software module/unit and a part of a hardware module/unit.
For example, for each device or product applied to or integrated into a chip, each module/unit included in the device or product may be implemented by hardware such as a circuit, or at least a part of the module/unit may be implemented by a software program running on a processor integrated within the chip, and the rest (if any) part of the module/unit may be implemented by hardware such as a circuit; for each device or product applied to or integrated with the chip module, each module/unit included in the device or product may be implemented by using hardware such as a circuit, and different modules/units may be located in the same component (e.g., a chip, a circuit module, etc.) or different components of the chip module, or at least some of the modules/units may be implemented by using a software program running on a processor integrated within the chip module, and the rest (if any) of the modules/units may be implemented by using hardware such as a circuit; for each device and product applied to or integrated in the terminal, each module/unit included in the device and product may be implemented by using hardware such as a circuit, and different modules/units may be located in the same component (e.g., a chip, a circuit module, etc.) or different components in the terminal, or at least part of the modules/units may be implemented by using a software program running on a processor integrated in the terminal, and the rest (if any) part of the modules/units may be implemented by using hardware such as a circuit.
An embodiment of the present invention further provides a computer-readable storage medium, which is a non-volatile storage medium or a non-transitory storage medium, and a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program performs the steps of the method provided in the embodiment corresponding to fig. 1, fig. 2, or fig. 3.
Another apparatus assisting model optimization is further provided in an embodiment of the present invention, including a memory and a processor, where the memory stores a computer program executable on the processor, and the processor executes the computer program to perform the steps of the method provided in the embodiment corresponding to fig. 1, fig. 2, or fig. 3.
The embodiment of the present invention further provides an electronic device, which includes a memory and a processor, where the memory stores a computer program executable on the processor, and the processor executes the steps of the method provided in the embodiment corresponding to fig. 1, fig. 2, or fig. 3 when executing the computer program.
In this embodiment of the present invention, the processor may be a Central Processing Unit (CPU), and the processor may also be other general-purpose processors, digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It will also be appreciated that the memory in embodiments of the invention may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash memory. Volatile memory can be Random Access Memory (RAM), which acts as external cache memory. By way of example and not limitation, many forms of Random Access Memory (RAM) are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (enhanced SDRAM), synchronous DRAM (SLDRAM), synchronous Link DRAM (SLDRAM), and direct bus RAM (DR RAM).
It should be understood that the term "and/or" herein is merely one type of association relationship that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein indicates that the former and latter associated objects are in an "or" relationship.
"plurality" appearing in the embodiments of the present invention means two or more.
The descriptions of the first, second, etc. appearing in the embodiments of the present invention are only for illustrating and differentiating the objects, and do not have any order or represent any special limitation to the number of devices in the embodiments of the present invention, and do not constitute any limitation to the embodiments of the present invention.
The term "connect" in the embodiments of the present invention refers to various connection manners, such as direct connection or indirect connection, to implement communication between devices, which is not limited in this respect.
Although the present invention is disclosed above, the present invention is not limited thereto. Various changes and modifications may be effected therein by one skilled in the art without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (13)
1. A method for assisting model optimization, the method comprising:
monitoring each interface in real time, and determining the actual application scene of the model to be optimized when the interface calls the model to be optimized;
determining and recording abnormal parameters in the actual application scene;
and generating a monitoring report corresponding to the interface, wherein the monitoring report comprises abnormal parameters recorded in a certain time.
2. The method of claim 1, further comprising: recording historical information of calling the model to be optimized by each interface;
the determining of the abnormal parameter in the actual application scenario includes:
and calling the historical data of the model to be optimized according to the interface to determine abnormal parameters in the actual application scene.
3. The method according to claim 2, wherein the recording of historical information of each interface call model to be optimized comprises:
when the interface calls the model to be optimized each time, recording the times of calling the model to be optimized by each interface, and determining whether the actual application scene of the model to be optimized is consistent with the applicable scene of the model to be optimized;
and recording the occurrence frequency of the abnormal scene and the abnormal parameters.
4. The method of claim 3, wherein the determining whether the actual application scenario of the model to be optimized is consistent with the applicable scenario of the model comprises:
acquiring transmission parameters when the interface calls the model to be optimized, wherein the transmission parameters comprise: input parameters, and/or output parameters;
and determining whether the actual application scene of the model to be optimized is consistent with the applicable scene of the model to be optimized according to the transmission parameters.
5. The method according to claim 4, wherein the determining whether the actual application scenario of the model to be optimized and the applicable scenario of the model to be optimized are consistent according to the transmission parameters comprises:
determining whether the actual application scene of the model to be optimized is consistent with the applicable scene of the model to be optimized according to any one or more of the following characteristics of the transmission parameters: parameter type, parameter quantity, parameter value range and whether the parameter is empty or not.
6. The method of claim 3, wherein the interface invoking the historical data of the model to be optimized comprises: and the interface recently calls the quantity of the model to be optimized, calls the proportion of an abnormal scene and abnormal parameters.
7. The method according to claim 2, wherein the determining abnormal parameters in the actual application scenario according to the historical data of the model to be optimized called by the interface comprises:
calling historical data of the model to be optimized according to the interface to determine alarm threshold values of all parameters;
and determining abnormal parameters in the actual application scene according to the alarm threshold values of the parameters.
8. The method according to any one of claims 1 to 7, further comprising:
generating training samples according to abnormal parameters in the monitoring reports corresponding to the interfaces;
and optimizing the model to be optimized by utilizing the training sample.
9. The method of claim 8, wherein generating training samples according to abnormal parameters in the monitoring reports corresponding to each interface comprises:
determining the priority of different application scenes according to the monitoring reports corresponding to each interface;
and generating a training sample according to the abnormal parameters in the application scenes with higher priority in the monitoring reports corresponding to the interfaces.
10. An apparatus that facilitates model optimization, the apparatus comprising:
the monitoring module is used for monitoring each interface in real time, and determining the actual application scene of the model to be optimized when the interface calls the model to be optimized;
the judging module is used for determining abnormal parameters in the actual application scene;
the recording module is used for recording the abnormal parameters;
and the report generating module is used for generating a monitoring report corresponding to the interface, and the monitoring report comprises abnormal parameters recorded in a certain time.
11. The apparatus of claim 10,
the recording module is also used for recording the historical information of the model to be optimized called by each interface;
the judging module is specifically configured to determine an abnormal parameter in the actual application scene according to the historical data of the model to be optimized called by the interface.
12. The apparatus of claim 10 or 11, further comprising:
the training sample generation module is used for generating training samples according to the abnormal parameters in the monitoring reports corresponding to the interfaces;
and the optimization processing module is used for optimizing the model to be optimized by utilizing the training sample.
13. The apparatus of claim 12, further comprising:
the priority determining module is used for determining the priorities of different application scenes according to the monitoring reports corresponding to the interfaces;
the training sample generation module is specifically configured to generate a training sample according to the abnormal parameter in the application scenario with a higher priority in the monitoring report corresponding to each interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210922178.6A CN115309638A (en) | 2022-08-02 | 2022-08-02 | Method and device for assisting model optimization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210922178.6A CN115309638A (en) | 2022-08-02 | 2022-08-02 | Method and device for assisting model optimization |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115309638A true CN115309638A (en) | 2022-11-08 |
Family
ID=83858218
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210922178.6A Pending CN115309638A (en) | 2022-08-02 | 2022-08-02 | Method and device for assisting model optimization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115309638A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115760494A (en) * | 2023-02-10 | 2023-03-07 | 深圳市思为软件技术有限公司 | Data processing-based service optimization method and device for real estate marketing |
-
2022
- 2022-08-02 CN CN202210922178.6A patent/CN115309638A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115760494A (en) * | 2023-02-10 | 2023-03-07 | 深圳市思为软件技术有限公司 | Data processing-based service optimization method and device for real estate marketing |
| CN115760494B (en) * | 2023-02-10 | 2023-05-05 | 深圳市思为软件技术有限公司 | Service optimization method and device for real estate marketing based on data processing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110275958B (en) | Website information identification method and device and electronic equipment | |
| WO2021174694A1 (en) | Operation and maintenance monitoring method and apparatus based on data center, device, and storage medium | |
| CN111552633A (en) | Interface abnormal call testing method and device, computer equipment and storage medium | |
| CN111522728A (en) | Method for generating automatic test case, electronic device and readable storage medium | |
| CN111597818B (en) | Call quality inspection method, device, computer equipment and computer readable storage medium | |
| CN111061948B (en) | User tag recommendation method and device, computer equipment and storage medium | |
| CN108074033A (en) | Processing method, system, electronic equipment and the storage medium of achievement data | |
| CN109409948B (en) | Transaction abnormity detection method, device, equipment and computer readable storage medium | |
| CN111626498A (en) | Equipment operation state prediction method, device, equipment and storage medium | |
| CN115309638A (en) | Method and device for assisting model optimization | |
| CN111047433A (en) | Method and device for analyzing reasons of user number abnormality, server and storage medium | |
| CN115130577A (en) | Method and device for identifying fraudulent number and electronic equipment | |
| CN114239296A (en) | Data processing method and device and electronic equipment | |
| CN113467867A (en) | Information processing method, information processing device, electronic equipment and storage medium | |
| US20170017706A1 (en) | Systems and methods for utilizing compliance drivers to conserve system resources and reduce compliance violations | |
| CN112232960A (en) | Transaction application system monitoring method and device | |
| CN110716856A (en) | Distributed system submodule interaction testing method and device | |
| EP4089592A1 (en) | Method for determining annotation capability information, related apparatus and computer program product | |
| CN115314404B (en) | Service optimization method, device, computer equipment and storage medium | |
| CN117421191A (en) | Intelligent question-answer pressure measurement method, device, equipment and medium | |
| CN113064599A (en) | Deployment method and device for online service predicted by machine learning model | |
| CN117252685A (en) | Method, system, device, equipment and storage medium for checking public accumulation information | |
| CN111125372A (en) | Text information publishing method and device, readable storage medium and electronic equipment | |
| CN116401140A (en) | Data processing method, device, equipment, readable medium and software product | |
| CN116976312A (en) | Financial business scene recognition method, device, equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |