JP2021043949A - Device and method for training target detection model, and electronic apparatus - Google Patents

Abstract

To provide a device and a method for training a target detection model, and an electronic apparatus.SOLUTION: A training device includes: a test unit that performs a test to obtain detection results of the test images by inputting multiple test images at the termination of the N-th training in a training process of a target detection model into the target detection model; a determination unit that determines the convergence level predicted value for each test image based on the detection results of the test images; and an adjustment unit that adjusts the parameters used for the N+1 training based on the convergence level predicted value of the detection results of the test images.SELECTED DRAWING: Figure 1

Description

The present invention relates to the field of information technology.

In recent years, research in the field of computer vision has made great progress behind deep learning. Deep learning refers to a set of algorithms that solve various problems such as image processing and text processing by using various machine learning algorithms on a hierarchical neural network. The core of deep learning is feature learning, and its purpose is to solve the conventional difficulty of manually (artificial) designing features by obtaining hierarchical feature information through a hierarchical neural network. It is in. Deep learning has been applied to various fields of artificial intelligence.

Target detection is one of the important applications of deep learning. For a target detection model based on deep learning, the performance of the model is determined by training on the model.

However, the quality of the training data collection is a severe constraint on target detection, and the training data collection is generally collected from the life scene and it is difficult to balance all aspects. For example, a traffic scene data collection is generally taken by a surveillance camera, and since the scene is relatively single, a target detection model for a traffic scene with strong robustness is obtained based on the data collection. Is difficult. With the conventional training method, it is difficult to find the imbalance existing in the training data collection, and it is not possible to eliminate the influence of such imbalance on the training effect.

An object of the present invention is to dynamically control the convergence direction of the model in the training process and effectively eliminate the influence of the imbalance of training data on the training effect to obtain a target detection model with good performance. To provide equipment and methods as well as electronic devices for training target detection models that can be trained.

According to the first aspect of the embodiment of the present invention, a training device for a target detection model is provided, wherein the device.
At the end of the Nth training in the training process of the target detection model, a plurality of test images are input to the target detection model to perform a test, the detection result of each test image is acquired, and N is a positive integer;
Determine the predicted convergence level of each test image based on the detection result of each test image; and the parameter used for the N + 1 training based on the predicted convergence level of the detection result of each test image. Is configured to make adjustments.

According to the second aspect of the embodiment of the present invention, an electronic device is provided, the electronic device including the device described in the first aspect of the embodiment of the present invention.

According to a third aspect of an embodiment of the present invention, a training method for a target detection model is provided, wherein the method
At the end of the Nth training in the training process of the target detection model, a plurality of test images are input to the target detection model to perform a test, and the detection result of each test image is obtained. N is a positive integer;
Determine the predicted convergence level of each test image based on the detection result of each test image; and the parameter used for the N + 1 training based on the predicted convergence level of the detection result of each test image. Including adjusting.

The beneficial effects of the present invention are as follows: At the end of the Nth training of the target detection model, the convergence level prediction value is determined based on the detection result of the test image, and the convergence level prediction value is determined. By adjusting the parameters used for the next training based on, the convergence direction of the model can be dynamically controlled during the training process, and the influence of the imbalance of training data on the training effect can be effectively eliminated. This allows you to train a well-performing target detection model.

It is a figure which shows the training apparatus of the target detection model in Example 1 of this invention. It is a figure which shows the determination unit 102 in Example 1 of this invention. It is a figure which shows the training process based on the training apparatus 100 in Example 1 of this invention. It is a figure which shows the electronic device in Example 2 of this invention. It is a figure which shows the system configuration of the electronic device in Example 2 of this invention. It is a figure which shows the training method of the target detection model in Example 3 of this invention.

Hereinafter, preferred examples for carrying out the present invention will be described in detail with reference to the attached drawings.

The embodiments of the present invention provide a training device for a target detection model. FIG. 1 is a diagram showing a training device for a target detection model according to the first embodiment of the present invention.

As shown in FIG. 1, the training device 100 of the target detection model includes the following.

Test unit 101: At the end of the Nth training in the training process of the target detection model, multiple test images are input to the target detection model for testing, and the detection result of each test image is obtained. N is a positive integer. Is;
Confirmation unit 102: Confirms the predicted convergence level of each test image based on the detection result of each test image; and Adjustment unit 103: N + 1 based on the predicted convergence level of the detection result of each test image. Adjust the parameters used for the training sessions.

In this way, at the end of the Nth training of the target detection model, the convergence level prediction value is determined based on the detection result of the test image, and the convergence level prediction value is used for the next training. By adjusting the parameters, the convergence direction of the model can be dynamically controlled during the training process, and the influence of the imbalance of the training data on the training effect can be effectively removed, which enables the target detection with good performance. You can train the model.

In this embodiment, the target detection model may be a target detection model based on various networks, for example, a target detection model based on a deep neural network (DNN).

The detection target of the target detection model may be determined according to the actual needs. For example, the detection target can be determined based on the application scene.

In this embodiment, the test unit 101 tests at the end of the Nth training, where N is a positive integer.

In this embodiment, the number of times the test is performed at the end of the training may be determined according to the actual situation. For example, testing and parameter adjustment may be performed at the end of each training after the target detection model has reached initial stability (some degree of stability) after multiple (N) trainings. For example, N is 5 or greater.

For example, regarding a target detection model based on a deep neural network, tests and parameter adjustments may be performed at the end of each training after the fifth training.

In this embodiment, the training image as training data and the test image for testing have a plurality of labels marking a plurality of features. By marking multiple features in this way, the content of the image can be reflected in multiple dimensions, helping to find imbalances in the training data, thereby training the target detection model. Can help optimize in the process.

For example, with respect to a traffic scene target detection model, the plurality of features may include three types of features: type, image size, and image brightness. In other words, at least the type of each training image and test image (vehicle or pedestrian), the size of the image (large or small size) and the brightness of the image (bright or dark) are marked.

Table 1 shows the statistical results after marking 10000 training images. As shown in Table 1, in terms of type characteristics, 7500 training images are marked positive (vehicle) and 2500 training images are marked negative (pedestrian), image size. As for the feature, 3500 training images are marked as positive (large size), 6500 training images are marked as negative (small size), and as for the feature of image brightness, 8000 training images. Images are marked "bright" and 2000 images are marked "dark".

In this embodiment, the number of test images is n, and the numerical value of n may be set according to the actual situation.

At the end of the Nth training in the training process of the target detection model, the test unit 101 inputs n test images into the target detection model for testing, and obtains n detection results of n test images. To do.

For example, for the first test image, the detection result is "vehicle, large size, bright", and for the second test image, the detection result is "pedestrian, small size, bright". , The detection result of the 3rd test image is "vehicle, small size, dark", and ..., the detection result of the nth test image is "pedestrian, small size, dark". Is.

In this embodiment, the confirmation unit 102 determines the predicted convergence level of each test image based on the detection result of each test image.

FIG. 2 is a diagram showing a determination unit 102 according to the first embodiment of the present invention. As shown in FIG. 2, the determination unit 102 includes the following.

Calculation unit 201: Calculates the average value of the detection performance index of each test image and the detection performance index of all test images based on the detection result of each test image; and acquisition unit 202: the detection performance index of each test image and Enter the mean value into the classifier and get the predicted convergence level of each test image.

In this embodiment, the calculation unit 201 calculates the detection performance index of each test image based on the detection result of each test image.

As the detection performance index, various commonly used indexes, for example, IOU (Intersection over Union) may be used.

For example, the calculation unit 201 can make a comparison based on the detection result of the test image and the true value (marked true value) of the test image, and can calculate the IOU based on the result of the comparison. For a specific calculation method, related techniques can be referred to, and detailed description thereof will be omitted here.

Further, the calculation unit 201 calculates the IOUs of all the test images, and then further calculates the average value of the IOUs of all the test images.

For example, the IOU of the first test image calculated by the calculation unit 201 is 0.92, the IOU of the second test image is 0.83, and the IOU of the third test image is 0.72. And ..., the IOU of the nth test image is 0.51, and the average value of the IOU of the n test images is 0.75.

After the calculation unit 201 calculates the detection performance index and the average value of each test image, the acquisition unit 202 inputs the detection performance index and the average value of each test image into the classifier and obtains the convergence level predicted value of each test image. get.

In this embodiment, the input of the classifier is the detection performance index of each test image and the average value of these detection performance indexes, and the output result is the predicted value of the convergence level of each test image. The predicted value reflects the effect of the test image on the convergence of the model in the next training process.

In this embodiment, the classifier may be included in the determinant unit 102 or may be installed separately from the determinant unit 102.

The classifier may be a variety of conventional classifiers capable of achieving the above functions, for example, the classifier is a random forest based classifier. The classifier can be obtained by training with conventional training methods.

In this embodiment, the classifier can determine the predicted convergence level based on the Gaussian distribution classification criteria.

For example, the convergence level prediction value can be determined based on the following formula (1).

Here, f (x) represents the degree of convergence (degree) of the test image, x represents the detection performance index of the test image, μ represents the average value of the detection performance index, and the coefficient σ is positive. For example, σ = 0.1.

Alternatively, in order to suppress excessive adjustment in the latter stage of training, restrictions may be made by increasing the standard deviation and additional coefficients based on the above formula (1). Specifically, the following formula may be applied. It is as shown in (2) and (3).

Here, f (x) indicates the degree of convergence of the test image, x indicates the detection performance index of the test image, μ indicates the average value of the detection performance index, and the coefficient σ is a positive number. Yes, λ indicates the constraint coefficient, for example, σλ = 0.1, and epoch indicates the number of trainings.

In this example, based on the calculated degree of convergence, it belongs to a certain level of convergence. The number of convergence levels may be set in the actual situation, for example, the convergence levels are divided into four levels, "very high", "high", "medium", and "low", respectively. It may be represented by level 2, level 3 and level 4. In other words, the predicted convergence level determined based on the degree of convergence is one of 1, 2, 3, and 4. The embodiment of the present invention is not limited to such a division method.

For example, Table 2 shows the predicted value of each convergence level and the numerical range of the corresponding degree of convergence.

After the confirmation unit 10 determines the predicted convergence level of each test image, the adjustment unit 103 determines the parameters used for the N + 1 training based on the predicted convergence level of the detection result of each test image. adjust.

In this embodiment, the higher the predicted convergence level of the test image, the better the convergence of the target detection model based on the data of the test image, and conversely, the lower the predicted convergence level of the test image, the better the test. The target detection model has relatively poor convergence due to the image data.

The adjustment unit 103 adjusts the parameters used for the next training based on the predicted convergence level of the test image so that the image with a relatively low predicted convergence level is used relatively frequently for training. Let me. In this way, the influence of the imbalance of training data on the training effect is removed, and the target detection model has good detection accuracy only for a certain image or a certain kind of image (an image having a high convergence level prediction value). You can avoid that. This allows the trained target detection model to have good Robertness.

For example, in the next training, the number of times the training image similar to the test image having a relatively low predicted convergence level is used can be set to a relatively high value.

For example, in the next training, the gradient coefficient used for backpropagation in the next training, that is, the backpropagation gradient coefficient can be determined according to the predicted convergence level.

Table 3 shows the correspondence between the predicted convergence level and the backpropagation gradient coefficient in the next training.

As can be seen from Table 3, the larger the predicted convergence level, the larger the corresponding backpropagation gradient coefficient in the next training. In other words, when it comes to images with relatively poor current convergence levels, we will increase the weight in the next training.

FIG. 3 is a diagram showing a training process based on the training device 100 in the first embodiment of the present invention. As shown in FIG. 3, the Nth and N + 1th trainings are explained as an example, and at the end of the Nth training, the test unit 101 of the training device 100 targets a plurality of test images as the current target. Input to the detection model 200 and acquire the detection result of each test image; the confirmation unit 102 determines the convergence level predicted value of each test image based on the detection result of each test image; the adjustment unit 103 determines each of them. Adjust the parameters used for the N + 1 training based on the predicted convergence level of the test image.

In this embodiment, the training device 100 of the target detection model performs tests and parameter adjustments at the end of the Nth training, and uses the adjusted parameters for the N + 1th training, as well. At the end of the N + 1 training, tests and parameter adjustments may be performed using the same method, and the adjusted parameters may be used for the N + 2 training, and inferred based on this. be able to. In other words, the parameters for the test and the next training use are adjusted at the end of each training after the Nth training, but in this embodiment, for convenience, one of the tests and parameter adjustments are performed. Explains the process.

In this embodiment, in addition to the contents described in the examples of the present invention in the training process for the target detection model, the specific training method and the end determination method that need to be used in the training process are related. The technique can be referred to, and a detailed description thereof will be omitted here.

As can be seen from the above embodiment, at the end of the Nth training of the target detection model, the convergence level predicted value is determined based on the detection result of the test image, and the next training is performed based on the convergence level predicted value. By adjusting the parameters used for, the convergence direction of the model can be dynamically controlled during the training process, and the effect of imbalance of training data on the training effect can be effectively eliminated, thereby improving the performance. A good target detection model can be trained.

An embodiment of the present invention further provides an electronic device, and FIG. 4 is a diagram showing the electronic device according to the second embodiment of the present invention. As shown in FIG. 4, the electronic device 400 includes the training device 401 of the target detection model, and the structure and function of the training device 401 of the target detection model are the same as those described in the first embodiment. , The detailed explanation is omitted.

FIG. 5 is a diagram showing a system configuration of an electronic device according to a second embodiment of the present invention. As shown in FIG. 5, the electronic device 500 may include a processor 501 and a storage device 502, which is connected to the processor 501. It should be noted that the figure is merely an example, and a telecommunications function or another function may be realized by supplementing or substituting the structure with a structure of another type.

As shown in FIG. 5, the electronic device 500 may further include an input unit 503, a display 504, and a power supply 505.

In one embodiment, the function of the training device of the target detection model described in Example 1 may be integrated into the processor 501. Among them, the processor 501 may be configured as follows, that is, at the end of the Nth training in the training process of the target detection model, a plurality of test images are input to the target detection model for testing. , Acquire the detection result of each test image, N is a positive integer; determine the predicted convergence level of each test image based on the detection result of each test image; and converge the detection result of each test image. Adjust the parameters used for the N + 1 training based on the predicted level.

For example, determining the convergence level prediction value of each test image based on the detection result of each test image described above is based on the detection result of each test image, the detection performance index of each test image, and all the test images. The average value of the detection performance index of each test image is calculated; and the detection performance index of each test image and the average value are input to the classifier, and the convergence level predicted value of each test image is acquired.

For example, the classifier is a random forest-based classifier.

For example, the classifier determines the predicted convergence level of each test image based on the Gaussian distribution classification criteria.

For example, the parameter is a backpropagation gradient coefficient.

For example, the training image and the test image for training the target detection model have a plurality of labels marking a plurality of features.

For example, the N is 5 or more.

In another embodiment, the training device of the target detection model according to the first embodiment may be arranged independently of the processor 501, for example, the training device of the target detection model is connected to the processor 501. It may be configured as a chip to be used, and the function of the training device of the target detection model may be realized by controlling the processor 501.

In this embodiment, the electronic device 500 need not include all the components shown in FIG.

As shown in FIG. 5, the processor 501 may be referred to as a controller or operation controller and may include a microprocessor or other processor and / or logic device, eg, the processor 501 is central. A processing unit (CPU, Central Processing Unit) and / or a graphic processing unit (GPU, Graphics Processing Unit) may be included. The processor 501 can receive the input and control the operation of each component of the electronic device 500.

The storage device 502 may be, for example, one or more of buffers, fresh memory, HDDs, mobile media, volatile storage devices, non-volatile storage devices or other suitable devices. Further, the processor 501 can also realize information storage, processing, and the like by executing a storage program in the storage device 502. The functions of the other parts are similar to those of the conventional ones, and detailed description thereof will be omitted here. Further, each component of the electronic device 500 may be realized by dedicated hardware, firmware, software or a combination thereof, all of which belong to the scope of the present invention.

As can be seen from the above embodiment, at the end of the Nth training of the target detection model, the convergence level predicted value is determined based on the detection result of the test image, and the next training is performed based on the convergence level predicted value. By adjusting the parameters used for, the convergence direction of the model can be dynamically controlled during the training process, and the effect of imbalance of training data on the training effect can be effectively eliminated, thereby improving the performance. A good target detection model can be trained.

The examples of the present invention further provide a training method for the target detection model, which corresponds to the training device for the target detection model in Example 1. FIG. 6 is a diagram showing a training method of the target detection model according to the third embodiment of the present invention. As shown in FIG. 6, the method includes the following steps.

Step 601: At the end of the Nth training in the training process of the target detection model, multiple test images are input to the target detection model for testing, and the detection result of each test image is obtained, where N is a positive integer. Yes;
Step 602: Determine the predicted convergence level of each test image based on the detection results of each test image;
Step 603: Adjust the parameters used for the N + 1 training based on the predicted convergence level of the detection result of each test image.

In this embodiment, the specific implementation method of each step described above is the same as that described in Example 1, and therefore detailed description thereof will be omitted here.

As can be seen from the above embodiment, at the end of the Nth training of the target detection model, the convergence level predicted value is determined based on the detection result of the test image, and the next training is performed based on the convergence level predicted value. By adjusting the parameters used for, the convergence direction of the model can be dynamically controlled during the training process, and the effect of imbalance of training data on the training effect can be effectively eliminated, thereby improving the performance. A good target detection model can be trained.

The embodiments of the present invention further provide a computer-readable program, and when the program is realized in a training device or electronic device of the target detection model, the program tells the computer the training device or electronic device of the target detection model. The training method of the target detection model described in Example 3 is executed in the device.

An embodiment of the present invention further provides a storage medium in which a computer-readable program is stored, wherein the computer-readable program causes a computer to detect a target according to the third embodiment in a training device or an electronic device of a target detection model. Have the model practice the training method.

In addition, the methods, devices, and the like described in the examples of the present invention can be realized by hardware, software modules executed by a processor, or a combination of both. For example, one or more functions in the functional block diagram shown in FIG. 1 and / or a combination of one or more functions in the functional block diagram may correspond to each software module in a computer program, and each hardware module. May correspond to. In addition, each of these software modules can correspond to each step shown in FIG. These hardware modules can be realized by solidifying these software modules using, for example, FPGA (field-programmable gate array).

Further, the apparatus, method, etc. according to the embodiment of the present invention may be realized by software, may be realized by hardware, or may be realized by a combination of hardware and software. The present invention also relates to such a computer-readable program, i.e., when the program is executed by a logic component, the logic component can realize the above-mentioned device or component, or the above-mentioned logic. The component can implement the method described above or a step thereof. Furthermore, the present invention also relates to a storage medium that stores the above-mentioned program, such as a hard disk, a magnetic disk, an optical disk, a DVD, or a fresh memory.

In addition, the following additional notes will be further disclosed with respect to the above examples.

(Appendix 1)
A training method for the target detection model
At the end of the Nth training in the training process of the target detection model, a plurality of test images are input to the target detection model to perform a test, and the detection result of each test image is obtained. N is a positive integer;
The convergence level prediction value of each test image is determined based on the detection result of each test image; and the parameter used for the N + 1 training is determined based on the convergence level prediction value of the detection result of each test image. Methods, including adjusting.

(Appendix 2)
The method described in Appendix 1
Determining the convergence level prediction value of each test image based on the detection result of each test image described above is not possible.
Based on the detection result of each test image, the average value of the detection performance index of each test image and the detection performance index of all test images is calculated; and the detection performance index of each test image and the average value are input to the classifier. And a method that involves obtaining a convergent level prediction value for each test image.

(Appendix 3)
The method described in Appendix 2
The method, wherein the classifier is a classifier based on a random forest.

(Appendix 4)
The method described in Appendix 2 or 3,
The classifier is a method of determining a predicted convergence level of each test image based on a Gaussian distribution classification criterion.

(Appendix 5)
The method described in Appendix 1
The method, wherein the parameter is a backpropagation gradient coefficient.

(Appendix 6)
The method described in Appendix 1
A method in which a training image for training the target detection model and the test image have a plurality of labels for marking a plurality of features.

(Appendix 7)
The method described in Appendix 1
The method, wherein N is 5 or more.

Although the preferred embodiment of the present invention has been described above, the present invention is not limited to this embodiment, and any modification to the present invention belongs to the technical scope of the present invention unless the gist of the present invention is deviated.

Claims (10)

A device that trains a target detection model
At the end of the Nth training in the training process of the target detection model, a plurality of test images are input to the target detection model to perform a test, the detection result of each test image is acquired, and N is a positive integer. unit;
A confirmation unit that determines the predicted convergence level of each test image based on the detection result of each test image; and is used for the N + 1 training based on the predicted convergence level of the detection result of each test image. A device that contains an adjustment unit that adjusts parameters. The device according to claim 1.
The confirmation unit is
A calculation unit that calculates the average value of the detection performance index of each test image and the detection performance index of all test images based on the detection result of each test image; and the detection performance index of each test image and the average value are classified. A device that includes an acquisition unit that inputs to and obtains a convergent level prediction for each test image. The device according to claim 2.
The classifier is a device that is a classifier based on Dunlam Forest. The device according to claim 2 or 3.
The classifier is a device that determines the predicted convergence level of each test image based on the classification criteria of the Gaussian distribution. The device according to claim 1.
The device, wherein the parameter is a backpropagation gradient coefficient. The device according to claim 1.
The training image for training the target detection model and the test image are devices having a plurality of labels marking a plurality of features. The device according to claim 1.
The device in which N is 5 or more. An electronic device including the device according to claim 1. A way to train a target detection model
At the end of the Nth training in the training process of the target detection model, a plurality of test images are input to the target detection model to perform a test, the detection result of each test image is acquired, and N is a positive integer;
Determine the predicted convergence level of each test image based on the detection result of each test image; and the parameter used for the N + 1 training based on the predicted convergence level of the detection result of each test image. Methods, including adjusting. The method according to claim 9.
Determining the convergence level prediction value of each test image based on the detection result of each test image is not possible.
Based on the detection result of each test image, the average value of the detection performance index of each test image and the detection performance index of all test images is calculated; and the detection performance index of each test image and the average value are input to the classifier. And a method that involves obtaining a convergent level prediction value for each test image.

Images