TWI750622B - Deep learning model training system, deep learning model training method, and non-transitory computer readable storage medium - Google Patents

Abstract

A deep learning model training system, deep learning model training method, and non-transitory computer readable storage medium are provided. The method includes: adjusting an original image set to obtain testing image sets according to testing values; inputting the testing training image sets to a first learning model for testing to obtain confidence values, wherein the confidence values correspond to the testing values in a one-to-one manner; classifying the test values into an invalid group or a valid group according to whether the confidence value corresponding to each of the test value is less than a threshold value; obtaining training values according to the test values in the invalid group; adjusting the original image set to obtain training image sets according to the training values; and inputting the original image set and the training image sets to the first deep learning model for training to obtain a second deep learning model.

Description

Deep learning model training system, deep learning model training method, and non-transitory computer-readable storage medium

This case is about the field of artificial intelligence, especially a deep learning model training system and its method.

In recent years, the research on artificial intelligence has changed rapidly, especially the application of deep learning in various fields, including the field of image recognition. Today's deep learning systems in the field of image recognition usually require users to use a huge amount of images as training data to train the deep learning system, so that the deep learning system can operate normally, that is, the output of the deep learning system has a higher level. Confidence (also known as Confidence). Although this training method is easy to understand in theory, it is not easy to implement because processing and obtaining a large number of images is time-consuming and usually requires a high-performance processor to execute. In addition to collecting the original images, the method to obtain a huge amount of images is to obtain similar images by adjusting the original images.

However, in the process of training a deep learning system with a huge amount of images as training data, it will be found that the training amount is not proportional to the increase in the confidence. For example, a large amount of training only increases the confidence slightly. In other words, the current method of obtaining similar images by adjusting the original image can increase the training data, but it is a time-consuming and inefficient training method.

In addition, in the current meter research, although the smart meter has been researched, the smart meter can measure and report the data electronically, so that the measurement data can be online in real time. However, in practice, smart meters are not common, and most users still use traditional meters. If the traditional meter is replaced with a smart meter, the water supply needs to be cut off, but the power supply needs to be cut off, which not only costs the cost of the meter itself, but also requires a 24-hour operation for the user. It has a huge time cost, so users are not happy to replace the traditional meter with a smart meter. However, the traditional meter needs manual meter reading to collect and count the measurement data, which is not only inefficient but also prone to errors. Therefore, the current meter still needs to be improved.

In view of the above, this case proposes a deep learning model training system and method.

According to some embodiments, the deep learning model training method includes: adjusting the original image set according to multiple test values to obtain multiple test image sets; inputting the test image set to the first deep learning model for testing to obtain multiple confidence levels, confidence levels Corresponding to the test values in a one-to-one manner; according to whether the confidence corresponding to each test value is less than the threshold, the test values are classified into valid groups or invalid groups; according to the test values in the invalid groups, multiple training values are obtained; The training value adjusts the original image set to obtain a plurality of training image sets for training; and, inputting the original image set and the training image set to the first deep learning model to obtain the second deep learning model.

According to some embodiments, a deep learning model training system includes a processor and a storage device. The processor is used to receive the original image set, a plurality of test values and threshold values. The processor is configured to adjust the original image set according to the test value to obtain multiple test image sets, and input the test image set to the first deep learning model for testing to obtain multiple confidence levels, where the confidence levels correspond to the test values in a one-to-one manner. The processor classifies the test values into a valid group or an invalid group according to whether the corresponding confidence level of each test value is smaller than a threshold value, and obtains a plurality of training values according to the test values in the invalid group. The processor is configured to adjust the original image set according to the training value to obtain a plurality of training image sets, and input the original image set and the training image set to the first deep learning model for training to obtain the second deep learning model.

According to some embodiments, a non-transitory computer-readable storage medium is used to store one or more software programs, the one or more software programs including a plurality of instructions, when the instructions are executed by one or more processors of the electronic device , which will make the electronic device perform the deep learning model training method.

To sum up, the deep learning model training system, the deep learning model training system method and the non-transitory computer-readable storage medium of some embodiments of the present application can adjust the original image set according to the test value to obtain the test image set, and input the test image set The first deep learning model is tested to obtain the confidence of the test image set, that is, the confidence of the test value. And the test value can be classified into the valid group or the invalid group according to the threshold value, and then the training value can be obtained from the test value in the invalid group. and adjusting the original image set according to the training value to obtain a training image set, and inputting the original image set and the training image set to the first deep learning model for training to obtain a second deep learning model.

Several embodiments of the present application will be disclosed in the following drawings. For the sake of clarity, many practical details will be described together in the following description. It should be understood, however, that these practical details should not be used to limit the present case. That is, in some embodiments of this case, these practical details are unnecessary. In addition, for the purpose of simplifying the drawings, some well-known and conventional structures and elements will be shown in a simple and schematic manner in the drawings, and the same reference numerals will be used to denote the same or similar elements in all the drawings. . And if possible in implementation, the features of different embodiments can be applied interactively.

FIG. 1 is a schematic diagram of a deep learning model training system 100 according to some embodiments of the present application. Please refer to FIG. 1 . In some embodiments, the deep learning model training system 100 includes a processor 120 and a storage device 140 . The processor 120 is configured to receive an original image set, a plurality of test values and threshold values, and adjust the original image set according to the test values to obtain a plurality of test image sets. The processor 120 is configured to input the test image set to the first deep learning model T1 for testing to obtain a plurality of confidence levels, and the confidence levels correspond to the test values in a one-to-one manner. The processor 120 is configured to classify the test values into a valid group or an invalid group according to whether the corresponding confidence level of the test value is smaller than a threshold value, and obtain a plurality of training values according to the test values in the invalid group. The processor 120 is configured to adjust the original image set according to the training value to obtain a plurality of training image sets. The processor 120 is configured to input the original image set and the training image set to the first deep learning model T1 for training to obtain the second deep learning model T2. The storage device 140 is used for storing the first deep learning model T1 and the second deep learning model T2. In some embodiments, the storage device 140 is electrically connected to the processor 120 .

In some embodiments, the deep learning model (ie, the first deep learning model T1 or the second deep learning model T2 ) is suitable for use in image recognition, eg, recognizing numerical values of digital images in images. Taking an image with a digital image "0011" as an example, the deep learning model is used to identify the image to obtain the value "0011".

In some embodiments, when the deep learning model is tested, the deep learning model outputs the test answer (eg, the value "0011") and the confidence of the test answer according to the input test question (eg, the image with the digital image "0011") Spend. Specifically, the confidence represents how much confidence the deep learning model has for the test answer to be correct, or how much confidence the deep learning model has for the input test question to answer correctly, that is, the correct rate of the test answer. Therefore, the confidence level is between 0 and 1. When the confidence level is 1, it means that the correct rate of the test answer is 100%. When the confidence level is 0.5, it means that the correct rate of the test answer is 50%. When the confidence level is 0, it means that the test answer is correct. The correct rate is 0%.

In some embodiments, the deep learning model is trained by supervised learning. Specifically, the supervised learning is inputting corresponding training questions and training answers to the deep learning model for training. After the training, the deep learning model can be trained according to The received training questions and training answers are used to process the test questions, that is, the deep learning model can compare the test questions and the training questions, and output the test answers corresponding to the test questions according to the training answers. Therefore, after the deep learning model is based on appropriate supervised learning, the deep learning model can improve the confidence of the output, that is, the correct rate of the test answer. When this specification refers to "input image (set) to the first deep learning model T1 for training, and obtain the second deep learning model T2", it means "input image (set) and corresponding values (set) to the first deep learning model T2" A deep learning model T1 is trained, and a second deep learning model T2 is obtained. The image (set) is the training question in supervised learning, and the corresponding value (set) is the training answer in supervised learning. The corresponding value (set) is not limited to be obtained in any form, for example, the file of the image (set) itself has the value (set), the value (set) is input to the deep learning model training system 100 from the outside, or the processor 120 is based on other data. The recognition model recognizes images (sets) to generate values (sets).

FIG. 2 is a flowchart of a deep learning model training method according to some embodiments of the present application. In order to clearly illustrate the operations of the various elements in FIG. 1 , the following detailed description is given in conjunction with the flowchart in FIG. 2 . However, those with ordinary knowledge in the technical field to which this case belongs can understand that the deep learning model training method of the embodiment of this case is not limited to the deep learning model training system 100 shown in FIG. item step sequence. In some embodiments, a non-transitory computer-readable storage medium is used to store one or more software programs, the one or more software programs including a plurality of instructions when executed by one or more processing circuits of an electronic device , which will make the electronic device perform the deep learning model training method. Specifically, the electronic device includes one or more processing circuits (also referred to as control circuits). The deep learning model training method is implemented by, for example, one or more software programs. The software program is stored on a CD-ROM, a hard disk or other non-transitory computer-readable storage medium. The software program includes a plurality of instructions related to the processing circuit. After these instructions or software programs are loaded by the electronic device, the electronic device will execute the deep learning model training method. A detailed description of each step of the deep learning model training method is listed below.

Please refer to FIG. 1 and FIG. 2 together. In some embodiments, the process of the deep learning model training method includes a test image set acquisition step (step S210 ), a model testing step (step S220 ), a grouping step (step S230 ), a training value The obtaining step (step S240 ), the training image set obtaining step (step S250 ), and the model training step (step S260 ).

In some embodiments, the step of obtaining a test image set (step S210 in FIG. 2 ) includes: adjusting the original image set according to a plurality of test values to obtain a plurality of test image sets. Specifically, the original image set includes a plurality of original images, and each test image set includes a plurality of test images respectively. The test value takes angle rotation as an example, the unit of test value is "degree", and each test value is "-10, -9.5, -9, -8.5, -8, -7.5, -7, -6.5, -6, -5.5, -5, -4.5, -4, -3.5, -3, -2.5, -2, -1.5, -1, -0.5, 0, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4 , 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10”. A positive test value indicates that the original image is rotated clockwise, a negative value indicates that the original image is rotated counterclockwise, and a test value of "0" indicates that the original image is not rotated, but not limited to this. In some embodiments, test image sets correspond to test values in a one-to-one manner, eg rotated "-10" degrees for one test image set, "-9.5" degrees for another test image set, and so on, so When there are "21" test values, the processor 120 adjusts the original image set according to the "21" test values to obtain corresponding "21" test image sets. In some embodiments, the number of original images in an original image set multiplied by a test scale factor equals the number of test images in a test image set, and the test scale factor is between 0 and 1. Specifically, the processor 120 selects a part of the original image set from the original image set according to the test scale factor, and adjusts the part of the original image set to obtain each test image set. In particular, divide "the number of original images in the partial original image set" (or, "the number of test images in a test image set") by "the number of original images in the original image set" ” is equal to the test scale factor. For example, when the number of original images is "1000", the test value is "21" in total, and the test scale factor is "0.1", the number of partial original images is "1000*0.1=100", and the test images in each test image set is "100", so the total number of test images in the "21" test image set is "21*100=2100".

In some embodiments, the test value is an arithmetic sequence. For example, in the above-mentioned angular rotation embodiment, the maximum value of the test value is "10" degrees, and the minimum value of the test value is "-10" degrees. The tolerance is "0.5".

In some embodiments, when the processor 120 adjusts the original image set according to the test value to obtain multiple test image sets, each test value only corresponds to the same adjustment type, for example, the adjustment type is angle rotation, but not limited thereto. In some embodiments, the adjustment categories are, for example, but not limited to, angle rotation, proportional scaling, height enlargement, width enlargement, contrast adjustment, brightness adjustment, and gamma (Gamma) value adjustment. "Angle rotation" can rotate the original image in any direction. For example, in a rectangular coordinate system, the original image includes a first axis and a second axis, where the first axis is the Y axis and the second axis is the Z axis . The original image can be rotated in X, Y, Z or any axis as a test image. "Height enlargement" means that the original image is stretched while maintaining the same width, and "width enlargement" means that the original image is widening while maintaining the same height.

In some embodiments, the test values include raw values corresponding to the original image set. Specifically, the original value is one of the test values, usually the original value is a benchmark of the test value, for example, the original value is the minimum value, the maximum value or the median among the test values. In terms of the adjustment category corresponding to the test value, the original value is the value corresponding to the adjustment category of the original image set. For example, when the adjustment type is angle rotation, the original value is "0" degrees, that is, the rotation angle of the original image set is "0" degrees (ie, not rotated). For example, when the adjustment category is proportional scaling, the original value is "1", that is, the scale of the original image set is "1" times (ie, the original size).

In some embodiments, the test image set corresponding to the original value is a partial original image set. Specifically, when the processor 120 adjusts the original image set according to the original value to obtain the test image set, the processor 120 uses the original image set as the test image set, that is, the processor 120 can obtain the test image without adjusting the original image set set. For example, when the adjustment type corresponding to the test value is an angle rotation, the original value is "0" degree, so the processor 120 can use the original image set as the test image set. In some embodiments, the processor 120 determines a portion of the original image set as the test image set according to the test scale factor.

In some embodiments, the model testing step (step S220 ) includes: inputting a test image set to the first deep learning model T1 for testing, and obtaining a plurality of confidence levels, where the confidence levels correspond to the test values in a one-to-one manner. As can be seen from the foregoing description about the deep learning model, the confidence level represents how much confidence the first deep learning model T1 has in the input test image set to correctly output the corresponding test answer. Therefore, when the processor 120 inputs the test image set to the first deep learning model T1 for testing, the processor 120 can obtain the confidence level corresponding to the test image set from the first deep learning model T1. Since the test image set is adjusted according to the test values, the confidence levels correspond to the test values in a one-to-one manner. In some embodiments, the confidence level corresponding to the test image set is an average of multiple sub-confidence levels. Specifically, the test image set includes a plurality of test images, the first deep learning model T1 correspondingly outputs confidence levels according to the input test image set, and the first deep learning model T1 outputs each sub-confidence correspondingly according to each input test image Spend. Therefore, when the processor 120 inputs any test image to the first deep learning model T1 for testing, the first deep learning model T1 can output the sub-confidence corresponding to the test image, so the processor 120 selects one of the test images in a certain test image set. The average calculation of all sub-confidences corresponding to each test image of , can obtain the confidence of this test image set. In some embodiments, the confidence level corresponding to the test image set may be the median, maximum value, and minimum value of multiple sub-confidence levels, but not limited thereto.

In some embodiments, the test value takes the angle rotation as an example, and the comparison table between the test value and the confidence is shown in Table 1 below:

Table 1 test value Confidence test value Confidence test value Confidence -10 0 -2.5 0.5 3 0.4 -9.5 0 -2 0.6 3.5 0.3 -9 0 -1.5 0.7 4 0.2 -8.5 0 -1 0.8 4.5 0.1 -8 0 -0.5 0.9 5 0 -7.5 0 0 1 5.5 0 -7 0 0.5 0.9 6 0 -6.5 0 1 0.8 6.5 0 -6 0 1.5 0.7 7 0 -5.5 0 2 0.6 7.5 0 -5 0 2.5 0.5 8 0 -4.5 0.1 8.5 0 -4 0.2 9 0 -3.5 0.3 9.5 0 -3 0.4 10 0

In some embodiments, the grouping step (step S230 in FIG. 2 ) includes: judging whether the confidence level corresponding to each test value is less than a threshold value, and classifying the test values into invalid groups or valid groups. Specifically, a higher confidence level means that the first deep learning model T1 can more correctly identify the input test image set, and a lower confidence level means that the first deep learning model T1 cannot be correctly identified. Therefore, the processor 120 determines whether the confidence level corresponding to the test value is smaller than the threshold value, distinguishes the valid confidence level and the invalid confidence level through the threshold value, classifies the test value whose confidence level is less than the threshold value into the invalid group, and classifies the confidence level greater than or Test values equal to the threshold are classified into valid groups. Taking the test value as an example of angular rotation and referring to Table 1, if the threshold value is "0.5", when the confidence level is less than "0.5", it means that the first deep learning model T1 cannot be effectively and correctly identified, so the confidence level is less than "0.5" corresponding to Test value "-10, -9.5, -9, -8.5, -8, -7.5, -7, -6.5, -6, -5.5, -5, -4.5, -4, -3.5, -3, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10" are classified in the invalid group. When the confidence level is greater than or equal to "0.5", it means that the first deep learning model T1 can be effectively and correctly identified, so the test value corresponding to the confidence level greater than or equal to "0.5" is "-2.5, -2, -1.5, -1, -0.5 , 0, 0.5, 1, 1.5, 2, 2.5" are classified as valid groups.

In some embodiments, the test value takes the angle rotation as an example, and the comparison table between the invalid group and the valid group is shown in Table 2 below:

Table 2 invalid group valid group invalid group test value Confidence test value Confidence test value Confidence -10 0 -2.5 0.5 3 0.4 -9.5 0 -2 0.6 3.5 0.3 -9 0 -1.5 0.7 4 0.2 -8.5 0 -1 0.8 4.5 0.1 -8 0 -0.5 0.9 5 0 -7.5 0 0 1 5.5 0 -7 0 0.5 0.9 6 0 -6.5 0 1 0.8 6.5 0 -6 0 1.5 0.7 7 0 -5.5 0 2 0.6 7.5 0 -5 0 2.5 0.5 8 0 -4.5 0.1 8.5 0 -4 0.2 9 0 -3.5 0.3 9.5 0 -3 0.4 10 0

In some embodiments, the step of obtaining the training value (step S240 in FIG. 2 ) includes: obtaining a plurality of training values according to the test values in the invalid group. Specifically, the test image set corresponding to the test values in the invalid group is the test image set that cannot be effectively and correctly identified by the first deep learning model T1, so the test image set corresponding to the test values in the invalid group is the first The deep learning model T1 needs to strengthen the training part, so the processor 120 obtains the training value according to the test value in the invalid group. In some embodiments, the training value is all the test values in the invalid group. Taking the test value as an example of angle rotation and referring to Table 2, the training value is "-10, -9.5, -9, -8.5, -8, -7.5 , -7, -6.5, -6, -5.5, -5, -4.5, -4, -3.5, -3, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10”.

In some embodiments, the step of obtaining the training value (step S240 in FIG. 2 ) further includes: obtaining a valid interval value according to the difference between the largest test value in the valid group and the original value; and according to a plurality of integers of the valid interval value times, the training values are obtained from the test values in the null group. Specifically, the process of obtaining the training value by the processor 120 according to the test value in the invalid group includes the above steps, wherein the processor 120 calculates the difference between the largest test value in the valid group and the original value, and then calculates the difference between the two values of the difference. The valid interval value is obtained multiple times (ie, the valid interval value is twice the difference between the largest test value in the valid group and the original value), and the processor 120 matches the tests in the invalid group with multiple integer multiples of the valid interval value. value as the training value. Taking the test value of angle rotation as an example and referring to Table 2, the largest test value in the valid group is "2.5", the original value is "0", and the difference between the largest test value in the valid group and the original value is "| 2.5-0|=2.5", twice the difference is "2*2.5=5", so the valid interval value is "5". Multiple integer multiples of the valid interval value are "5*m, m=0, ±1, ±2, ...", so the test values of multiple integer multiples of the valid interval value in the invalid group are "-10, -5 , 5, 10", so the training value is "-10, -5, 5, 10".

In some embodiments, the step of obtaining the training image set (step S250 in FIG. 2 ) includes: adjusting the original image set according to the training value to obtain a plurality of training image sets. Specifically, the training image set includes a plurality of training images. Taking the test value as an example of an angle rotation and referring to Table 2, the unit of the training value is "degree", and each training value is "-10, -5, 5, 10". In some embodiments, training image sets correspond to training values in a one-to-one manner, eg, rotating the original image set by "-10" degrees to obtain one training image set, and rotating the original image set by "-5" degrees to obtain another One training image set, and so on, so when there are "4" training values, the processor 120 adjusts the original image set according to the "4" training values to obtain "4" training image sets. In some embodiments, the number of original images in an original image set multiplied by a training scale factor equals the number of training images in a training image set, and the training scale factor is between 0 and 1. Specifically, the training scale coefficient is similar to the test scale coefficient, except that the training scale coefficient is, for example, but not limited to, equal to the test scale coefficient. The processor 120 selects parts of the original image sets from the original image sets according to the training scale factor, and adjusts the part of the original image sets to obtain each training image set. In particular, divide "the number of original images in the partial original image set" (or, "the number of training images in a training image set") by "the number of original images in the original image set" ” is equal to the training scale factor. For example, when the number of original images is "1000" and the training scale factor is "0.5", the number of training images in one training image set is "1000*0.5=500", and similarly for "4" training image sets The total number of training images is "4*500=2000".

In some embodiments, the model training step (step S260 in FIG. 2 ) includes: inputting the original image set and the training image set to the first deep learning model T1 for training to obtain the second deep learning model T2 . Specifically, when the processor 120 inputs the original image set and the training image set to the first deep learning model T1 for training, the first deep learning model T1 uses the original image set and each training image set as training data. In other words, the second deep learning model T2 can improve the confidence of each test image set of the invalid group according to the original image set and each training image set. Therefore, compared with the first deep learning model T1, without using all the test values in the invalid group as training values, the second deep learning model T2 can use each training image set as training data to improve the invalid group. The confidence level corresponding to each test image set of , so it can reduce the amount of training data of the deep learning model. Taking the angle rotation as the test value as an example and referring to Table 2, the processor 120 only needs to input the original image set and the training image set corresponding to the training value "-10, -5, 5, 10" (4 training image sets in total) to the first deep learning model T1 for training.

In some embodiments, taking the training scale factor of 1 as an example, the number of training images in the training image set is equal to the number of original images in the original image set, for all test values in the null set (ie, -10, -9.5 , -9, -8.5, -8, -7.5, -7, -6.5, -6, -5.5, -5, -4.5, -4, -3.5, -3, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10) as training values (30 training image sets in total, excluding the original image set) for comparison, the deep learning model training system 100 And the deep learning model training method only needs the amount of training data of "(1+4)/(1+30)=16.1%". For comparison with all test values as training values (41 training image sets in total, including original image sets), the deep learning model training system 100 and the deep learning model training method only need "(1+4)/( 1+40) = 12.2%" of the amount of training data. The above-mentioned embodiment of reducing the amount of training data is only an example and not limited thereto. The amount of training data reduced by the learning model training system 100 and the deep learning model training method will be based on different adjustment categories, different test values, and different values. Confidence and different thresholds have different results.

In some embodiments, the test value takes the angle rotation as an example, and the comparison table between the test value and the confidence is shown in Table 3 below:

table 3 valid group invalid group test value Confidence test value Confidence 0 1 3 0.4 0.5 0.9 3.5 0.3 1 0.8 4 0.2 1.5 0.7 4.5 0.1 2 0.6 5 0 2.5 0.5 5.5 0 6 0 6.5 0 7 0 7.5 0 8 0 8.5 0 9 0 9.5 0 10 0

In some embodiments, each test value according to Table 3 is "0, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10", the original value is "0" degrees. Since the threshold is "0.5", the test values "3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10" with confidence less than "0.5" are classified In the invalid group, test values "0, 0.5, 1, 1.5, 2, 2.5" with confidence greater than or equal to "0.5" are classified as valid. In some embodiments, the training value is "3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10 when the training value is all test values in the null group. ". In some embodiments, the step of obtaining the training value (step S250 in FIG. 2 ) further includes: obtaining a valid interval value according to the difference between the largest test value in the valid set and the original value; and according to a plurality of integers of the valid interval value times, the training values are obtained from the test values in the null group. The largest test value in the valid group is "2.5", the original value is "0", and the difference between the largest test value in the valid group and the original value is "|2.5-0|=2.5", twice the difference is "2*2.5=5", so the valid interval value is "5". Multiple integer multiples of the valid interval value are "5*m, m=0, ±1, ±2, ...", so the test values of multiple integer multiples of the valid interval value in the invalid group are "5, 10", So the training value is "5, 10".

In some embodiments, the deep learning model training method further includes an importance obtaining step, and the importance obtaining step is used to obtain the importance corresponding to the test value. In some embodiments, the processor 120 obtains the importance corresponding to the test value according to the importance obtaining step. Specifically, when the processor 120 adjusts the original image set into a plurality of test image sets according to the test value, each test value corresponds to only one adjustment category, so the importance obtaining step is used to obtain the importance of the adjustment category. The adjustment categories therefore correspond to the importance in a one-to-one manner.

In some embodiments, the deep learning model training system 100 and the deep learning model training method can determine which adjustment category is used to train the first deep learning model T1 according to the importance of each adjustment category. For example, take the rotation with the least important angle as the adjustment category. In some embodiments, the deep learning model training system 100 and the deep learning model training method can sequentially train the first deep learning model T1 with each adjustment category according to the importance of each adjustment category.

In some embodiments, when the height of the original image is greater than the width of the original image, the importance of the adjustment categories is arranged as follows: proportional scaling > angle rotation > contrast adjustment > brightness adjustment > gamma value adjustment > height enlargement > width enlarge.

In some embodiments, when the height of the original image is smaller than the width of the original image, the importance of the adjustment categories is arranged as follows: scaling>angle rotation>contrast adjustment>brightness adjustment>gamma value adjustment>width enlargement>height enlarge.

FIG. 3 is a flow chart of the steps of obtaining the importance according to some embodiments of the present application. Referring to FIG. 3 , in some embodiments, the flow of the importance obtaining step includes a range value obtaining step (step S310 ), a grouping step (step S320 ), an invalid boundary value obtaining step (step S330 ), and an importance determining step (step S340 ) ).

In some embodiments, the step of obtaining the range value (step S310 in FIG. 3 ) includes: obtaining the range value according to the difference between the largest test value and the smallest test value. Specifically, the range value is the range of the test value, so the processor 120 can obtain the range value according to the difference between the largest test value and the smallest test value. Taking the test value of angle rotation as an example and referring to Table 1, the maximum test value is "10", and the minimum test value is "-10", so the range value is "|10-(-10)|=20".

In some embodiments, the grouping step (step S320 in FIG. 3 ) includes: classifying the test values into invalid groups or valid groups according to whether the confidence levels corresponding to each test value are smaller than a threshold. The grouping step (step S320 in FIG. 3 ) is similar to the grouping step (step S230 in FIG. 2 ), which will not be repeated here. The difference between step S320 and step S230 is that the thresholds of both may be the same (for example, is "0.5"), or may be different (for example, the threshold value in step S320 may be "0.6", and the threshold value in step S230 may be "0.5"). Taking the test value of angle rotation as an example and referring to Table 2, if the threshold value is "0.5", the confidence level is less than "0.5" and the test value classified in the invalid group is "-10, -9.5, -9, -8.5, - 8, -7.5, -7, -6.5, -6, -5.5, -5, -4.5, -4, -3.5, -3, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10”. When the confidence level is greater than or equal to "0.5", the test value classified as a valid group is "-2.5, -2, -1.5, -1, -0.5, 0, 0.5, 1, 1.5, 2, 2.5".

In some embodiments, the step of obtaining the invalid boundary value (step S330 in FIG. 3 ) includes: obtaining the invalid boundary value according to the smallest difference between the test value in the invalid group and the original value. Specifically, the processor 120 can compare the difference between the test value in the invalid group and the original value, obtain each difference value corresponding to each test value in the invalid group and the original value, and compare the difference between the difference values. The minimum difference value is obtained from the size, and then the invalid boundary value is obtained according to the test value corresponding to the minimum difference value, that is, the invalid boundary value is the test value corresponding to the minimum difference value. It should be noted that the difference between the test value and the original value is "|original value-test value|", that is, each difference value is a positive value. The invalid boundary value represents the threshold that the test value cannot be correctly identified by the first deep learning module T0. In other words, the invalid boundary value is the boundary between the invalid group and the valid group, and the invalid boundary value is the test value in the invalid group. . In some embodiments, taking the test value as an example of angular rotation and referring to Table 2, if the original value is "0", the difference between the test value and the original value in the invalid group is "3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10", so the smallest difference is "3", and the smallest difference corresponds to a test value of "3" or "-3" ", and "3" or "-3" is an invalid boundary value. It should be noted that using "3" or "-3" as the invalid boundary value does not affect the importance, because the difference between "3" and "-3" and the original value is the same (both are the smallest difference. value "3").

In some embodiments, the step of determining the importance level (step S340 in FIG. 3 ) includes: dividing the difference between the original value and the invalid boundary value by the range value to obtain the importance level. Specifically, the processor 120 can obtain a quotient value by dividing the difference between the original value and the invalid boundary value by the range value, and the quotient value is the importance. The original value is the value of the original image set corresponding to the adjustment category, and the importance is the difference between the original value and the invalid boundary value divided by the range value, that is, the importance corresponds to the difference between the valid group (invalid group) and the range value. Proportion. Therefore, when the importance is lower, the proportion of the test values representing the valid group to all the test values is lower (the proportion of the test values of the invalid group to all the test values is higher), that is, the more likely this adjustment category affects the confidence. . On the contrary, when the importance is higher, the proportion of the test values representing the valid group to all the test values is higher (the proportion of the test values of the invalid group to all the test values is lower), that is, the less likely this adjustment category will affect the confidence. Spend. Take the test value of angle rotation as an example and refer to Table 2, and take the test value of angle rotation as an example and refer to Table 2, the original value is "0", the difference between the invalid boundary value and the original value is "3", the range The value is "20" and the quotient is "3/20=0.15", so the importance is "0.15".

In some embodiments (eg, Table 2), the importance determination step (step S340 in FIG. 3 ) can be represented by a first importance obtaining formula, and the first importance obtaining equation is as follows:

是代表在

與

之中取最小值，B₁ 、B₂ 是無效邊界值，R為範圍值。where I is the importance, O is the original value,

is represented in

and

Take the minimum value among them, B ₁ and B ₂ are invalid boundary values, and R is the range value.

In some embodiments (eg, Table 3), the importance determination step (step S340 in FIG. 3 ) can be represented by a second importance obtaining formula, and the second importance obtaining equation is as follows:

Among them, I is the importance, O is the original value, B is the invalid boundary value, and R is the range value.

FIG. 4 is a schematic diagram of a deep learning model training system 100 according to some embodiments of the present application. Referring to FIG. 4 , in some embodiments, the deep learning model training system 100 can be equipped with a meter 400 and an image capture device 500 . The meter 400 is used to display the value, and the image capture device 500 is used to capture the original image from the meter 400, for example, to capture the "area of the meter 400 for displaying the value" or the "display screen of the meter 400" as the original image image. The deep learning model training system 100 then obtains the original images from the image capturing device 500, so the deep learning model training system 100 can obtain a set of original images (ie, a plurality of original images). In some embodiments, the image capture device 500 is connected to the deep learning model training system 100 through a wireless network. The meter 400 is, for example, but not limited to, mechanically rotating digital or electronically displaying values, and the meter 400 is, for example, but not limited to, used in an electric power transmission network or a water supply network.

In some embodiments, in addition to the processor 120 and the storage device 140, the deep learning model training system 100 can further include a meter 400 and an image capturing device 500 (the deep learning model is not shown in the figure). The training system 100 can further include a meter 400 and an image capture device 500).

FIG. 5 is a partial schematic diagram of a meter 400 according to some embodiments of the present application. Referring to FIG. 5 , in some embodiments, the area of the meter 400 for displaying values can display four decimal digits, that is, the value displayed by the meter 400 is between “0000 to 9999”. Taking FIG. 5 as an example, the value displayed on the meter 400 is "0011".

In some embodiments, the architecture of the deep learning model used by the deep learning model training system 100 and the deep learning model training method is, for example, but not limited to, a convolutional neural network (CNN), a recurrent neural network (Recurrent Neural Network, RNN), Deep Neural Network (DNN), or yolo (You Only Look Once). In other words, the deep learning model training system 100 and the deep learning model training method, for example, but not limited to, obtain the confidence level according to the above-listed architectures.

FIG. 6 is a schematic diagram of an object OB and a frame according to some embodiments of the present application. FIG. 7 is a schematic diagram of the intersection and ratio IOU according to some embodiments of the present application. 6 and 7, in some embodiments, the image IM input to the deep learning model has an object OB, and the deep learning model sets the first frame A1 and the second frame A2 according to the position of the object OB in the image IM . The deep learning model obtains the intersection area A3 and the union area A4 according to the first frame A1 and the second frame A2, wherein the intersection area A3 is the intersection area between the first frame A1 and the second frame A2, and the union area A4 is an area of union between the first frame A1 and the second frame A2. The deep learning model obtains the confidence of the image IM according to the image IM, the first frame A1 and the second frame A2, and the formula for obtaining the confidence by the deep learning model is, for example, but not limited to the following:

C=Pr(ob)*IOU, IOU=A3/A4.

Among them, C is the confidence level, Pr(ob) is the probability that the object OB belongs to a category (taking Figure 6 as an example, the category can be "0", so Pr(ob) is the probability that the object OB belongs to "0"). And the ratio of IOU is the intersection area A3 divided by the union area A4.

In some embodiments, specifically, the deep learning model divides the image IM into a plurality of array blocks (there are S*S array blocks in total), calculates the probability that the object OB in each array block belongs to a class, and The maximum value is obtained from the probability that the object OB belongs to a category in each array block, and the maximum value is used as the probability that the object OB belongs to a category. The first frame A1 is the reference standard block (ground truth box), that is, the block in which the deep learning model marks the standard answer of the object OB belonging to a category according to the result of the previous training. The second frame A2 is a candidate block (bounding box), that is, the deep learning model marks the image IM as a block that predicts whether the object OB belongs to a category according to the input image IM.

To sum up, the deep learning model training system, deep learning model training method and non-transitory computer-readable storage medium of some embodiments of this case can adjust the original image set according to the test value to obtain the test image set, and input the test image set to the The first deep learning model is tested to obtain the confidence of the test image set, that is, the confidence of the test value. And the test value can be classified into the valid group or the invalid group according to the threshold value, and then the training value can be obtained from the test value in the invalid group. and adjusting the original image set according to the training value to obtain a training image set, and inputting the original image set and the training image set to the first deep learning model for training to obtain a second deep learning model. In some embodiments, the deep learning model training system and method can use the training image set to strengthen the training of the first deep learning model without training all the test values corresponding to the confidence levels lower than the threshold, thus achieving The effect of reducing the amount of training data of the deep learning model, and still obtaining a second deep learning model with high confidence. In some embodiments, deep learning model training systems and methods can be adapted to identify images of areas of meters used to display values.

100: Deep Learning Model Training System 120: Processor 140: Storage Device 400: Meter 500: Image Capture Device T1: The first deep learning model T2: Second Deep Learning Model IM: Video OB:Object A1: The first frame A2: Second frame A3: Intersection area A4: Union area S210-S260: Steps S310-S340: Steps

FIG. 1 is a schematic diagram of a deep learning model training system according to some embodiments of the present application. FIG. 2 is a flowchart of a deep learning model training method according to some embodiments of the present application. FIG. 3 is a flow chart of the steps of obtaining the importance according to some embodiments of the present application. FIG. 4 is a schematic diagram of a deep learning model training system according to some embodiments of the present application. FIG. 5 is a partial schematic diagram of a meter according to some embodiments of the present application. FIG. 6 is a schematic diagram of objects and frames according to some embodiments of the present application. FIG. 7 is a schematic diagram of an intersection ratio according to some embodiments of the present application.

S210-S260: Steps

Claims (15)

A deep learning model training method, including: a step of obtaining a test image set, adjusting an original image set according to a plurality of test values to obtain a plurality of test image sets; a model testing step, inputting the test image sets to a first deep learning model for testing to obtain a plurality of confidence levels, the confidence levels corresponding to the test values in a one-to-one manner; a grouping step of classifying the test values into an invalid group or a valid group according to whether the confidence level corresponding to each of the test values is less than a threshold; a training value obtaining step, obtaining a plurality of training values according to the test values in the invalid group; a training image set obtaining step, adjusting the original image set according to the training values to obtain a plurality of training image sets; and In a model training step, the original image set and the training image sets are input to the first deep learning model for training to obtain a second deep learning model. The deep learning model training method according to claim 1, wherein the step of obtaining the training value further comprises: obtaining a valid interval value according to the difference between the largest test value in the valid set and an original value; and The training values are obtained from the test values in the invalid group according to a plurality of integer multiples of the valid interval value. The deep learning model training method according to claim 1, wherein the test image sets correspond to the test values in a one-to-one manner, and the training image sets correspond to the training values in a one-to-one manner. The deep learning model training method according to claim 1, wherein the test values are an arithmetic sequence. The deep learning model training method according to claim 1, wherein the test values include an original value corresponding to the original image set. The deep learning model training method according to claim 5, wherein the test image set corresponding to the original value is a part of the original image set. The deep learning model training method according to claim 1, further comprising an importance obtaining step, the importance obtaining step is used to obtain an importance corresponding to the test values, wherein the importance obtaining step includes: obtaining a range value according to the difference between the maximum test value and the minimum test value; classifying the test values into the invalid group or the valid group according to whether the confidence level corresponding to each of the test values is less than the threshold value; obtaining a null boundary value according to the smallest difference between the test values in the null set and an original value; and The importance is obtained by dividing the difference between the original value and the invalid boundary value by the range value. A deep learning model training system, including: a processor for receiving an original image set, a plurality of test values and a threshold, adjusting the original image set according to the test values to obtain a plurality of test image sets, and inputting the test image sets to a first depth The learning model is tested to obtain a plurality of confidence levels, the confidence levels correspond to the test values in a one-to-one manner, and the test values are classified into a an invalid group or a valid group, obtain a plurality of training values according to the test values in the invalid group, adjust the original image set according to the training values to obtain a plurality of training image sets, and input the original image set and the training the image set to the first deep learning model for training to obtain a second deep learning model; and a storage device for storing the first deep learning model and the second deep learning model. The deep learning model training system according to claim 8, wherein the processor obtains a valid interval value according to the difference between the largest test value in the valid set and an original value, and according to the plurality of valid interval values The training values are obtained in integer multiples from the test values in the null set. The deep learning model training system of claim 8, wherein the test image sets correspond to the test values in a one-to-one manner, and the training image sets correspond to the training values in a one-to-one manner. The deep learning model training system according to claim 8, wherein the test values are an arithmetic sequence. The deep learning model training system of claim 8, wherein the test values include an original value corresponding to the original image set. The deep learning model training system of claim 12, wherein the test image set corresponding to the original value is a part of the original image set. The deep learning model training system according to claim 8, wherein the processor obtains a range value according to the difference between the maximum test value and the minimum test value, and according to whether the confidence corresponding to each test value is less than The threshold classifies the test values into the invalid group or the valid group, obtains an invalid boundary value according to the smallest difference between the test values in the invalid group and an original value, and obtains an invalid boundary value according to the original value and the original value. The importance is obtained by dividing the difference between the invalid boundary values by the range value. A non-transitory computer-readable storage medium for storing one or more software programs, the one or more software programs including a plurality of instructions when the instructions are executed by one or more processing circuits of an electronic device , will make the electronic device perform a deep learning model training method, and the deep learning model training method includes: a step of obtaining a test image set, adjusting an original image set into a plurality of test image sets according to a plurality of test values; a model testing step, inputting the test image sets to a first deep learning model for testing to obtain a plurality of confidence levels, the confidence levels corresponding to the test values in a one-to-one manner; a grouping step of classifying the test values into an invalid group or a valid group according to whether the confidence level corresponding to each of the test values is less than a threshold; a training value obtaining step, obtaining a plurality of training values according to the test values in the invalid group; a training image set obtaining step, adjusting the original image set according to the training values to obtain a plurality of training image sets; and In a model training step, the original image set and the training image sets are input to the first deep learning model for training to obtain a second deep learning model.

Images