JP7168485B2 - LEARNING DATA GENERATION METHOD, LEARNING DATA GENERATION DEVICE, AND PROGRAM - Google Patents

Description

The present invention relates to a learning data generation method, a learning data generation device, and a program.

In order to build a model for image recognition by machine learning, it is necessary to prepare a large amount of still images of people, objects, and other targets for image recognition as training data.

In order to improve the accuracy of image recognition using such a model, it is necessary to learn the model using high-quality training data that includes a wide variety of still images with different shooting conditions such as shooting direction, distance, and tilt. It is preferable to

Therefore, techniques for efficiently generating such learning data have been developed (see, for example, Patent Document 1).

JP 2016-062524 A

On the other hand, learning data containing a large amount of still images is also generated by taking a picture of an object for image recognition with a video camera and extracting a still image of each frame from the video data.

However, in this case, the still images included in the learning data are captured every several tenths of a second. Therefore, the generated learning data has almost the same shooting conditions such as shooting direction and distance, and contains a large amount of redundant still images, which increases the amount of data and the time required for learning.

Conversely, if the images are captured in a short period of time in an attempt to reduce the amount of learning data, there is a possibility that sufficient still images required for model learning may not be obtained.

The present invention has been made in view of such problems, and provides a learning data generation method, a learning data generation device, and a program for efficiently generating good quality learning data for learning a model for image recognition. One of the purposes is to provide

A learning data generating method according to an embodiment of the present invention is a learning data generating method for performing learning of a model for image recognition, wherein a computer generates video data in which the target for image recognition is photographed. and obtaining a feature amount of a still image of each frame included in the moving image data, and for each combination of selecting two still images from the still images, the features of the two still images forming the combination The difference in quantity is obtained as an index value representing the degree of difference between the two still images, and the learning data is generated by removing redundant still images from the still images included in the moving image data based on the index value. Execute the processing to be performed.

In addition, the problems disclosed by the present application and their solutions will be clarified by the descriptions in the description of the mode for carrying out the invention, the descriptions in the drawings, and the like.

It is possible to efficiently generate good quality learning data for learning a model for image recognition.

1 is an overall configuration diagram of an information system; FIG. It is a figure which shows the hardware constitutions of a user terminal. It is a figure which shows the hardware constitutions of a learning data generation apparatus. FIG. 3 is a diagram showing a storage device; FIG. 4 is a diagram showing a neural network model; It is a figure which shows the outline|summary of learning data generation processing. It is a figure which shows the functional structure of a learning data generation apparatus. 4 is a flowchart showing the flow of learning data generation processing; 10 is a flowchart showing the flow of similar image deletion processing; It is a figure for demonstrating similar image deletion processing. 10 is a flowchart showing the flow of similar image deletion processing; It is a figure for demonstrating similar image deletion processing. It is a figure for demonstrating similar image deletion processing. 4 is a flowchart showing the flow of learning data generation processing; FIG. 4 is a diagram for explaining grouping of still images; FIG.

At least the following matters will become apparent from the description of the present specification and the accompanying drawings. DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will now be described in accordance with one embodiment thereof with reference to the accompanying drawings.

[First embodiment]
== Overall configuration ==
FIG. 1 shows an information system 1000 including a learning data generation device 200 and a user terminal 100 according to one embodiment of the present invention. The learning data generation device 200 and the user terminal 100 are communicably connected through a network 500 such as the Internet, a LAN (Local Area Network), or a telephone network.

The learning data generation device 200 is a server, personal computer, cloud computer, or the like that generates learning data for learning an image recognition model (hereinafter also referred to as an image recognition model 610 or a first neural network model 610). It is a computer or an information processing device.

The image recognition model 610 includes formulas such as discriminants or functions for determining from image data whether or not a subject appearing in an image is a specific recognition target. As the image recognition model 610 is trained, the coefficients of these equations are adjusted to change the accuracy of image recognition.

Although the details will be described later, as shown in FIG. 6, the learning data generation device 200 according to the present embodiment generates still images 621 of each frame included in moving image data 620 in which an image recognition target such as a person or an object is photographed. Learning data 640 for learning the image recognition model 610 is efficiently generated by removing the redundant still image 630 from the .

The redundant still images 630 are selected from similar still images 621 among the still images 621 included in the moving image data 620, for example. Although the details will be described later, in this embodiment, the still images 62 in the moving image data 620 are compared by comparing the feature amounts 650 of the respective still images 621 with each other.
1's similarity (degree of difference) to identify redundant still images 630 .

By generating learning data 640 using moving image data 620, the learning data generation device 200 can efficiently collect a large amount of image data for learning, and remove redundant still images 630 from the moving image data 620. , high-quality learning data 640 with high learning efficiency can be generated. Therefore, it is possible to efficiently construct the image recognition model 610 with high recognition accuracy.

Returning to FIG. 1, the user terminal 100 is a computer used by a user. It is a stationary type information processing apparatus such as.

When the user uses the user terminal 100 to transmit the video data 620 to the learning data generation device 200 , learning data 640 is transmitted from the learning data generation device 200 to the user terminal 100 . The user then uses the user terminal 100 or a computer (not shown) to learn the image recognition model 610 based on this learning data 640 .

A detailed description will be given below.

== user terminal ==
First, the user terminal 100 will be described with reference to FIG.

An example of a hardware configuration diagram of the user terminal 100 is shown in FIG. The user terminal 100 according to the present embodiment includes a CPU (Central Processing Unit) 110, a memory 120, a communication device 1
30 , a storage device 140 , an input device 150 , an output device 160 , and a recording medium reading device 170 .

The storage device 140 stores various data such as the user terminal control program 710 and video data 620 that are executed or processed by the user terminal 100 .

Various functions of the user terminal 100 are realized by reading the user terminal control program 710 and various data stored in the storage device 140 into the memory 120 and executing or processing them by the CPU 110 . For example, the user terminal 100 transmits the moving image data 620 to the learning data generation device 200 .

Here, the storage device 140 is a nonvolatile storage device such as a hard disk, SSD (Solid State Drive), flash memory, or the like.

The user terminal control program 710 is a general term for programs for realizing various functions of the user terminal 100 according to the present embodiment. Including various libraries.

The recording medium reading device 170 reads various programs and data recorded in a recording medium 800 such as an SD card and DVD, and stores them in the storage device 140 .

The communication device 130 exchanges various programs and data with the learning data generation device 200 and other computers (not shown) via the network 500 . For example, the above-described user terminal control program 710 and video data 620 can be stored in another computer, and the user terminal 100 can download the user terminal control program 710 and video data 620 from this computer.

The input device 150 is a device that receives commands and data input by the user, and includes various buttons, switches, a keyboard, a touch sensor that detects a touch position on a touch panel display, an input interface such as a microphone, an acceleration sensor, a temperature sensor, It includes a GPS receiver, a position detection sensor such as a compass, and a camera.

The output device 160 is, for example, a display device such as a display, a speaker, a vibrator, an output user interface such as lighting.

==Learning Data Generator==
The learning data generation device 200 is a computer that generates learning data 640 from video data 620 . As shown in FIG. 3, the learning data generation device 200 includes a CPU 210, a memory 220, a communication device 230, a storage device 240, an input device 250, an output device 260, and a recording medium reader 270. FIG. Although the hardware configuration of these learning data generating devices 200 is not necessarily the same as that of the user terminal 100, they have a common basic configuration. Therefore, redundant description of these hardware configurations will be omitted.

In the storage device 240 of the learning data generation device 200, as shown in FIG. Various programs and data such as 600 (hereinafter also referred to as a second neural network model 600), learning data 640, and feature amounts 650 (vector data 651) are stored.

Various functions of the learning data generation device 200 are performed by reading various data such as the learning data generation device control program 720 and the video data 620 stored in the storage device 240 into the memory 220 and being executed or processed by the CPU 210. is realized.

The moving image data 620 is data obtained by photographing an image recognition target. The specifications (standard, frame rate, resolution, screen size, etc.) of the video data 620 are not particularly limited in this embodiment, and the video data 620 may have any specifications. For example, the moving image data 620 may be data recording a moving image captured using the moving image capturing function of the user terminal 100, or data recording a moving image captured using a moving image capturing camera (not shown). It can be.

The neural network model 600 is used to acquire the feature quantity 650 of each still image 621 within the moving image data 620 . In this embodiment, as an example, the type of neural network model 600 is a CNN (Convolution Neural Network), and as shown in FIG. The neural network model 600 is designed to correctly identify redundant still images 630 based on the similarity (degree of difference) of each still image 621 in the moving image data 620, if the still images 621 are similar. A certain amount of learning is performed in advance so that the vector data 651 obtained by the calculation is output.

In this embodiment, the similarity of the still images 621 is determined by comparing the Euclidean distance of each vector data 651 of the two still images 621 to be compared with a threshold value A (predetermined determination value) described later. . Specifically, when the Euclidean distance of each vector data 651 of two still images 621 is equal to or less than the threshold value A, it is determined that these still images 621 are similar.

Further, the neural network model 600 may be another type of model such as RNN (Recurrent Neural Network) as long as it is a model capable of extracting vector data 651 (feature quantity 650) from the intermediate layer.

The vector data 651 may be output from any intermediate layer among the plurality of intermediate layers forming the neural network model 600. This is preferable because the feature to be recognized is more clearly quantified as the feature amount 650 .

As described above, the similarity of each still image 621 in the moving image data 620 is determined, and redundant still images 630 are removed to generate learning data 640, which is stored in the storage device 240 as shown in FIG. be.

<Functional configuration>
Next, FIG. 7 shows an example of a functional block diagram of the learning data generation device 200. As shown in FIG. A learning data generation device 200 according to this embodiment includes functions of a video data acquisition unit 201 and a learning data generation unit 202 .

Each of these functions is implemented by the hardware of the learning data generation device 200 executing the learning data generation device control program 720 according to the present embodiment.

The moving image data acquisition unit 201 acquires moving image data 620 in which a recognition target such as a person or an object to be recognized by the image recognition model 610 is photographed. Note that the moving image data acquisition unit 201 not only acquires the moving image data 620 from the user terminal 100, but also, if the moving image data 620 is stored in another computer (not shown), acquires this computer according to an instruction from the user terminal 100. The moving image data 620 may be acquired from.

Also, as described above, there are no particular restrictions on the specifications of the moving image data 620 such as the standard and the frame rate, and the moving image data acquisition unit 201 can acquire the moving image data 620 with various specifications.

The learning data generation unit 202 selects two still images 621 from among the still images 621 of each frame included in the moving image data 620. For each combination, an index representing the degree of difference (similarity) between these two still images 621 Learning data 640 is generated by removing redundant still images 630 from still images 621 included in moving image data 620 based on these index values.

With such an aspect, it is possible to efficiently generate good quality learning data 640 for learning the image recognition model 610 .

Note that when obtaining the index value, the learning data generation unit 202 obtains a feature amount 650 (vector data 651 in this embodiment) of each still image 621 included in the moving image data 620, and calculates a combination for each combination. A difference between the feature amounts 650 of the two still images 621 formed may be obtained as the index value.

In this manner, two still images 621 having a smaller difference in the feature amount 650 can be determined to have a smaller degree of difference. 630 can be identified. This makes it possible to more accurately identify the redundant still image 630 .

Note that, as the feature quantity 650 of the still image 621, in addition to the vector data 651 obtained by retrieving from the intermediate layer of the neural network model 600, the HOG feature quantity, the EOH feature quantity, the Haar-like feature quantity, the pixel difference feature quantity , or a combination thereof. It can be determined that two still images 621 having a smaller difference in the feature amount 650 have a smaller degree of difference with each other.

In addition, as described above, the learning data generation unit 202 inputs each still image 621 included in the moving image data 620 to the neural network model 600 such as CNN, and uses the output data from the intermediate layer in the neural network model 600 , vector data 651 is obtained as the feature quantity 650 of the still image 621. At this time, if the image recognition model 610 and the neural network model 600 are the same type of neural network (CNN in this embodiment), Since the characteristics of the image recognition model 610 and the characteristics of the neural network model 600 are common, it is possible to generate learning data 640 that matches the characteristics of the image recognition model 610 . This makes it possible to learn the image recognition model 610 more efficiently.

For example, in a CNN, when two still images are input in which the same object appears in a position that has been translated in the screen, the feature values (vector data) of these still images obtained from the intermediate layer are almost the same. Since these objects have the characteristics, they can be correctly recognized as the same object. Since it has the characteristic that the difference in the feature amount (vector data) tends to be large, it is easy to mistakenly recognize that it is a different object.

Therefore, even if two still images 621 with substantially the same vector data 651 output from the neural network model 600 are left in the training data 640, the image recognition model 610 cannot correctly recognize the subject from any of the still images 621. It is preferable to remove at least one of these still images 621 as a redundant still image 630 because it does not contribute much to learning.

Conversely, two still images 621 with a large difference in vector data 651 output from the neural network model 600 (even though the subjects are the same, they are likely to be erroneously recognized as non-identical) are left in the learning data 640. This is preferable because it allows the image recognition model 610 to learn that it is the same subject.

Note that the learning data generation unit 602 detects the difference between the two still images 621 in the combination of one still image 621 and the other still image 621 among the combinations of the two still images 621 in the moving image data 620. The moving image data 620 includes a process of removing the one still image 621 as a redundant still image 630 when there is a combination whose index value representing the degree (similarity) is equal to or less than a predetermined judgment value (threshold A). It is preferable to generate the learning data 640 by repeating each of the still images 621 in order as the one still image 621 .

For example, as shown in FIG. 12, there are five still images (A, B, C, D, E) 621, and an index representing the degree of difference between two still images 621 selected from these still images 621. If the values are the values shown in FIG. 12 (for example, the index value of still images A and B is 70), first the still image A 621 is taken as one still image 621, and then the other still images (B, C, D, E) Determine whether or not there is a combination with 621 whose index value is equal to or less than a determination value (for example, 100). In the example shown in FIG. 12, the still image A621 and the still image B621 have an index value of 70 (100 or less), so the still image A621 is removed.

Next, whether or not there is a combination of the still image B 621 as one still image 621 and the other still images (C, D, E) 621 whose index value is equal to or less than the judgment value (100) judge. Since the index value of still image B621 and still image C621 is 60, still image B621 is also removed.

Thereafter, the same processing is performed with the still images C, D, and E 621 as one still image 621 in order. As a result, the still image C621 is removed, but the still images D and E621 remain without being removed. Therefore, the learning data generation unit 602 generates learning data 640 composed of still images D and E621.

This aspect makes it possible to generate learning data 640 that does not include redundant still images 630 .

Returning to FIG. 7, the learning data generation unit 602 identifies the first combination with the smallest index value representing the degree of difference from each combination of the two still images 621 in the moving image data 620, and then Furthermore, a second combination with the smallest index value is specified among other combinations including one of the two still images 621 forming the first combination, and the first combination and the second combination are identified. The learning data 640 may be generated by repeatedly performing the process of removing the common still image 621 as the redundant still image 630 until there is no combination in which the index value is equal to or less than a predetermined determination value.

Again referring to the example of FIG. 12, the learning data generation unit 602 first identifies the combination (index value 40) of the still image B621 and the still image E621 as the combination (first combination) with the smallest index value. . This combination is the combination indicated by “α” in the list of index values for each combination of still images 621 shown in FIG. 13(a).

At this time, the learning data generation unit 602 identifies a combination (second combination) with the smallest index value among other combinations including the still image B621 and other combinations including the still image E621. . In the example shown in FIG. 12, among the other combinations including the still image B621, the combination of the still image B621 and the still image C621 has the smallest index value (index value 60), and the combination including the still image E621 has the lowest index value. The combination with the smallest index value is the combination of the still image E621 and the still image C621 (index value 90). Therefore, the learning data generation unit 602 identifies the combination of the still image B621 and the still image C621 with the smallest index value as the second combination. This combination is the combination indicated by "β" in the list of index values for each combination of still images 621 shown in FIG. 13(a).

Then, the learning data generation unit 602 removes the still image B621 common to the first combination (α) and the second combination (β) as a redundant still image 630 in FIG. 13(a).

Subsequently, as shown in FIG. 13B, the learning data generation unit 602 selects the still image C621 and A combination of still images D621 (index value 50) is specified. The learning data generation unit 602 then identifies the combination (index value 90) of the still image C621 and the still image E621 as the second combination.

The learning data generation unit 602 then removes the still image C621 common to the first combination (α) and the second combination (β) as a redundant still image 630 .

Here, as shown in FIG. 13C, in each combination from which the still image B621 and the still image C621 are removed, all index values are greater than the judgment value (100).

Therefore, the learning data generation unit 602 generates learning data 640 composed of still images A, D, and E 621 .

In this way, by preferentially deleting the still image 621 with the smallest degree of difference between the still images 621 in the video data 620, it is possible to more appropriately generate the learning data 640 that does not include the redundant still images 630. can be done.

== Process flow ==
Next, the flow of processing by the information system 1000 according to this embodiment will be described with reference to FIGS. 8 to 15. FIG.

First, the learning data generation device 200 acquires the moving image data 620 in which the target of image recognition is captured (S1000). The learning data generation device 200 may acquire the video data 620 from the user terminal 100 or may acquire it from another computer (not shown) according to an instruction from the user terminal 100 .

The learning data generation device 200 then extracts the still image 621 of each frame from the moving image data 620 (S1010). For example, the learning data generation device 200 extracts 9000 (30×60×5) still images 621 from five-minute video data 620 with a frame rate of 30 fps.

Next, the learning data generating device 200 inputs each still image 621 to the neural network model 600 to obtain respective vector data 651 (S1020).

Then, the learning data generation device 200 obtains the threshold A (predetermined determination value described above) (S1030). The threshold A is two still images 621 selected from the still images 621 in the moving image data 620.
is a judgment value when judging the similarity (degree of difference) between . In this embodiment, when the norm of the difference between the vector data 651 of the two still images 621 (for example, the Euclidean distance of the vector data 651) is equal to or less than the threshold A, the two still images 621 are similar. is determined to be

Note that the threshold A is preferably determined based on the vector data 651 of each still image 621 . For example, the learning data generation device 200 may obtain the average value of the L2 norm of each vector data 651 as the threshold value A. FIG. One of the reasons for this is that the better the neural network model 600 can recognize the recognition target in the still image 621, the larger the magnitude (L2 norm) of the vector data 651, that is, the feature quantity 650.

In other words, for example, if the neural network model 600 is properly trained, the vector data 651 when the object X to be recognized appears in the still image 621 is the vector data 651 when the object X does not appear in the still image 621. This is because the value should be larger than the vector data 651 (learning is performed to achieve this).

Note that if the value of the threshold A is increased, the number of still images 621 that are judged to be similar increases. Data volume is reduced. Conversely, if the value of the threshold A is decreased, the number of still images 621 that are judged to be similar decreases. data volume increases.

Therefore, it is more preferable to adjust the threshold A according to the number of still images included in the learning data 640 or the number of still images 621 to be removed as redundant still images 630 . Such an aspect makes it possible to appropriately adjust the data size of the learning data 640 .

Next, the learning data generation device 200 executes similar image deletion processing (S1040). As a result, the redundant still image 630 can be removed from the moving image data 620. FIG.

As shown in FIG. 10, in the similar image deletion process, one still image 621 (still image 6 denoted by i in FIG.
21) and other still images 621 (still images 621 denoted by symbols j, j+1, . is equal to or less than a predetermined judgment value (threshold value A), the process of removing the one still image 621 as a redundant still image 630 is performed by sequentially removing each of the still images 621 included in the moving image data 620 from the one This is a process for generating learning data 640 by repeating the still image 621 .

The flow of similar image deletion processing will be described with reference to the flowchart of FIG. 9. First, the learning data generation device 200 sets i=1 and j=i+1 as control variables (S2000, S2010). A control variable i indicates one still image 621 and a control variable j indicates another still image 621 .

Next, the learning data generation device 200 calculates the norm of the difference between the vector data 651 of the i-th still image 621 and the j-th still image 621 (S2020). Specifically, the Euclidean distance of each vector data 651 is calculated.

If these norms (Euclidean distance) are equal to or less than the threshold A (YES in S2030), the learning data generation device 200 determines that the i-th still image 621 is similar to the j-th still image 621, The i-th still image 621 is deleted (S2060).

On the other hand, if these norms are not equal to or less than the threshold A (NO in S2030), the learning data generation device 200 adds 1 to the control variable j (S2040), the i-th still image 621 and the next j-th still image Similar processing is performed with the image 621 (S2020, S2030).

However, as a result of adding 1 to the control variable j in S2040, if j exceeds MAX, the comparison with all the still images 621 is completed, so the learning data generation device 200 adds 1 to i ( S2070).

Then, the learning data generation device 200 compares the Euclidean distance of the i-th still image 621 and the j-th still image 621 with the threshold A until i exceeds MAX (S2080). The processing of deleting the i-th still image 621 when it is smaller than is repeated.

Returning to FIG. 8, the learning data generation device 200 thus removes the redundant still image 630 from the video data 620 to generate the learning data 640 (S1050).

With such an aspect, the learning data 640 for learning the image recognition model 610 can be efficiently generated.

Note that the learning data generation device 200 can also perform similar image deletion processing in accordance with the procedure shown in the flowchart of FIG. 11 .

In this case, the learning data generation device 200 selects the first combination (indicated by α in FIG. combination) is specified, and then a second combination (shown in FIG. 13 β), and remove the still image 621 common to the first combination (α) and the second combination (β) as a redundant still image 630. Repeat until there are no more combinations.

In FIG. 11, the learning data generation device 200 first determines the norm of the difference (for example, each vector Euclidean distance of data 651) is calculated (S3000).

If there is no combination in which the norm is equal to or less than threshold A (NO in S3000), learning data generation device 200 ends the process, advances to S1050 in FIG. 8, and outputs learning data.

On the other hand, if there is a combination whose norm is equal to or less than the threshold A in S3010, the learning data generation device 200 identifies the combination with the smallest norm among those combinations (the combination indicated by α in the above example). (S3020).

Next, the learning data generation device 200 selects the combination (the combination indicated by β described above) with the smallest index value among the other combinations including one of the two still images 621 forming the combination (α). is identified (S3030).

The learning data generation device 200 then deletes the still images 621 common to these combinations (α, β) as redundant still images 630 (S3040).

After that, learning data generation device 200 repeats the processing of S3020 to S3040 until there is no combination whose norm is equal to or less than threshold A (NO in S3000).

With such an aspect, the learning data generation device 200 can more appropriately generate the learning data 640 that does not include the redundant still image 630 .

[Second embodiment]
Note that the learning data generation device 200 may perform processing in a mode as shown in FIGS. 14 and 15. FIG.

In this embodiment, the learning data generation device 200 divides the still images 621 included in the moving image data 620 into a plurality of groups in chronological order, and performs the process of removing the redundant still images 630 described in the first embodiment for each group. conduct. FIG. 15 shows how still images 621 included in moving image data 620 are divided into N groups.

Then, the learning data generation device 200 generates intermediate data by performing a process of removing redundant still images 630 on a group-by-group basis. 630 is removed. The learning data generation device 200 generates the learning data 640 in this manner.

With this aspect, the number of combinations for selecting two still images 621 from the moving image data 620 can be reduced, so the processing time for generating the learning data 640 can be shortened.

In addition, the still images 621 in each group are likely to be similar because the timings at which they were shot are close to each other. Therefore, by once determining the similarity of each still image 621 within a group as in the present embodiment, redundant still images 630 can be removed efficiently.

The flow of processing according to this embodiment will be described along the flowchart of FIG. 14 .

First, learning data generation device 200 acquires video data 620 in which an object for image recognition is captured (S4000).

The learning data generation device 200 then extracts the still images 621 of each frame from the video data 620 (S4010), inputs each still image 621 to the neural network model 600, and obtains vector data 651 (S4020). The learning data generation device 200 then obtains a threshold A (S4030). The above processing is the same as in the first embodiment.

The learning data generation device 200 divides each still image 621 into N groups in chronological order (S4040).

Then, the learning data generation device 200 generates intermediate data by performing similar image deletion processing (processing for removing redundant still images 630) on a group-by-group basis (S4050).

The learning data generation device 200 further performs similar image deletion processing on the entire intermediate data (S4060).

The learning data generation device 200 then generates learning data 640 (S4070). The learning data generation device 200 then transmits the learning data 640 to the user terminal 100 .

Such an aspect makes it possible to further shorten the processing time for generating the learning data 640 .

The method of generating learning data 640, the learning data generation device 200, and the program have been described above. not from The present invention may be modified and improved without departing from its spirit, and the present invention also includes equivalents thereof.

For example, in the above embodiment, the learning data generation device 200 generates the learning data 640 and then transmits the learning data 640 to the user terminal 100. The learning data 640 may be transmitted. With this aspect, the user terminal 100 can save the trouble of transmitting the learning data 640 to a computer (not shown) storing the image recognition model 610, and the work efficiency when learning the image recognition model 610 can be reduced. can be improved.

Alternatively, the learning data generating device 200 may store the image recognition model 610 and the learning data generating device 200 may learn the image recognition model 610 by itself. This aspect eliminates the need for the learning data generation device 200 to transmit the learning data 640 to the user terminal 100 or another computer, thereby further improving work efficiency when learning the image recognition model 610. becomes possible.

Further, in the above embodiment, the case where each still image 621 in the moving image data 620 is input to the neural network model 600 prepared separately from the image recognition model 610 to obtain the vector data 651 has been described. Each still image 621 of the moving image data 620 may be input to the image recognition model 610 without using , and the vector data 651 may be acquired from the intermediate layer of the image recognition model 610 .

In this case, the vector data 651 is obtained using the image recognition model 610 whose learning has not been completed, and the redundant still image 630 is removed from the video data 620 using this vector data 651. In many cases, the image recognition model 610 used is distributed in a trained state to the extent that image recognition of general objects is possible with a certain degree of accuracy. , the vector data 651 can be obtained without using the neural network model 600 . With such an aspect, it is possible to reduce the burden on the operator when creating the learning data 640 because it saves the trouble of various setting work and the like that are necessary when using the neural network model 600 separately. .

In the learning data generation method according to the present embodiment, in the process of generating the learning data, the computer obtains the feature amount of each still image included in the moving image data, and determines the combination for each of the combinations. A difference between the feature amounts of the two still images may be obtained as the index value.

According to this, redundant still images can be specified more accurately.

Further, in the method of generating learning data according to the present embodiment, the model for image recognition is a first neural network model, and the computer generates the learning data in the process of generating the first neural network model. Each still image included in the moving image data is input to a second neural network model of the same type as and the feature amount is obtained using the output data from the intermediate layer in the second neural network model. good.

According to this, it is possible to obtain learning data that matches the characteristics of the model.

Further, in the method of generating learning data according to the present embodiment, in the process of generating the learning data, the computer determines that the index value is a predetermined value in the combination of one still image and another still image. When there is a combination that is equal to or less than the value, the processing of removing the one still image as the redundant still image is repeatedly performed as the one still image, and the learning is performed by repeatedly performing each still image included in the moving image data in order. may generate data.

According to this, it is possible to generate learning data that does not include redundant still images.

Further, in the method of generating learning data according to the present embodiment, in the process of generating the learning data, the computer identifies a first combination with the smallest index value from among the combinations, and further , identifying a second combination having the smallest index value among other combinations including one of the two still images forming the first combination, and determining the first combination and the second combination The learning data may be generated by repeatedly performing the processing of removing the still images common to the above as the redundant still images until there is no combination in which the index value is equal to or less than a predetermined judgment value.

According to this, it is possible to more appropriately generate learning data that does not include redundant still images.

Further, in the method of generating learning data according to the present embodiment, in the process of generating the learning data, the computer divides the still images included in the moving image data into a plurality of groups in chronological order, and for each group, the After intermediate data is generated by performing a process of removing redundant still images, the learning data may be generated by further performing a process of removing the redundant still images on the entire intermediate data. .

According to this, it becomes possible to generate learning data in a shorter time.

100 User terminal 110 CPU
120 memory 130 communication device 140 storage device 150 input device 160 output device 170 recording medium reading device 200 learning data generation device 201 video data acquisition unit 202 learning data generation unit 210 CPU
220 memory 230 communication device 240 storage device 250 input device 260 output device 270 recording medium reader 500 network 600 neural network model 610 image recognition model 620 video data 621 still image 630 redundant still image 640 learning data 650 feature quantity 651 vector data 710 User terminal control program 720 Learning data generation device control program 800 Recording medium 1000 Information system

Claims (7)

A method of generating learning data for learning a model for image recognition, comprising:
the computer
a process of acquiring moving image data in which the target of image recognition is captured;
A feature amount of a still image of each frame included in the moving image data is obtained, and for each combination in which two still images are selected from the still images, the difference in the feature amount of the two still images forming the combination is calculated as the 2. A process of generating the learning data by obtaining an index value representing the degree of difference between the two still images, and removing redundant still images from the still images included in the moving image data based on the index value;
A method of generating training data that runs The learning data generation method according to claim 1 ,
The model for image recognition is a first neural network model,
In the process of generating the learning data, the computer inputs each still image included in the moving image data to a second neural network model of the same type as the first neural network model, and the second neural network A learning data generation method for determining the feature amount using output data from an intermediate layer in a model. The learning data generation method according to claim 1,
In the process of generating the learning data, if the combination of one still image and another still image includes a combination in which the index value is equal to or less than a predetermined judgment value, the one A method of generating learning data, wherein the learning data is generated by repeatedly performing a process of removing still images as the redundant still images from each of the still images included in the moving image data in turn as the one still image. The learning data generation method according to claim 1,
In the process of generating the learning data, the computer identifies, from among the combinations, a first combination with the smallest index value, and then further selects two still images that form the first combination. A second combination having the smallest index value among other combinations including one of the above is specified, and a still image common to the first combination and the second combination is defined as the redundant still image. A method of generating learning data, wherein the learning data is generated by repeatedly performing the removing process until there is no combination in which the index value is equal to or less than a predetermined judgment value. The learning data generation method according to claim 1,
In the process of generating the learning data, the computer divides the still images included in the moving image data into a plurality of groups in chronological order, and removes the redundant still images from each group to obtain intermediate data. is generated, the learning data is generated by further performing a process of removing the redundant still image from the entire intermediate data. A learning data generation device for generating learning data for learning a model for image recognition,
a moving image data acquisition unit that acquires moving image data in which the image recognition target is captured;
A feature amount of a still image of each frame included in the moving image data is obtained, and for each combination in which two still images are selected from the still images, the difference in the feature amount of the two still images forming the combination is calculated as the 2. a learning data generation unit that generates the learning data by obtaining an index value representing the degree of difference between the two still images, and removing redundant still images from the still images included in the moving image data based on the index value; ,
A learning data generation device comprising: A computer that generates training data for training a model for image recognition,
a procedure for acquiring moving image data in which the target for image recognition is captured;
A feature amount of a still image of each frame included in the moving image data is obtained, and for each combination in which two still images are selected from the still images, the difference in the feature amount of the two still images forming the combination is calculated as the 2. a step of obtaining an index value representing the degree of difference between two still images, and removing redundant still images from the still images included in the moving image data based on the index value to generate the learning data;
program to run the

Images