JP2023029006A - Information processing device, generation method of teacher data, generation method of learnt model and program - Google Patents

Abstract

To generate teacher data including a label indicating a position and a range of a detection object correctly.SOLUTION: An information processing device (1) comprises: a teacher image generation unit (101) that generates a teacher image by applying conversion that enlarges or shrinks an original image on which a detection object is photographed in a height direction and a width direction; a coordinate conversion unit (102) that converts a coordinate of each apex indicating a position and a range of the detection object photographed on the original image into a coordinate of each apex indicating a position and a range of the detection object photographed on the teacher image based on the magnification of the conversion; and a teacher data generation unit (103) that generates a label indicating the position and the range of the detection object photographed on the teacher image based on the converted coordinate to make it associated with the teacher image to be teacher data (112).SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing apparatus and the like that generate teacher data used for machine learning.

Conventionally, machine learning using teacher data has been performed. Machine learning generally requires a large amount of teacher data, but sometimes it is difficult to obtain a large amount of teacher data. As a technique for solving such a problem, a technique is known in which new teacher data is generated based on existing teacher data to increase the total number of teacher data.

For example, in Patent Document 1 below, a teacher image is classified into a plurality of patterns, a pattern with a small number is specified, and by spatially inverting the teacher image or changing the color tone, it is possible to solve the missing pattern. A machine learning system is disclosed that generates a new training image to belong to.

JP 2018-169672

In addition, in machine learning to generate a trained model that detects a detection target in an image, training data that associates a label indicating the position and range of the detection target with a teacher image in which the detection target is captured is used as training data. and In general, representative coordinates of a rectangular area surrounding a detection target (for example, coordinates of the center position of the rectangular area) and the width and height of the rectangular area are associated with the teacher image as labels. Note that width and height can also be read as horizontal length and vertical length.

FIG. 4 is a diagram showing a conventional technique, showing an example of label setting indicating the position and range of a detection target in a teacher image. A detection target OB1 is shown in the teacher image IMG1 shown on the left side of FIG. 4, and a rectangular area R1 surrounding the detection target OB1 is set. The representative coordinates, width and height of this rectangular area R1 are associated with the teacher image IMG1 as a label.

Here, as in the teacher image IMG1 in FIG. 4, when the detection target OB1 is a rod-shaped object and the detection target OB1 is photographed at an angle, the background region in which the detection target OB1 is not shown in the rectangular region R1 is The ratio of (upper right area and lower left area of rectangular area R1) is high. If learning is performed using such labels, the influence of the background area of the detection target OB1 becomes large, which is not preferable. For example, in the teacher image IMG1 of FIG. 4, part of a plate-like object is reflected in the rectangular region R1, which may affect the learning result.

Therefore, it is conceivable to set an inclined rectangular region R2 like the teacher image IMG2 on the right side of FIG. As a result, the ratio of the background area can be greatly reduced compared to the rectangular area R1. In addition to the representative coordinates, width, and height of the rectangular region R2, the teacher image IMG2 may be associated with the tilt angle α as a label.

However, when the rectangular area is tilted, the aspect ratio (ratio of height and width; synonymous with aspect ratio) of the original image is changed in order to increase the variation of the teacher image. There is a problem that the position and range of the detection target indicated by the label in the teacher image may deviate from the position and range of the detection target shown in the teacher image.

This will be described with reference to FIG. FIG. 5 is a diagram for explaining problems of the conventional technology. In the example of FIG. 5, a cut-out area indicated by a dashed line is cut out from an image IMG3 in which the detection target OB1 is captured, as with the teacher images IMG1 and IMG2 described above. Then, the clipped region portion is used as the original image IMG3', and the teacher image IMG4 is generated by adjusting the size of the original image IMG3'.

More specifically, the size of the image IMG3 is 608 pixels in width and height, and a rectangular area R3 inclined at an angle α is set in the object OB1 shown in the image IMG3 as in the case of the IMG2 in FIG. ing. Also, the size of the original image IMG3' cut out from the image IMG3 is 566 pixels wide and 383 pixels high. Although the lower end of the original image IMG3' protrudes from the image IMG3, such a clipping method is also possible in generating the teacher image.

Then, the original image IMG3' is enlarged/reduced to become a teacher image IMG4 having a width and height of 544 pixels. Specifically, the teacher image IMG4 is generated by multiplying the original image IMG3' by (544/566) in the width direction and by (544/383) in the height direction. Since the enlargement ratios in the width direction and height direction are different, the original image IMG3' and the teacher image IMG4 have different aspect ratios. Changing the aspect ratio itself is preferable from the viewpoint of increasing variations of the teacher image.

In IMG4, a rectangular area R4 indicating the position and range of the detection target OB1 is obtained by multiplying the width of the rectangular area R2 by (544/566) and by multiplying the height by (544/383). In this way, when the rectangular region R4 is set by enlarging/reducing the width and height of the rectangular region R3 in the same manner as the original image IMG3′, a portion of the detection target OB1 extends from the rectangular region R4 as shown in FIG. It may stick out.

This is because the tilt angle of the detection target OB1 changes due to the change in the aspect ratio of the original image IMG3′, whereas the tilt angle α of the rectangular region R4 is the same as the tilt angle of the rectangular region R3 of the original image IMG3′. because it is the same as

In this way, when the rectangular area set in the original image is slanted and the teacher image is generated with an aspect ratio different from that of the original image, the label associated with the teacher image is detected. The problem arises that it does not correctly indicate the position and extent of the object.

An object of one aspect of the present invention is to realize an information processing apparatus or the like capable of generating teacher data including a label that correctly indicates the position and range of a detection target in a teacher image even in the case described above. do.

In order to solve the above-described problems, an information processing apparatus according to an aspect of the present invention converts an original image in which a detection target is shown so as to expand or contract in at least one of the height direction and the width direction. a teacher image generating unit that generates an image; and based on the magnification of the conversion, coordinates of each vertex of a rectangular area indicating the position and range of the detection target appearing in the original image are converted to the detection target appearing in the teacher image. A coordinate transformation unit that transforms the coordinates of each vertex of a rectangular area indicating the position and range, and a label that indicates the position and range of the detection target appearing in the teacher image based on the coordinates after conversion by the coordinate conversion unit. and a teacher data generation unit that generates teacher data in association with the teacher image.

In order to solve the above-described problems, a teaching data generating method according to an aspect of the present invention is a teaching data generating method executed by one or a plurality of information processing apparatuses, wherein an original image in which a detection target is captured is , a training image generation step of generating a training image by applying a transformation that enlarges or shrinks in at least one of the height direction and the width direction, and based on the magnification of the transformation, the position and a coordinate transformation step of transforming the coordinates of each vertex of a rectangular area indicating a range into the coordinates of each vertex of a rectangular area indicating the position and range of the detection target in the teacher image; and a teacher data generating step of generating a label indicating the position and range of the detection target appearing in the teacher image based on the coordinates and associating the label with the teacher image as teacher data.

In order to solve the above problems, a method for generating a trained model according to an aspect of the present invention is a method for generating a trained model executed by one or more information processing apparatuses, the method comprising: a teacher image generating step of generating a teacher image by applying a transformation that enlarges or shrinks in at least one of the height direction and the width direction to the target image; a coordinate conversion step of converting the coordinates of each vertex of a rectangular area indicating the position and range into the coordinates of each vertex of the rectangular area indicating the position and range of the detection target appearing in the teacher image; and after conversion by the coordinate conversion step. a teacher data generation step of generating a label indicating the position and range of the detection target appearing in the teacher image based on the coordinates of the teacher image and using the label as teacher data in association with the teacher image; and a learning step of generating a learned model for detecting the detection target from the image by machine learning using the teacher data.

According to one aspect of the present invention, even when a teacher image having an aspect ratio different from that of the original image is generated from an original image in which a slanted rectangular area is set, the position and range of the detection target in the teacher image can be correctly set. Teacher data can be generated that includes labels that indicate

1 is a block diagram showing an example of the main configuration of an information processing apparatus according to an embodiment of the present invention; FIG. It is a figure which shows the generation example of the teacher image and label by the said information processing apparatus. It is a flow chart which shows an example of processing which the above-mentioned information processor performs. FIG. 10 is a diagram showing a conventional technique, and is a diagram showing a setting example of labels indicating the position and range of a detection target in a teacher image. It is a figure explaining the problem of a prior art.

〔Device configuration〕
A configuration of an information processing apparatus 1 according to one embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing an example of the main configuration of an information processing apparatus 1. As shown in FIG. The information processing device 1 may be, for example, a personal computer, a server, or a workstation.

As shown in the figure, the information processing apparatus 1 includes a control section 10 that centrally controls each section of the information processing apparatus 1 and a storage section 11 that stores various data used by the information processing apparatus 1 . The information processing apparatus 1 also includes a communication unit 12 for communicating between the information processing apparatus 1 and other apparatuses, an input unit 13 for receiving input of various data to the information processing apparatus 1, and various data output by the information processing apparatus 1. An output unit 14 is provided for doing so.

The control unit 10 also includes a teacher image generator 101 , a coordinate transform unit 102 , a teacher data generator 103 , a learning unit 104 and an inference unit 105 . An image 111, teacher data 112, and a trained model 113 are stored in the storage unit 11. FIG.

An image 111 is an image showing a detection target. The image 111 is associated with a label indicating the position and range of the detection target appearing in the image 111 . This label may indicate, for example, representative coordinates, width, height, and tilt angle of a rectangular area surrounding the detection target. Note that width and height can also be read as horizontal length and vertical length. When generating a trained model that not only detects but also identifies a detection target, the identifier of the detection target may be added to the label in addition to the information described above.

The image 111 may be input via the input unit 13 or received via the communication unit 12 . For example, the input unit 13 may be a USB (Universal Serial Bus) interface. In this case, the input unit 13 and a photographing device that photographs the detection target may be connected via a USB interface. Then, the information processing device 1 may receive the image captured by the image capturing device via the input unit 13 and store it as the image 111 . Also, an image captured by the imaging device is transmitted via a LAN (Local-Area Network), a wireless LAN, or the like, and the information processing device 1 receives the image via the communication unit 12 and stores it as the image 111. good too.

The teacher image generation unit 101 generates a teacher image by performing a conversion to enlarge or reduce the original image cut out from the image 111 showing the detection target in at least one of the height direction and the width direction. Note that the teacher image may be generated using the image 111 as the original image. Further, the details of the method of generating the teacher image will be described later with reference to FIG.

The coordinate transformation unit 102 converts the coordinates of each vertex of a rectangular area representing the position and range of the detection target in the original image based on the transformation magnification of the teacher image (magnification for enlargement/reduction in the height direction and width direction). are transformed into the coordinates of the vertices of a quadrangular area indicating the position and range of the detection target in the teacher image. The details of the coordinate conversion method will be described later with reference to FIG.

The teacher data generation unit 103 generates a label indicating the position and range of the detection target appearing in the teacher image generated by the teacher image generation unit 101 based on the coordinates after conversion by the coordinate transformation unit 102, and associates the label with the teacher image. be training data. Then, the teacher data generation unit 103 stores the generated teacher data as the teacher data 112 in the storage unit 11 . The details of the label generation method will be described later with reference to FIG.

The learning unit 104 generates a trained model for detecting a detection target from an image by machine learning using the training data 112 generated by the training data generation unit 103 . Then, the learning unit 104 stores the generated trained model as a trained model 113 in the storage unit 11 . The model may be, for example, a neural network model (including a deep neural network model), or the like. In addition, learning calculates the error between the inference result obtained by inputting the teacher image contained in the teacher data into the model under learning and the correct answer shown in the teacher data, and updates the model by error back propagation. This process may be performed repeatedly while changing the teacher data (error backpropagation method).

The inference unit 105 uses the trained model 113 to detect detection targets from the image. More specifically, the inference unit 105 first reads the target image. The image may be stored in the storage unit 11 in advance, or may be input from a photographing device or the like via the communication unit 12 or the input unit 13 . Then, the inference unit 105 inputs the read image to the trained model 113, and causes the output unit 14 to output the detection results whose accuracy is equal to or higher than the threshold among the obtained detection results.

The trained model 113 is generated by learning using the teacher data 112 including labels that correctly indicate the position and range of the detection target. , high-precision object detection is possible. Note that the detection result indicates the position and range of the detection target appearing in the image. As the detection result, the representative coordinates, width, height, and tilt angle of the area surrounding the detection target may be output, or a rectangle indicating the area may be drawn on the target image. .

As described above, the information processing apparatus 1 includes the teacher image generation unit 101 that generates a teacher image by performing a transformation that enlarges or reduces an original image in which a detection target is captured in at least one of the height direction and the width direction. , based on the conversion magnification, the coordinates of each vertex of a rectangular area showing the position and range of the detection target in the original image are converted to the coordinates of each vertex of a rectangular area showing the position and range of the detection target in the teacher image. A coordinate conversion unit 102 for conversion and a label indicating the position and range of a detection target appearing in the teacher image are generated based on the coordinates after conversion by the coordinate conversion unit 102, and are associated with the teacher image to form teacher data 112. and a generation unit 103 .

According to the above configuration, the original image in which the detection target is captured is subjected to a transformation that enlarges or reduces it in at least one of the height direction and the width direction, thereby generating the teacher image. As a result, it is possible to generate a wide variety of teacher images, including those with different aspect ratios (ratio of height and width; synonymous with aspect ratio) from the original image.

Further, according to the above configuration, the labels associated with the generated teacher image are the four vertices of the rectangular area obtained by transforming the coordinates of the four vertices of the rectangular area of the original image based on the magnification of the transformation applied when the teacher image was generated. Generate based on coordinates. Then, the teacher data is obtained by associating the label with the teacher image.

The coordinates of the four vertices of the quadrilateral area are obtained based on the magnification of the conversion applied when generating the teacher image, so they correctly indicate the position and range of the detection target appearing in the teacher image. As a result, it is possible to generate teacher data 112 associated with labels that correctly indicate the position and range of the detection target appearing in the teacher image.

As described above, according to the above configuration, even when the rectangular area set in the original image is slanted and the teacher image is generated with an aspect ratio different from that of the original image, the The effect is that it is possible to generate teacher data 112 including labels that correctly indicate the position and range of the detection target.

The information processing apparatus 1 also includes a learning unit 104 that generates a trained model 113 for detecting a detection target from an image by machine learning using the teacher data 112 generated by the teacher data generation unit 103 . With this configuration, the learned model 113 can be automatically generated.

[Generation example of teacher image and label]
FIG. 2 is a diagram showing an example of generation of teacher images and labels by the information processing apparatus 1. As shown in FIG. In FIG. 2, an image IMG5 with a width and height of 608 pixels is cut out from an image IMG5 whose width and height are both 608 pixels, and an original image IMG5′ is obtained by cutting out the cutout region indicated by the dashed line. A pixel teacher image IMG6 is used. Note that the width and height of the teacher image IMG6 may be determined by using random numbers or the like. Alternatively, a teacher image may be generated using an original image obtained by cutting out a portion of the detection target OB1 instead of the entire detection target OB1. By performing learning using such a teacher image, it becomes possible to detect the detection target OB1 from an image showing only a part of the detection target OB1.

In this way, the teacher image generation unit 101 performs conversion to enlarge or reduce the original image IMG5′ cut out from the image IMG5 including the detection target in at least one of the height direction and the width direction, thereby creating the teacher image IMG6. Generate. The teacher image generation unit 101 may determine the position and range of the clipping region and the conversion magnification using random numbers, for example.

When teacher image IMG6 is generated, coordinate transformation unit 102 transforms each of rectangular regions R5 indicating the position and range of detection target OB1 appearing in original image IMG5′ based on the magnification of transformation when teacher image IMG6 is generated. Find the coordinates of the vertices P5a to P5d. These coordinates are calculated from the representative coordinates, width, height, and tilt angle of the rectangular area R5 indicated on the label associated with the image IMG5.

Coordinate transformation unit 102 transforms the coordinates of vertices P5a to P5d into the coordinates of vertices P6a to P6d of quadrangular region R6 indicating the position and range of detection target OB1 appearing in teacher image IMG6. For example, the coordinate transformation unit 102 multiplies the x-coordinates of the vertices P5a to P5d by a transformation magnification in the width direction when generating the teacher image IMG6, and converts the y-coordinates of the vertices P5a to P5d to the heights when generating the teacher image IMG6. The above conversion is performed by multiplying the direction conversion magnification.

The aspect ratios of the original image IMG5' and the teacher image IMG6 are different because the magnifications of the transforms in the height direction and the width direction that are applied when the teacher image IMG6 is generated are different. In this case, the detection target OB1 appearing in the teacher image IMG6 has a distorted shape due to a change in the balance between height and width. At this time, a quadrangular region R6 corresponding to the inclined rectangular region R5 becomes a parallelogram region along the outer edge of the distorted detection object OB1. Note that if the aspect ratios of the original image and the teacher image do not change, the detection target appearing in the teacher image is enlarged or reduced without being distorted. In this case, since the rectangular area of the original image is enlarged or reduced without being distorted, the area after conversion is also rectangular.

Next, training data generation unit 103 generates a label indicating the position and range of detection target OB1 appearing in training image IMG6 based on the coordinates after conversion by coordinate conversion unit 102, that is, the coordinates of vertices P6a to P6d. Specifically, the teacher data generation unit 103 calculates the following four values from the coordinates of the vertices P6a to P6d.

(1) Representative coordinates of quadrilateral region R6 (2) Length of long side of quadrilateral region R6 (3) Inclination angle of long side of quadrilateral region R6 (4) Distance between opposite long sides of quadrilateral region R6 (1) ) is information indicating the position of the detection target OB1 appearing in the teacher image IMG6. The representative coordinates may be coordinates that can specify the position of the quadrangular region R6. For example, the teacher data generation unit 103 may calculate the coordinates of the intersection point P6e of the diagonal lines of the rectangular region R6 as the representative coordinates. Further, for example, the teacher data generation unit 103 may calculate the coordinates of the upper left corner vertex of the quadrangular region R6, that is, the coordinates of the vertex P6a as the representative coordinates.

The above (2) is information indicating the height H of the detection target appearing in the teacher image IMG6. The teacher data generator 103 may calculate the distance between the vertices P6a and P6d or the distance between the vertices P6b and P6c. The length of the short side of the quadrilateral region R6 may be calculated instead of the length of the long side of the quadrilateral region R6.

The above (3) is information indicating the tilt angle of the detection target appearing in the teacher image IMG6. The training data generation unit 103 determines the angle β formed by the straight line connecting the vertices P6a and P6d with respect to the width direction of the training image IMG6, or the angle β formed by the straight line connecting the vertices P6b and P6c with respect to the width direction of the training image IMG6. should be calculated. In addition, in the above (2), when the length of the short side of the quadrilateral region R6 is calculated instead of the length of the long side of the quadrilateral region R6, the inclination angle of the short side may be calculated.

The above (4) is information indicating the width W of the detection target appearing in the teacher image IMG6. The teacher data generation unit 103 may calculate the distance between the straight line connecting the vertices P6a and P6d and the straight line connecting the vertices P6b and P6c. In the above (2), when the length of the short side of the quadrilateral region R6 is calculated instead of the length of the long side of the quadrilateral region R6, the distance between the short sides may be calculated.

Although the actual width of OB1 is slightly wider than the actual width, the training data generation unit 103 calculates the distance between the vertices P6a and P6b or the distance between the vertices P6c and P6d as the width of the rectangular area R6. You may

Further, as shown in FIG. 2, the training data generation unit 103 corrects the vertices P6a to P6d to P6a′ to P6d′, specifies a rectangle having four vertices P6a′ to P6d′, and determines the position of the rectangle. and a label indicating the range. Specifically, the training data generation unit 103 corrects the positions of the vertices P6a to P6d so that the area is the same as that of a parallelogram having four vertices P6a to P6d and the intersection of the diagonals is P6e. For example, the long sides of the rectangle may have the same length as the long sides of the parallelogram, and the short sides of the rectangle may have the same length as the height of the parallelogram. As a result, a rectangle having four vertices P6a' to P6d' as shown in FIG. 2 is specified. In this case, coordinates P6a' to P6d', for example, may be labels indicating the position and range of the rectangle. Of course, the short sides of the rectangle may be the same length as the short sides of the parallelogram, and the length of the long sides of the rectangle may be the same as the height of the parallelogram.

Finally, the teacher data generation unit 103 associates the labels indicating the four calculated values with the teacher image IMG6 to generate teacher data. In this way, the teacher data generation unit 103 (1) calculates the representative coordinates of the rectangular area as information indicating the position of the detection target from the coordinates of each vertex obtained by the coordinate conversion unit 102, and (2) calculates the representative coordinates of the detection target. (3) calculating the inclination angle of the one side as information indicating the inclination angle of the detection target; and (4) information indicating the width of the detection target. , the distance between the one side and the side opposite to the one side may be calculated.

According to the above configuration, in addition to representative coordinates, height, and width, which are general labels for object detection, labels indicating tilt angles useful for learning rod-shaped detection targets are associated. Teacher data can be generated.

[Process flow]
The flow of processing executed by the information processing apparatus 1 will be described with reference to FIG. FIG. 3 is a flowchart showing an example of processing executed by the information processing apparatus 1. As shown in FIG. It is assumed that the setting information necessary for learning has been read before the processing of FIG. 3 is started. The setting information includes, for example, the storage location of the image 111, the range of random numbers used when generating the teacher image, the number of images 111 to be read at one time, and learning termination conditions.

In S<b>1 , the teacher image generation unit 101 acquires the image 111 from the storage unit 11 . The number of images 111 to be acquired may be set in the setting information as described above. The number of images 111 to be acquired may be one or plural. Also, the teacher image generation unit 101 may determine which image 111 to acquire using a random number.

In S2, the teacher image generation unit 101 determines the clipping range of the image 111 acquired in S1. For example, the teacher image generation unit 101 may determine the position, width, and height of the extraction range using random numbers.

In S3 (teaching image generating step), the teaching image generating unit 101 cuts out the image 111 read out in S1 within the cutting range determined in S2, and uses it as an original image to be the basis of the teaching image. Then, the teacher image generating unit 101 generates a teacher image by performing a conversion to enlarge or reduce the original image in at least one of the height direction and the width direction. The magnification for enlargement or reduction is determined by the size of the teacher image to be generated in the height direction and width direction and the size of the original image in the height direction and width direction. The teacher image generation unit 101 may determine the size of the teacher image in the height direction and width direction using random numbers.

In S4 (coordinate conversion step), first, the coordinate conversion unit 102 calculates the coordinates of the four vertices indicating the detection target range in the original image. Then, the coordinate conversion unit 102 converts the calculated coordinates based on the magnification of the conversion applied to the original image, and calculates the coordinates of the four vertices indicating the detection target range in the teacher image generated in S3. . Since the method of calculating the coordinates of each vertex is as described with reference to FIG. 2, the description will not be repeated here.

In S5, the teacher data generation unit 103 converts the coordinates of each vertex calculated by the coordinate transformation unit 102 in S4 into representative coordinates of the detection target as information indicating the position and range of the detection target appearing in the teacher image generated in S3. , width, height, and tilt angle. The method for calculating these pieces of information is as described with reference to FIG. 2, so description thereof will not be repeated here.

In S6 (teaching data generating step), the teaching data generating unit 103 generates a label from each information calculated in S5, and associates the label with the teaching image generated in S3 to make the teaching data. Then, the teacher data generation unit 103 stores the generated teacher data as the teacher data 112 in the storage unit 11 . The method of generating labels from coordinates after conversion is as described with reference to FIG. 2, so description thereof will not be repeated here.

In S7 (learning step), the learning unit 104 performs machine learning using the teacher data 112 generated in S6. For example, when learning a neural network model, the learning unit 104 inputs a teacher image included in the teacher data 112 to the model being learned, and outputs values output from the model and correspondences to the teacher image. Calculate the error with the attached label value. Then, the learning unit 104 updates the weight values of the neural network model by the error back propagation method based on the calculated error.

In S8, the learning unit 104 determines whether or not to end learning. If it is determined to end here (YES in S8), the learning unit 104 stores the latest updated model in the storage unit 11 as the learned model 113, and the processing in FIG. 3 ends. On the other hand, if it is determined not to end (NO in S8), the process returns to S1 and a new image 111 is read. The termination condition of S8 may be determined in advance as setting information. For example, the termination condition may be that the model has been updated a predetermined number of times or more, or that the error has become equal to or less than a threshold.

If a plurality of images 111 have been read out in S1, teacher data is generated from each of the plurality of images 111 read out in S1 and stored as teacher data 112 by the processes of S2 to S6. Then, in S7, learning using the teacher data 112 is performed.

The processing of S1 to S6 in FIG. 3 is a method of generating teacher data. In addition, S1 to S8, which are obtained by adding the processing of S7 to S8 for generating a learned model by machine learning to this teacher data generation method, are methods for generating a trained model. It should be noted that the processing of S1 to S6 and the processing of S7 to S8 do not necessarily have to be performed consecutively. In S1, the number of images 111 necessary for learning is acquired, and after the necessary number of teacher data 112 for learning is generated from those images 111 by the processing of S2 to S6, the processing of S7 to S8 is performed. may

Also, in generating the training data, the color and rotation angle of the training image may be changed from the original image in order to increase the variation of the training data. Various conventionally applied methods can be applied as methods of color conversion and rotation conversion for increasing the variation of teacher data.

As described above, the method of generating teacher data executed by the information processing apparatus 1 is to convert an original image in which a detection target is shown so as to expand or contract in at least one of the height direction and the width direction, thereby producing a teacher image. Based on the teacher image generation step (S3) and the conversion magnification, the coordinates of each vertex of a rectangular area indicating the position and range of the detection target appearing in the original image are converted to the position and range of the detection target appearing in the teacher image. A coordinate transformation step (S4) for transforming the coordinates of each vertex of the quadrilateral area shown, and a label indicating the position and range of the detection target appearing in the teacher image based on the coordinates after the transformation in the coordinate transformation step, are generated in the teacher image. and a teacher data generating step (S6) for making teacher data in association with each other. This makes it possible to generate teacher data including labels that correctly indicate the position and range of the detection target in the teacher image.

As described above, the method of generating a trained model executed by the information processing apparatus 1 is to convert an original image in which a detection target is shown so as to expand or contract in at least one of the height direction and the width direction, and generate a teacher image. and the coordinate of each vertex of a rectangular area indicating the position and range of the detection target appearing in the original image, based on the magnification of the conversion, to the position and range of the detection target appearing in the teacher image. a coordinate transformation step (S4) for transforming into the coordinates of each vertex of a quadrangular area showing ; Learning to generate a trained model for detecting a detection target from an image by a teacher data generation step (S6) that uses the teacher data as teacher data in association with and machine learning using the teacher data generated in the teacher data generation step. and a step (S7). As a result, it is possible to automatically generate a trained model for detecting a detection target from an image using teacher data including labels that correctly indicate the position and range of the detection target.

[Modification]
The execution subject of each process described in the above embodiments is arbitrary, and is not limited to the above examples. That is, an information processing system having functions similar to those of the information processing apparatus 1 can be constructed by using a plurality of information processing apparatuses that can communicate with each other. For example, a configuration may be adopted in which different information processing apparatuses execute the generation of the teacher image, the conversion of the coordinates, the generation of the teacher data, the learning, and the inference. Also, for example, an information processing device that executes processing from generation of a teacher image to generation of teacher data, that is, a method of generating teacher data, an information processing device that generates a trained model using the generated teacher data, It is also possible to construct an information processing system including an information processing device that performs inference using the generated trained model. In this way, the method of generating teacher data described above can be implemented by one or more information processing devices. The same applies to the method of generating the trained model described above. Also, a photographing device for photographing a detection target may be included in the information processing system.

[Example of realization by software]
The function of the information processing device 1 (hereinafter referred to as "device") is a program for causing a computer to function as the device, and the computer is used as each control block (especially each part included in the control unit 10) of the device. It can be realized by a program for functioning.

In this case, the apparatus comprises a computer having at least one control device (eg processor) and at least one storage device (eg memory) as hardware for executing the program. Each function described in each of the above embodiments is realized by executing the above program using the control device and the storage device.

As the control device, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a combination thereof may be applied. Further, the storage device may include a high-speed storage unit capable of writing and reading data at high speed, and a large-capacity storage unit having a larger data storage capacity than the high-speed storage unit. As the high-speed storage unit, a high-speed access memory such as SDRAM (Synchronous Dynamic Random-Access Memory) can be applied. Moreover, as a large-capacity storage unit, for example, a HDD (Hard Disk Drive), SSD (Solid-State Drive), SD (Secure Digital) card, eMMC (embedded Multi-Media Controller), or the like can be applied.

The program may be recorded on one or more computer-readable recording media, not temporary. The recording medium may or may not be included in the device. In the latter case, the program may be supplied to the device via any transmission medium, wired or wireless.

Also, part or all of the functions of each control block can be realized by a logic circuit formed in an integrated circuit (IC chip) or the like. For example, integrated circuits in which logic circuits functioning as the control blocks described above are formed are also included in the scope of the present invention. In addition, it is also possible to implement the functions of the control blocks described above by, for example, a quantum computer.

The present invention is not limited to the above-described embodiments, but can be modified in various ways within the scope of the claims, and can be obtained by appropriately combining technical means disclosed in different embodiments. is also included in the technical scope of the present invention.

1 Information Processing Device 101 Teacher Image Generation Unit 102 Coordinate Transformation Unit 103 Teacher Data Generation Unit 104 Learning Unit

Claims (6)

a teacher image generating unit for generating a teacher image by performing a conversion to enlarge or reduce an original image in which a detection target is captured in at least one of the height direction and the width direction;
Coordinates of each vertex of a rectangular area indicating the position and range of the detection target in the original image are obtained based on the conversion magnification, and each vertex of a rectangular area indicating the position and range of the detection target in the teacher image. a coordinate transformation unit that transforms the coordinates of
a teacher data generating unit that generates a label indicating the position and range of the detection target appearing in the teacher image based on the coordinates converted by the coordinate transforming unit, and associates the label with the teacher image as teacher data; information processing device. The training data generation unit, from the coordinates of each vertex of the quadrilateral area,
(1) calculating representative coordinates of the rectangular area as information indicating the position of the detection target;
(2) calculating the length of one side of the rectangular region as information indicating the height of the detection target;
(3) calculating the tilt angle of the one side as information indicating the tilt angle of the detection target;
(4) The information processing apparatus according to claim 1, wherein the distance between the one side and the side opposite to the one side is calculated as the information indicating the width of the detection target. 3. The information processing apparatus according to claim 1, further comprising a learning unit that generates a trained model for detecting the detection target from an image by machine learning using the teacher data generated by the teacher data generation unit. . A method of generating teacher data executed by one or more information processing devices, comprising:
a teacher image generating step of generating a teacher image by applying a transformation that enlarges or shrinks an original image in which a detection target is captured in at least one of the height direction and the width direction;
Coordinates of each vertex of a rectangular area indicating the position and range of the detection target in the original image are obtained based on the conversion magnification, and each vertex of a rectangular area indicating the position and range of the detection target in the teacher image. a coordinate transformation step of transforming to the coordinates of
a teacher data generation step of generating a label indicating the position and range of the detection target appearing in the teacher image based on the coordinates after the transformation by the coordinate transformation step, and associating the label with the teacher image as teacher data; How to generate teacher data including. A method for generating a trained model executed by one or more information processing devices,
a teacher image generating step of generating a teacher image by applying a transformation that enlarges or shrinks an original image in which a detection target is captured in at least one of the height direction and the width direction;
Coordinates of each vertex of a rectangular area indicating the position and range of the detection target in the original image are obtained based on the conversion magnification, and each vertex of a rectangular area indicating the position and range of the detection target in the teacher image. a coordinate transformation step of transforming to the coordinates of
a teacher data generation step of generating a label indicating the position and range of the detection target appearing in the teacher image based on the coordinates after the transformation by the coordinate transformation step, and associating the label with the teacher image as teacher data;
and a learning step of generating a learned model for detecting the detection target from an image by machine learning using the teacher data generated in the teacher data generating step. 2. A program for causing a computer to function as the information processing apparatus according to claim 1, the program for causing the computer to function as the teacher image generating section, the coordinate transforming section, and the teacher data generating section.

Images