WO2023062784A1

WO2023062784A1 - Dataset creation device, dataset creation method, and recording medium

Info

Publication number: WO2023062784A1
Application number: PCT/JP2021/038080
Authority: WO
Inventors: 智一金子; 真寺尾
Original assignee: 日本電気株式会社
Priority date: 2021-10-14
Filing date: 2021-10-14
Publication date: 2023-04-20
Also published as: JPWO2023062784A1

Abstract

In this dataset creation device, an acquisition means acquires imaged images of an object. A data processing means generates sample images from the imaged images, said sample images being images of sections of the object, and generates a training dataset including a sample image among the sample images that is useful for satisfying a prescribed criterion. A data quality estimation means estimates the quality of the dataset on the basis of the imaged images and the dataset. A display control means displays, on a display device, imaging assistance information including dataset quality information.

Description

DATASET CREATION DEVICE, DATASET CREATION METHOD, AND RECORDING MEDIUM

This disclosure relates to creating a training dataset.

In stores, etc., there is a known method of identifying products by photographing the products and performing image recognition. In order to perform product image recognition, it is necessary to learn a recognition model for the target product. Also, when a new product is released, it is necessary to learn an existing recognition model so that the new product can be recognized. In order to train the recognition model, it is necessary to prepare a training data set for the target product.

Patent Literature 1 describes a method of evaluating labelers in order to improve the quality when creating datasets for machine learning.

International publication 2019/187421

When creating a data set for learning, the appearance of objects such as target products is photographed from various directions, and sample images for learning are generated from the captured images. However, experience is required to capture images that can efficiently create data effective for learning. Specifically, inexperienced users have problems such as not knowing the amount of data necessary for learning a recognition model and not being able to collect useful appearance variations for learning.

One object of the present disclosure is to provide a dataset creation device that enables even an inexperienced person to capture images suitable for creating learning data and create a high-quality learning dataset. to do.

In one aspect of the present disclosure, the dataset creation device
Acquisition means for acquiring a photographed image of an object;
data processing means for generating a sample image, which is an image of a portion of the object, from the captured image, and generating a learning data set including effective sample images satisfying a predetermined criterion among the sample images;
data quality estimation means for estimating the quality of the dataset based on the captured image and the dataset;
display control means for displaying shooting support information including quality information of the data set on a display device;
Prepare.

In another aspect of the present disclosure, a dataset creation method includes:
Acquire a photographed image of an object,
generating a sample image, which is an image of a portion of the object, from the captured image, generating a learning data set containing valid sample images satisfying a predetermined criterion among the sample images;
estimating the quality of the dataset based on the captured image and the dataset;
A display device displays shooting support information including quality information of the data set.

In yet another aspect of the present disclosure, the recording medium comprises
Acquire a photographed image of an object,
generating a sample image, which is an image of a portion of the object, from the captured image, generating a learning data set containing valid sample images satisfying a predetermined criterion among the sample images;
estimating the quality of the dataset based on the captured image and the dataset;
A program for causing a computer to execute a process of displaying shooting support information including quality information of the data set on a display device is recorded.

According to the present disclosure, even an inexperienced person can capture images suitable for generating learning data and create a high-quality learning data set.

1 shows a schematic configuration of a dataset creation system according to a first embodiment; 2 is a block diagram showing the hardware configuration of the data set creation device of the first embodiment; FIG. 2 is a block diagram showing the functional configuration of the data set creation device of the first embodiment; FIG. 4 is a block diagram showing the configuration of a data quality estimation unit according to the first example; FIG. 4 shows an example of a shooting support screen according to the first embodiment; 9 is a flowchart of display processing of shooting support information; FIG. 11 is a block diagram showing the configuration of a data quality estimation unit according to the second embodiment; FIG. 9 shows an example of a shooting support screen according to the second embodiment; It is a block diagram which shows the functional structure of the dataset creation apparatus of 2nd Embodiment. 9 is a flowchart of processing by the data set creation device of the second embodiment;

Preferred embodiments of the present disclosure will be described below with reference to the drawings.
<First embodiment>
[overall structure]
FIG. 1 shows a schematic configuration of a data set creation system according to the first embodiment. The data set creation system 1 is a system that creates a learning data set that is used when learning a recognition model for recognizing products from photographed images.

As illustrated, the dataset creation system 1 includes a dataset creation device 100, a camera 2, and a display device 5. The camera 2 and the display device 5 are each connected to the data set creation device 100 . The camera 2 is fixedly arranged at a predetermined position such as a shelf. An operator brings an object (product) for which a data set is to be created into the photographing range of the camera 2 and photographs the object with the camera 2 . At this time, the operator rotates the object or changes its grip to photograph the appearance of the object from various directions. The camera 2 outputs a captured image (moving image) of the captured object to the data set creation device 100 .

The data set creation device 100 creates a learning data set for the target object based on the captured image input from the camera 2 . The created dataset is used to train a recognition model that recognizes objects. By learning the recognition model using the created data set, the recognition model can recognize the object.

The data set creation device 100 generates shooting support information and displays it on the display device 5 while the worker is shooting an object with the camera 2 . The shooting support information is information that tells the operator how the object is currently being shot and how the data set is created, and if necessary, gives instructions and advice on how to shoot the object. The data set creation device 100 generates shooting support information based on the captured images captured by the camera 2 and analysis results such as the number and quality of data created based on the captured images, and outputs the information to the display device 5. do. Details of the shooting assistance information will be described later.

In this way, by displaying the shooting support information while the worker is shooting an object and feeding back the current shooting state and the data creation state to the worker, even an inexperienced worker can recognize a recognition model. It is possible to capture images suitable for learning and efficiently create a high-quality data set.

[Hardware configuration]
FIG. 2 is a block diagram showing the hardware configuration of the dataset creation device 100 of the first embodiment. As illustrated, the data set creation device 100 includes an interface (I/F) 11, a processor 12, a memory 13, a recording medium 14, and a database (DB) 15.

The interface 11 performs data input/output with an external device. Specifically, the interface 11 acquires a photographed image from the camera 2 while the operator is photographing, and outputs photographing support information to the display device 5 . In addition, the interface 11 outputs the created data set for learning to a learning device for learning the recognition model.

The processor 12 is a computer such as a CPU (Central Processing Unit), and controls the entire data set creation device 100 by executing a program prepared in advance. The processor 12 may be a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array). The processor 12 executes processing for displaying shooting support information, which will be described later.

The memory 13 is composed of ROM (Read Only Memory), RAM (Random Access Memory), and the like. Memory 13 is also used as a working memory during execution of various processes by processor 12 .

The recording medium 14 is a non-volatile, non-temporary recording medium such as a disk-shaped recording medium or semiconductor memory, and is configured to be detachable from the data set creation device 100 . The recording medium 14 records various programs executed by the processor 12 . When the data set creation device 100 executes various processes, a program recorded on the recording medium 14 is loaded into the memory 13 and executed by the processor 12 . The DB 15 stores captured images input from the camera 2 and created data sets.

[Function configuration]
FIG. 3 is a block diagram showing the functional configuration of the dataset creation device 100 of the first embodiment. The data set creation device 100 includes a captured image input unit 21 , a data processing unit 22 , a storage unit 23 , a data quality estimation unit 24 and a display control unit 25 . The captured image input unit 21 is composed of the interface 11 , the data processing unit 22 , the data quality estimation unit 24 and the display control unit 25 are mainly composed of the processor 12 , and the storage unit 23 is composed of the DB 15 .

The photographed image input unit 21 acquires the photographed image of the object photographed by the worker from the camera 2 and outputs it to the data processing unit 22 . A photographed image is a moving image obtained by continuously photographing an object.

The data processing unit 22 uses the input captured image to generate learning data for learning the recognition model. Specifically, the data processing unit 22 detects an object from the captured image using an object detection model or the like, cuts out an image of the object portion, and generates an image of the object (hereinafter referred to as a “sample image”). Generate. The object detection model detects an object from a captured image, and outputs position information of a rectangle containing the object and a score indicating the likelihood of the object. The data processing unit 22 extracts the rectangular area detected by the object detection model from the captured image and uses it as a sample image. The data processing unit 22 then outputs the generated sample image to the storage unit 23 together with the captured image.

The storage unit 23 receives the captured image generated by the camera 2 and a plurality of sample images extracted from the captured image from the data processing unit 22 and stores them.

The data quality estimation unit 24 estimates the quality of the sample images generated by the data processing unit 22, selects sample images that satisfy predetermined criteria as valid sample images, and stores them in the storage unit 23 as learning data. In this way, a training data set is created by collecting a plurality of valid sample images determined to satisfy the criteria. Of the sample images generated by the data processing unit 22, those that do not satisfy the above criteria are not adopted as learning data and discarded.

In addition, the data quality estimation unit 24 generates shooting support information based on the captured image and the result of estimating the quality of the sample image, and outputs it to the display control unit 25 . The shooting support information includes a captured image captured by the camera 2, an effective sample image, and the like, the details of which will be described later. The display control unit 25 creates a shooting assistance screen using the shooting assistance information input from the data quality estimation unit 24 and displays it on the display device 5 .

[First embodiment]
(Configuration of data quality estimation unit)
Next, a first example of the data set creation device 100 will be described. In the first embodiment, the data quality estimation unit 24 estimates the importance of the sample images in learning as the quality of the sample images. FIG. 4 is a block diagram showing the configuration of the data quality estimator 24a according to the first embodiment. As illustrated, the data quality estimator 24a includes an importance estimator 26. FIG.

A sample image is input from the storage unit 23 to the importance estimation unit 26 . The importance estimation unit 26 estimates the importance of the input sample image using an importance estimation model. In one example, an object detection model can be used as the importance estimation model. Since the object detection model outputs a score of object-likeness of the detected object based on the input sample image, the importance estimation unit 26 uses this score as the importance. The higher the object-likeness score, the higher the probability that an object is included in the sample image, and the higher the suitability as learning data. Therefore, the data quality estimating unit 24a selects a sample image whose object-likeness score is higher than a predetermined value as an effective sample image.

In another example, a model for estimating image quality as an image can be used as the importance estimation model. As a model for estimating image quality, for example, a model for estimating camera shake, blurring, brightness, degree of hiding of objects (ratio of objects hidden behind other objects), etc. can be used. . Specifically, based on the estimation result of the above model, the importance estimating unit 26 determines that the less camera shake, the less blurring, the more appropriate brightness, and the smaller the degree of hiding of the object, the more important it is. increase the degree.

If the image quality of the sample image is not good, specifically if there is a lot of camera shake, the image is blurred, the image is dark, or most of the object is hidden, the sample image is used as training data. Considered to be less suitable. Therefore, the data quality estimating unit 24a selects sample images whose image quality is determined to be equal to or higher than the reference level by the model for estimating image quality as described above, as valid sample images.

It should be noted that both the above object-likeness score and image quality may be combined and used as the degree of importance. For example, the degree of importance may be calculated by adding a score of object-likeness and a value indicating the degree of camera shake or brightness using a predetermined weight.

The data quality estimation unit 24a stores the valid sample images selected as described above in the storage unit 23. A set of effective sample images accumulated in this manner serves as a data set for learning.

In addition, the data quality estimation unit 24a calculates the degree of attainment, the effective image ratio, etc. calculated based on the photographed images input from the storage unit 23, the valid sample images selected by the importance estimation unit 26, and the number of valid sample images. is output to the display control unit 25 as shooting assistance information. The display control unit 25 generates a shooting support screen using the input shooting support information and displays it on the display device 5 .

(Shooting support screen)
FIG. 5 shows an example of a shooting assistance screen according to the first embodiment. As illustrated, the shooting support screen is roughly divided into a captured image display area 30 and an effective image display area 40 . In the example of FIG. 5, the captured image display area 30 displays a captured image 31, a rectangle 32, a degree of importance 33, and a degree of attainment .

The captured image 31 is a captured image (moving image) captured by the camera 2 and displayed in real time. A rectangle 32 indicates the position of the object detected by the object detection model from the captured image 31 . As described above, the portion of the rectangle 32 surrounding the object is cut out from the captured image 31 as a sample image. The degree of importance 33 is the degree of importance of the detected object, specifically, the value of the degree of importance calculated by the degree-of-importance estimator 26 described above. Therefore, in the first embodiment, the importance value is the object-likeness score or the image quality estimate of the sample image as described above.

The attainment level 34 is the ratio of the number of valid sample images already obtained to the total number of images required for learning. Note that the total number of images required for learning is determined in advance based on experience and the like. The example of FIG. 5 shows that currently "120 (frames)" of effective sample images have been acquired for the total number of images "300 (frames)" required for learning. This allows the operator to know how many sample images required for learning have been acquired and how much more is required. It should be noted that when the necessary total number of valid sample images have been acquired, the fact may be notified to the operator by display or voice.

On the other hand, in the effective image display area 40, a thumbnail display area 41 and an effective image ratio 42 are displayed. In the thumbnail display area 41, thumbnails of a plurality of effective sample images cut out from the captured image 31 are displayed side by side. That is, each thumbnail image 43, as indicated by the rectangle 32 in the captured image display area 30, is determined by the data quality estimating unit 24a to satisfy the predetermined criteria among the sample images cut out from the captured image 31. It is a sample image. Instead of displaying only effective sample images as shown in FIG. 5, all sample images cut out from the photographed image 31 are displayed in the thumbnail display area 41, and the effective sample images are displayed, for example, in the color of the frame. may be emphasized and displayed by changing

The valid image ratio 42 is the ratio of valid sample images to the total number of sample images cut out from the captured image 31 . In the example of FIG. 5, since the effective image ratio is 90%, it indicates that 90% of the sample images cut out from the photographed image 31 so far have been adopted as effective sample images. . It may also display the main determining factors for an ineffective image, such as 5% blur, 3% lighting conditions, and 2% hidden objects. If the effective image ratio is lower than a predetermined standard, the operator may be notified of this by means of display, voice, or the like. As a result, the operator can notice a problem such as the object being out of the photographing range of the camera 2.例文帳に追加Furthermore, if the effective image ratio is lower than a predetermined standard, instructions or advice may be given to the operator. For example, if the sample image has a lot of camera shake and the effective image ratio is low, a message such as "Please move a little more slowly" may be displayed or output by voice.

In the example of FIG. 5, the attainment level 34 and the effective image ratio 42 are displayed numerically, but they may be displayed as graphs or meters instead.

In addition, the operator may be notified of whether or not sufficient valid sample images have been acquired for all objects at the time when all objects have been photographed. For example, for an object for which effective sample images are insufficient due to interruption of photography, etc., a message such as "Product X: 30 insufficient images" may be displayed to notify the worker that the required number of sample images has not been reached. good. This allows the operator to take additional shots of missing sample images.

(Display processing)
Next, display processing of shooting support information will be described. FIG. 6 is a flowchart of display processing of shooting support information. This processing is realized by the processor 12 shown in FIG. 2 executing a program prepared in advance and operating as each element shown in FIGS. 3 and 4. FIG.

First, the captured image input unit 21 acquires a captured image from the camera 2 (step S11). Next, the data processing unit 22 cuts out the object portion from the captured image to generate a sample image (step S12). Next, the data quality estimator 24a estimates the importance of each sample image and extracts the sample image that satisfies a predetermined criterion as an effective sample image. The data quality estimator 24a also calculates the degree of reach, the effective image ratio, etc., based on the number of effective sample images. Then, the data quality estimation unit 24a generates shooting support information including the shot image, the effective sample image, the reach, the effective image ratio, etc., and outputs it to the display control unit 25 (step S13).

Next, the display control unit 25 uses the input shooting assistance information to generate a shooting assistance screen (step S14), and displays the shooting assistance screen on the display device 5 (step S15). In this way, a photographing support screen such as that illustrated in FIG. 5 is displayed on the display device 5 .

Next, the data set creation device 100 determines whether or not to end shooting (step S16). For example, when an instruction to end photography is input to the display device 5, or when the reach reaches 100%, the data set creation device 100 determines to end photography. If it is determined not to end the shooting (step S16: No), the process returns to step S11, and steps S11 to S16 are repeated. On the other hand, if it is determined to end the shooting (step S16: Yes), the display process ends.

[Second embodiment]
(Configuration of data quality estimation unit)
Next, a second embodiment of the data set creation device 100 will be described. In the second embodiment, the data quality estimation unit 24 estimates the existing object similarity as the quality of the sample image. FIG. 7 is a block diagram showing the configuration of the data quality estimator 24b according to the second embodiment. As illustrated, the data quality estimation unit 24b includes an existing object similarity estimation unit 27. FIG. The existing object similarity estimation unit 27 estimates the quality of the sample image using the existing recognition model.

"Existing object similarity" refers to the similarity of a sample image to an existing object that has already been learned in an existing recognition model. That is, the existing object similarity indicates the degree of similarity between the sample image and other objects (products) already registered in the recognition model. The degree of similarity is calculated, for example, by cosine similarity with respect to feature quantities extracted using a pre-learned feature extraction model. When learning a recognition model so that a certain new product A can be recognized, the new product A may be similar to product B already registered in the recognition model. In this case, for learning the recognition model, it is necessary to use a sample image of the new product A that is not similar to the existing product B, that is, has distinctiveness.

The data quality estimating unit 24b calculates the attainment level, effective image ratio, etc. calculated based on the captured image input from the storage unit 23, the valid sample images selected by the existing object similarity estimating unit 27, and the number of valid sample images. is output to the display control unit 25 as shooting assistance information. Further, the existing object similarity estimation unit 27 selects an image of an existing object determined to have a high similarity (hereinafter also referred to as a "similar object image") for a sample image whose similarity to the existing object is higher than a predetermined standard. ) is output to the display control unit 25 as shooting support information. The display control unit 25 generates a shooting support screen using the input shooting support information and displays it on the display device 5 .

(Shooting support screen)
FIG. 8 shows an example of a shooting assistance screen according to the second embodiment. As in the first embodiment shown in FIG. 5, the photographing support screen includes a photographed image display area 30 and an effective image display area 40. As shown in FIG. Since the display contents of the effective image display area 40 are the same as those of the first embodiment, description thereof will be omitted. A photographed image 31, a rectangle 32, and an attainment level 34 are displayed in the photographed image display area 30, as in the first embodiment.

Further, in the second embodiment, when the existing object similarity estimating unit 27 determines that the similarity of the sample image to the existing object is higher than a predetermined standard, the image of the existing object, that is, the similar object image 35 is displayed. be done. In the example of FIG. 8, the similar object image 35 is displayed side by side with the rectangle 32 corresponding to the sample image. Furthermore, the similarity 36 between the sample image and the similar object image calculated by the existing object similarity estimation unit 27 is displayed near the similar object image 35 .

As a result, the worker can see that there are existing objects similar to the currently created sample image, the image of the existing object, the degree of similarity with the existing object, and so on. Therefore, the worker can actively photograph the surface or part of the object currently being photographed that is highly identifiable from the existing object with a high degree of similarity. can be created efficiently. Specifically, the operator may change the photographing location of the product so that the displayed similarity value decreases and photograph the product. Note that when there are a plurality of existing objects with high similarities, the similar object images 35 may be displayed for a predetermined number of existing objects with the highest similarities.

Also, in the vicinity of the similar object image 35, the product name of the product indicated by the similar object image 35 may be displayed. Furthermore, if the degree of similarity between the sample image and the existing object is higher than a predetermined value, the operator may be notified that there is a high possibility that the product currently being photographed has already been registered. As a result, it is possible to prevent duplicate registration of an object (product) that has already been registered in the existing recognition model.

(Display processing)
Next, display processing of shooting support information will be described. The display processing of the shooting assistance information in the second embodiment is basically the same as that in the first embodiment shown in FIG. However, in the second embodiment, in step S13, the existing object similarity estimating unit 27 generates shooting support information including the similar object image 35 and the similarity 36 with the existing object. Output.

[Modification]
Next, a modified example of the data set creation device 100 according to the first embodiment will be described. This modification can be applied to the first or second embodiment described above. In the first embodiment described above, the data quality estimation unit 24a estimates the importance of sample images, and adds sample images whose importance satisfies the criteria to the data set as valid sample images. At this time, in the modified example, the estimated importance is added to the effective sample image as attribute information. As a result, the sample images included in the data set accumulated in the storage unit 23 are attached with their importance as attribute information. Therefore, when learning a recognition model using that data set, it is possible to select sample images to be used for learning and determine priorities using the importance added to the sample images. Specifically, by learning a recognition model by preferentially selecting from sample images with a high degree of importance, improvement in learning efficiency can be expected.

Similarly, in the second embodiment, the data quality estimation unit 24b estimates the existing object similarity of the sample image. At this time, in the modified example, the estimated existing object similarity is added as attribute information to the effective sample image and added to the data set. As a result, the existing object similarity is attached as attribute information to the sample images included in the data set accumulated in the storage unit 23 . Therefore, when learning a recognition model using the data set, the existing object similarity added to the sample images can be used to select and prioritize sample images to be used for learning. Specifically, by learning a recognition model by preferentially selecting from sample images with low similarity to existing objects, it is expected to improve the efficiency of learning.

<Second embodiment>
FIG. 9 is a block diagram showing the functional configuration of the data set creation device of the second embodiment. The data set creation device 70 includes acquisition means 71 , data processing means 72 , data quality estimation means 73 , and display control means 74 .

FIG. 10 is a flowchart of processing by the data set creation device 70 of the second embodiment. First, the acquisition means 71 acquires a photographed image of an object (step S21). Next, the data processing means 72 generates a sample image, which is an image of the part of the object, from the photographed image, and generates a data set for learning including effective sample images satisfying a predetermined criterion among the sample images ( step S22). Next, the data quality estimation means 73 estimates the quality of the dataset based on the captured image and the dataset (step S23). Then, the display control means 74 displays the shooting support information including the quality information of the dataset on the display device (step S24). Then the process ends.

According to the data set creation device 70 of the second embodiment, even an inexperienced person can capture images suitable for learning and create a high quality learning data set.

Some or all of the above embodiments can also be described as the following additional remarks, but are not limited to the following.

(Appendix 1)
Acquisition means for acquiring a photographed image of an object;
data processing means for generating a sample image, which is an image of a portion of the object, from the captured image, and generating a learning data set including effective sample images satisfying a predetermined criterion among the sample images;
data quality estimation means for estimating the quality of the dataset based on the captured image and the dataset;
display control means for displaying shooting support information including quality information of the data set on a display device;
A dataset creation device comprising:

(Appendix 2)
2. The data set creation device according to appendix 1, wherein the quality information includes importance of sample images included in the data set.

(Appendix 3)
3. The data set creation device according to appendix 2, wherein the importance includes a score of object-likeness of an object detected from the sample image.

(Appendix 4)
4. The data set creation device according to appendix 2 or 3, wherein the degree of importance includes information about image quality of the sample image.

(Appendix 5)
1. The data set creation apparatus according to appendix 1, wherein the quality information includes a similarity to an existing object obtained by recognizing a sample image included in the data set using an existing recognition model.

(Appendix 6)
2. The data set creation device according to appendix 1, wherein the quality information includes a similarity between a sample image included in the data set and an existing object.

(Appendix 7)
The shooting support information includes the captured image, information indicating a position where the sample image is extracted from the captured image, valid sample images included in the data set, and valid samples for the total number of the sample images. 7. The data set creation device according to any one of Appendices 1 to 6, comprising: an effective image ratio, which is a ratio of images.

(Appendix 8)
8. The data set creation device according to any one of appendices 1 to 7, wherein the shooting support information includes an attainment level indicating a ratio of the number of valid sample images already obtained to the total number of images required for learning.

(Appendix 9)
9. The data set creation device according to any one of additional notes 1 to 8, further comprising storage means for adding quality information estimated based on the sample image to the effective sample image and storing the data set.

(Appendix 10)
Acquire a photographed image of an object,
generating a sample image, which is an image of a portion of the object, from the captured image, generating a learning data set containing valid sample images satisfying a predetermined criterion among the sample images;
estimating the quality of the dataset based on the captured image and the dataset;
A data set creation method for displaying shooting assistance information including quality information of the data set on a display device.

(Appendix 11)
Acquire a photographed image of an object,
generating a sample image, which is an image of a portion of the object, from the captured image, generating a learning data set containing valid sample images satisfying a predetermined criterion among the sample images;
estimating the quality of the dataset based on the captured image and the dataset;
A recording medium recording a program for causing a computer to execute processing for displaying shooting support information including quality information of the data set on a display device.

Although the present disclosure has been described above with reference to the embodiments and examples, the present disclosure is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

1 data set creation system 12 processor 21 captured image input unit 22 data processing unit 23

storage unit

24, 24a, 24b data quality estimation unit 25 display control unit 26 importance estimation unit 27 existing object similarity estimation unit 100 data set creation device

Claims

Acquisition means for acquiring a photographed image of an object;
data processing means for generating a sample image, which is an image of a portion of the object, from the captured image, and generating a learning data set including effective sample images satisfying a predetermined criterion among the sample images;
data quality estimation means for estimating the quality of the dataset based on the captured image and the dataset;
display control means for displaying shooting support information including quality information of the data set on a display device;
A dataset creation device comprising:
The dataset creation device according to claim 1, wherein the quality information includes importance of sample images included in the dataset.
The data set creation device according to claim 2, wherein the degree of importance includes a score of object-likeness of an object detected from the sample image.
The data set creation device according to claim 2 or 3, wherein the degree of importance includes information about image quality of the sample image.
The dataset creation device according to claim 1, wherein the quality information includes the degree of similarity to an existing object obtained by recognizing a sample image included in the dataset using an existing recognition model.
The dataset creation device according to claim 1, wherein the quality information includes a degree of similarity between the sample image included in the dataset and an existing object.
The shooting support information includes the captured image, information indicating a position where the sample image is extracted from the captured image, valid sample images included in the data set, and valid samples for the total number of the sample images. 7. The data set creation device according to any one of claims 1 to 6, further comprising an effective image ratio which is a ratio of images.
The data set creation device according to any one of claims 1 to 7, wherein the shooting support information includes an attainment level indicating the ratio of the number of valid sample images already obtained to the total number of images required for learning.
The dataset creation device according to any one of claims 1 to 8, further comprising storage means for adding quality information estimated based on the sample image to the effective sample image and storing the dataset.
Acquire a photographed image of an object,
generating a sample image, which is an image of a portion of the object, from the captured image, generating a learning data set containing valid sample images satisfying a predetermined criterion among the sample images;
estimating the quality of the dataset based on the captured image and the dataset;
A data set creation method for displaying shooting assistance information including quality information of the data set on a display device.
Acquire a photographed image of an object,
generating a sample image, which is an image of a portion of the object, from the captured image, generating a learning data set containing valid sample images satisfying a predetermined criterion among the sample images;
estimating the quality of the dataset based on the captured image and the dataset;
A recording medium recording a program for causing a computer to execute processing for displaying shooting support information including quality information of the data set on a display device.