JP6489005B2 - Information processing system, information processing method, and program - Google Patents

Description

  The present invention relates to an information processing system, an information processing method, and a program.

  In recent years, research on deep learning has been actively conducted as an image recognition technique based on supervised learning. Deep learning is one type of machine learning in which conventional neural networks are stacked in multiple layers, and solves problems such as image classification by learning in advance using learning images.

  In order to improve accuracy by image recognition by deep learning, it is important to secure and select learning images. In order to use learning images as additional learning for advance learning or recognition accuracy improvement, a large amount of images and their content information must be prepared as a data set. For example, in the case of a data set of “cat”, it is necessary to prepare several hundred sets or more of a pair of an image showing a cat and its content information (such as breed name, label).

  However, it is often very expensive to prepare such a data set. The reason for this is that a large amount of images are prepared, the content information needs to be associated, and the association of the content information must be performed manually.

  From the above, securing and selecting images for pre-learning and additional learning for deep learning is one of the major issues, and reduction of their creation costs is strongly desired.

  Patent Document 1 describes a technique for selecting additional image candidates in order to improve the quality of a learning dictionary used for image / video similarity search.

JP 2013-250881 A

  In the technique of Patent Document 1, the similarity is calculated from the average feature amount of the image group that is a learning dictionary and the feature amount of the candidate image, and is selected as an additional image when the similarity is larger than a threshold value.

  According to this method, the number of images in the learning dictionary can be increased without changing the tendency of the feature amount originally possessed by the learning dictionary, but an image having a tendency different from the average feature amount cannot be selected as a candidate. .

  Therefore, it cannot be applied to a system that performs feedback to a false detection result caused by a difference in feature amount tendency when image recognition using deep learning or the like is performed.

  Therefore, an object of the present invention is to provide a mechanism that can select an image as a candidate for additional learning for the purpose of feedback to an erroneous detection result in image recognition.

  An information processing system according to the present invention is an information processing system for classifying verification images by a supervised image classifier, and includes an image input means for inputting a verification image, and a verification input input by the image input means. Similar image extraction means for extracting an image similar to an image, teacher data creation means for creating teacher data by giving a label to the image extracted by the similar image extraction means, and created by the teacher data creation means Learning means for learning the supervised image classifier using teacher data.

  The information processing method of the present invention is an information processing method in an information processing system for classifying verification images by a supervised image classifier, and includes an image input step for inputting verification images, and the image input step. A similar image extraction step for extracting an image similar to the input verification image, a teacher data creation step for creating teacher data by giving a label to the image extracted by the similar image extraction step, and the teacher data creation And a learning step of learning the supervised image classifier using the teacher data created in the step.

  The program of the present invention is a program that can be executed in an information processing system that classifies verification images by a supervised image classifier, and the information processing system includes image input means that inputs verification images. Similar image extraction means for extracting an image similar to the verification image input by the image input means, and teacher data creation means for creating teacher data by assigning a label to the image extracted by the similar image extraction means And using the teacher data created by the teacher data creating means to function as a learning means for learning the supervised image classifier.

  According to the present invention, it is possible to select an image as a candidate for additional learning for the purpose of feedback to an erroneous detection result in image recognition.

The figure which shows the structure of the information processing system of this invention 1 is a block diagram showing the hardware configuration of an information processing apparatus applicable to the operation terminal PC 101, application server 102, supervised image classification server 103, unsupervised image classification server 104, and image database 105 shown in FIG. Functional block diagram for explaining the functions of the operation terminal PC 101, application server 102, supervised image classification server 103, unsupervised image classification server 104, and image database 105 Flowchart showing selection of learning image to supervised image classifier using unsupervised image classifier Flowchart showing similar image acquisition processing (step S409) Data structure example of image data management record information table An example of a screen displayed on operation terminal PC101 in step S411 An example of a screen displayed on operation terminal PC101 in step S405

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a system configuration diagram showing an example of the configuration of an information processing system to which the additional learning image selection support function of the present invention is applied.

  The information processing system includes one or more operation terminals PC101, one or more application servers 102, one or more supervised image classification servers 103, one or more unsupervised image classification servers 104, and one or more image databases 105. Are connected via a network.

  In this embodiment, the application server 102, the supervised image classification server 103, the supervised image classification server 104, and the database 105 are described as separate housings. However, these servers and databases are stored in the same housing. You may comprise so that a function may be provided.

  The operation terminal PC101 is a terminal for performing each operation of image classification, image classifier learning, and candidate image registration, and exists on the network. A web operation screen (operated on a web browser) is used for image classification and learning operations, and connected to the application server 102 via a network.

  The application server 102 is a server for controlling image classification, image classifier learning, and candidate image registration, and is constructed on a network. The operation terminal PC 101 is connected to the application server 102. When an image classification operation is performed from the operation terminal PC 101, the application server 102 transmits an image to the supervised image classification server 103 and receives a classification result. When a classifier learning operation is performed from the operation terminal PC 101, an image is transmitted to the unsupervised image classification server 104, a similar image class as a learning candidate is received, and from the image database 105 based on the class information. Image data used for learning is acquired, and the acquired image data is transmitted to the supervised image classification server 103. When a candidate image registration operation is performed from the operation terminal PC 101, the image data is transmitted to the image database 105.

  The supervised image classification server 103 is a server for performing image classification and learning of an image classifier, and is constructed on a network. When the supervised image classification server 103 receives image data to be classified from the operation terminal PC 101, the supervised image classification server 103 transmits the classification result. When image data for classifier learning is received from the operation terminal PC101, the classifier is learned using the received image data.

  The unsupervised image classification server 104 is a server for selecting candidate images for learning the image classifier in the supervised image classification server 103, and is constructed on a network. When the unsupervised image classification server 104 receives image data to be learned from the operation terminal PC 101, the unsupervised image classification server 104 transmits a similar image class similar to the image data.

  The image database 105 is a server that stores and manages image data based on the operation of the application server 102, and is constructed on a network. The image database 105 stores image management record information in addition to image data.

  Hereinafter, the hardware configuration of the information processing apparatus applicable to the operation terminal PC 101, the application server 102, the supervised image classification server 103, the unsupervised image classification server 104, and the image database 105 shown in FIG. To do.

  FIG. 2 is a block diagram showing a hardware configuration of an information processing apparatus applicable to the operation terminal PC 101, application server 102, supervised image classification server 103, unsupervised image classification server 104, and image database 105 shown in FIG. is there.

  As shown in FIG. 2, in the information processing apparatus, a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, a storage device 204, an input controller 205, An audio input controller 206, a video controller 207, a memory controller 208, and a communication I / F controller 209 are connected.

  The CPU 201 comprehensively controls each device and controller connected to the system bus 200.

  The ROM 202 or the storage device 204 is a control program executed by the CPU 201 such as a basic input / output system (BIOS) or an operating system (OS), a computer-readable program for realizing this information processing method, and various necessary programs. Holds data (including data table).

  The RAM 203 functions as a main memory, work area, and the like for the CPU 201. The CPU 201 implements various operations by loading a program or the like necessary for executing the processing from the ROM 202 or the storage device 204 to the RAM 203 and executing the loaded program.

  The input controller 205 controls input from an input device such as a keyboard / touch panel 210. The input device is not limited to this, and may be a touch panel capable of detecting a position touched by a plurality of fingers, such as a mouse or a multi-touch screen.

  Various instructions can be performed when the user presses (touches with a finger or the like) in accordance with an icon, a cursor, or a button displayed on the touch panel.

  This input device is used to input a destination for a communication destination that can be used by various communication devices.

  The voice input controller 206 controls input from the microphone 211. Voice input from the microphone 211 can be recognized.

  The video controller 207 controls display on an external output device such as the display 212. The display includes a display of a notebook computer integrated with the main body. The external output device is not limited to a display, and may be a projector, for example. Further, for devices that can be received by the touch operation described above, it is also possible to receive input from the keyboard / touch panel 210.

  Note that the video controller 207 can control a video memory (VRAM) for display control, and a part of the RAM 203 can be used as a video memory area, or a dedicated video memory can be provided separately. Is possible.

  In the present invention, the first video memory area used for display when the user normally uses the information processing apparatus and the display overlaid on the display content of the first video memory area when a predetermined screen is displayed. A second video memory area used in The number of video memory areas is not limited to two, and a plurality of video memory areas can be provided as long as the resources of the information processing apparatus permit.

  The memory controller 208 controls access to the external memory 213. The external memory is connected via an adapter to an external storage device (hard disk), flexible disk (FD), or PCMCIA card slot that stores boot programs, various applications, font data, user files, editing files, and various data. A compact flash (registered trademark) memory or the like can be used.

  The communication I / F controller 209 is connected to and communicates with an external device via the network 214, and executes communication control processing in the network. For example, communication using TCP / IP, telephone lines such as ISDN, and communication using 3G lines of mobile phones are possible.

  The storage device 204 is a medium for permanently storing information, and the form is not limited to a storage device such as a hard disk. For example, a medium such as SSD (Solid State Drive) may be used.

  It can also be used as a temporary memory area during various processes performed by the communication terminal in the present embodiment.

  FIG. 3 is a functional block diagram for explaining the functions of the operation terminal PC 101, application server 102, supervised image classification server 103, unsupervised image classification server 104, and image database 105 shown in FIG.

In the information processing system, operation terminal PC101 mainly displays image classification and classifier learning operations and processing result display, application server 102 controls image classification processing and classifier learning processing, and supervised image classification server 103 performs image classification. The classifier learning is performed, the unsupervised image classification server 104 executes image candidate selection used for classifier learning, and the image database 105 manages image data used for classifier learning.
The operation terminal PC 101 has an operation result display unit 301.

  The operation result display unit 301 is realized by a web management screen that accesses a dedicated URL from a web browser. The operation result display unit 301 receives display data by transmitting an operation classification display request input to the application server 102, and receives image classification information. Is displayed. In addition, on the web management screen, in addition to image classification information, it is possible to confirm classifier learning operation, confirmation of candidate images used for classifier learning, and the like.

  The application server 102 includes an operation control unit 311, an image classification control unit 312, and a classifier learning control unit 313.

  When the application server 102 receives a display request for each operation from the operation terminal PC 101, the operation control unit 311 analyzes the operation content. From the received request, the operation control unit 311 determines whether the operation is a classification display operation of an image, a learning operation of an image classifier, or a confirmation operation of a candidate image used for classification learning.

  If the image classification control unit 312 determines that the operation of the image classification display has been performed by the operation control unit 311, the image classification control unit 311 transmits the image data to be classified to the supervised image classification server 103, and the resulting classification information is displayed. Receive. The received classification information is transmitted to the operation terminal PC 101 via the operation control unit 311.

  If the classifier learning control unit 313 determines that the operation control unit 311 has confirmed the candidate image used for classification learning, the classifier learning control unit 313 uses the unsupervised image classification server 104 to select image data as a basis for selecting a candidate image. The image classification process is performed by transmission, and a similar image class is received. The received similar image class is transmitted to the operation terminal PC 101 via the operation control unit 311. When the operation control unit 311 determines that a learning operation to the supervised image classification server 103 has been performed, the corresponding image data is acquired from the image database 105 based on the similar image class, and the acquired image data and the similar image class are determined. The data for classifier learning is transmitted to the supervised image classification server 103, and the classifier learning process is executed. It is also possible to perform a learning operation on the unsupervised image classification server 104. In this case, image data received from the operation terminal PC 101 via the operation control unit 311 is used as learning data, and the unsupervised image classification server 104 is used. Send to.

  The supervised image classification server 103 includes a data transmission / reception control unit 321, an image classification analysis unit 322, a classifier control unit 323, and a supervised learning control unit 324.

  The supervised image classification server 103 receives image data subject to image classification or classifier learning data from the application server 102. The data transmission / reception control unit 321 determines whether the received data is the image classification target image data or the classifier learning data.

  When the data transmission / reception control unit 321 determines that the image classification target image data has been received, the image classification analysis unit 322 performs image classification analysis. In the image classification analysis, image data to be classified is input to the classifier control unit 323, and the classification result is acquired. The acquired result is transmitted to the application server 102 via the data transmission / reception control unit 321.

  The classifier control unit 323 outputs a classification result when image data is input from the image classification analysis unit 322. When classifier learning data is input from the supervised learning control unit 324, supervised learning of the classifier is performed.

  If the supervised learning control unit 324 determines that the data transmission / reception control unit 321 has received classifier learning data, the supervised learning control unit 324 performs supervised learning of the classifier. In supervised learning of the classifier, learning data is input to the classifier control unit 323 to learn the classifier.

  The unsupervised image classification server 104 includes a data transmission / reception control unit 331, an image selection control unit 332, a classifier control unit 333, and an unsupervised learning control unit 334.

  The unsupervised image classification server 104 receives, from the application server 102, image data and classifier learning data as a basis for selecting candidate images for supervised learning data. The data transmitting / receiving unit 331 determines whether the received data is image data that is a basis for selecting a candidate image or classifier learning data.

  The image selection control unit 332 performs candidate image selection processing when the data transmission / reception control unit 331 determines that it has received image data as a basis for selecting candidate images for supervised learning data. In selecting a candidate image, image data of the selection base is input to the classification control unit 333, and a similar image class as a result is acquired. The acquired class information is transmitted to the application server 102 via the data transmission / reception control unit 331.

  When image data is input from the image selection control unit 332, the classifier control unit 333 outputs a similar image class of the image. When the classifier learning data is input from the unsupervised learning control unit 334, the classifier performs unsupervised learning.

  When the data transmission / reception control unit 321 determines that the classifier learning data has been received, the unsupervised learning control unit 334 performs unsupervised learning of the classifier. In unsupervised learning of the classifier, learning data is input to the classifier control unit 333, and the classifier is trained.

  The image database 105 includes a data transmission / reception control unit 341, a record management control unit 342, an image data storage area 343, and a management record storage area 344.

  The image database 105 receives a request for registration or acquisition of image data and management information update from the application server 102. The data transmission / reception unit 341 determines whether the received data is an image data registration or acquisition request or a management information update request.

  The record management control unit 342 performs registration / acquisition of image data and update of management information according to the determination contents of the data transmission / reception unit 341. If it is determined that the image data is registered, the image data is stored in the image data storage area 343 and new management record information is created in the management record storage area 344. If it is determined that the image data is to be acquired, the image data that meets the conditions is searched from the management record storage area 344, the image data is acquired from the image data storage area 343 based on the search result, and the data transmission / reception control unit 341 is executed. The acquired image data is transmitted to the application server 102. When it is determined that the management information is updated, the corresponding management record is updated from the management record storage area 344.

  Next, the learning image selection processing for the supervised image classifier using the unsupervised image classifier will be described using the flowchart shown in FIG.

  Although the application server 102 is not shown in FIG. 4, the communication between the operation terminal PC101 and the supervised image classification server 103 and the communication between the operation terminal PC101 and the unsupervised image classification server 104 are as follows. The application server 102 is interposed.

  In step S <b> 401, the CPU 201 of the operation terminal PC 101 receives a verification image according to an instruction from the user, and transmits the image to the supervised image classification server 103.

  The verification image is image data for verifying the accuracy of image classification.

  In step S402, the CPU 201 of the supervised image classification server 103 acquires the verification image transmitted from the operation terminal PC 101 in step S401.

  In step S403, the CPU 201 of the supervised image classification server 103 performs classification estimation on the image acquired in step S402.

  Classification estimation uses a supervised image classifier. The supervised image classifier uses a known technique such as CNN (Convolutional Neural Network).

  In step S404, the CPU 201 of the supervised image classification server 103 transmits the result of the process in step S403 to the operation terminal PC101.

  In step S405, the CPU 201 of the operation terminal PC101 receives the result transmitted from the supervised image classification server 103 and displays it on the display unit.

  In step S406, the CPU 201 of the operation terminal PC101 receives an evaluation on the result received in step S405 from the user. Specifically, selection of whether or not the classification estimation result is appropriate is accepted via the screen shown in FIG.

  In step S407, the CPU 201 of the operation terminal PC 101 determines whether the evaluation received in step S406 is an evaluation that the classification estimation result is appropriate.

  In the case of evaluation that the classification estimation result is appropriate (step S407: YES), the process of this flowchart is terminated.

  In the case of evaluation that the classification estimation result is not appropriate (step S407: NO), the process proceeds to step S408.

  In step S <b> 408, the CPU 201 of the operation terminal PC 101 transmits an instruction for extracting an image used for learning of the supervised image classifier to the unsupervised image classification server 104.

  In step S409, the CPU 201 of the unsupervised image classification server 104 extracts image candidates used for learning of the supervised image classifier from the image database 105. The candidate images are extracted using an unsupervised image classifier and an image similar to the verification image is extracted. The feature used for determining as a similar image may be other features representing similarity in addition to color, texture, shape, and the like. Assume that an unsupervised image classifier is learned in advance so that images can be automatically extracted from these features. The unsupervised image classifier uses a known technique such as SOM (Self Organizing Map).

  Details of the processing in step S409 will be described later with reference to FIG.

  In step S410, the CPU 201 of the unsupervised image classification server 104 transmits the candidate image data extracted in step S409 to the operation terminal PC101.

  In step S411, the CPU 201 of the operation terminal PC101 receives the image transmitted from the unsupervised image classification server 104 in step S410 and displays it on the display unit. An example of the screen displayed in this step is shown in FIG.

  In step S412, the CPU 201 of the operation terminal PC101 creates teacher data in accordance with a user instruction. Specific processing will be described later with reference to FIG.

  In step S413, the CPU 201 of the operation terminal PC101 transmits the teacher data created in step S412 to the supervised image classification server 103.

  In step S414, the CPU 201 of the supervised image classification server 103 receives the teacher data transmitted from the operation terminal PC 101 in step S413, and inputs the supervised data to the supervised classifier for learning.

  In step S415, the CPU 201 of the supervised image classification server 103 records the image data used for learning in step S414 in the management record of the image database 105 so as to be removed from the learning image candidates. By recording what has been used for learning, it is excluded from the extraction target of candidate images for subsequent learning, and is prevented from being reused by mistake for learning.

  Then, the process returns to step S403. The reason why the process returns to step S403 is to verify the effect of learning in step S414. Therefore, in the process of the next step S403, the classification estimation process is executed again using the same verification image.

  Next, with reference to FIG. 5, the similar image acquisition process (step S409) of the present embodiment will be described.

  In step S501, the CPU 201 of the unsupervised image classification server 104 calculates the feature amount of the verification image. The feature amount to be calculated may be a color, a texture, a shape, or the like, or another feature representing image similarity. These are set by the user.

  For example, the feature amount of the verification image is calculated by weighting the color feature, and a similar image is extracted using the feature amount, so that the supervised image classifier learns the classification based on the color feature. Is possible. In this way, which feature is used is determined depending on what kind of learning is desired for the supervised image classifier.

  In step S502, the CPU 201 of the unsupervised image classification server 104 classifies the verification image into a similar image class using an unsupervised image learner based on the feature amount calculated in step S501. The similar image class is information for collecting similar images according to the feature amounts of the images, and images determined to be similar belong to the same class. This information is assumed to be classified and recorded in advance as similar image class information in the management record of each image data stored in the image database 105. A description example of the management record will be described later with reference to FIG.

  The unsupervised image classifier used for classification of similar image classes may be any classifier based on unsupervised learning such as SOM.

  In step S503, the CPU 201 of the unsupervised image classification server 104 acquires image data belonging to the same class from the image database 105 based on the similar image class classified in step S502, and creates a candidate image list. The candidate image list is composed of a list in which each image data and management record are paired.

  In addition to image data belonging to the same class, image data of a class having a higher degree of similarity than the threshold is listed as a candidate image. By this process, in FIG. 7, it is possible to display only images belonging to the same similar image class or display candidate images including similar image classes whose similarity is closer than the threshold.

  In this embodiment, the candidate images are extracted based on the similar image class information. However, other information (for example, image category information) is given to each image in advance, and a condition based on a combination thereof is used. May be extracted.

  In step S504, the CPU 201 of the unsupervised image classification server 104 determines whether each image data registered in the candidate image list created in step S503 is already used for learning of the supervised classifier.

  It is assumed that information indicating whether or not it has been used for learning is recorded in a management record of each image data (FIG. 6).

  If it is already used for learning (step S504: YES), the process proceeds to step S505.

  If learning is not being performed (step S504: NO), the process of this flowchart is terminated, and the process proceeds to step S410.

In step S505, the CPU 201 of the unsupervised image classification server 104 deletes each image data determined to be used for learning in step S504 from the candidate image list.
Then, the process proceeds to step S410.

  As described above, in the present invention, in learning for improving the classification accuracy of the supervised image classifier, an image similar to the verification image is extracted by the unsupervised image classifier.

  By such processing, it is possible to extract an image suitable for improving the classification accuracy of the supervised image classifier from a large number of candidate images.

  Specifically, it is possible to increase the recognition accuracy of an automobile of the form by learning an image of an automobile having a form similar to the automobile for an “automobile” having a form with poor recognition accuracy. As a result, various forms of automobiles can be recognized, and the accuracy of image classification can be improved.

  In addition, since the user selects a learning image from the extracted images, it is possible to improve the efficiency of the work of selecting the learning image.

  Next, a data configuration example of the management record information table for image data will be described with reference to FIG.

  FIG. 6 is a data configuration example of a management record for managing the image data recorded in the image database 105.

  FIG. 6A is a data configuration example that represents a similar image class to which image data belongs.

  An image ID, which is information for identifying image data, and a node ID for identifying a similar image class (node) to which the image data belongs are registered in association with each other.

  Image data is managed by associating similar images with each other based on similar image class information. If they belong to the same similar image class, those images are handled as similar objects.

  FIG. 6B is an example of a data configuration that represents a use situation of supervised image classifier learning of image data.

  Information indicating whether or not the image ID has been learned is registered in association with each other. In the example of FIG. 6B, if true is registered, it means that learning has been completed, and if false is registered, it means that learning has not been completed.

  FIG. 6C is a data configuration example that represents similar image class information of image data.

  A node ID, which is information for identifying a similar image class, and information (features A to Z) indicating features of image data belonging to the similar image class are registered in association with each other.

  The feature in FIG. 6C is a feature amount that serves as an index when classification is performed by an unsupervised image classifier. The number of feature quantities is arbitrary. The feature value may be either an average value of image data belonging to the same similar image class or a value calculated from another index. This feature quantity may be used as a classification dictionary depending on the type of unsupervised image classifier.

  Next, the screen displayed on the operation terminal PC 101 in step S411 will be described with reference to FIG.

  The screen shown in FIG. 7 is a screen that accepts an instruction to create teacher data from the user. Teacher data for use in learning of the supervised image classifier is created according to the information input on the screen of FIG.

  The screen shown in FIG. 7 accepts an area 701 for displaying the verification image received in step S401, an area 702 for displaying the classification estimation result for the image, and a selection of a label given to the verification image. An area 703, an area 704 for accepting designation of an image display range, an area 705 for displaying an image extracted by the processing in step S409, and the like.

  In a region 703 for receiving a label selection, selection of a correct classification result of the verification image is received. For example, in the example of FIG. 7, it is shown that the labels “automobile” and “passenger car” are selected. Labels that can be specified can be classified by a supervised classifier. In addition to the single designation, a plurality of designations can be designated.

  Also, an image displayed in the image display area and checked is learned as an image of the label selected in the area 703.

  An image display range designation area 704 can designate a range of an image displayed in the image display area 705.

  Candidate images are selected based on similar image class information that is the result of unsupervised image classification of verification images.

  At that time, according to the conditions specified in the range specification area 704, for example, only images belonging to the same similar image class are displayed, or candidate images including similar image classes whose similarity is closer than the threshold are displayed. To do.

  The image display area 705 displays a list of candidate images used for learning of the supervised image classifier according to the content specified in the display range specification 704. An image to be used for learning is designated by selecting an image (by being checked). When the collective selection button 706 is pressed, all displayed images are selected. When the learning execution button 707 is pressed after the designation, the label selected in the region 703 is assigned to the selected image, and learning of the supervised image classifier is executed.

  FIG. 8 is an example of a screen displayed on the operation terminal PC 101 in step S405.

  Through the screen shown in FIG. 8, an instruction is received from the user as to whether or not the classification estimation result is appropriate and whether learning is necessary (S406).

  As shown in FIG. 8, the verification image 801 that has received the input in step S401 and the classification estimation result 802 are displayed.

  When the appropriate button 803 is pressed, a result indicating that the classification estimation result is appropriate is accepted (step S407: YES). When the learning button 804 is pressed, a result indicating that the classification estimation result is not appropriate is received (step S407: NO).

  The program in the present invention is a program that causes a computer to execute the processes of FIGS. The program in the present invention may be a program for each process in FIGS. 4 and 5.

  As described above, a recording medium that records a program that implements the functions of the above-described embodiments is supplied to a system or apparatus, and a computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by reading and executing.

  In this case, the program itself read from the recording medium realizes the novel function of the present invention, and the recording medium recording the program constitutes the present invention.

  As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, nonvolatile memory card, ROM, EEPROM, silicon A disk or the like can be used.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on an instruction of the program is actually It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the processing and the processing is included.

  Furthermore, after the program read from the recording medium is written to the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is based on the instructions of the program code. It goes without saying that the case where the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  The present invention may be applied to a system constituted by a plurality of devices or an apparatus constituted by a single device. Needless to say, the present invention can be applied to a case where the present invention is achieved by supplying a program to a system or apparatus. In this case, by reading a recording medium storing a program for achieving the present invention into the system or apparatus, the system or apparatus can enjoy the effects of the present invention.

  Furthermore, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or apparatus can enjoy the effects of the present invention. In addition, all the structures which combined each embodiment mentioned above and its modification are also included in this invention.

101 Operation terminal PC
102 application server 103 supervised image classification server 104 unsupervised image classification server 105 image database

Claims (8)

Image input means for inputting a classification target image to be classified by the first classifier ;
Similar image acquisition means for acquiring an image similar to the classification target image classified by the first classifier ;
And the teacher data generating means for generating teaching data by accepting the label against the image obtained by the similar image acquisition unit,
Learning means for learning the first classifier using the teacher data created by the teacher data creating means;
An information processing system comprising:   An evaluation receiving means for receiving an evaluation result for the classification result by the first classifier;
  The similar image acquisition means acquires an image similar to the classification target image classified by the first classifier when the evaluation result received by the evaluation reception means is a result indicating that learning is necessary. The information processing system according to claim 1.   Of the images acquired by the similar image acquisition means, further comprising a selection reception means for receiving selection of an image used for learning by the learning means,
  The information processing system according to claim 1, wherein the teacher data creating unit creates the teacher data by receiving a label for the image selected by the selection receiving unit.   Display control means for displaying a list of images acquired by the similar image acquisition means;
  The information processing system according to claim 3, wherein the selection receiving unit receives a selection of an image used for learning by receiving a selection operation on the image displayed by the display control unit.   5. The display control unit further displays the classification target image classified by the first classifier on a screen on which a list of images acquired by the similar image acquisition unit is displayed. Information processing system described in 1.   The image acquired by the similar image acquisition unit is an image similar to the classification target image, and is an image extracted by an unsupervised image classifier. Information processing system described in 1. An image input step in which the image input means of the information processing system inputs a classification target image to be classified by the first classifier ;
Similar image acquisition unit of the information processing system, a similar image acquisition step of acquiring an image similar to the classified grouping target image by the first classifier,
Teaching data creation means of the information processing system, and the teacher data creating step of creating teacher data by accepting the label against the image obtained by the similar image acquisition step,
A learning step in which learning means of the information processing system learns the first classifier using the teacher data created by the teacher data creation step;
An information processing method comprising: Information processing device
Image input means for inputting a classification target image to be classified by the first classifier ;
Similar image acquisition means for acquiring an image similar to the classification target image classified by the first classifier ;
And the teacher data generating means for generating teaching data by accepting the label against the image obtained by the similar image acquisition unit,
A program for causing a teacher to learn the first classifier using the teacher data created by the teacher data creating means.

Images