JP2019109924A - Information processing system, information processing method, and program - Google Patents

Abstract

To provide an information processing system, an information processing method, and a program thereof which can select an image as a candidate for additional learning for the purpose of feedback to an error detection result in an image recognition.SOLUTION: The present invention is directed to an information processing system for classifying verification images by a supervised image classification device. The verification images are input, and an image similar to the input verification images is extracted. The extracted image is provided with a label to thereby generate supervised data. With the generated supervised data, the supervised image classification device carries out learning.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an information processing system, an information processing method, and a program.

  In recent years, deep learning has been actively researched as an image recognition technology by supervised learning. Deep learning is one of machine learning in which a conventional neural network is stacked in multiple layers, and solves problems such as image classification by learning in advance using images for learning.

  In order to improve the accuracy in image recognition by deep learning, it is important to secure and select images for learning. A large number of images and their content information must be prepared as a data set in order to be used as preliminary learning and additional learning for improving recognition accuracy. For example, in the case of a data set of "cat", it is necessary to prepare several hundred sets or more of an image showing a cat and its content information (such as a product name or the like) as a pair.

  However, preparing the above-mentioned data set is often very expensive. The reason is that it is necessary to prepare a large number of images, to associate their content information, and to basically associate the content information manually.

  From the above, securing and selecting images for deep learning in advance / additional learning has become one of the major issues, and reduction of their creation costs is strongly desired.

  Patent Document 1 describes a technique for selecting additional image candidates in order to improve the quality of a learning dictionary for use in image / image similarity search.

JP, 2013-250881, A

  In the technique of Patent Document 1, the similarity is calculated from the average feature amount of the image group which is a learning dictionary and the feature amount of the candidate image, and when the similarity is larger than a threshold, it is selected as an additional image.

  According to this method, the number of images in the learning dictionary can be increased without changing the tendency of the characteristic amount originally possessed by the learning dictionary, but an image having a different tendency from the average characteristic amount can not be selected as a candidate .

  Therefore, it can not be applied to a system that performs feedback to a false detection result due to a difference in the tendency of feature amounts when performing image recognition using deep learning or the like.

  Therefore, in the present invention, it is an object of the present invention to provide a mechanism capable of selecting an image to be a candidate for additional learning for the purpose of feedback to a false detection result in image recognition.

  The information processing system according to the present invention is an information processing system that classifies a verification image by a supervised image classifier, and includes an image input unit that inputs a verification image, and the verification input by the image input unit. Similar image extraction means for extracting an image similar to an image, teacher data creation means for creating teacher data by giving a label to the image extracted by the similar image extraction means, and the teacher data creation means And learning means for learning the supervised image classifier using teacher data.

  The information processing method according to the present invention is an information processing method in an information processing system for classifying a verification image by a supervised image classifier, and includes an image input step of inputting a verification image and the image input step. A similar image extracting step of extracting an image similar to the input verification image; a teacher data creating step of creating teacher data by applying a label to the image extracted by the similar image extracting step; And a learning step of training the supervised image classifier using the teacher data created by the step.

  Further, a program according to the present invention is a program executable in an information processing system for classifying a verification image by a supervised image classifier, and the information processing system comprises an image input unit for inputting the verification image A similar image extracting unit that extracts an image similar to the verification image input by the image input unit; and a teacher data creating unit that creates teacher data by attaching a label to the image extracted by the similar image extracting unit And using the teacher data created by the teacher data creation means as a learning means for making the supervised image classifier learn.

  According to the present invention, it is possible to select an image to be a candidate for additional learning for the purpose of feedback to a false detection result in image recognition.

The figure which shows the structure of the information processing system of this invention A block diagram showing a hardware configuration of an information processing apparatus applicable to the operation terminal PC 101, the application server 102, the supervised image classification server 103, the unsupervised image classification server 104, and the image database 105 shown in FIG. Functional block diagram for explaining the functions of the operation terminal PC 101, the application server 102, the supervised image classification server 103, the unsupervised image classification server 104, and the image database 105 Flow chart showing selection process of learning image to supervised image classifier using unsupervised image classifier Flowchart showing similar image acquisition process (step S409) Data configuration example of management record information table of image data An example of a screen displayed on operation terminal PC 101 in step S411 An example of a screen displayed on operation terminal PC 101 in step S405

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a system configuration diagram showing an example of the configuration of an information processing system to which the additional learning image selection support function of the present invention is applied.

  The information processing system includes one or more operation terminals PC 101, one or more application servers 102, one or more supervised image classification servers 103, one or more unsupervised image classification servers 104, one or more image databases 105 Are connected via a network.

  In the present embodiment, the application server 102, the supervised image classification server 103, the supervised image classification server 104, and the database 105 are described as separate cases respectively. You may comprise so that a function may be provided.

  The operation terminal PC 101 is a terminal for performing operations of image classification, image classifier learning, and candidate image registration, and exists on the network. A web operation screen (operation on a web browser) is used for image classification and learning operations, and connection to the application server 102 is made via a network.

  The application server 102 is a server for controlling image classification, image classifier learning, and candidate image registration, and is built on a network. The operation terminal PC 101 is connected to the application server 102. When the image classification operation is performed from the operation terminal PC 101, the application server 102 transmits an image to the supervised image classification server 103, and receives the classification result. When the learning operation of the classifier is performed from the operation terminal PC 101, the image is transmitted to the unsupervised image classification server 104, the similar image class to be the learning candidate is received, and the image database 105 is generated based on the class information. The image data to be used for learning is acquired, and the acquired image data is transmitted to the supervised image classification server 103. When a candidate image registration operation is performed from the operation terminal PC 101, image data is transmitted to the image database 105.

  The supervised image classification server 103 is a server for performing image classification and image classifier learning, and is built on a network. When the supervised image classification server 103 receives image data to be classified from the operation terminal PC 101, the supervised image classification server 103 transmits the classification result. When image data for classifier learning is received from the operation terminal PC 101, the classifier is learned using the received image data.

  The unsupervised image classification server 104 is a server for selecting candidate images for learning the image classifier in the supervised image classification server 103, and is constructed on a network. When receiving image data to be learned from the operation terminal PC 101, the unsupervised image classification server 104 transmits a similar image class similar to the image data.

  The image database 105 is a server that stores and manages image data based on the operation of the application server 102, and is built on a network. The image database 105 stores management record information as well as image data.

  Hereinafter, the hardware configuration of the information processing apparatus applicable to the operation terminal PC 101, the application server 102, the supervised image classification server 103, the unsupervised image classification server 104, and the image database 105 shown in FIG. Do.

  FIG. 2 is a block diagram showing a hardware configuration of an information processing apparatus applicable to the operation terminal PC 101, the application server 102, the supervised image classification server 103, the unsupervised image classification server 104, and the image database 105 shown in FIG. is there.

  As shown in FIG. 2, in the information processing apparatus, a central processing unit (CPU) 201, a read only memory (ROM) 202, a random access memory (RAM) 203, a storage device 204, an input controller 205, and a system bus 200. An audio input controller 206, a video controller 207, a memory controller 208, and a communication I / F controller 209 are connected.

  The CPU 201 centrally controls the devices and controllers connected to the system bus 200.

  The ROM 202 or the storage device 204 is a control program executed by the CPU 201, such as a BIOS (Basic Input / Output System) or an OS (Operating System), a computer readable and executable program for realizing the information processing method, and various necessary items. It holds data (including data tables).

  The RAM 203 functions as a main memory, a work area, and the like of the CPU 201. The CPU 201 loads programs and the like necessary for execution of processing from the ROM 202 or the storage device 204 to the RAM 203, and implements various operations by executing the loaded programs.

  The input controller 205 controls an input from an input device such as the keyboard / touch panel 210. The input device is not limited to this, and may be a touch panel capable of detecting a position touched with a plurality of fingers, such as a mouse or a multi touch screen.

  Various instructions can be given by the user pressing (touching with a finger or the like) according to an icon, a cursor or a button displayed on the touch panel.

  This input device is used to input a destination for a communication destination that can be used by various communication devices.

  The voice input controller 206 controls an input from the microphone 211. It is possible to perform voice recognition of the voice input from the microphone 211.

  The video controller 207 controls display on an external output device such as the display 212. The display shall also include the display of a notebook computer integrated with the main unit. The external output device is not limited to the display, and may be, for example, a projector. Further, with regard to devices that can be received by the above-described touch operation, it is also possible to receive an input from the keyboard / touch panel 210.

  The video controller 207 can control a video memory (VRAM) for display control, and can use a part of the RAM 203 as a video memory area, or can separately provide a dedicated video memory. It is possible.

  In the present invention, the first video memory area used for display when the user normally operates the information processing apparatus and the display content superimposed on the display content of the first video memory area when the predetermined screen is displayed And a second video memory area used for The video memory area is not limited to two, and can be plural as long as the resource of the information processing apparatus permits.

  The memory controller 208 controls access to the external memory 213. The external memory is connected via an adapter to an external storage device (hard disk) that stores a boot program, various applications, font data, user files, editing files, various data, etc., a flexible disk (FD), or a PCMCIA card slot. Compact flash (registered trademark) memory etc. can be used.

  The communication I / F controller 209 connects and communicates with an external device via the network 214, and executes communication control processing in the network. For example, communications using TCP / IP, telephone lines such as ISDN, and 3G lines of mobile phones are possible.

  The storage device 204 is a medium for permanently storing information, and the form thereof is not limited to a storage device such as a hard disk. For example, a medium such as a solid state drive (SSD) may be used.

  Moreover, it can also be used as a temporary memory area at the time of various processes performed by the communication terminal in the present embodiment.

  FIG. 3 is a functional block diagram for explaining the functions of the operation terminal PC 101, the application server 102, the supervised image classification server 103, the unsupervised image classification server 104, and the image database 105 shown in FIG.

In the information processing system, the operation terminal PC 101 mainly displays operations of image classification and classifier learning and displays processing results, the application server 102 controls image classification processing and classifier learning processing, and the supervised image classification server 103 image classification. And, execution of classifier learning, unsupervised image classification server 104 executes selection of image candidates used for classifier learning, and image database 105 manages image data used for classifier learning.
The operation terminal PC 101 has an operation result display unit 301.

  The operation result display unit 301 is realized by a web management screen for accessing a dedicated URL from a web browser, and transmits display request for classification of an image input by operation to the application server 102 to receive display data, and image classification information Display In addition to the image classification information, the web management screen can confirm classifier learning operation, confirmation of candidate images used for classifier learning, and the like.

  The application server 102 includes an operation control unit 311, an image classification control unit 312, and a classifier learning control unit 313.

  When the application server 102 receives a display request for each operation from the operation terminal PC 101, the operation control unit 311 analyzes the contents of the operation. From the received request, the operation control unit 311 determines whether the operation is the display operation of the image classification, the learning operation of the image classifier, or the confirmation operation of the candidate image used for the classification learning.

  When the image classification control unit 312 determines that the operation of the classification display of the image is performed by the operation control unit 311, the image classification control unit 312 transmits the image data of the image classification target to the supervised image classification server 103, and the classification information resulting therefrom. To receive. The received classification information is transmitted to the operation terminal PC 101 via the operation control unit 311.

  If the classifier learning control unit 313 determines that the confirmation operation of the candidate image used for classification learning has been performed by the operation control unit 311, the image data serving as a basis for selecting candidate images to the unsupervised image classification server 104 is used. Send to perform image classification processing and receive similar image class. The received similar image class is transmitted to the operation terminal PC 101 via the operation control unit 311. If the operation control unit 311 determines that a supervised learning operation to the image classification server 103 is performed, the corresponding image data is acquired from the image database 105 based on the similar image class, and the acquired image data and the similar image class are The data is transmitted to the supervised image classification server 103 as classifier learning data, and the learning process of the classifier is executed. In addition, it is possible to perform a learning operation on the unsupervised image classification server 104. In this case, the unsupervised image classification server 104 uses the image data received from the operation terminal PC 101 via the operation control unit 311 as learning data. Send to

  The supervised image classification server 103 includes a data transmission / reception control unit 321, an image classification analysis unit 322, a classifier control unit 323, and a supervised learning control unit 324.

  The supervised image classification server 103 receives image data as a target of image classification or data for classifier learning from the application server 102. The data transmission / reception control unit 321 determines whether the received data is image data as an image classification target or classifier learning data.

  The image classification analysis unit 322 performs image classification analysis when it is determined by the data transmission / reception control unit 321 that the image data of the image classification target has been received. In image classification analysis, image data to be classified is input to the classifier control unit 323, and the classification result is acquired. The acquired result is transmitted to the application server 102 via the data transmission / reception control unit 321.

  When the image data is input from the image classification analysis unit 322, the classifier control unit 323 outputs the classification result. When data for classifier learning is input from the supervised learning control unit 324, supervised learning of the classifier is performed.

  When the supervised learning control unit 324 determines that the data transmission / reception control unit 321 has received the classifier learning data, the supervised learning control unit 324 performs supervised learning of the classifier. In supervised learning of a classifier, learning data is input to the classifier control unit 323 to make the classifier learn.

  The unsupervised image classification server 104 includes a data transmission / reception control unit 331, an image selection control unit 332, a classifier control unit 333, and an unsupervised learning control unit 334.

  The unsupervised image classification server 104 receives, from the application server 102, image data serving as a basis for selecting candidate images of supervised learning data, and classifier learning data. The data transmitting / receiving unit 331 determines whether the received data is image data to be a basis of candidate image selection or classifier learning data.

  The image selection control unit 332 performs candidate image selection processing when it is determined by the data transmission / reception control unit 331 that image data serving as a basis for selecting candidate images of supervised learning data has been received. In the selection of candidate images, the image data of the selection base is input to the classification control unit 333, and the similar image class as a result is acquired. The acquired class information is transmitted to the application server 102 via the data transmission / reception control unit 331.

  When image data is input from the image selection control unit 332, the classifier control unit 333 outputs a similar image class of the image. When classifier learning data is input from the unsupervised learning control unit 334, unsupervised learning of the classifier is performed.

  The unsupervised learning control unit 334 performs unsupervised learning of the classifier when the data transmission / reception control unit 321 determines that the classifier learning data is received. In unsupervised learning of a classifier, learning data is input to the classifier control unit 333 to learn the classifier.

  The image database 105 includes a data transmission / reception control unit 341, a record management control unit 342, an image data storage area 343, and a management record storage area 344.

  The image database 105 receives a request for registration or acquisition of image data and update of management information from the application server 102. The data transmission / reception unit 341 determines whether the received data is an image data registration or acquisition request or a management information update request.

  The record management control unit 342 registers and acquires image data, and updates management information according to the determination content of the data transmission / reception unit 341. When it is determined that the image data is registered, the image data is stored in the image data storage area 343, and new management record information is created in the management record storage area 344. When it is determined that the image data is to be acquired, the management record storage area 344 is searched for the image data meeting the conditions, the image data is acquired from the image data storage area 343 based on the search result, and the data transmission / reception control unit 341 The acquired image data is transmitted to the application server 102 via the interface. If it is determined that the management information is updated, the corresponding management record is updated from the management record storage area 344.

  Next, processing for selecting a learning image to a supervised image classifier using an unsupervised image classifier will be described using the flowchart shown in FIG.

  Although the application server 102 is not shown in FIG. 4, the communication between the operation terminal PC 101 and the supervised image classification server 103 and the communication between the operation terminal PC 101 and the unsupervised image classification server 104 are as follows. The application server 102 intervenes.

  In step S401, the CPU 201 of the operation terminal PC 101 receives the verification image according to an instruction from the user, and transmits the image to the supervised image classification server 103.

  The verification image is image data for performing accuracy verification of the image classification.

  In step S402, the CPU 201 of the supervised image classification server 103 acquires the verification image transmitted from the operation terminal PC 101 in step S401.

  In step S403, the CPU 201 of the supervised image classification server 103 performs classification estimation on the image acquired in step S402.

  Classification estimation uses a supervised image classifier. The supervised image classifier uses a known technology such as CNN (Convolutional Neural Network).

  In step S404, the CPU 201 of the supervised image classification server 103 transmits the processing result of step S403 to the operation terminal PC 101.

  In step S405, the CPU 201 of the operation terminal PC 101 receives the result transmitted from the supervised image classification server 103 and displays it on the display unit.

  In step S406, the CPU 201 of the operation terminal PC 101 receives an evaluation of the result received in step S405 from the user. Specifically, the selection of whether or not the classification estimation result is appropriate is accepted through the screen shown in FIG.

  In step S407, the CPU 201 of the operation terminal PC 101 determines whether the evaluation accepted in step S406 is an evaluation that the classification estimation result is appropriate.

  In the case of evaluation that the classification estimation result is appropriate (step S407: YES), the processing of this flowchart ends.

  In the case of the evaluation that the classification estimation result is not appropriate (step S407: NO), the process proceeds to step S408.

  In step S408, the CPU 201 of the operation terminal PC 101 transmits, to the unsupervised image classification server 104, an instruction to extract an image used for learning of the supervised image classifier.

  In step S409, the CPU 201 of the unsupervised image classification server 104 extracts image candidates used for learning of the supervised image classifier from the image database 105. The candidate image extraction uses an unsupervised image classifier to extract an image similar to the verification image. The features used to determine as similar images may be colors, textures, shapes, etc., as well as other features representing similarity. An unsupervised image classifier should be trained in advance so that images can be extracted automatically from those features. Note that the unsupervised image classifier uses a known technique such as SOM (Self Organizing Map).

  Details of the process of step S409 will be described later with reference to FIG.

  In step S410, the CPU 201 of the unsupervised image classification server 104 transmits the data of the candidate image extracted in step S409 to the operation terminal PC 101.

  In step S411, the CPU 201 of the operation terminal PC 101 receives the image transmitted from the unsupervised image classification server 104 in step S410, and displays the image on the display unit. An example of the screen displayed in this step is shown in FIG.

  In step S412, the CPU 201 of the operation terminal PC 101 creates teacher data in accordance with the user's instruction. Specific processing will be described later with reference to FIG.

  In step S413, the CPU 201 of the operation terminal PC 101 transmits the teacher data created in step S412 to the supervised image classification server 103.

  In step S414, the CPU 201 of the supervised image classification server 103 receives the teacher data transmitted from the operation terminal PC 101 in step S413, and inputs the teacher data to the supervised classifier for learning.

  In step S415, the CPU 201 of the supervised image classification server 103 records the image data used for learning in step S414 in the management record of the image database 105 so as to be removed from the learning image candidates. By recording what has been used for learning, it is excluded from the extraction targets of candidate images for learning from the next time on, and it is prevented from being erroneously reused for learning.

  Then, the process returns to step S403. The processing is returned to step S403 in order to verify the effect of learning in step S414. Therefore, in the process of the next step S403, the classification estimation process is executed again using the same verification image.

  Next, the similar image acquisition process (step S409) according to the present embodiment will be described with reference to FIG.

  In step S501, the CPU 201 of the unsupervised image classification server 104 calculates the feature amount of the verification image. The feature quantity to be calculated may be not only the color, the texture, the shape, etc., but also another feature representing the similarity of the image. These are set by the user.

  For example, the feature amount of the verification image is calculated by weighting the color feature, and the similar image is extracted using the feature amount to make the supervised image classifier learn the classification by the color feature. Is possible. In this way, which feature is used is determined depending on what kind of learning is desired for the supervised image classifier.

  In step S502, the CPU 201 of the unsupervised image classification server 104 classifies the image for verification into the similar image class using the unsupervised image learning device based on the feature quantity calculated in step S501. The similar image class is information for organizing similar images according to the feature amount of the image, and the images determined to be similar belong to the same class. It is assumed that this information is classified and recorded in advance as similar image class information in the management record of each image data stored in the image database 105. A description example of the management record will be described later in the description of FIG.

  The unsupervised image classifier used to classify similar image classes may be any classifier by unsupervised learning such as SOM.

  In step S503, the CPU 201 of the unsupervised image classification server 104 acquires image data belonging to the same class from the image database 105 based on the similar image class classified in step S502, and creates a candidate image list. The candidate image list is composed of a list in which each image data and management record are paired.

  Further, not only image data belonging to the same class, but also image data of a class having a higher degree of similarity than a threshold value is listed as a candidate image. By this processing, in FIG. 7, it is possible to display only images belonging to the same similar image class or to display candidate images including similar image classes whose similarity is closer than the threshold.

  Further, in the present embodiment, the candidate image is extracted based on the similar image class information, but other information (for example, image category information etc.) is added to each image in advance, and a condition combining these is May be extracted.

  In step S504, the CPU 201 of the unsupervised image classification server 104 determines whether each image data registered in the candidate image list created in step S503 is already used for learning of a supervised classifier.

  Information as to whether it has been used for learning is recorded in the management record of each image data (FIG. 6).

  If it has been used for learning (step S504: YES), the process proceeds to step S505.

  If learning is not desired (step S504: NO), the processing of this flowchart ends, and the process proceeds to the processing of step S410.

In step S505, the CPU 201 of the unsupervised image classification server 104 deletes each image data determined to be used for learning in step S504 from the candidate image list.
Then, the process proceeds to step S410.

  As described above, in the present invention, in learning for improving classification accuracy of a supervised image classifier, an image similar to the verification image is extracted by the unsupervised image classifier.

  Such a process makes it possible to extract an image suitable for improving the classification accuracy of the supervised image classifier from among a large number of candidate images.

  Specifically, it is possible to improve the recognition accuracy of the car of the form by learning the image of the car of the form similar to the car with respect to the “car” of the form of the recognition accuracy which is conventionally poor. As a result, it becomes possible to recognize various forms of vehicles and to improve the accuracy of image classification.

  In addition, since the user selects a learning image from the extracted images, it is possible to improve the efficiency of the work of selecting a learning image.

  Next, a data configuration example of a management record information table of image data will be described using FIG.

  FIG. 6 shows an example of the data configuration of a management record for managing image data stored in the image database 105. As shown in FIG.

  FIG. 6A shows an example of data configuration representing a similar image class to which image data belongs.

  An image ID, which is information for identifying image data, and a node ID for identifying a similar image class (node) to which the image data belongs are registered in association with each other.

  The image data is managed by associating similar images with similar image class information. If they belong to the same similar image class, those images are treated as similar ones.

  FIG. 6 (B) is an example of a data configuration representing a use situation of supervised image classifier learning of image data.

  The image ID and the information indicating whether it has been learned are registered in association with each other. In the example of FIG. 6B, it means that learning is completed if true is registered, and learning is not performed if false is registered.

  FIG. 6C shows an example of data configuration that expresses similar image class information of image data.

  A node ID, which is information for identifying a similar image class, and information (a feature A to a feature Z) indicating features of image data belonging to the similar image class are registered in association with each other.

  The feature in FIG. 6C is a feature that serves as an index when classifying with the unsupervised image classifier. The number of feature quantities is arbitrary. Further, the value of the feature amount may be either an average value of image data belonging to the same similar image class or a value calculated from another index. This feature may also be used as a classification dictionary, depending on the type of unsupervised image classifier.

  Next, the screen displayed on the operation terminal PC 101 in step S411 will be described using FIG.

  The screen shown in FIG. 7 is a screen for receiving an instruction of teacher data creation from the user. According to the information input on the screen of FIG. 7, creation of teacher data to be used for learning of the supervised image classifier is performed.

  The screen shown in FIG. 7 includes an area 701 for displaying the verification image received at step S401, an area 702 for displaying the classification estimation result for the image, and a selection of a label attached to the verification image. An area 703, an area 704 for receiving designation of an image display range, and an area 705 for displaying an image extracted by the process of step S409 are included.

  An area 703 for receiving the selection of the label receives the selection of the correct classification result of the verification image. For example, the example of FIG. 7 indicates that the labels "car" and "passenger car" are selected. Labels that can be specified are those that can be classified by a supervised classifier. In addition to single designation, multiple designations can also be made for the classification labels to be designated.

  Further, the image displayed in the image display area and checked is also learned as the image of the label selected in the area 703.

  The designated area 704 of the image display range can designate the range of the image displayed in the image display area 705.

  The candidate images are selected based on similar image class information which is a result of unsupervised image classification of the verification image.

  At that time, for example, only images belonging to the same similar image class may be displayed, or candidate images may be displayed including similar image classes whose similarity is closer than a threshold, according to the conditions specified in the range specification region 704. Do.

  The image display area 705 displays a list of candidate images used for learning of the supervised image classifier in accordance with the content designated by the designation of the display range 704. The image used for learning is designated (selected) by selecting the image. When the batch selection button 706 is pressed, all the displayed images are selected. After designation, when the learning execution button 707 is pressed, the label selected in the area 703 is given to the selected image, and learning of the supervised image classifier is executed.

  FIG. 8 is an example of a screen displayed on the operation terminal PC 101 in step S405.

  An instruction is received from the user via the screen shown in FIG. 8 as to whether or not the classification estimation result is appropriate and whether learning is necessary (S406).

  As shown in FIG. 8, the verification image 801 for which the input has been received in step S401 and the classification estimation result 802 are displayed.

  By pressing the appropriate button 803, a result indicating that the classification estimation result is appropriate is received (step S407: YES). By pressing the learning button 804, a result indicating that the classification estimation result is not appropriate is received (step S407: NO).

  Furthermore, a program according to the present invention is a program that causes a computer to execute the processing of FIGS. The program in the present invention may be a program for each process of FIGS. 4 and 5.

  As described above, the recording medium recording the program for realizing the functions of the above-described embodiments is supplied to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by reading and executing.

  In this case, the program itself read out from the recording medium realizes the novel function of the present invention, and the recording medium recording the program constitutes the present invention.

  As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, non-volatile memory card, ROM, EEPROM, silicon A disk etc. can be used.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiment are realized, but also an operating system (OS) or the like running on the computer is actually executed based on the instructions of the program. It goes without saying that the processing is partially or entirely performed, and the processing realizes the functions of the above-described embodiments.

  Furthermore, after the program read from the recording medium is written to the memory provided to the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is read based on the instruction of the program code. It goes without saying that the case where the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiment are realized by the processing.

  Further, the present invention may be applied to a system constituted by a plurality of devices or to an apparatus comprising a single device. It goes without saying that the present invention can also be applied to the case where it is achieved by supplying a program to a system or apparatus. In this case, by reading a recording medium storing a program for achieving the present invention into the system or apparatus, the system or apparatus can receive the effects of the present invention.

  Further, by downloading and reading out a program for achieving the present invention from a server on a network, a database or the like by a communication program, the system or apparatus can receive the effects of the present invention. In addition, the structure which combined each embodiment mentioned above and its modification is also contained in this invention altogether.

101 Operation terminal PC
102 Application server 103 Supervised image classification server 104 Unsupervised image classification server 105 Image database

Claims (8)

An information processing system for classifying a verification image by a supervised image classifier, comprising:
Image input means for inputting a verification image;
A similar image extraction unit that extracts an image similar to the verification image input by the image input unit;
Teacher data creation means for creating teacher data by giving a label to the image extracted by the similar image extraction means;
Learning means for learning the supervised image classifier using the teacher data created by the teacher data creation means;
An information processing system comprising:   The information processing system according to claim 1, wherein the similar image extraction unit extracts an image similar to the verification image using an unsupervised image classifier. The image processing apparatus further comprises selection accepting means for accepting selection of an image used for learning by the learning means from the image extracted by the similar image extracting means by the user.
The information processing system according to claim 1 or 2, wherein the teacher data creation unit creates teacher data by attaching a label to the image selected by the selection receiving unit.   The information processing system according to any one of claims 1 to 3, further comprising an evaluation receiving unit that receives, from the user, an evaluation result on a classification result in the supervised image classifier for the verification image.   The similar image extraction means is characterized in that, when the evaluation result received by the evaluation reception means is a result indicating that learning is necessary, the similar image extraction means extracts an image similar to the verification image. Information processing system.   The information processing system according to any one of claims 1 to 5, wherein the similar image extraction means does not extract an image which has already been used to create teacher data by the teacher data creation means. An information processing method in an information processing system for classifying a verification image by a supervised image classifier,
An image input process for inputting a verification image;
A similar image extraction step of extracting an image similar to the verification image input in the image input step;
A teacher data creation step of creating teacher data by applying a label to the image extracted in the similar image extraction step;
A learning step of learning the supervised image classifier using the teacher data generated by the teacher data generation step;
An information processing method comprising: A program executable in an information processing system for classifying a verification image by a supervised image classifier,
The information processing system,
Image input means for inputting a verification image;
A similar image extraction unit that extracts an image similar to the verification image input by the image input unit;
Teacher data creation means for creating teacher data by giving a label to the image extracted by the similar image extraction means;
A program characterized in that it functions as a learning means for learning the supervised image classifier using the teacher data created by the teacher data creation means.

Images