JP6902150B1 - Image recognition device, learning system, image storage device, image recognition method, image storage method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To have high accuracy in recognizing a captured image in the prior art. SOLUTION: An optical signal acquisition unit 31 for photographing and acquiring an optical signal, an original image acquisition unit 32 for acquiring two or more different original images using an optical signal, and two or more original images are used. An image recognition device A including a recognition unit 35 that performs recognition processing related to an optical signal and acquires a recognition result and an output unit 4 that outputs the recognition result can recognize an image with high accuracy. [Selection diagram] Fig. 1

Description

The present invention relates to an image recognition device or the like that performs recognition processing on a captured image.

Conventionally, there has been a technique for identifying an object to be detected by performing image recognition on a captured image by machine learning (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2016-218760

However, in the prior art, it has been difficult to recognize an image taken with high accuracy. The captured image is appropriately referred to as a captured image.

The image recognition device of the first invention has an optical signal acquisition unit that photographs and acquires an optical signal, an original image acquisition unit that acquires two or more different original images using the optical signal, and two or more elements. It is an image recognition device including a recognition unit that performs recognition processing related to an optical signal using an image and acquires a recognition result, and an output unit that outputs the recognition result.

With such a configuration, the captured image can be recognized with high accuracy.

Further, the image recognition device of the second invention further includes a composite image acquisition unit that synthesizes two or more original images and acquires a composite image with respect to the first invention, and the recognition unit is at least composite. It is an image recognition device that performs image recognition processing using an image and acquires the recognition result.

With such a configuration, the captured image can be recognized with higher accuracy.

Further, the image recognition device of the third invention has one or more candidate images used for recognition processing from candidate images including two or more original images acquired by the original image acquisition unit for the first or second invention. A selection unit for selecting a target image is further provided, and the recognition unit is an image recognition device that performs image recognition processing using one or more target images and acquires a recognition result.

With such a configuration, the captured image can be recognized with high accuracy.

Further, in the image recognition device of the fourth invention, with respect to the third invention, the selection unit selects one or more target images satisfying a predetermined condition from two or more candidate images. It is a recognition device.

With such a configuration, the captured image can be recognized with high accuracy.

Further, in the image recognition device of the fifth invention, with respect to the fourth invention, the selection unit has at least two or more original images and two or more teacher data having image identification information for identifying the selected image. This is an image recognition device that performs machine learning prediction processing using the selective learning device acquired by performing learning processing and two or more candidate images, and identifies one or more target images.

With such a configuration, the captured image can be recognized with high accuracy.

Further, in the image recognition device of the sixth invention, for any one of the first to fifth inventions, the recognition unit performs a learning process using a set including two or more original images and a recognition result. It is an image recognition device that performs machine learning prediction processing and acquires a recognition result by using the recognition learner acquired by performing the operation and two or more original images.

With such a configuration, the captured image can be recognized with higher accuracy.

Further, the learning system of the seventh invention is a learning system including an image storage device and a learning device, and the image storage device has an optical signal acquisition unit for photographing and acquiring an optical signal, and an optical signal. An original image acquisition unit that acquires two or more different original images, a recognition result reception unit that receives a recognition result for an optical signal, and one or more of two or more candidate images including two or more different original images. A teacher data storage unit that stores teacher data having a candidate image and a recognition result is provided, and the learning device performs learning processing using two or more teacher data accumulated by the image storage device, and uses the recognition learner. It is a learning system including a cognitive learning unit for acquisition and a learning device storage unit for accumulating cognitive learning devices.

With such a configuration, it is possible to acquire a recognition learner that enables highly accurate recognition of captured images.

Further, in the learning system of the eighth invention, in contrast to the seventh invention, the image storage device further includes a composite image acquisition unit that synthesizes two or more original images and acquires the composite image, and provides teacher data. The storage unit is a learning system that stores teacher data having one or more candidate images and recognition results among three or more candidate images including two or more different original images and composite images.

With such a configuration, it is possible to acquire a recognition learner that enables more accurate recognition of a captured image by using a composite image as well.

Further, in the learning system of the ninth invention, with respect to the seventh or eighth invention, the image storage device outputs a set including two or more candidate images including two or more different original images. And, among the two or more candidate images included in the set, a selection reception unit that accepts the selection of one candidate image is further provided, and one candidate image corresponding to the selection accepted by the selection reception unit is used as a normal example for selection. The learning device further includes a distinguishing unit that performs a process of distinguishing one or more candidate images that have not been performed as a negative example, and the learning device includes one candidate image of the positive example and one or more candidate images of the negative example. Using the above set, a selective learning unit that performs learning processing and acquires a selective learning device is further provided, and the learning device storage unit is a learning system that stores selective learning devices.

With such a configuration, the target image used for image recognition can be selected with high accuracy.

According to the image recognition device according to the present invention, an image can be recognized with high accuracy.

Block diagram of the image recognition device A according to the first embodiment A flowchart illustrating a first operation example of the image recognition device A. Flowchart explaining an example of the same original image acquisition process A flowchart illustrating an example of the composite image acquisition process. A flowchart illustrating the first example of the selection process. A flowchart illustrating a second example of the selection process. A flowchart illustrating a second operation example of the image recognition device A. The figure which shows the example of the candidate image Conceptual diagram of the learning system B in the second embodiment Block diagram of the learning system B A flowchart illustrating an operation example of the image storage device 5. A flowchart for explaining an example of the selective learning process. Diagram showing the teacher data management table Block diagram of the computer system in the above embodiment

Hereinafter, embodiments of the image recognition device and the like will be described with reference to the drawings. In addition, since the components with the same reference numerals perform the same operation in the embodiment, the description may be omitted again.

(Embodiment 1)
In the present embodiment, an image that acquires an optical signal by photographing, acquires two or more original images using the optical signal, performs recognition processing using two or more original images, and outputs a recognition result. The recognition device will be described.

Further, in the present embodiment, an image recognition device that synthesizes two or more original images, acquires a composite image, performs recognition processing using the composite image, and outputs a recognition result will be described.

Further, in the present embodiment, the target image to be the target image of the image recognition processing is automatically selected from the set of the candidate images including two or more original images, the recognition processing is performed on the target image, and the recognition result is output. The image recognition device to be used will be described.

Further, in the present embodiment, an image recognition device that selects a target image by a machine learning algorithm will be described.

Further, in the present embodiment, an image recognition device that performs recognition processing by a machine learning algorithm will be described.

FIG. 1 is a block diagram of the image recognition device A according to the present embodiment. The image recognition device A includes a storage unit 1, a reception unit 2, a processing unit 3, and an output unit 4. The processing unit 3 includes an optical signal acquisition unit 31, an original image acquisition unit 32, a composite image acquisition unit 33, a selection unit 34, and a recognition unit 35.

Various types of information are stored in the storage unit 1. The various types of information are, for example, a selection learner described later, a recognition learner described later, two or more original image identification information, one or more composite image identification information, and a composite image flag.

The original image identification information is information for identifying the type of the original image to be acquired. The original image identification information is, for example, "RGB image", "IR image", and "NIR image". The original image identification information is, for example, an identifier of a program for acquiring the original image (for example, an execution module name, a function name, or a method name). The original image identification information is, for example, an image processing identifier performed to acquire the original image. The image processing identifier is information that identifies image processing, and is, for example, an ID, an identifier of a program that performs one or more image processing (for example, an execution module name, a function name, or a method name).

The original image is an image acquired by using an optical signal. The original image is an uncombined image. The original image is, for example, a spectroscopic image obtained by dispersing an optical signal. The original image is, for example, an image obtained by performing a predetermined one or more image processing on one spectroscopic image. The original image is an image that can be a candidate image.

The composite image identification information is information for identifying the type of composite image to be acquired. The composite image identification information is, for example, an identifier of a program for acquiring a composite image (for example, an execution module name, a function name, or a method name). The composite image identification information includes, for example, the original image identification information of the original image used when acquiring the composite image. The original image identification information in such a case is, for example, "RGB image", "IR image", and "NIR image".

A composite image is an image obtained by synthesizing two or more images. The synthesis method does not matter. A composite image is an image acquired from two or more images.

The composite image flag is information indicating whether or not to acquire the composite image.

The reception unit 2 receives various instructions, information, and the like. The various instructions, information, and the like are, for example, setting information. The setting information is information for identifying one image acquired by the selection unit 34.

Here, the acceptance usually means acceptance of information input from an input device such as a touch panel, a keyboard, or a mouse. However, acceptance may be a concept including reception of information transmitted via a wired or wireless communication line, acceptance of information read from a recording medium such as an optical disk, a magnetic disk, or a semiconductor memory.

The input means of the user's instruction may be any means such as a touch panel, a keyboard, a mouse, and a menu screen.

The processing unit 3 performs various processes. Various processes are performed by the optical signal acquisition unit 31, the original image acquisition unit 32, the composite image acquisition unit 33, the selection unit 34, and the recognition unit 35.

The optical signal acquisition unit 31 takes a picture and acquires an optical signal. Since the optical signal acquisition unit 31 is a known technique, detailed description thereof will be omitted.

The original image acquisition unit 32 acquires two or more different original images by using the optical signal acquired by the optical signal acquisition unit 31. The optical signals that are the basis of each of the two or more different original images are the same optical signal.

Also, the objects included in two or more different original images are usually the same, but may be different. That is, for example, one original image may be an image in the same region as the optical signal, and the other original image may be an image in a part of the region (so-called zoomed image). For example, the original image acquisition unit 32 may disperse the optical signal acquired by the optical signal acquisition unit 31 to acquire a short-distance “RGB image” and a long-distance “IR image”. In such a case, the region of the long-distance "IR image" is narrower than the region of the short-distance "RGB image". Further, for example, the original image acquisition unit 32 includes a beam splitter, and the beam splitter divides the light from the single lens into two, and the light having the same spectral characteristics divided into the two is used as a sensor having a different focal length. Input and acquire two images with different focal lengths (for example, "near RGB image" and "far RGB image").

The original image acquisition unit 32 acquires, for example, two or more spectroscopic images obtained by extracting a part of the wavelength from the optical signal acquired by the optical signal acquisition unit 31. In such a case, the original image is a spectroscopic image. The two or more spectroscopic images are, for example, two or more images of an RGB image (color image), an IR image (infrared image), and an NIR image (near infrared ray).

For example, the original image acquisition unit 32 refers to a sensor capable of simultaneously photographing RGB and NIR (for example, "http://www.optronics-media.com/news/20160606/42937/" (November 1, 2nd year of Reiwa). It can be realized by using search).

The original image acquisition unit 32 acquires, for example, an RGB image from the optical signal acquired by the optical signal acquisition unit 31, and acquires an image obtained by performing predetermined image processing on the RGB image. In such a case, the original image is, for example, an RGB image and an image subjected to predetermined image processing. The predetermined image processing is, for example, sharpness processing, noise reduction processing, brightness improvement processing, and the like, and various known image processings may be applicable.

The original image acquisition unit 32 acquires, for example, an IR image from the optical signal acquired by the optical signal acquisition unit 31, and acquires an image obtained by performing predetermined image processing on the IR image. In such a case, the original image is, for example, an IR image and an image subjected to predetermined image processing. The predetermined image processing is, for example, sharpness processing, noise reduction processing, brightness improvement processing, and the like, and various known image processings may be applicable.

The original image acquisition unit 32, for example, disperses the optical signal acquired by the optical signal acquisition unit 31 and acquires an RGB image and an IR image. Then, the original image acquisition unit 32 acquires, for example, an image in which the RGB image has been subjected to predetermined image processing and an image in which the IR image has been subjected to predetermined image processing. In such a case, the original image is, for example, an RGB image, an IR image, an image obtained by performing predetermined image processing on the RGB image, and an image obtained by performing predetermined image processing on the IR image.

The shooting targets of two or more different original images acquired by the original image acquisition unit 32 are the same.

The composite image acquisition unit 33 synthesizes two or more original images and acquires the composite image. The composite image acquisition unit 33 may synthesize the original image and the composite image and acquire a new composite image. The method of synthesizing the original image does not matter.

The composite image acquisition unit 33 acquires, for example, a composite image in which a partial original image of a part of one or more of the two or more original images is adopted as a region corresponding to a part of the original image. To do.

The composite image acquisition unit 33 is, for example, a composite image in which the first partial original image of the first region of the first original image of the two or more original images is adopted as the region corresponding to the first region. Yes, a composite image in which the second partial original image of the second region of the second original image of the two or more original images is adopted in the region corresponding to the second region is acquired.

The composite image acquisition unit 33 selects, for example, a pixel having a strong signal strength from two or more original images, and acquires one composite image. For example, the composite image acquisition unit 33 synthesizes two or more different original images (for example, an RGB image and an IR image) by a NAM circuit. That is, the composite image acquisition unit 33 preferentially outputs a high-level pixel value among the pixels at the same position of the two original images by using, for example, a NAM circuit, and acquires the composite image.

The composite image acquisition unit 33 divides, for example, two or more original images into predetermined regions, determines a region having a stronger signal strength for each region at the same position, and sets pixels in the determined region. Combine to get a composite image. The region has two or more pixels.

The composite image acquisition unit 33 may, for example, give two or more original images and a learning device to a module of machine learning prediction processing, and acquire a composite image in which two or more original images are combined. The learning device in such a case is a learning device in which two or more teacher data composed of two or more original images and a composite image are given to a module of learning processing of machine learning, and the module is executed and acquired. As described above, the machine learning algorithm does not matter. Further, such a learner is used in a prediction process in which two or more original images are input and a composite image is output.

The composite image and the objects in the two or more original images are usually the same object, but may be different.

The selection unit 34 selects one or more target images to be used for the recognition process from the candidate images including the two or more original images acquired by the original image acquisition unit 32.

It is preferable that the selection unit 34 acquires one target image from three or more candidate images including two or more original images and a composite image.

The selection unit 34 automatically selects one or more target images satisfying a predetermined condition from the two or more candidate images.

It should be noted that the predetermined condition is selected, for example, by the prediction processing of machine learning described later. The predetermined condition is, for example, that the score obtained by the prediction processing of machine learning described later is the maximum. The predetermined condition is, for example, that the representative value (for example, average value, median value) of each pixel of the candidate image is the maximum. The predetermined condition is that, for example, the representative values (for example, average value and median value) of the attribute values (for example, brightness and brightness) of each pixel of the candidate image are the maximum.

The selection unit 34 performs machine learning prediction processing using the selection learning device and two or more candidate images, and identifies one or more target images. The selective learner is information acquired by performing a learning process using at least two or more original images and two or more teacher data having image identification information for identifying the selected image. It is preferable that the selective learning device is a learning device acquired by the learning device 6 described later.

The selection unit 34 performs machine learning prediction processing using, for example, a selection learner and two or more candidate images, acquires image identification information that identifies one target image, and is specified by the image identification information. Acquire one target image. The machine learning algorithm is, for example, random forest, decision tree, deep learning, SVM, or the like, and the machine learning algorithm does not matter. Further, various machine learning functions such as a TensorFlow library, tinySVM, and R language random forest module, and various existing libraries can be used for the machine learning prediction processing. Further, the selective learning device is, for example, a learning device acquired by the learning device 6 in the second embodiment described later. The learning device may be called a classifier or a model.

Further, the selection learner here is information for inputting two or more candidate images and outputting one candidate image or an identifier of one candidate image. The selection learner inputs, for example, one of two or more candidate images, and outputs a flag (true or false) indicating whether or not the candidate image is selected as the target image. Is. The selection learner inputs, for example, one of two or more candidate images, and outputs a flag (true or false) and a score indicating whether or not the candidate image is selected as the target image. Information. The target image is an image that is the target of image recognition.

That is, for example, the selection unit 34 acquires the selection learning device from the storage unit 1, gives the selection learning device and two or more candidate images to the module of the prediction processing of machine learning, executes the module, and then executes the module. Candidate image of is determined as a target image.

Further, the selection unit 34 acquires, for example, the selection learning device from the storage unit 1. Then, the selection unit 34, for example, gives a pair of a candidate image of one of two or more candidate images and a selection learner to the module of the prediction processing of machine learning in order, and a flag indicating whether or not to select the candidate image. And get the score. Then, the selection unit 34 determines, for example, a candidate image from which a flag indicating that the image is selected is obtained and has the maximum score as the target image.

Further, for example, the selection unit 34 acquires the selection learning device from the storage unit 1, gives the selection learning device and two or more candidate images to the module of the prediction processing of machine learning, executes the module, and executes the module. The above candidate images are determined as target images.

Further, the selection unit 34 acquires, for example, the selection learning device from the storage unit 1. Then, the selection unit 34, for example, gives a pair of a candidate image of one of two or more candidate images and a selection learner to the module of the prediction processing of machine learning in order, and a flag indicating whether or not to select the candidate image. And get the score. Then, the selection unit 34 determines, for example, a candidate image from which a flag indicating that the image is selected is obtained, and one or two or more candidate images having a score equal to or higher than the threshold value or larger than the threshold value as the target image.

The selection unit 34 selects, for example, one candidate image according to the user's instruction from two or more candidate images as the target image. For example, when the user's instruction is the setting information, the selection unit 34 selects the candidate image corresponding to the setting information of the storage unit 1 as the target image from the two or more candidate images. The setting information is, for example, a type identifier indicating one type of candidate image among two or more types of candidate images. The type identifiers are, for example, "RGB image", "IR image", and "composite image".

The recognition unit 35 performs recognition processing on an optical signal using two or more original images, and acquires a recognition result. The recognition process related to an optical signal is usually an image recognition process for a target image.

The recognition unit 35 performs image recognition processing using at least a composite image, for example, and acquires a recognition result. The recognition unit 35 performs image recognition processing using, for example, two or more original images and a composite image, and acquires a recognition result.

The recognition unit 35 usually performs image recognition processing using one or more target images and acquires a recognition result. The one or more target images are images selected by the selection unit 34 from two or more candidate images.

The recognition unit 35 uses, for example, a recognition learner acquired by performing a learning process using a set including two or more original images and a recognition result, and a machine learning prediction process using two or more original images. And get the recognition result. To use two or more original images, it is also possible to use a target image acquired from two or more original images.

The recognition unit 35 may perform recognition processing on the target image and acquire the recognition result by using a technique other than the machine learning algorithm. The techniques other than the machine learning algorithm are, for example, known object recognition techniques and character recognition techniques. That is, the recognition unit 35 may recognize the object in the target image by the image recognition process and acquire the object identifier that identifies the object. Further, the recognition unit 35 may acquire the color and shape of the object in the target image by the image recognition process.

The recognition result is, for example, the object name of the object included in the photographed optical signal, the character string (may be a number string) included in the photographed optical signal, and the attribute value (color, shape, of the object) of the object. Size etc.). The information in the optical signal is the information in the target image.

The recognition unit 35 performs recognition processing on each of two or more target images, acquires different types of recognition results from each of the two or more target images, and outputs the recognition results using the two or more types of recognition results. You may get the result. For example, the recognition unit 35 may acquire the recognition result having the vehicle model name and the color of the automobile from one target image, and may acquire the license plate information from the other target image. Further, when the recognition unit 35 performs recognition processing on each of two or more target images and obtains different recognition results from each of the two or more target images, two or more different target images are obtained. The recognition process may be performed using an algorithm. For example, the recognition unit 35 performs machine learning prediction processing on one target image to acquire a vehicle model name, performs image recognition on the one target image, extracts the outline of the vehicle, and obtains the vehicle model name. Acquires color information (for example, "white") of an automobile area, extracts the outline of the license plate area from another target image, performs character recognition processing on the area, and acquires the license plate number. You may. As described above, the recognition unit 35 may acquire two or more recognition results for one target image by two or more different algorithms.

The recognition unit 35 detects, for example, a fingerprint area in the target image, performs fingerprint authentication processing, and acquires identification information of a person corresponding to the fingerprint from a storage unit (not shown). In such a case, the storage unit stores information for associating the fingerprint information with the person's identification information.

For example, the recognition unit 35 detects a face area of a person in the target image, performs face recognition processing using the image of the face area, and illustrates the identification information of the person corresponding to the face image. Do not get from the storage. In such a case, information for associating a face image with a person's identification information is stored in the storage unit.

For example, the recognition unit 35 recognizes a pattern from an image signal obtained by two images having different focal lengths (the image on the near focus side is W and the image on the far focus side is T) acquired by the original image acquisition unit 32. Alternatively, the object in the image (for example, the vehicle in front) is recognized by machine learning or the like. Then, assuming that the difference between the images of W and T is X, the recognition unit 35 compares the signal amounts of W and T with reference to X, and calculates the distance to the object. If the focal length that is the maximum signal of W is set to 1.0 meter and the focal length that is the maximum signal of T is set to 3.0 meters in advance, the relationship with X can be obtained. , The recognition unit 35 can acquire the distance. That is, the recognition unit 35 can acquire the distance from the optical signal acquisition unit 31 to the object in the image as information constituting the recognition result.

The output unit 4 outputs the recognition result acquired by the recognition unit 35. Here, the output means display on a display, projection using a projector, printing by a printer, sound output, transmission to an external device, storage on a recording medium, storage on another processing device, another program, or the like. It is a concept that includes delivery of processing results.

A non-volatile recording medium is suitable for the storage unit 1, but a volatile recording medium can also be used.

The process in which the information is stored in the storage unit 1 does not matter. For example, the information may be stored in the storage unit 1 via the recording medium, the information transmitted via the communication line or the like may be stored in the storage unit 1, or The information input via the input device may be stored in the storage unit 1.

The reception unit 2 can be realized by a device driver for input means such as a touch panel or a keyboard, control software for a menu screen, or the like.

The processing unit 3, the original image acquisition unit 32, the composite image acquisition unit 33, the selection unit 34, and the recognition unit 35 can usually be realized from a processor, a memory, or the like. The processing procedure of the processing unit 3 and the like is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit). The processor is, for example, a CPU, an MPU, a GPU, or the like, and the type thereof does not matter.

The optical signal acquisition unit 31 is realized by, for example, a so-called camera optical component and an image sensor.

The output unit 4 may or may not include an output device such as a display or a speaker. The output unit 4 can be realized by the driver software of the output device, the driver software of the output device, the output device, or the like.

Next, a first operation example of the image recognition device A will be described with reference to the flowchart of FIG.

(Step S201) The optical signal acquisition unit 31 acquires an optical signal.

(Step S202) The original image acquisition unit 32 acquires two or more different original images using the optical signal acquired by the optical signal acquisition unit 31. An example of such an original image acquisition process will be described with reference to the flowchart of FIG.

(Step S203) The composite image acquisition unit 33 determines whether or not to acquire the composite image. If the composite image is acquired, the process goes to step S205, and if the composite image is not acquired, the process goes to step S206. The composite image acquisition unit 33 may always acquire the composite image. Further, the composite image acquisition unit 33 may determine that the composite image is acquired, for example, when the composite image flag of the storage unit 1 is information indicating that the composite image is acquired. However, the conditions for determining that a composite image is to be acquired are not limited.

(Step S204) The composite image acquisition unit 33 acquires a composite image. An example of such a composite image acquisition process will be described with reference to the flowchart of FIG.

(Step S205) The selection unit 34 acquires one target image from the candidate images including the two or more original images acquired by the original image acquisition unit 32. It is preferable that the selection unit 34 acquires one target image out of three or more candidate images including two or more original images and a composite image. An example of such a selection process will be described with reference to the flowcharts of FIGS. 5 and 6.

(Step S206) The recognition unit 35 performs image recognition processing on one target image acquired in step S205, and acquires the recognition result.

(Step S207) The output unit 4 outputs the recognition result acquired in step S206.

(Step S208) The processing unit 3 determines whether or not to end the processing. If it is determined that the process is terminated, the process is terminated, and if it is determined that the process is not terminated, the process returns to step S201. Here, the conditions for ending the process do not matter. When the image recognition device A is, for example, an in-vehicle device mounted on an automobile or the like, the condition for ending the process is, for example, that the engine of a moving body such as an automobile is turned off.

Next, an example of the original image acquisition process in step S202 will be described with reference to the flowchart of FIG.

(Step S301) The original image acquisition unit 32 substitutes 1 for the counter i.

(Step S302) The original image acquisition unit 32 determines whether or not the i-th original image identification information for acquiring the original image exists in the storage unit 1.

(Step S303) The original image acquisition unit 32 acquires the i-th original image corresponding to the i-th original image identification information and temporarily stores it in a buffer (not shown).

(Step S304) The original image acquisition unit 32 increments the counter i by 1. Return to step S302.

Next, an example of the composite image acquisition process in step S204 will be described with reference to the flowchart of FIG.

(Step S401) The composite image acquisition unit 33 substitutes 1 for the counter i.

(Step S402) The composite image acquisition unit 33 determines whether or not to acquire the i-th composite image. If the i-th composite image is acquired, the process proceeds to step S403, and if the i-th composite image is not acquired, the process returns to higher processing. For example, the composite image acquisition unit 33 determines whether or not to acquire the i-th composite image depending on whether or not the i-th composite image identification information exists in the storage unit 1.

(Step S403) The composite image acquisition unit 33 acquires two or more original images used for acquiring the i-th composite image from a buffer (not shown).

(Step S404) The composite image acquisition unit 33 acquires the i-th composite image using the two or more original images acquired in step S403, and temporarily stores the composite image in a buffer (not shown).

(Step S405) The composite image acquisition unit 33 increments the counter i by 1. Return to step S402.

Next, a first example of the selection process in step S205 will be described with reference to the flowchart of FIG.

(Step S501) The selection unit 34 acquires the selection learning device from the storage unit 1.

(Step S502) The selection unit 34 substitutes 1 for the counter i.

(Step S503) The selection unit 34 determines whether or not the i-th candidate image exists in a buffer (not shown). If the i-th candidate image exists, the process goes to step S504, and if the i-th candidate image does not exist, the process goes to step S508.

(Step S504) The selection unit 34 acquires the i-th candidate image from a buffer (not shown).

(Step S505) The selection unit 34 gives the selection learner and the i-th candidate image to the prediction module of machine learning, executes the prediction module, and acquires the prediction result. Here, the prediction result is a flag (prediction value) and a score indicating whether or not the selection is made. The higher the score, the greater the likelihood of being selected.

(Step S506) The selection unit 34 temporarily stores the predicted value and the score in a buffer (not shown) in association with the i-th candidate image.

(Step S507) The selection unit 34 increments the counter i by 1. Return to step S503.

(Step S508) The selection unit 34 is a predicted value indicating that the image is selected, and determines a candidate image paired with the maximum score as the target image. Return to higher-level processing.

Next, a second example of the selection process in step S205 will be described with reference to the flowchart of FIG. In the flowchart of FIG. 6, the same steps as those of the flowchart of FIG. 5 will be omitted.

(Step S601) The selection unit 34 acquires two or more candidate images from a buffer (not shown).

(Step S602) The selection unit 34 gives the selection learner and two or more candidate images to the prediction module of machine learning, executes the prediction module, and acquires the prediction result. The prediction result is information for specifying the target image here. The information that identifies the target image may be the target image, an identifier of the target image (for example, a file name), or the like.

(Step S603) The selection unit 34 determines a candidate image corresponding to the prediction result as the target image.

Next, a second operation example of the image recognition device A will be described with reference to the flowchart of FIG. 7. In the flowchart of FIG. 7, the same steps as those of the flowchart of FIG. 2 will be omitted.

(Step S701) The recognition unit 35 substitutes 1 for the counter i.

(Step S702) The recognition unit 35 determines whether or not the i-th target image, which is the target of the image recognition process, exists. If the i-th target image exists, the process goes to step S703, and if the i-th target image does not exist, the process goes to step S705.

(Step S703) The recognition unit 35 performs image recognition processing on the i-th target image, acquires the i-th recognition result, and temporarily stores the i-th recognition result in a buffer (not shown).

(Step S704) The recognition unit 35 increments the counter i by 1. Return to step S702.

(Step S705) The recognition unit 35 acquires a recognition result to be output by using one or more recognition results in a buffer (not shown). Go to step S207.

The recognition unit 35 acquires, for example, a recognition result corresponding to the maximum score acquired as a result of the image recognition process. The recognition unit 35 acquires, for example, the most recognition result among two or more recognition results as the recognition result to be output.

In the flowchart of FIG. 7, image recognition processing was performed on all the candidate images. However, one or two or more target images may be selected from the candidate images, and image recognition processing may be performed on the one or more target images.

Hereinafter, the specific operation of the photographing apparatus A in the present embodiment will be described. The appearance of the photographing device A is, for example, a camera.

Now, it is assumed that the storage unit 1 stores two original image identification information of "RGB image" and "IR image". Further, it is assumed that the storage unit 1 stores the composite image identification information which is the module name of one program for acquiring the composite image. The first program adopts the original image of the "RGB image" and "IR image" with the higher average brightness as the base, detects the license plate area, and has the larger sharpness of the license plate area. It is assumed that the program adopts the license plate area of the original image of.

Further, the storage unit 1 stores a selection learning device that selects one candidate image from three candidate images.

Further, it is assumed that the storage unit 1 stores a recognition learning device that acquires vehicle type information for identifying the vehicle type of the vehicle shown in the target image when the target image is given. The recognition learner is information acquired by performing machine learning learning processing on two or more teacher data having an image of a vehicle and vehicle type information.

In the above situation, it is assumed that the optical signal acquisition unit 31 of the photographing apparatus A has photographed and acquired the optical signal.

Next, the original image acquisition unit 32 acquires two original images, an “RGB image” and an “IR image”, using the optical signal acquired by the optical signal acquisition unit 31. The “RGB image” is 81 in FIG. The “IR image” is 82 in FIG.

Further, the composite image acquisition unit 33 executes the module identified by the module name indicated by the composite image identification information, and acquires the composite image. Such a composite image is 83 in FIG.

Next, the selection unit 34 acquires the selection learning device of the storage unit 1. The selection unit 34 gives three candidate images (“RGB image 81”, “IR image 82”, and “composite image 83”) and a selection learner to the machine learning prediction module, and gives one image (here, a composite image). It is assumed that 83) has been acquired.

Next, the recognition unit 35 performs image recognition processing on the composite image 83 to acquire the license plate number “20-20”. It also acquires the color "white" of the car.

Further, the recognition unit 35 acquires the recognition learning device of the storage unit 1. Then, the recognition unit 35 gives the composite image 83 and the recognition learner to the prediction module of machine learning, executes the prediction module, and acquires the vehicle model "XXX" of the automobile shown in the composite image 83. To do.

Next, the output unit 4 outputs the recognition result "<vehicle type> XXX <color> white <license plate> 20-20".

As described above, according to the present embodiment, the image can be recognized with high accuracy. That is, according to the present embodiment, an image can be recognized with high accuracy by selecting a target image to be image-recognized from two or more candidate images and performing image recognition processing on the target image.

Needless to say, the use of the image recognition device A in the present embodiment does not matter. The image recognition device A can be used as, for example, a surveillance camera. The surveillance camera is, for example, a surveillance camera for automatic driving of a car or a drive recorder. Further, the surveillance camera is, for example, a camera for monitoring the inside of a drive recorder, a camera for monitoring the rear seats, and a camera for monitoring the driver's movement or falling asleep. The image recognition device A is, for example, a camera for automatic driving of a vehicle and a camera for monitoring the outside of a drive recorder.

Further, the processing in the present embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded on a recording medium such as a CD-ROM and disseminated. This also applies to other embodiments herein. The software that realizes the image recognition device A in this embodiment is the following program. That is, this program includes an optical signal acquisition unit that photographs a computer and acquires an optical signal, an original image acquisition unit that acquires two or more different original images using the optical signal, and the two or more elements. This is a program for functioning as a recognition unit that performs recognition processing related to the optical signal and acquires the recognition result and an output unit that outputs the recognition result using an image.

(Embodiment 2)
In the present embodiment, the selective learning device and the learning system for acquiring the recognition learning device that can be used by the image recognition device A will be described.

FIG. 9 is a conceptual diagram of the learning system B according to the present embodiment. The learning system B includes one or more image storage devices 5 and a learning device 6. The learning system B may be realized by one device, or may be realized by three or more devices.

The image storage device 5 takes an image, acquires a set of two or more candidate images using the image, and accepts one selection from the set. Then, the image storage device 5 stores the set in a state in which the selected candidate image and the non-selected candidate image can be distinguished from each other in the set. Further, the image storage device 5 receives object information regarding the objects in the image. The object information is information corresponding to the above-mentioned recognition result. The image storage device 5 is, for example, a camera or a computer with a camera. The camera may be one that can shoot a still image or a camera that can shoot a moving image.

The learning device 6 is a device that constitutes a selective learning device for performing learning processing using two or more sets and selecting one image from two or more candidate images. Further, the learning device 6 is a device that learns two or more teacher data having an image and object information to form a recognition learner.

FIG. 10 is a block diagram of the learning system B according to the present embodiment. The image storage device 5 constituting the learning system B includes a storage unit 51, a reception unit 52, a processing unit 53, and an output unit 54. The reception unit 52 includes a selection reception unit 521 and a recognition result reception unit 522. The processing unit 53 includes an optical signal acquisition unit 31, an original image acquisition unit 32, a composite image acquisition unit 33, a teacher data storage unit 531 and a discrimination unit 532. The output unit 54 includes a set output unit 541.

The learning device 6 includes a learning storage unit 61, a selective learning unit 62, a recognition learning unit 63, and a learning device storage unit 64.

Various types of information are stored in the storage unit 51 that constitutes the image storage device 5. The various information is, for example, a set of two or more candidate images.

The reception unit 52 receives various instructions and information. The various instructions and information are, for example, a shooting instruction, a selection instruction, and a recognition result described later. The selection instruction is an instruction for selecting an image. The selection instruction may be simply called selection.

The means for inputting various instructions and information may be any means such as a touch panel, a keyboard, a mouse, and a menu screen.

The selection reception unit 521 accepts the selection of one candidate image from the two or more candidate images included in the set.

The recognition result reception unit 522 receives the recognition result for the optical signal. The recognition result for an optical signal has the same meaning as the recognition result for an image. The recognition result is object information about the object in the image for the optical signal.

The processing unit 53 performs various processes. The various processes are performed by, for example, the optical signal acquisition unit 31, the original image acquisition unit 32, the composite image acquisition unit 33, the set storage unit 531 and the discrimination unit 532.

The teacher data storage unit 531 stores two or more teacher data. The teacher data includes one or more candidate images out of two or more candidate images including two or more different original images acquired by the original image acquisition unit 32. The teacher data includes, for example, a set of candidate images capable of distinguishing a candidate image corresponding to the selection accepted by the selection reception unit 521 and one or more candidate images not corresponding to the selection. The teacher data has, for example, the recognition result received by the recognition result reception unit 522.

The one or more candidate images possessed by the teacher data may be one candidate image corresponding to the selection accepted by the selection reception unit 521, or may be a set of two or more candidate images. The two or more candidate images are, for example, two or more original images acquired by the original image acquisition unit 32. The two or more candidate images are, for example, two or more original images acquired by the original image acquisition unit 32 and one or more composite images acquired by the composite image acquisition unit 33. The two or more candidate images are, for example, two or more candidate images corresponding to the selection received by the selection reception unit 521.

In the teacher data storage unit 531, for example, among two or more candidate images including two or more different original images acquired by the original image acquisition unit 32, the candidate image received by the selection reception unit 521 and the recognition result reception unit 522 Accumulate teacher data with the received recognition result.

The teacher data storage unit 531 stores teacher data having one or more candidate images and a recognition result among three or more candidate images including two or more different original images and composite images.

The discrimination unit 532 performs a process of distinguishing one candidate image corresponding to the selection received by the selection reception unit 521 as a positive example and one or more candidate images not selected as a negative example. The process of distinguishing is, for example, a process of associating a regular example flag with one selected candidate image. The process of distinguishing is, for example, a process of associating a negative example flag with each one or more candidate images that have not been selected. The process of distinguishing is, for example, a process of associating a positive example flag with one selected candidate image and associating a negative example flag with each of one or more unselected candidate images. The process of distinguishing is, for example, a process of accumulating one selected candidate image and one or more unselected candidate images in different folders. It suffices if one selected candidate image and one or more non-selected candidate images can be distinguished, and the processing and method for distinguishing them are not limited.

The output unit 54 outputs various information. Various information is a set of two or more candidate images. Here, the output means display on a display, projection using a projector, printing by a printer, transmission to an external device, storage on a recording medium, processing to another processing device or another program, or the like. It is a concept that includes delivery of results.

The set output unit 541 outputs a set including two or more candidate images including two or more different original images.

Various types of information are stored in the learning storage unit 61 that constitutes the learning device 6. Various types of information are two or more teacher data. The two or more teacher data are the data accumulated by the image accumulator 5.

The teacher data includes, for example, a positive example image and one or more negative example images. The regular image is a candidate image of the regular example. The negative example image is a candidate image of the negative example. Further, the teacher data is data for acquiring a selective learner.

The teacher data includes, for example, an image and a recognition result. Such teacher data is data for acquiring a recognition learner.

The selective learning unit 62 performs learning processing using two or more sets including one positive example image and one or more negative example images, and acquires a selective learning device. The selective learner is a learner for determining one or more target images from two or more candidate images.

Such a learning process is a learning process using a machine learning algorithm. The machine learning algorithm is, for example, random forest, decision tree, deep learning, SVM, or the like, and the machine learning algorithm does not matter. Further, for the machine learning learning process, various machine learning functions such as a TensorFlow library, tinySVM, and a R language random forest module, and various existing libraries can be used.

The selective learning unit 62 gives, for example, two or more sets including one positive example image and one or more negative example images to the learning module of machine learning, executes the learning module, and acquires the selective learning device. ..

The selection learning unit 62 comprises, for example, a set of one positive example image and one negative example image of the same shooting target as the one positive example image from two or more sets. The selective learning unit 62 gives the configured set of two or more images to the learning module of machine learning, executes the learning module, and acquires the selective learning device.

The recognition learning unit 63 performs learning processing using two or more teacher data accumulated by the image storage device 5, and acquires a recognition learning device. The teacher data here is one target image and the recognition result. The teacher data may be two or more target images and a recognition result.

The cognitive learning unit 63 gives two or more teacher data to a machine learning learning module, executes the learning module, and acquires a cognitive learning device.

The learning process is also a learning process using a machine learning algorithm, and the algorithm does not matter.

The learning device storage unit 64 stores the selective learning device acquired by the selective learning unit 62. The learning device storage unit 64 stores the recognition learning device acquired by the recognition learning unit 63. The storage destination of the learning device does not matter.

The storage unit 51 and the learning storage unit 61 are preferably non-volatile recording media, but can also be realized by volatile recording media.

The process of storing information in the storage unit 51 or the like does not matter. For example, the information may be stored in the storage unit 51 or the like via the recording medium, or the information transmitted via the communication line or the like may be stored in the storage unit 51 or the like. Alternatively, the information input via the input device may be stored in the storage unit 51 or the like.

The reception unit 52, the selection reception unit 521, and the recognition result reception unit 522 can be realized by a device driver of an input means such as a touch panel or a keyboard, a menu screen control software, or the like.

The processing unit 53, the teacher data storage unit 531, the discrimination unit 532, the selective learning unit 62, the recognition learning unit 63, and the learner storage unit 64 can usually be realized from a processor, a memory, or the like. The processing procedure of the processing unit 53 or the like is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit). The processor is, for example, a CPU, an MPU, a GPU, or the like, and the type thereof does not matter.

Next, an operation example of the learning system B will be described. First, an operation example of the image storage device 5 will be described with reference to the flowchart of FIG. In the flowchart of FIG. 11, the same steps as the flowchart of FIG. 2 will be omitted.

(Step S1101) The processing unit 53 determines whether or not to perform shooting. If shooting is performed, the process proceeds to step S202, and if shooting is not performed, the process returns to step S201. The processing unit 53 determines that shooting is performed when, for example, the reception unit 52 receives a shooting instruction. Further, the processing unit 53 determines that, for example, after the reception unit 52 receives the shooting instruction, the shooting is performed until the instruction to end the shooting is received. The conditions under which the processing unit 53 determines to perform photographing are not limited.

(Step S1102) The output unit 54 outputs two or more acquired candidate images. The two or more candidate images are, for example, two or more original images. The two or more candidate images are, for example, two or more original images and one or more composite images. It should be noted that such an output is usually an output to a display.

(Step S1103) The selection reception unit 521 determines whether or not the selection from the user has been accepted. If the selection is accepted, the process goes to step S1104, and if the selection is not accepted, the process returns to step S1103.

(Step S1104) The distinguishing unit 532 associates a regular example flag with the candidate image selected in step S1103.

(Step S1105) The recognition result reception unit 522 determines whether or not the recognition result has been received. If the recognition result is accepted, the process proceeds to step S1106, and if the recognition result is not accepted, the process returns to step S1105. The recognition result is usually information input by the user.

(Step S1106) The teacher data storage unit 531 constitutes teacher data. The teacher data storage unit 531 constitutes, for example, teacher data having one positive example image, one or more negative example images, and a recognition result. The teacher data storage unit 531 constitutes, for example, teacher data having two or more positive example images, one or more negative example images, and a recognition result.

(Step S1107) The teacher data storage unit 531 stores the teacher data configured in step S1106 in a buffer (not shown). Return to step S1101.

In the flowchart of FIG. 11, the process ends when the power is turned off or an interrupt for the end of the process occurs.

Next, an example of the selective learning process in which the learning device 6 acquires the selective learning device will be described with reference to the flowchart of FIG. It is assumed that two or more sets are stored in the learning storage unit 61. The set is, for example, a set of one selected candidate image of a positive example and one or more candidate images of a negative example. That is, the set has two or more candidate images in which the positive example image and the negative example image are distinguished.

(Step S1201) The learning device 6 determines whether or not to start learning. If learning is to be started, the process goes to step S1202, and if learning is not started, the process returns to step S1201. The conditions for starting learning do not matter. For example, the learning device 6 determines that the learning is started according to the instruction of the user.

(Step S1202) The selection learning unit 62 substitutes 1 for the counter i.

(Step S1203) The selective learning unit 62 determines whether or not the i-th set exists in the learning storage unit 61.

(Step S1204) The selective learning unit 62 acquires one positive example image of the i-th set. The regular image is a candidate image corresponding to the regular flag.

(Step S1205) The selection learning unit 62 substitutes 1 for the counter j.

(Step S1206) The selective learning unit 62 acquires the j-th negative example image of the i-th set.

(Step S1207) The selective learning unit 62 acquires a set of the positive example image acquired in step S1204 and the j-th negative example image acquired in step S1206, and temporarily stores the pair in a buffer (not shown).

(Step S1208) The selection learning unit 62 increments the counter j by 1. Return to step S1206.

(Step S1209) The selective learning unit 62 increments the counter i by 1. Return to step S1203.

(Step S1210) The selective learning unit 62 gives a pair of two or more positive example images and negative example images temporarily stored in a buffer (not shown) to the learning module, executes the module, and acquires the selective learning device.

(Step S1211) The learning device storage unit 64 stores the selective learning device acquired in step S1210. Return to step S1201.

In the flowchart of FIG. 11, the selective learning unit 62 may give two or more sets to the learning module of machine learning, execute the learning module, and acquire the selective learning device.

Further, in the flowchart of FIG. 11, the process is terminated by the power off or the interrupt of the process end.

Next, an example of the recognition learning process in which the learning device 6 acquires the recognition learning device will be described. The recognition learning unit 63 acquires two or more teacher data from the learning storage unit 61. Next, the cognitive learning unit 63 gives the two or more teacher data to the machine learning learning module, executes the learning module, and acquires the cognitive learning device. The teacher data here has, for example, one target image and a recognition result.

Hereinafter, the specific operation of the learning system B in the present embodiment will be described.

The output unit 54 of the image storage device 5 outputs four candidate images of each set in the storage unit 51 by the above-described processing. It is assumed that the four candidate images are two original images and two composite images. Further, it is assumed that the two original images are an "RGB image" and an "IR image".

Then, the user selects one candidate image for each set. Then, the selection reception unit 521 accepts such a selection. Next, the discrimination unit 532 accumulates the regular example flag in association with the selected candidate image.

In addition, the user inputs the name of the object shown in the selected candidate image. Then, the recognition result reception unit 522 of the image storage device 5 receives the object name. Then, the processing unit 53 stores the object name in association with the selected candidate image.

It is assumed that the teacher data management table shown in FIG. 13 has been accumulated in the storage unit 51 by the above processing. The teacher data management table has two or more teacher data having "ID", "original image 1", "original image 2", "composite image 1", "composite image 2", and "recognition result". The "ID" is information that identifies the set. The "original image 1" is an RGB image, and the "original image 2" is an IR image. It is assumed that the "composite image 1" and the "composite image 2" are images synthesized by different algorithms, and are images synthesized by using the original image 1 and the original image 2. The "recognition result" is the name of the object shown in the selected candidate image.

Further, in FIG. 13, a “◯” indicating that the candidate image is a positive example is given to the candidate image selected by the user, and a “×” indicating that the candidate image is not selected by the user is a negative example. "Is given.

Then, the teacher data management table is also accumulated in the learning storage unit 61 of the learning device 6.

Next, it is assumed that the learning device 6 determines that the learning is started.

Next, the selective learning unit 62 of the learning device 6 performs the selective learning process according to, for example, the flowchart of FIG. That is, the selective learning unit 62 acquires a set having one positive example image and three negative example images from each teacher data in the teacher data management table. Then, the selective learning unit 62 gives two or more sets to the learning module of machine learning, executes the learning module, acquires the selective learning device, and stores the selective learning device in the learning storage unit 61. It is preferable that the selection learner is used by the above-mentioned photographing apparatus A for the selection process.

Further, the recognition learning unit 63 acquires, for example, one regular example image (candidate image corresponding to ◯) and the recognition result from each teacher data in the teacher data management table. Then, the cognitive learning unit 63 gives a set of one positive example image and the recognition result to the learning module of machine learning, executes the learning module, acquires the cognitive learning device, and performs the cognitive learning. The vessel is stored in the learning storage unit 61. It is preferable that the recognition learner is used by the above-mentioned photographing apparatus A for image recognition processing.

As described above, according to the present embodiment, a selection learning device for selecting an image necessary for image recognition can be obtained.

Further, according to the present embodiment, a recognition learner necessary for image recognition can be obtained.

The software that realizes the image storage device 5 in this embodiment is the following program. That is, this program includes an optical signal acquisition unit that photographs a computer and acquires an optical signal, an original image acquisition unit that acquires two or more different original images using the optical signal, and the original image acquisition unit. An output unit that outputs two or more candidate images including two or more original images acquired by the user, a selection reception unit that accepts the selection of one candidate image from the user, and one corresponding to the selection received by the selection reception unit. As a positive example, one or more candidate images that were not selected as a negative example, a distinguishing unit that performs discrimination processing, a recognition result receiving unit that accepts a recognition result for an optical signal, and a regular image and one or more. This is a program for functioning as a teacher data storage unit that stores teacher data including negative example images and recognition results.

The software that realizes the learning device 6 is the following program. That is, this program is a set of two or more computers that can access the learning storage unit in which two or more teacher data including a positive example image, one or more negative example images, and a recognition result are stored. The selective learning unit that performs learning processing and acquires the selective learning device using, and the cognitive learning unit that performs learning processing and acquires the cognitive learning device using two or more teacher data, and the selective learning device and cognitive learning. It is a program to function as a learning device storage unit that stores devices. The software that realizes the learning device 6 may acquire and store only one of the selective learning device and the cognitive learning device.

Further, in the above-described embodiment, the recognition result may be used in any way. That is, for example, the image recognition device A may be used for automatic operation. For example, when it is determined that the distance is within the threshold value by using the distance of the recognition result output by the output unit 4 of the image recognition device A, the control to apply the brake of the automobile may be performed.

Moreover, the above-described embodiment can be realized by computer hardware and a computer program executed on the computer hardware. FIG. 14 is a block diagram of a computer system 300 capable of realizing the photographing device A, the image storage device 5, and the learning device 6.

In FIG. 14, the computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

In FIG. 14, in addition to the CD-ROM drive 3012, the computer 301 is connected to the MPU 3013, the bus 3014 connected to the CD-ROM drive 3012 and the like, the ROM 3015 for storing a program such as a bootup program, and the MPU 3013. It includes a RAM 3016 that is connected and for temporarily storing instructions of an application program and providing a temporary storage space, and a hard disk 3017 for storing an application program, a system program, and data. Although not shown here, the computer 301 may further include a network card that provides a connection to the LAN.

The program for causing the computer system 300 to execute the functions of the photographing apparatus A and the like according to the above-described embodiment may be stored in the CD-ROM 3101, inserted into the CD-ROM drive 3012, and further transferred to the hard disk 3017. Alternatively, the program may be transmitted to the computer 301 via a network (not shown) and stored on the hard disk 3017. The program is loaded into RAM 3016 at run time. The program may be loaded directly from the CD-ROM3101 or network.

The program does not necessarily include an operating system (OS) that causes the computer 301 to execute the functions of the photographing apparatus A according to the above-described embodiment, a third-party program, or the like. The program need only include a portion of the instruction that calls the appropriate function (module) in a controlled manner to obtain the desired result. It is well known how the computer system 300 works, and detailed description thereof will be omitted.

Further, the number of computers that execute the above program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed. That is, the image storage device 5 and the like may be a stand-alone device, or may be composed of two or more devices.

Further, in each of the above-described embodiments, each process may be realized by centralized processing by a single device, or may be realized by distributed processing by a plurality of devices.

It goes without saying that the present invention is not limited to the above embodiments, and various modifications can be made, and these are also included in the scope of the present invention.

As described above, the image recognition device according to the present invention has an effect of being able to recognize an image with high accuracy, and is useful as an image recognition device or the like.

A Image recognition device B Learning system 1, 51 Storage unit 2, 52 Reception unit 3, 53 Processing unit 4, 54 Output unit 5 Image storage device 6 Learning device 31 Optical signal acquisition unit 32 Original image acquisition unit 33 Composite image acquisition unit 34 Selection unit 35 Recognition unit 61 Learning storage unit 62 Selection learning unit 63 Recognition learning unit 64 Learner storage unit 521 Selection reception unit 522 Recognition result reception unit 531 Teacher data storage unit 531 Set storage unit 532 Discrimination unit 541 Set output unit

Claims (11)

An optical signal acquisition unit that shoots and acquires an optical signal,
An original image acquisition unit that acquires two or more different original images, which are two or more spectroscopic images obtained by extracting a part of wavelengths from the optical signal.
A selective learner acquired by performing a learning process using at least two or more original images and two or more teacher data having image identification information for identifying the selected image, and 2 acquired by the original image acquisition unit. A selection unit that performs machine learning prediction processing using two or more candidate images including the above original images and specifies one or more target images to be used for recognition processing.
A recognition unit that performs recognition processing on the optical signal using the one or more target images and acquires the recognition result.
An image recognition device including an output unit that outputs the recognition result. A composite image acquisition unit that synthesizes the two or more original images and acquires the composite image is further provided.
The selection unit
The image recognition device according to claim 1, wherein machine learning prediction processing is performed using the selective learning device and at least two or more candidate images including the composite image to specify one or more target images. The recognition unit
Using the recognition learner acquired by performing learning processing using a set including two or more original images and the recognition result, machine learning prediction processing is performed using the two or more original images, and the recognition result is obtained. The image recognition device according to claim 1 or 2 , to be acquired. The recognition unit
Any one of claims 1 to 3 that performs recognition processing on the optical signal using the one or more target images, recognizes an object, and acquires a recognition result that is an object identifier that identifies the object. The image recognition device described in 1. A learning system including an image storage device and a learning device.
The image storage device is
An optical signal acquisition unit that shoots and acquires an optical signal,
An original image acquisition unit that acquires two or more different original images using the optical signal,
A recognition result reception unit that receives the recognition result for the optical signal,
It includes a teacher data storage unit that stores teacher data having one or more candidate images of two or more candidate images including the two or more different original images and the recognition result.
The learning device is
A recognition learning unit that performs learning processing and acquires a recognition learner using two or more teacher data accumulated by the image storage device.
It is provided with a learning device storage unit that stores the recognition learning device.
The image storage device is
A set output unit that outputs a set including two or more candidate images including the two or more different original images, and a set output unit.
A selection reception unit that accepts the selection of one candidate image among the two or more candidate images included in the set is further provided.
A distinguishing unit for performing a distinguishing process is further provided, with one candidate image corresponding to the selection accepted by the selection receiving unit as a positive example and one or more candidate images not selected as a negative example.
The learning device is
A selective learning unit that performs learning processing and acquires a selective learning device by using two or more sets including one candidate image of a positive example and one or more candidate images of a negative example is further provided.
The learning device storage unit
A learning system that accumulates the selective learning device. The image storage device is
A composite image acquisition unit that synthesizes the two or more original images and acquires the composite image is further provided.
The teacher data storage unit
The learning system according to claim 5 , wherein teacher data having one or more candidate images out of three or more candidate images including the two or more different original images and the composite image and the recognition result is accumulated. An optical signal acquisition unit that shoots and acquires an optical signal,
An original image acquisition unit that acquires two or more different original images using the optical signal,
A recognition result reception unit that receives the recognition result for the optical signal,
A teacher data storage unit that stores teacher data having one or more candidate images of two or more candidate images including the two or more different original images and the recognition result, and a teacher data storage unit.
A set output unit that outputs a set including two or more candidate images including the two or more different original images, and a set output unit.
A selection reception unit that accepts the selection of one candidate image among the two or more candidate images included in the set is further provided.
It is provided with a distinguishing unit that performs a process of distinguishing one candidate image corresponding to the selection received by the selection receiving unit as a positive example and one or more candidate images not selected as a negative example.
Learning processing is performed using the above two or more teacher data, and a recognition learner is acquired and accumulated.
An image storage device in which two or more sets including one candidate image of the positive example and one or more candidate images of the negative example are used, learning processing is performed, and a selective learning device is acquired and stored. It is an image recognition method realized by an optical signal acquisition unit, an original image acquisition unit, a selection unit, a recognition unit, and an output unit.
An optical signal acquisition step in which the optical signal acquisition unit takes a picture and acquires an optical signal,
An original image acquisition step in which the original image acquisition unit acquires two or more different original images, which are two or more spectroscopic images extracted from the optical signal with a part of wavelengths.
A selection learner acquired by performing a learning process by the selection unit using at least two or more original images and two or more teacher data having image identification information for identifying the selected image, and the original image acquisition. A selection step of performing machine learning prediction processing using two or more candidate images including two or more original images acquired by the department and specifying one or more target images to be used for recognition processing.
A recognition step in which the recognition unit performs recognition processing on the optical signal using the one or more target images and acquires a recognition result.
An image recognition method in which the output unit includes an output step for outputting the recognition result. An image storage method realized by an optical signal acquisition unit, an original image acquisition unit, a recognition result reception unit, a teacher data storage unit, a set output unit, a selection reception unit, and a distinction unit.
An optical signal acquisition step in which the optical signal acquisition unit takes a picture and acquires an optical signal,
An original image acquisition step in which the original image acquisition unit acquires two or more different original images using the optical signal.
A set output step in which the set output unit outputs a set including two or more candidate images including the two or more different original images, and a set output step.
A selection reception step in which the selection reception unit accepts the selection of one candidate image from the two or more candidate images included in the set.
A distinction step in which the discrimination unit performs a process of distinguishing one candidate image corresponding to the selection received in the selection acceptance step as a positive example and one or more candidate images not selected as a negative example. Equipped with
Learning processing is performed using the above two or more teacher data, and a recognition learner is acquired and accumulated.
An image storage method in which a learning process is performed, a selective learner is acquired, and a storage is performed using two or more sets including one candidate image of the positive example and one or more candidate images of the negative example. Computer,
An optical signal acquisition unit that shoots and acquires an optical signal,
An original image acquisition unit that acquires two or more different original images, which are two or more spectroscopic images obtained by extracting a part of wavelengths from the optical signal.
A selective learner acquired by performing a learning process using at least two or more original images and two or more teacher data having image identification information for identifying the selected image, and 2 acquired by the original image acquisition unit. A selection unit that performs machine learning prediction processing using two or more candidate images including the above original images and specifies one or more target images to be used for recognition processing.
A recognition unit that performs recognition processing on the optical signal using the one or more target images and acquires the recognition result.
A program for functioning as an output unit that outputs the recognition result. Computer,
An optical signal acquisition unit that shoots and acquires an optical signal,
An original image acquisition unit that acquires two or more different original images using the optical signal,
A recognition result reception unit that receives the recognition result for the optical signal,
A teacher data storage unit that stores teacher data having one or more candidate images of two or more candidate images including the two or more different original images and the recognition result, and a teacher data storage unit.
A set output unit that outputs a set including two or more candidate images including the two or more different original images, and a set output unit.
A program for functioning as a selection reception unit that accepts the selection of one candidate image among two or more candidate images included in the set.
It is provided with a distinguishing unit that performs a process of distinguishing one candidate image corresponding to the selection received by the selection receiving unit as a positive example and one or more candidate images not selected as a negative example.
Learning processing is performed using the above two or more teacher data, and a recognition learner is acquired and accumulated.
A program in which a learning process is performed, a selective learner is acquired, and accumulated by using two or more sets including one candidate image of the positive example and one or more candidate images of the negative example.

Images