JP2018045359A - Image processing device and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device that can perform machine learning including a peripheral area of an area in which an object exists while making an object area to recognize clear.SOLUTION: In an image processing device, detection means detects an area where an object exists from an image, expansion means expands the area, cut-out means cuts out a first image from the image in the expanded area, generation means generates a second image from an object in the area detected by the detection means, and learning means performs machine learning using the first image and the second image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus and an image processing program.

  In Patent Document 1, it is an object to provide an image recognition method for recognizing an action of a person or the like based on image information, and an image processing apparatus using the image recognition method. An image pickup means for picking up an image of a subject such as a person, a frame separation means for separating the moving image data into frame images, an area dividing means for dividing the frame image into a plurality of small areas, and background image data are stored. Background image storage means, conversion means for determining whether each small area includes a feature region representing a person or the like by using the background image data, and converting it into numeric string data, and the numeric string data as a neural network It is disclosed that it comprises knowledge processing means for generating a pattern code by applying to the above and an action recognition means for recognizing the contents of the action based on the pattern code.

  Patent Document 2 has an object to provide a hand-to-hand pointing point positioning method and a finger gesture determination method in an HCI system. The hand positioning method continuously acquires images of a controlled region, and a foreground object is obtained. Obtaining a sequence of included images; extracting a foreground image from the captured image; performing binarization processing on the extracted foreground image to generate a binarized foreground image; and binarization Obtaining a set of pixel vertices of the minimum convex hull of the foreground image, constructing a region of interest as a candidate region including a hand around each pixel vertex of the minimum convex hull, and hand from the constructed regions of interest Extracting an image feature, and determining a region of interest in which the hand is included in the candidate region by a pattern recognition method.

  In Patent Document 3, it is an object to provide an image object detection device capable of obtaining an objective detection result when a visual target object is detected from an image including an object. By inputting an image containing multiple objects to the pulse neural network, the gazing point detection unit that detects the gaze point of the viewer that changes over time in the image as time series data, and the gazing point detected by the gazing point detection unit A gazing point visual field setting unit that sets a gazing point visual field region in an image based on the viewpoint, and an object included in the gazing point visual field region set by the gazing point visual field setting unit among a plurality of objects is detected as a visual target object. And a detection information storage unit that accumulates and stores predetermined detection information related to the visual target object detected each time the object detection unit detects the visual target object. To have.

JP 2002-123834 A JP 2012-059271 A Japanese Unexamined Patent Publication No. 2015-210695

Machine learning (hereinafter also simply referred to as learning) for images is performed. By including the area around the object, it may be proved to be the object.
However, in the technique in which only the target area is learned from the image, the learning is not performed in consideration of the area around the area. On the other hand, if learning is performed by simply enlarging a region, it is unclear which region should be recognized.

The gist of the present invention resides in the inventions of the following items.
The invention of claim 1 is a detection means for detecting an area where an object exists from an image, an enlargement means for enlarging the area, and a cutout means for cutting out a first image from the image at the enlarged area. An image processing apparatus comprising: generating means for generating a second image from an object in the region detected by the detecting means; and learning means for performing machine learning using the first image and the second image. It is.

  The invention according to claim 2 is the image processing apparatus according to claim 1, wherein the generation unit generates a second image in which the first part of the target and the second part other than the target are separated. .

  According to a third aspect of the present invention, the generating means generates a second image as a binary image having the first portion as a first value and the second portion as a second value. The image processing apparatus according to Item 2.

  The invention according to claim 4 is the image processing apparatus according to claim 3, wherein the generation means generates a second image having the same size as the first image.

  The invention according to claim 5 is the image processing apparatus according to claim 1, wherein the object is a character, and the learning unit learns whether or not the image in the region detected by the detection unit is a character. is there.

  The invention according to claim 6 is the image processing apparatus according to claim 5, wherein the learning means outputs a numerical value indicating the possibility of being a character and a numerical value indicating a possibility of not being a character.

  According to a seventh aspect of the present invention, there is provided a detecting means for detecting an area where an object is present from an image, an enlarging means for enlarging the area, and a cutting means for extracting a first image from the image at the enlarged area. Recognition that performs recognition using a learning result by the image processing apparatus according to any one of claims 1 to 6, and a generation unit that generates a second image from an object in an area detected by the detection unit An image processing apparatus having means.

  According to an eighth aspect of the present invention, there is provided a computer for detecting a region where a target exists from an image, an enlarging unit for enlarging the region, and a cutout for cutting out a first image from the image at the enlarged region. Functioning as output means, generation means for generating a second image from the object in the region detected by the detection means, and learning means for performing machine learning using the first image and the second image This is an image processing program.

  According to the ninth aspect of the present invention, there is provided a computer for detecting a region in which an object exists from an image, a magnifying unit for enlarging the region, and a cutout for cutting out a first image from the image at the enlarged region. And a generating unit that generates a second image from an object within the region detected by the detecting unit, and a recognizing unit that performs recognition using a learning result obtained by the image processing program according to claim 8. This is an image processing program.

  According to the image processing apparatus of the first aspect, it is possible to perform machine learning including a region around a region where the target exists while clarifying a target region to be recognized.

  According to the image processing apparatus of the second and third aspects, the second image obtained by dividing the first part of the target and the second part other than the target is the target of machine learning.

  According to the image processing apparatus of the fourth aspect, learning is performed using an image of a region having a uniform size.

  According to the image processing apparatus of the fifth and sixth aspects, learning is performed as to whether or not the image in the detected area is a character.

  According to the image processing apparatus of the seventh aspect, it is possible to perform recognition including the area around the area where the target exists, while clarifying the target area to be recognized.

  According to the image processing program of the eighth aspect, it is possible to perform machine learning including a region around a region where the target exists while clarifying a target region to be recognized.

  According to the image processing program of the ninth aspect, it is possible to perform recognition including the area around the area where the target exists while clarifying the target area to be recognized.

It is a conceptual module block diagram about the structural example of this Embodiment (image recognition learning apparatus). It is a conceptual module block diagram about the structural example of this Embodiment (image recognition apparatus). It is explanatory drawing which shows the system configuration example using this Embodiment. It is a flowchart which shows the process example by this Embodiment (image recognition learning apparatus). It is a flowchart which shows the process example by this Embodiment (image recognition apparatus). It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is a block diagram which shows the hardware structural example of the computer which implement | achieves this Embodiment.

Hereinafter, an example of a preferred embodiment for realizing the present invention will be described with reference to the drawings.
FIG. 1 shows a conceptual module configuration diagram of a configuration example of the present embodiment (image recognition learning apparatus 100).
The module generally refers to components such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment is a computer program for causing these modules to function (a program for causing a computer to execute each procedure, a program for causing a computer to function as each means, and a function for each computer. This also serves as an explanation of the program and system and method for realizing the above. However, for the sake of explanation, the words “store”, “store”, and equivalents thereof are used. However, when the embodiment is a computer program, these words are stored in a storage device or stored in memory. This means that control is performed so as to be stored in the apparatus. Modules may correspond to functions one-to-one, but in mounting, one module may be configured by one program, or a plurality of modules may be configured by one program, and conversely, one module May be composed of a plurality of programs. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. Hereinafter, “connection” is used not only for physical connection but also for logical connection (data exchange, instruction, reference relationship between data, etc.). “Predetermined” means that the process is determined before the target process, and not only before the process according to this embodiment starts but also after the process according to this embodiment starts. Also, if it is before the target processing, it is used in accordance with the situation / status at that time or with the intention to be decided according to the status / status up to that point. When there are a plurality of “predetermined values”, they may be different values, or two or more values (of course, including all values) may be the same. In addition, the description of “do B when A” is used to mean “determine whether or not A and do B when A”. However, the case where it is not necessary to determine whether or not A is excluded.
In addition, the system or device is configured by connecting a plurality of computers, hardware, devices, and the like by communication means such as a network (including one-to-one correspondence communication connection), etc., and one computer, hardware, device. The case where it implement | achieves by etc. is also included. “Apparatus” and “system” are used as synonymous terms. Of course, the “system” does not include a social “mechanism” (social system) that is an artificial arrangement.
In addition, when performing a plurality of processes in each module or in each module, the target information is read from the storage device for each process, and the processing result is written to the storage device after performing the processing. is there. Therefore, description of reading from the storage device before processing and writing to the storage device after processing may be omitted. Here, the storage device may include a hard disk, a RAM (Random Access Memory), an external storage medium, a storage device via a communication line, a register in a CPU (Central Processing Unit), and the like.

  The image recognition learning apparatus 100 according to the present embodiment performs machine learning for image recognition. As shown in the example of FIG. 1, the image reception module 105, the character cutout module 110, and the region extraction module 115 are used. , An area enlargement module 120, a map image generation module 125, an image cutout module 130, a correct answer assignment module 135, a machine learning module 140, and a learning result output module 145.

The image receiving module 105 is connected to the character cutting module 110 and the image cutting module 130. The image reception module 105 receives an image and passes the image to the character cut module 110. Accepting an image means, for example, reading an image with a scanner, a camera, etc., receiving an image from an external device via a communication line by fax, etc., a hard disk (in addition to what is built in a computer, via a network) And the like, and the like read out the images stored in the device etc.). The image may be a binary image or a multi-value image (including a color image). One image may be received or a plurality of images may be received. The image received by the image receiving module 105 includes the target image. Further, the contents of the image may be a photographic image in which the object is reflected, a pamphlet for advertising, or the like.
In addition, here, for example, a character, a person (a face, a part of a face, a body, a part of a body, etc.), an object (an artificial object such as a car or an airplane, a natural object such as a mountain, a river, a dog, a cat, etc.) Etc.). Hereinafter, although a character is illustrated as an object, a person, an object, etc. may be sufficient. As an image including a character image, for example, an image obtained by photographing a signboard, a sign, a price tag, or the like is applicable.

The character cutting module 110 is connected to the image receiving module 105, the region extracting module 115, and the map image generating module 125. The character cutting module 110 detects an area where the target character exists from the image. For example, the character cutting module 110 performs MSER (Maximally Stable External Regions) processing. The MSER process is a process for acquiring a bounding box (also called a bounding box) of an area where characters exist.
The area extraction module 115 is connected to the character cutting module 110, the area enlargement module 120, and the map image generation module 125. The area extraction module 115 extracts the area detected by the character cutout module 110. Specifically, the area is specified by a value indicating the position and size of the rectangular area in the original image received by the image receiving module 105. For example, there are combinations of the position of the upper left corner (X coordinate, Y coordinate) of the rectangular area, the width and height of the rectangular area, and the like.

  The area enlargement module 120 is connected to the area extraction module 115 and the image cutout module 130. The area enlargement module 120 enlarges the area. Note that the area after the enlargement is hereinafter also referred to as a “cut-out area”. The enlargement here means enlargement of the area, not so-called image enlargement processing (processing for increasing the resolution). The enlargement may be to enlarge the rectangular area by a predetermined value (X dot for width, Y dot for height, etc.) or a predetermined magnification (width and height for the rectangular area). 1 or more) may be multiplied, or the rectangular area may be enlarged so that at least a part of the adjacent rectangular areas is included. The center of the enlargement may be the center of the rectangular area, the upper left corner of the rectangular area, or the like.

The map image generation module 125 is connected to the character cutting module 110, the region extraction module 115, and the machine learning module 140. The map image generation module 125 generates a second image from the object in the area detected by the character cutout module 110.
Further, the map image generation module 125 may generate a second image in which a first part of the target (hereinafter also referred to as a region of interest) and a second part other than the target are separated. This second image is a so-called map image (mask image). The “first part” is a part indicating a shape indicating the character itself. The “second part” is a part other than the first part and is a so-called background.
Further, the map image generation module 125 may generate the second image as a binary image having the first portion as the first value and the second portion as the second value. Since it is a binary image, it can be cut out by performing EOR processing with the original image as mask processing.
Further, the map image generation module 125 may generate a second image having the same size (size) as the first image. Here, “the same size as the first image” refers to the size of the area after being enlarged by the area enlargement module 120. As a result, learning can be performed using an image having a uniform size. Handling images of the same size means that the dimensions of the vectors are the same. Of course, the positions of the first image and the second image in the original image received by the image receiving module 105 are also the same. Therefore, the first image and the second image are intended for the same partial image (an image of a region where the target exists) in the original image, and the first image is the partial image itself as described later. The second image is a map image of the partial image.

  The image cutout module 130 is connected to the image reception module 105, the area enlargement module 120, and the machine learning module 140. The image cutout module 130 cuts out a first image (hereinafter also referred to as a candidate area image) from the image received by the image receiving module 105 in the area enlarged by the area enlargement module 120. Of course, the area enlarged by the area enlargement module 120 includes the area cut out by the character cutout module 110, and thus the target image is included, and the surrounding image is also included. Will be. There may be other objects in the peripheral image, and it can be determined more clearly whether or not the object detected by the character cutting module 110 is a true object. When the target is a character, the image cutout module 130 cuts out the character including the character adjacent to the character detected by the character cutout module 110. For example, when the character image detected by the character cutout module 110 is “1”, it is often difficult to determine whether or not the character image is based on its shape alone. However, if there are other characters (“8”, “day”, etc.) in the vicinity, it is easy to determine that there is a character “1”. That is, the accuracy of determination can be increased. On the other hand, simply having a plurality of objects in a region does not clearly indicate which is the object of learning. Therefore, the second image generated by the map image generation module 125 is also an input object for learning by the machine learning module 140.

  The correct answer giving module 135 is connected to the machine learning module 140. The correct answer giving module 135 gives the machine learning module 140 the correct answer (teacher data) for the image in the area detected by the character cutout module 110. When the target is a character, there is information indicating whether or not the image in the area detected by the character cutting module 110 is a character, the character code of the character, and the like.

The machine learning module 140 is connected to the map image generation module 125, the image cutout module 130, the correct answer assignment module 135, and the learning result output module 145. The machine learning module 140 performs machine learning using the first image cut out by the image cut-out module 130 and the second image generated by the map image generation module 125. This machine learning is learning for recognizing a target cut out by the character cutout module 110, and is learning for determining whether or not there is a target and what the target is. Specifically, when the target is a character, learning is performed to determine whether or not the image detected by the character cutting module 110 is a character, what character is the character (general character recognition), and the like. It is. Then, when learning whether or not it is a character, the machine learning module 140 may output a numerical value indicating the possibility of being a character and a numerical value indicating the possibility of not being a character.
Examples of the machine learning performed by the machine learning module 140 include CNN (Convolutional Neural Networks), support vector machine (SVM) and the like. Hereinafter, CNN will be described as an example. In the CNN, an image feature extraction method is automatically learned from a large amount of learning data. After learning, feature amounts can be extracted and recognized (labeled) simply by inputting an image to the CNN. Note that the CNN incorporates processing (convolution processing and pooling processing to be described later) that absorbs the shift even if the position of the object in the image is slightly shifted. In particular, in R-CNN (Regions with Evolutionary Neural Network), a target region is extracted from an image by a character cutting module 110 and a region extracting module 115, and the image is cut based on the extracted region. The cut image (candidate area image) is input to the CNN (machine learning module 140) to recognize the target. When applied to a character area, as described above, the area is extracted by MSER, the extracted area is cut out, input to the CNN, and labeled. The output of CNN may be a score or a character recognition result. Of course, unsupervised learning may be performed without using the correct answer assignment module 135.

  The learning result output module 145 is connected to the machine learning module 140. The learning result output module 145 passes the learning result to the image recognition apparatus 200. Specifically, the learning result is a CNN for which machine learning has been completed, and functions as the recognition module 240 of the image recognition apparatus 200.

FIG. 2 is a conceptual module configuration diagram of a configuration example of the present embodiment (image recognition apparatus 200).
The image recognition apparatus 200 according to the present embodiment performs image recognition using the learning result of the image recognition learning apparatus 100. As shown in the example of FIG. A region extraction module 215, a region enlargement module 220, a map image generation module 225, an image cutout module 230, a recognition module 240, and a recognition result output module 245. .
The image receiving module 205, the character cutting module 210, the area extracting module 215, the area expanding module 220, the map image generating module 225, and the image cutting module 230 in the image recognition apparatus 200 are the image recognition learning shown in the example of FIG. The image receiving module 105, the character cutting module 110, the area extracting module 115, the area expanding module 120, the map image generating module 125, and the image cutting module 130 of the apparatus 100 have the same processing and functions. Therefore, the image recognition learning device 100 and the image recognition device 200 may share these modules. Note that description of these modules is omitted.
The image receiving module 205 is connected to the character cutting module 210 and the image cutting module 230.
The character cutting module 210 is connected to the image receiving module 205, the region extracting module 215, and the map image generating module 225.
The area extraction module 215 is connected to the character cutting module 210, the area enlargement module 220, and the map image generation module 225.
The area enlargement module 220 is connected to the area extraction module 215 and the image cutout module 230.
The map image generation module 225 is connected to the character cutout module 210, the region extraction module 215, and the recognition module 240.
The image cutout module 230 is connected to the image reception module 205, the area enlargement module 220, and the recognition module 240.

The recognition module 240 is connected to the map image generation module 225, the image cutout module 230, and the recognition result output module 245. The recognition module 240 performs recognition using the learning result obtained by the machine learning module 140 of the image recognition learning apparatus 100. For example, recognition is performed using CNN machine-learned by the image recognition learning apparatus 100. Specifically, when the target is a character, the image detected by the character cutting module 210 recognizes whether or not the image is a character, what character is the character (general character recognition), and the like.
The recognition result output module 245 is connected to the recognition module 240. The recognition result output module 245 outputs the recognition result by the recognition module 240. For example, the recognition result is displayed on a display device such as a display, printed on a printing device such as a printer, written in a storage device such as a database, stored in a storage medium such as a memory card, or other information processing. Including delivery to the device.

FIG. 3 is an explanatory diagram showing a system configuration example using the present embodiment.
In the example illustrated in FIG. 3A, the image recognition learning device 100 is connected to the image acquisition device 310. This is a so-called stand-alone system configuration. The image acquisition device 310 is a scanner, a camera, or the like. Then, the learning result is embedded in the image recognition apparatus 200.
In the example shown in FIG. 3B, the image recognition device 200 is connected to the image capturing device 320. This is a so-called stand-alone system configuration. The image capturing device 320 is a scanner, a camera, or the like. The image recognition apparatus 200 and the image capturing apparatus 320 may be a recognition apparatus having an integrated configuration.
In the example illustrated in FIG. 3C, the user terminal 330 </ b> A, the user terminal 330 </ b> B, the user terminal 330 </ b> C, and the image processing device 350 are connected via a communication line 390. The communication line 390 may be wireless, wired, or a combination thereof, and may be, for example, the Internet or an intranet as a communication infrastructure. Further, the function of the image processing apparatus 350 may be realized as a cloud service.
The image processing device 350 includes an image recognition learning device 100 and an image recognition device 200. For example, the user terminal 330A transmits an image for learning to the image processing apparatus 350, and after learning with the image recognition learning apparatus 100, the captured image is processed with the user terminal 330A, the user terminal 330B, the user terminal 330C, and the like. The result of recognition by the image recognition apparatus 200 may be sent back to the apparatus 350. Further, as described above, since the image recognition learning apparatus 100 and the image recognition apparatus 200 have a common module, the image recognition learning apparatus 100 and the image recognition apparatus 200 are used as the image recognition learning apparatus 100 as an integrated configuration. After that, the image recognition apparatus 200 may be used.

FIG. 4 is a flowchart showing an example of processing according to this embodiment. It is an example of a learning process by the image recognition learning apparatus 100.
In step S402, the image reception module 105 receives an image. The image here is a learning target.
In step S404, the character cutting module 110 performs a character cutting process. For example, MSER processing is performed.
In step S406, the region extraction module 115 extracts a region (obtained from the XY coordinates of the input image).
In step S408, the area expansion module 120 expands the range of the extracted area (cut area).

In step S410, the image cutout module 130 cuts out an image (candidate area image) in the cutout area from the input image.
In step S412, the map image generation module 125 creates an image (map image) in which the region of interest and the other region are filled with binary values.
In step S414, the machine learning module 140 receives a candidate area image and a map image.
In step S416, the machine learning module 140 performs machine learning. Of course, the processing from step S402 to step S416 is repeated a plurality of times.
In step S418, the learning result output module 145 outputs the learning result.

FIG. 5 is a flowchart showing an example of processing according to the present embodiment. It is an example of recognition processing by the image recognition apparatus 200.
In step S502, the image reception module 205 receives an image. The image here is a recognition target.
In step S504, the character cutting module 210 performs a character cutting process. For example, MSER processing is performed.
In step S506, the region extraction module 215 extracts a region (obtained from the XY coordinates of the input image).
In step S508, the region enlargement module 220 expands the range of the extracted region (cut region).

In step S510, the image cutout module 230 cuts out an image (candidate area image) in the cutout area from the input image.
In step S512, the map image generation module 225 creates an image (map image) in which the region of interest and the other region are filled with binary values.
In step S514, the recognition module 240 receives a candidate area image and a map image.
In step S516, the recognition module 240 performs a recognition process.
In step S518, the recognition result output module 245 outputs the recognition result.

FIG. 6 is an explanatory diagram showing a processing example according to the present embodiment. A specific example of recognition processing by the image recognition apparatus 200 is shown.
In step S602, the image receiving module 205 receives the image 650.
In step S604, the character cutout module 210 performs MSER processing to generate an MSER processed image 652.
In step S606, the region extraction module 215 extracts a region and acquires the XY coordinates (for example, the coordinates, width, and height of the upper left corner) of the MSER region image 654.
In step S608, the area enlargement module 220 enlarges the cut area. Here, the MSER region 656 is expanded as an MSER expansion region 658, and the region 660 is expanded as an expansion region 662.
In step S610-1, the image cutout module 230 cuts out the candidate area image 664 from the image 650.
In step S610-2, the map image generation module 225 creates a map image 666 in the MSER enlarged region.
In step S612, the recognition module 240 performs CNN. Then, the recognition result output module 245 outputs a score 670 that is a recognition result. The process of step S612 will be described using the example of FIG.

FIG. 7 is an explanatory diagram illustrating a processing example according to the present exemplary embodiment. A score 670 is calculated based on the candidate area image 664 and the map image 666. The candidate area image 664 is an image including the extracted area and the peripheral area. The map image 666 is an image representing a region of interest.
In step S702, conv1 processing is performed on the candidate area image 664. That is, the first Convolution process is performed. Specifically, a filter process is performed to create a Convolution Layer (conv1 process result).
In step S704, a pool1 process is performed on the conv1 process result. That is, the first Pooling (layer reduction) process is performed. Specifically, the image is compressed to create a Pooling Layer (pool 1 processing result).

In step S706, the conv2 process is performed on the pool1 process result. That is, the second Convolution process is performed.
In step S708, a pool2 process is performed on the conv2 process result. That is, the second pooling process is performed.
In step S710, conv3 processing is performed on the pool2 processing result. That is, the third Convolution process is performed.
In step S712, conv4 processing is performed on the conv3 processing result. That is, the fourth Convolution process is performed.
In step S714, conv5 processing is performed on the conv4 processing result. That is, the fifth Convolution process is performed.

The processing of step S702a to step S714a is the same as the processing of step S702 to step S714 except that the map image 666 is targeted instead of the candidate area image 664.
In step S702a, conv1-a processing is performed on the map image 666.
In step S704a, a pool1-a process is performed on the conv1-a process result.
In step S706a, the conv2-a process is performed on the pool1-a process result.
In step S708a, pool2-a processing is performed on the conv2-a processing result.
In step S710a, conv3-a processing is performed on the pool2-a processing result.
In step S712a, conv4-a processing is performed on the conv3-a processing result.
In step S714a, conv5-a processing is performed on the conv4-a processing result.

In step S716, the fc1 process is performed on the conv5 process result and the conv5-a process result.
In step S718, fc2 processing is performed on the fc1 processing result.
In step S720, the fc3 process is performed on the fc2 process result. Then, a character area score 772 and a non-character area score 774 are output as the score 670.
In steps S716 to S720, a three-layer Full Connect process is performed. That is, the feature map generated in steps S702 to S714 is input for identification.

Here, the meaning of enlarging the region will be described using a specific example. FIG. 8 is an explanatory diagram showing a processing example according to the present embodiment.
In step S <b> 802, the image reception module 105 receives the image 850. In this image 850, a plurality of characters (“alarm” in the example of FIG. 8) are photographed.
In step S804, the character cutting module 110 performs MSER processing on the image 850. Then, the MSER region image 852 (“l” in the example of FIG. 8) is detected as one character.
In step S806, the region extraction module 115 extracts the MSER region image 854 from the MSER region image 852. Specifically, the XY coordinates, width, and height of the upper left corner of the MSER region image 854 that is the bounding box of the MSER region image 852 are extracted.
The region image 856 is an image cut out from the image 850 corresponding to the MSER region image 854. With only this area image 856, it cannot be determined whether the character is “I” or an object similar thereto. In other words, in the technique that does not use the present embodiment (conventional technique), only the region image 856 is input as machine learning, so whether the image such as the region image 856 is a character or an image? The judgment accuracy of was low.

In step S808, the area enlargement module 120 enlarges the cut area. Specifically, the MSER region image 854 is shaped (squared) according to the larger width and height to generate a square MSER region image 858.
The square area image 860 is an image cut out from the image 850 corresponding to the square MSER area image 858.
In step S810, the area enlargement module 120 enlarges the cut area. Specifically, the map image 862 is generated by doubling the square MSER region image 858 both vertically and horizontally.
The candidate area image 864 is an image cut out from the image 850 corresponding to the map image 862. In other words, by enlarging the area, the peripheral area of the area image 856 corresponding to the area extracted by the character cutting module 110 can be added to the machine learning. Specifically, the image “a” is present in the vicinity of the image “I”, and the image “I” is information that assists the determination that the image is “I”. Specifically, it is machine-learned that “there is a high possibility that there are characters around the characters”. However, if the area is simply enlarged (only with the candidate area image 864), information on which part should be determined is lost. Therefore, a map image 862 showing a portion that is an original determination target is also used as an input for machine learning. That is, since the part to be determined by the map image 862 is clear, the other part (image “a” in the example of FIG. 8) can be handled as auxiliary information.

  A hardware configuration example of the image recognition learning device 100 and the image recognition device 200 according to the present embodiment will be described with reference to FIG. The configuration illustrated in FIG. 9 is configured by, for example, a personal computer (PC) or the like, and illustrates a hardware configuration example including a data reading unit 917 such as a scanner and a data output unit 918 such as a printer.

  A CPU (Central Processing Unit) 901 includes various modules described in the above-described embodiments, that is, the image receiving module 105, the character cutting module 110, the region extracting module 115, the region expanding module 120, the map image generating module 125, the image Cutout module 130, correct answer assignment module 135, machine learning module 140, learning result output module 145, image reception module 205, character cutout module 210, area extraction module 215, area enlargement module 220, map image generation module 225, image cutout The processing according to the computer program describing the execution sequence of each module such as the module 230, the recognition module 240, and the recognition result output module 245 is executed. It is a control unit to execute.

  A ROM (Read Only Memory) 902 stores programs used by the CPU 901, operation parameters, and the like. A RAM (Random Access Memory) 903 stores programs used in the execution of the CPU 901, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 904 including a CPU bus or the like.

  The host bus 904 is connected to an external bus 906 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 905.

  A keyboard 908 and a pointing device 909 such as a mouse are devices operated by an operator. The display 910 includes a liquid crystal display device or a CRT (Cathode Ray Tube), and displays various types of information as text or image information. Further, a touch screen or the like having both functions of the pointing device 909 and the display 910 may be used.

  An HDD (Hard Disk Drive) 911 includes a hard disk (may be a flash memory or the like), drives the hard disk, and records or reproduces a program executed by the CPU 901 or information. The hard disk stores a target image, a clipped image, an image to be learned, and the like. Further, various other data, various computer programs, and the like are stored.

  The drive 912 reads out data or a program recorded in a removable recording medium 913 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and the data or program is read out from the interface 907 and the external bus 906. , To the RAM 903 connected via the bridge 905 and the host bus 904. Note that the removable recording medium 913 can also be used as a data recording area.

  The connection port 914 is a port for connecting the external connection device 915 and has a connection unit such as USB or IEEE1394. The connection port 914 is connected to the CPU 901 and the like via the interface 907, the external bus 906, the bridge 905, the host bus 904, and the like. The communication unit 916 is connected to a communication line and executes data communication processing with the outside. The data reading unit 917 is a scanner, for example, and executes document reading processing. The data output unit 918 is, for example, a printer, and executes document data output processing.

  Note that the hardware configurations of the image recognition learning device 100 and the image recognition device 200 shown in FIG. 9 show one configuration example, and the present embodiment is not limited to the configuration shown in FIG. Any configuration capable of executing the module described in the embodiment may be used. For example, some modules may be configured with dedicated hardware (for example, Application Specific Integrated Circuit (ASIC), etc.), and some modules are in an external system and connected via a communication line In addition, a plurality of systems shown in FIG. 9 may be connected to each other via communication lines so as to cooperate with each other. In particular, in addition to personal computers, portable information communication devices (including mobile phones, smartphones, mobile devices, wearable computers, etc.), information appliances, robots, copiers, fax machines, scanners, printers, multifunction devices (scanners, printers, An image processing apparatus having two or more functions such as a copying machine and a fax machine) may be incorporated.

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standard “DVD + R, DVD + RW, etc.”, compact disc (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), Blu-ray disc ( Blu-ray (registered trademark) Disc), magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM (registered trademark)) )), Flash memory, Random access memory (RAM) SD (Secure Digital) memory card and the like.
Then, the whole or a part of the program may be recorded on the recording medium for storage or distribution. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, or a wireless communication It may be transmitted using a transmission medium such as a network or a combination of these, or may be carried on a carrier wave.
Furthermore, the program may be a part or all of another program, or may be recorded on a recording medium together with a separate program. Moreover, it may be divided and recorded on a plurality of recording media. Further, it may be recorded in any manner as long as it can be restored, such as compression or encryption.

DESCRIPTION OF SYMBOLS 100 ... Image recognition learning apparatus 105 ... Image reception module 110 ... Character cutting module 115 ... Area extraction module 120 ... Area expansion module 125 ... Map image generation module 130 ... Image extraction module 135 ... Correct grant module 140 ... Machine learning module 145 ... Learning result output module 200 ... Image recognition device 205 ... Image reception module 210 ... Character cutout module 215 ... Area extraction module 220 ... Area enlargement module 225 ... Map image generation module 230 ... Image cutout module 240 ... Recognition module 245 ... Recognition result output Module 310 ... Image acquisition device 320 ... Image photographing device 330 ... User terminal 350 ... Image processing device 390 ... Communication line

Claims (9)

Detecting means for detecting a region where the target exists from the image;
Enlarging means for enlarging the area;
Cutting means for cutting out a first image from the image in the enlarged region;
Generating means for generating a second image from the object in the area detected by the detecting means;
An image processing apparatus comprising learning means for performing machine learning using the first image and the second image. The generating means generates a second image obtained by dividing the first part of the object and the second part other than the object;
The image processing apparatus according to claim 1. The generating means generates a second image as a binary image having the first portion as a first value and the second portion as a second value.
The image processing apparatus according to claim 2. The generating means generates a second image having the same size as the first image;
The image processing apparatus according to claim 3. Character as the object,
The learning means learns whether the image in the area detected by the detection means is a character;
The image processing apparatus according to claim 1. The learning means outputs a numerical value indicating the possibility of being a character and a numerical value indicating a possibility of not being a character,
The image processing apparatus according to claim 5. Detecting means for detecting a region where the target exists from the image;
Enlarging means for enlarging the area;
Cutting means for cutting out a first image from the image in the enlarged region;
Generating means for generating a second image from the object in the area detected by the detecting means;
An image processing apparatus comprising: a recognition unit configured to perform recognition using a learning result obtained by the image processing apparatus according to claim 1. Computer
Detecting means for detecting a region where the target exists from the image;
Enlarging means for enlarging the area;
Cutting means for cutting out a first image from the image in the enlarged region;
Generating means for generating a second image from the object in the area detected by the detecting means;
An image processing program for causing a machine to perform machine learning using the first image and the second image. Computer
Detecting means for detecting a region where the target exists from the image;
Enlarging means for enlarging the area;
Cutting means for cutting out a first image from the image in the enlarged region;
Generating means for generating a second image from the object in the area detected by the detecting means;
The image processing program for functioning as a recognition means which performs recognition using the learning result by the image processing program of Claim 8.

Images