JP7037875B2 - Image normalization equipment, methods, and computer-readable recording media - Google Patents

Description

The present invention relates to image normalization devices, methods, and computer-readable recording media, and in particular, image normalization devices, methods, and computer-readable images that produce normalized images for input to a recognizer. Regarding recording media.

In the field of image recognition, when the distance between an object and the camera that shoots the object changes, or the object rotates with respect to the camera, and the image of the object in the image is rotated / enlarged / reduced. , It is difficult to recognize / identify them invariantly with a simple image recognition technique. In order to recognize the rotation / enlargement / reduction pattern only in the image recognition part, it is necessary to prepare a pattern recognition / classifier corresponding to each change in advance, and a huge number of pattern recognition / classifiers are required. Become. As a method of avoiding these, a method of calculating the feature amount of the image pattern to be recognized and comparing it with the feature amount of the input image pattern has been devised. For example, SIFT (Histogram of Oriented Gradients) / SURF (Speed Up Robot Features) [SIFT / SURF], which are feature quantities of images described in Non-Patent Document 1, depend on deformation of rotation, enlargement, and reduction. It is possible to discriminate / match the image pattern by using the feature amount that does not exist.

Fujiyoshi Laboratory, Department of Computer Science, Faculty of Engineering, Chubu University, "Image Local Features and Specific Object Recognition-SIFT and Recent Approaches-", [Search June 20, 2016], Internet <http://www.vision.cs. chubu.ac.jp/cvtutorial/PDF/02SIFTandMore.pdf>

However, these features are two-dimensional planes with respect to enlargement and reduction of the target image in order to obtain the scale of the target image using a filter by a two-dimensional isotropic Gaussian function called Orthogonality-of-Gaussian (DoG). Only the isotropic case with two orthogonal axes is assumed. Therefore, SIFT / SURF cannot be used for invariant discrimination / matching of images that are anisotropically enlarged / reduced on a two-dimensional orthogonal axis. In general, the appearance of a three-dimensional object in a two-dimensional plane on an image is a projective transformation, and the trapezoidal transformation, which is a part of it, is a transformation of two orthogonal axes on the image within a certain rotation angle range. It can be approximated to independent enlargement / reduction. If invariant recognition that is independent enlargement / reduction is possible on the two axes, some trapezoidal transformations of the projective transformation can be invariantly recognized, and the application to trapezoidal transformations that was difficult with SIFT / SURF will be expanded to some extent.

Further, feature quantities such as SIFT / SURF are used for performing template matching (see Reference 1) for comparing a local region in an input image with a specific object image called a template and determining whether they are the same. Is the main thing. For example, it is effective for identifying / detecting specific objects such as road signs, but it is a SIFT / SURF feature as it is for class classification (image classification) of objects in images and general object recognition / detection for class classification. There was the problem that it was difficult to use the quantity.

[Reference 1] Chukyo University Faculty of Engineering Manabu Hashimoto, "Charm of Template Matching", Internet <http://isl.sist.chukyo-u.ac.jp/Archives/SSII2013TS-Hashimoto.pdf>

Due to this problem, it has been proposed to perform image classification by a method called bag-of-keypoints (BoK), but this method ignores the positional relationship of multiple local features on the image. , Has important side effect drawbacks in image recognition. That is, if the set of local features is the same, even if the positions of the local images are exchanged, they are classified as the same class. The method called bag-of-keypoints regards an image as a collection of local features (keypoints) used in SIFT / SURF (bag-of-keypoints), and divides the images into categories according to the statistic. (See Reference 2).

[Reference 2] Fujiyoshi Laboratory, Department of Computer Science, Faculty of Engineering, Chubu University, "Object Detection by Local Features and Statistical Learning Methods", [Search on June 20, 2016], Internet <http://www.vision.cs. chubu.ac.jp/CVTutorial/PDF/03ObjectDetection.pdf>

Therefore, the positional relationship of the local features on the image is ignored. In general, image classification is a function to classify what category an object in an image belongs to, and if it is in the same category, it ignores the individual difference and classifies it based on common characteristics, and it is in a different category. In the meantime, it is necessary to classify according to the feature difference between the categories. Therefore, for image classification, it is necessary to configure an image recognizer by a statistical learning method such as machine learning, and the template matching used for the above-mentioned identification / detection of a specific object may be used for classification. It is usually difficult except by introducing special ideas such as forming categories with the template of.

As mentioned above, it is difficult to simultaneously recognize the two points of invariantly recognizing an image that is enlarged / reduced in an image two-dimensional plane and using SIFT / SURF features for general object recognition. There was a challenge.

The present invention has been made in view of the above problems, and is an image normalization device and a method capable of generating a normalized image by anisotropic enlargement or reduction according to recognition using a recognizer. , And to provide a recording medium.

In order to achieve the above object, the image normalization device according to the first invention is an image normalization device that generates a normalized image to be input to a recognizer from an input image, and is the input image. A candidate for acquiring a candidate image by surrounding a diagram pattern, which is an area representing an object reflected in the image, with a rectangular frame so that the diagram pattern does not protrude and is in contact with the diagram pattern. It is configured to include an acquisition unit and an image normalization unit that normalizes the candidate image by enlarging or reducing the candidate image so that the rectangular frame has the aspect ratio of the object recognized by the recognizer. There is.

Further, in the image normalization device according to the first invention, the candidate acquisition unit is a predetermined of a plurality of rectangular frames obtained by rotating either one of the figure pattern and the rectangular frame. The candidate images may be acquired by using the rectangular frame that satisfies the criteria for the rectangular frame.

Further, in the image normalization apparatus according to the first invention, the reference condition may be such that the outer peripheral length, the diagonal length, or the area of the rectangular frame is the minimum. ..

The image normalization method according to the second invention is an image normalization method in an image normalization device that generates a normalized image for input to a recognizer from an input image, and the candidate acquisition unit is described above. A diagram pattern, which is an area representing an object shown in an input image, is surrounded by a rectangular frame so that the diagram pattern does not protrude and is in contact with the diagram pattern, and a candidate image is acquired using the rectangular frame. The image normalization unit includes a step of normalizing the candidate image by enlarging or reducing the candidate image so that the rectangular frame has the aspect ratio of the object recognized by the recognizer. It is characterized by executing.

Further, in the image normalization method according to the second invention, the step of acquiring the candidate image by the candidate acquisition unit is a plurality of rectangular frames obtained by rotating either one of the diagram pattern and the rectangular frame. Of these, the candidate images may be acquired by using the rectangular frame that satisfies the predetermined criteria for the rectangular frame.

Further, in the image normalization method according to the second invention, the reference condition may be such that the outer peripheral length, the diagonal length, or the area of the rectangular frame is the minimum. ..

Further, the recording medium according to the third invention is a computer-readable recording medium on which a program for making a computer function as each part constituting the image normalization apparatus according to the first invention is recorded.

According to the image normalization device, method, and recording medium of the present invention, a rectangular frame is provided so that the diagram pattern, which is an area representing an object in the input image, does not protrude from the diagram pattern and is in contact with the diagram pattern. Recognized by enclosing with and using a rectangular frame to acquire candidate images and normalizing the candidate image by enlarging or reducing it so that the rectangular frame has the aspect ratio of the object recognized by the recognizer. The effect that a normalized image can be generated by anisotropic enlargement or reduction according to recognition using a vessel can be obtained.

It is a block diagram which shows the structure of the image normalization apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the input image which copied the character. It is a figure which shows an example of a character template. It is a figure which shows an example of the binary image which separated the map area of the background area and the character area. It is a figure which shows an example of each of the figure pattern cut out by the rectangular frame which was first fitted after labeling the connecting area. It is a figure which shows an example of each of the rotated figure patterns. It is a figure which shows an example of each of the candidate images. , Is a diagram showing an example of a candidate image including a rectangular frame having the smallest area and a certain outer area. It is a figure which shows an example of the case where each of the candidate images is normalized. It is a figure which shows an example of the case where the rectangular frame is fitted to the input image. It is a figure which shows an example of the case where the figure pattern which can be read as a character E is recognized. It is a figure which shows an example of the case of rotating a rectangular frame. It is a flowchart which shows the image normalization processing routine in the image normalization apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the input image which showed the fish. It is a figure which shows an example which separated the area of a fish and the area of a background. It is a figure which shows the detection example of the spine of a fish.

<Overview of Embodiments of the Present Invention>

First, an outline of the embodiment of the present invention will be described.

The method of this embodiment provides a method of solving the difficulty of the above-mentioned problem. Of the above two issues, first of all, the function that could be normalized only in the case of an image that is isotropically enlarged / reduced in a two-dimensional plane with respect to the deformation of enlargement / reduction is anisotropically enlarged / reduced. The image to be reduced can also be normalized independently on each 2D axis. As a result, the normalized image is used as an image to be sent to the recognition / discriminator, and the recognition / discrimination process that does not depend on the enlargement / reduction deformation can be performed.

At the same time, the recognition part enables the use of conventional local features so that not only template matching but also input to the recognizer composed of statistical learning can be performed.

<Principle of the Embodiment of the present invention>

Next, the principle in the embodiment of the present invention will be described.

In an embodiment of the present invention, when a two-dimensional image of an object can be separated into two regions, a diagram pattern inside a closed curve (closed region with respect to the background) and another background ground, the diagram pattern protrudes. The above problem is solved by a method of generating a rectangular frame that surrounds the frame so as to be in contact with each other, normalizing the input image including the two-dimensional image in and around the frame, cutting it out, and transmitting it to the recognizer. do. The necessary figure-ground separation is as long as the object to be recognized is in the figure pattern when the figure-ground is separated, and the object to be recognized and the others do not have to be completely separated (if they can be separated). There is no need to recognize it again), and the recognizer makes a judgment by inputting all the figure pattern candidates including the object to be recognized into the recognizer. The size of the image of the object is grasped by a rectangular frame that can be surrounded so that the figure pattern of the object does not protrude. As a result of this processing, the length of each side of the frame is known, so that the enlargement / reduction deformation can be normalized. There are various rectangular frames that surround one figure pattern of a two-dimensional image of an object, but in order to uniquely determine it, one reference condition is set. Naturally, the rectangular frame needs to be able to be surrounded so that the figure pattern of the object does not protrude and touches. It is a standard on that. For example, the outer peripheral length of the rectangular frame is the minimum, the diagonal length of the rectangular frame is the minimum, the area of the rectangular frame is the minimum, and so on. In some cases, the conditions are the same rectangular frame. Depending on the reference condition of one of these, the number of rectangular frames satisfying the reference condition can be at most several and can surround the figure pattern of the target. After that, the images in the rectangular frame and within a certain range around it are normalized according to the size of the rectangular frame, and input to the recognizer at most several times. Of these, the one that most applies is the judgment of the recognizer. A certain area around a rectangular frame is ideally such that the area of the map pattern is halved in and around the rectangular frame, and in reality, it is rectangular from the beginning. You may decide how many times each side of the frame should be used.

Normalization for rotation can be achieved by inputting all cases into the recognizer, taking into account only the four-way possibilities when normalizing the enlargement / reduction according to the size of the rectangular frame. Enter the four possible normalized images into the recognizer and select the one that best fits your needs. As a result, an invariant input can be made for rotation conversion. It is also possible to use the various features of the four normalized images to select only the direction that seems most suitable before inputting everything into the recognizer. The features that can be used vary.

The frame can be a circle like SIFT / SURF, or a polygon such as an ellipse or a triangle, but the reason for making it a rectangle is as follows. Since the image is represented by a two-dimensional plane, there are two independent axes, the x and y axes. Therefore, by using a rectangle having sides parallel to this independent axis, it is possible to express a simple enlargement / reduction deformation of the image. With other polygons, it is difficult to represent a simple enlargement / reduction transformation of an image. A circle can only represent isotropic enlargement / reduction transformations. Also, in the case of an ellipse, it is difficult to find one that touches the target figure pattern. Therefore, the frame is made rectangular.

The specific method of normalization is shown below. Specifically, for example, the center of the figure pattern in a closed area by an object is arbitrarily set, the figure pattern is rotated at various angles, surrounded by a rectangular frame, and its outer peripheral length, diagonal length, or area is set. You can choose the one with the minimum angle. Here, a method of rotating the image of the object without rotating the rectangular frame will be described. Of course, the reverse is also possible as a method. Since the rectangular frame does not protrude from the image pattern of the object and surrounds it so that it touches it, the y coordinate should be a straight line constituting the frame as well as the minimum and maximum x coordinates of the rotated closed area diagram pattern. It is sufficient to compose and make a frame in the same manner at various rotation angles of the image of the object, and select the one that satisfies the reference condition. At that time, the frame corresponding to the rotated figure pattern of 0 degree, 90 degree, 180 degree, and 270 degree as the relative angle difference is selected as satisfying the standard. After normalizing the enlargement / reduction according to the size of the four rectangular frames, it may be input to the recognition / discriminator to determine which rotation angle image is most applicable. At this time, when the recognition / discriminator is a dichotomizer, even if it is considered to be the closest, it cannot be judged only by being relatively close, and it is necessary to make a judgment based on the absolute closeness threshold value. Judgment uses a threshold that can be judged to be correct if this classification is so close.

In the above description, it is assumed that the two-dimensional image of the target object can be separated into two areas, the figure pattern inside the closed curve (closed area with respect to the background) and the other background ground. In this situation, as an image, the scalar value (feature amount, for example, luminance or one component amount of RGB color) assigned to each pixel is divided into binary values (two groups), and the figure and the ground are divided into two groups. This is the case when it can be separated into two areas. The target object in the image can be cut out two-dimensionally in the closed curve area (closed area) in the image as the area including the figure, and the other areas can be separated as the ground. However, if the figure pattern of the two-dimensional image by the object is not a closed area, it becomes impossible to surround it with a rectangular frame. For example, in an image in which adjacent pixels have features having close continuous values, particularly in a black-and-white grayscale luminance image, the brightness of adjacent pixels has close continuous values. In these cases, the image itself, not the brightness value of the pixel, but the feature amount of each pixel is calculated, binarized, etc., and then the figure and the ground are separated, and the image part of the target object is displayed. It is possible to use this method if it is possible to make it a closed region. In the case of luminance, it is a bright / dark region where the luminance value is equal to or higher than a certain threshold value. It is also possible to use the difference between. When the background image is obtained by some method, the difference between the input image and the feature amount of the background image is divided into an area where there is a change of a certain threshold value or more and an area where the difference is not.

The above method can target a closed region obtained by performing image processing using various features. However, there may be cases where it is difficult even with the above method. For example, when targeting a part such as the face of a human or animal, there are cases where the boundary between the face and the neck and torso cannot be obtained by image processing. .. Depending on the lighting conditions, it may be applicable by a method such as binarization. However, in image recognition targeting such a specific category, it is possible to normalize the input image, particularly by utilizing the prior knowledge of the target object. The task of detecting human faces, animal faces, etc. from input images uses eye features, and if the positions of the two eyes can be detected, the size of the face can be estimated from the distance between the eyes, and the area can be estimated. It may be an image of an object. If it is uncertain, a plurality of image candidates of the object may be considered. Even when only one eye or two or more eyes are detected, it is sufficient to consider a plurality of image candidates of the object by a plurality of combinations. In order to detect eyes from an image, a method such as template matching can be used. In addition, there is an example in which the prior knowledge of the object can be used without using the method using the frame. A frame can also be used, but in the case of a fish, for example, a part characteristic of the object such as the spine may be used, so various possibilities can be considered.

In addition, it is conceivable that the ground pattern enters the closed region as the figure pattern of the two-dimensional image by the object. For example, there is a case where an object having a hole such as a donut or a pixel originally classified as a figure pattern in the process of being binarized is classified into a ground pattern for some reason. In this case, the above frame may be applied while it is still inserted. Similar to the above, it can be normalized for rotation, enlargement, and reduction.

In the following description as well, in order to simplify the explanation, an image pattern in which the target object can be separated into a map area as a region in a closed curve is used. However, various conventional image processing is used for the image separation. It can often be achieved by using technology.

The image input to the recognizer is a normalized two-dimensional image including the inside and the periphery of the fitted frame, and the area ratio between the target area and the non-target area is set to be about 1: 1. However, it is desirable for subsequent recognizers and template matching.

In addition, the input image with frame information to be transmitted to the recognizer can be considered in the following two ways. One is a binary feature quantity obtained from the result of performing threshold processing or the like used to make a figure pattern of a two-dimensional image of an object to be recognized into a closed region. Next, it is the feature amount of each pixel calculated from the image cut out with the same size from the original image before the processing. Generally, the recognizer usually recognizes using the input feature image.

Further, since the method of the present embodiment is a method of normalizing the input image, it is not necessary to specify the recognizer / template matching in particular. However, for the template and training image used for the recognizer to be used, a rectangular frame is applied according to the above criteria, and an image normalized to include a certain area outside the rectangular frame is used for recognizer / template matching. Need to be configured. Regarding the angle of rotation, the angle of the characteristic axis of the object is unified, determined and arranged as one, and enlargement / reduction is performed by making the vertical and horizontal lengths of the learning images the same among multiple learning images. Can achieve the conditions for recognizer / template matching.

The input image can be normalized according to the template matching and the recognizer by the above method, but when multiple target 3D objects approach each other and the 2D images overlap, the overlapped image or figure pattern is obtained. Therefore, even if it is normalized by the above method and input to the recognizer, correct recognition cannot be obtained. In this case, it should be noted that recognition cannot be performed without some ingenuity in addition to the above processing.

As described above, the input image to the recognition / classifier can be normalized by the method of the embodiment of the present invention, and invariant pattern recognition / identification can be performed by enlargement / reduction in the two-dimensional independent axis by normalization. .. In general, the appearance of a three-dimensional object in a two-dimensional plane on an image is a projective transformation, and the trapezoidal transformation, which is a part of it, is a transformation of two orthogonal axes on the image within a certain rotation angle range. It can be approximated to independent enlargement / reduction. If invariant recognition that is independent enlargement / reduction is possible on the two axes, some trapezoidal transformations of the projective transformation can be invariantly recognized, and the application to trapezoidal transformations that was difficult with SIFT / SURF will be expanded to some extent.

Further, the method of the embodiment of the present invention can be any feature quantity used for image recognition that can be calculated from an image. Therefore, an example of feature quantity that can be used not only for template matching of specific object recognition but also for general object recognition method with categorization of objects with large individual changes by using statistical recognition / classifier by machine learning. These include a histogram of the number of pixels in the figure of the binarized local region, a histogram of the RGB color component values of the original image before binarization, and the gradient direction of brightness called HOG (Histogram of Oriented Radients) and its. Various things such as an intensity histogram [HOG] can be considered (see Reference 3). As a result, as a statistical recognition / discriminator, not only template matching but also conventional techniques such as AdaBoost, NN (Neural Networks), SVM (Support Vector Machine) [AdaBoost / NN / SVM] can be freely selected. ..

[Reference 3] Fujiyoshi Laboratory, Department of Computer Science, Faculty of Engineering, Chubu University, "People detection using HOG features and Boosting", [Search on June 20, 2016], Internet <http://www.vision.cs. chubu.ac.jp/joint_hog/pdf/HOG+Boosting_LN.pdf>

Further, in the method of the embodiment of the present invention, the position / region of the candidate is determined by using the existing region of the object such as a binary image. Therefore, as a side effect of this method, the input image can be raster-scanned and the candidates for the local image to be input to the recognizer / classifier can be narrowed down without processing a large amount, and the processing time can be significantly reduced. There is an advantage that it can be done.

Hereinafter, the configuration according to the embodiment of the present invention will be described in detail with reference to the drawings.

<Structure of image normalization device according to the first embodiment of the present invention>

Next, the configuration of the image normalization device according to the first embodiment of the present invention will be described.

As shown in FIG. 1, the image normalization device 100 according to the first embodiment of the present invention stores a CPU, a RAM, a program for executing an image normalization processing routine described later, and various data. It can be configured with a computer including a ROM. The image normalization device 100 functionally includes an input unit 10, a calculation unit 20, and an output unit 50 as shown in FIG.

The input unit 10 accepts an image in which characters are reflected as an object as an input image as shown in FIG. The luminance value of the input image is grayscale. In the present embodiment, a case is considered in which a character pattern in an input image is rotated / enlarged / reduced and continuously recognized / detected in order to perform template matching with the character E which is a recognition target shown in FIG.

The calculation unit 20 includes a region extraction unit 30, a candidate acquisition unit 32, an image normalization unit 34, an image recognition unit 36, and a recognizer 40.

The recognizer 40 stores a recognizer that recognizes by template matching with characters. Any recognizer can be used, not just template matching.

As described below, the area extraction unit 30 has a rectangular shape so that the diagram pattern does not protrude and is in contact with the diagram pattern, which is an area representing an object reflected in the input image received by the input unit 10. The target area image surrounded by a rectangular frame is cut out. In the present embodiment, the characters appearing in the input image are objects.

The area extraction unit 30 first separates the object area having the character figure pattern from the background area not having the character figure pattern. The input image has image patterns of various characters whose brightness is grayscale, and when the brightness of the pixels of the input image is binarized for the purpose of separating the figure ground, it becomes as shown in FIG. FIG. 4 is a binary image after the threshold value processing for the luminance value, and the background area and the character area are separated from each other.

Next, the area extraction unit 30 labels the connected area with respect to the figure pattern of the character part separated from the ground part, and examines the number of figure patterns. The number of figure patterns is the number of objects to be determined. In the case of FIG. 4, the number of figure patterns is eight. Next, the target figure pattern is surrounded by a rectangular frame, and the target area images surrounded by the rectangular frame are cut out one by one. First, FIG. 5 is a cutout of a target area image with a rectangular frame fitted so as to be in contact with each of the target diagram patterns. FIG. 5 shows each of the diagram patterns cut out by the rectangular frame first fitted after labeling the connecting region.

As described below, the candidate acquisition unit 32 acquires candidate images from the target area image cut out by the area extraction unit 30 using a rectangular frame that satisfies the reference condition for the rectangular frame. In the present embodiment, the reference condition is that the area of the rectangular frame is the minimum. Further, the target area image is rotated, and the candidate images are acquired by using the rectangular frame having the smallest area among the plurality of fitted rectangular frames.

First, the candidate acquisition unit 32 applies a rotation transformation of various angles with an arbitrary center to the cut out target area image. In FIG. 6, only rotations every 15 degrees (excluding 90 degree rotations) are displayed for rotations from 0 degrees to 90 degrees, but rotation conversions at all angles of 360 degrees are candidates.

Next, the candidate acquisition unit 32 creates a rectangular frame for each of the rotated target region images by using the minimum and maximum x-coordinates of the closed region of the diagram pattern and the same minimum and maximum y-coordinates. A rectangular frame is constructed so that each of the constituent sides is a vertical / horizontal straight line. Next, for example, a rectangular frame is selected under the reference condition that the area is the minimum, and an image including the inside of the selected rectangular frame and a certain area outside is set as a candidate image. As shown in FIG. 7, the numbers written under each figure pattern of the rotated target area image are the rotation angle and the area of the rectangular frame. Among the rectangular frames fitted to each of the rotated target area images, the image including the rectangular frame having the smallest area and the outer fixed area is each of the candidate images shown in FIG.

The image normalization unit 34 enlarges or reduces the candidate image so that the size of the rectangular frame of the candidate image is the aspect ratio of the characters recognized by the recognizer 40 for each of the candidate images. Normalize. Since one candidate image is obtained at a rotation angle of 90 degrees, four candidate images having the smallest area in each of the diagram patterns are obtained at 360 degrees, as shown in FIG. In these four candidate images, the squared difference value when template matching is performed by normalizing the aspect ratio of the fitted frame to match the aspect ratio of the character E in FIG. 3 is in the lower left of each of the candidate images. It is displayed. The one with the smallest difference is surrounded by a square.

The image recognition unit 36 recognizes the characters of the input image from each of the candidate images normalized by the image normalization unit 34 using the recognizer 40, and outputs the characters to the output unit 50. When the candidate image is displayed on the original binary image, it becomes as shown in FIG. Further, assuming that the value having a small difference is the same as the template of the character E of the recognizer 40, the figure pattern surrounded by the frame in FIG. 11 corresponds. The figure pattern surrounded by these frames is recognized as being readable as the character E. It can be seen that by the above method, characters can be recognized using an image in which the figure pattern is normalized by rotation, enlargement, and reduction.

Further, in the present embodiment, the problem is set that the pattern in which the character pattern is turned inside out is different from the original character pattern.

The reference condition may be that the length of the outer circumference or the length of the diagonal line of the rectangular frame is the minimum. Further, although the method of rotating the target area image without rotating the rectangular frame has been described, the reverse may be used as the method. When rotating the rectangular frame, it becomes as shown in FIG.

<Operation of the image normalization device according to the first embodiment of the present invention>

Next, the operation of the image normalization device 100 according to the first embodiment of the present invention will be described. When the input unit 10 receives the input image, the image normalization device 100 executes the image normalization processing routine shown in FIG.

First, in step S100, the figure pattern, which is an area representing the characters appearing in the input image received by the input unit 10, is surrounded by a rectangular frame so that the figure pattern does not protrude and is in contact with the figure pattern, and is rectangular. Cut out the target area image surrounded by the frame.

Next, in step S102, candidate images are acquired by using the rectangular frame having the smallest area among the plurality of rectangular frames fitted by rotating the target area image cut out in step S100.

In step S104, for each of the candidate images acquired in step S102, the candidate image is enlarged or reduced so that the length of the rectangular frame of the candidate image is the aspect ratio of the characters recognized by the recognizer 40. Normalize by.

In step S106, the characters of the input image are recognized from each of the candidate images normalized in step S104 by using the recognizer 40, and the characters are output to the output unit 50 to end the process.

As described above, according to the image normalization apparatus according to the first embodiment, the diagram pattern, which is an area representing the object reflected in the input image, is formed into the diagram pattern without the diagram pattern protruding. Enclose it in a rectangular frame so that it touches, obtain a candidate image using the rectangular frame, and normalize the candidate image by enlarging or reducing it so that the rectangular frame has the aspect ratio of the object recognized by the recognizer. By doing so, it is possible to generate a normalized image by eccentric enlargement or reduction according to the recognition using the recognizer.

<Structure of image normalization device according to the second embodiment of the present invention>

Next, the configuration of the image normalization device according to the second embodiment of the present invention will be described. The parts having the same configuration as that of the first embodiment are designated by the same reference numerals and the description thereof will be omitted.

In the second embodiment, the image obtained from the input image representing the fish in the water is rotated, enlarged, or reduced to be normalized and recognized.

Similar to the first embodiment, the image normalization device 100 according to the second embodiment of the present invention has an input unit 10, a calculation unit 20, and an output unit 50 as shown in FIG. I have.

The input unit 10 accepts a moving image of an underwater fish as shown in FIG. 14 as an input image group.

Similar to the first embodiment, the calculation unit 20 of the second embodiment includes a region extraction unit 30, a candidate acquisition unit 32, an image normalization unit 34, an image recognition unit 36, and a recognizer 40. It is composed including and.

In the area extraction unit 30, for each of the input image groups received by the input unit 10, the diagram pattern protrudes from the diagram pattern which is an region representing the object reflected in the input image, as in the first embodiment. The target area image surrounded by the rectangular frame is cut out by surrounding it with a rectangular frame so as to be in contact with the figure pattern. In the present embodiment, the fish shown in the input image group is used as an object.

Specifically, the area extraction unit 30 first separates the map area using the fish area as a figure pattern and the background as a ground pattern. Therefore, the background image is calculated from the input image group shot as a moving image, the difference from the image of the scene where the fish is shown is calculated for each pixel, and if the value is above a certain threshold value, it changes. There is, and it is a candidate for the area where the fish is reflected.

The value that takes the difference in each pixel can be considered variously, such as only the H component value of the HSV component of the luminance or color. Here, the case of the luminance value will be described as an example. In the background image, for example, a histogram of the luminance value can be composed of moving images for each pixel, and can be configured by the mode of the histogram. In addition, it can be configured by various methods such as the Eign Background method (see Reference 4). Here, for the sake of explanation, the only moving object is the fish, and assuming that there is no movement such as the flow of water, it is relatively easy to take a candidate for the area where the fish is clearly reflected. By the process of dividing into two values of the change area and the background area, unlike the background, the changed area can be detected and separated into the area where the fish seems to be present and the background area as shown in FIG.

[Reference 4] Yasutomo Kawanishi, Masayuki Hibiki, Michihiko Mino, "Background image estimation based on time-series filter in eigenspace focusing on time change of background", Institute of Electronics, Information and Communication Engineers

In addition, among the areas with the figure pattern that seems to be fish, those with a small area are ignored, and it is judged that they are not fish and the processing is omitted. In addition, the entire area where the fish is shown may not be completely in the figure pattern, but it suffices if it has the shape of a fish to some extent. It is the role of the subsequent recognizer / discriminator to determine whether it is a fish, and the amount of information required there may be included in the diagram pattern of the area.

Next, the region extraction unit 30 labels the diagram patterns that can separate the regions where the fish are supposed to be present, and examines the number of the diagram patterns. Next, the target figure pattern is surrounded by a rectangular frame, and the target area images surrounded by the rectangular frame are cut out one by one. It is possible to cut out a plurality of target area images for the figure pattern of one fish from each of the input images of the input image group, but in the present embodiment, the target area images cut out from any input image are candidates. It may be output to the acquisition unit 32.

As in the first embodiment, the candidate acquisition unit 32 uses a rectangular frame that satisfies the reference condition for the rectangular frame from the target area image cut out by the area extraction unit 30 for each of the input image groups. Acquire each candidate image.

First, the candidate acquisition unit 32 applies a rotation transformation with an arbitrary center to each of the cut out target region images, as in the case of the character diagram pattern of the first embodiment. For each of the rotated target area images, each side constituting the rectangular frame becomes a vertical / horizontal straight line using the minimum and maximum x-coordinates of the closed area and the same minimum and maximum y-coordinates. As such, construct a rectangular frame. For example, a rectangular frame is selected under the criterion that the area is the smallest, and an image including the inside of the selected rectangular frame and a certain area outside is set as a candidate image. The rectangular frame in FIG. 15 is the selected frame.

In the image normalization unit 34, for each of the input image groups, the size of the rectangular frame of the candidate image for each of the candidate images is the vertical and horizontal directions of the fish recognized by the recognizer 40, as in the first embodiment. Normalize by enlarging or reducing the candidate image so that it becomes a ratio.

In the image recognition unit 36, as in the first embodiment, for each of the input image groups, the fish of the input image is used from each of the candidate images normalized by the image normalization unit 34 using the recognizer 40. Is recognized and output to the output unit 50.

Since the other configurations and operations of the second embodiment are the same as those of the first embodiment, detailed description thereof will be omitted.

In addition, in the example of the fish of the present embodiment, if the dichotomous device recognizer is used due to the request that even the image in which the face of the fish is turned inside out must be recognized as a fish, the recognizer for the side with the fish and the opposite side thereof. It is necessary to prepare two.

Also, in the case of fish, the image can be normalized by detecting the spine instead of the frame. For example, the line segment in FIG. 16 is the site detected as the spine. Two points on the contour line of the connecting area of the figure pattern detected as the fish area, which have the longest distance, are described. In this way, the same object, which may be normalized in a frame or other than the frame, which is difficult in the frame, can be normalized by detecting the most characteristic part of the object.

As described above, according to the image normalization device according to the second embodiment, the diagram pattern, which is an area representing the object reflected in the input image of the input image group, is formed into the diagram pattern without the diagram pattern protruding. Enclose it in a rectangular frame so that it touches, obtain a candidate image using the rectangular frame, and normalize the candidate image by enlarging or reducing it so that the rectangular frame has the aspect ratio of the object recognized by the recognizer. By doing so, it is possible to generate a normalized image by eccentric enlargement or reduction according to the recognition using the recognizer.

The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention.

For example, in the above-described embodiment, among a plurality of rectangular frames to which the target area image is rotated and fitted based on the reference condition for the rectangular frame, the rectangular frame that satisfies the reference condition for the rectangular frame is used. , The case where each candidate image is acquired has been described as an example, but the present invention is not limited to this. For example, when it is not necessary to consider the rotation of the object, the candidate image may be acquired by using the rectangular frame fitted without rotating the target area image.

10 Input unit 20 Calculation unit 30 Area extraction unit 32 Candidate acquisition unit 34 Image normalization unit 36 Image recognition unit 40 Recognizer 50 Output unit 100 Image normalization device

Claims (5)

An image normalization device that generates a normalized image for input to a recognizer from an input image.
From the input image , the two-dimensional image on the image of the three-dimensional object shown in the input image and the background area that is not the two-dimensional image of the three-dimensional object are separated by binarization and labeled. For each connected area, a region extraction unit that extracts a diagram pattern, which is an region representing a three-dimensional object shown in the input image separated from the map, and a region extraction unit.
For each of the diagram patterns by the one connecting region, the diagram pattern is surrounded by a rectangular frame so that the diagram pattern does not protrude and is in contact with the diagram pattern, and a candidate image is obtained using the rectangular frame. Candidate acquisition department to acquire and
The candidate image includes an image normalization unit that normalizes the candidate image by enlarging or reducing the rectangular frame so as to have the aspect ratio of the three-dimensional object recognized by the recognizer.
The candidate acquisition unit uses the rectangular frame that satisfies the criteria for the predetermined rectangular frame among the plurality of rectangular frames obtained by rotating either one of the figure pattern and the rectangular frame. And acquire each of the candidate images.
Image normalization device. The image normalization apparatus according to claim 1, wherein the reference condition is that the length of the outer circumference, the length of the diagonal line, or the area of the rectangular frame is the minimum. It is an image normalization method in an image normalization device that generates a normalized image for input to a recognizer from an input image.
The area extraction unit separates the two-dimensional image on the image of the three-dimensional object shown in the input image and the background region that is not the two-dimensional image of the three-dimensional object from the input image by binarization. , A step of extracting a diagram pattern, which is an region representing a three-dimensional object reflected in the input image separated from the map, for each connected region of the labeled pixels.
The candidate acquisition unit surrounds the figure pattern with a rectangular frame so that the figure pattern does not protrude and is in contact with the figure pattern for each of the figure patterns formed by the one connecting region, and the rectangular frame is used. And the steps to get the candidate image,
The image normalization unit includes a step of normalizing the candidate image by enlarging or reducing the rectangular frame so as to have the aspect ratio of the three-dimensional object recognized by the recognizer.
In the step of acquiring the candidate image by the candidate acquisition unit, a predetermined reference condition regarding the rectangular frame among a plurality of rectangular frames obtained by rotating either one of the figure pattern and the rectangular frame is set. Each of the candidate images is acquired using the rectangular frame that fills.
Image normalization method. The image normalization method according to claim 3, wherein the reference condition is that the length of the outer circumference, the length of the diagonal line, or the area of the rectangular frame is the minimum. A computer-readable recording medium on which a program for operating a computer as a component of the image normalization device according to claim 1 or 2 is recorded.

Images