JP2013250868A - Image recognition device, image recognition method and image recognition program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image recognition device that can enhance recognition performance of a target object by a bilateral symmetry (or diphycercal symmetry) determination process.SOLUTION: The image recognition device (as one example, a camera electronic control unit (ECU) 1) comprises: an image acquisition unit 12 that acquires an image; and an object recognition unit 13 that performs a process of identifying determination of a window capturing a predetermined target object regarding a window to be set for an image acquired by the image acquisition unit 12 and a process of clustering the determined window including a symmetry determination rejecting a window determined non symmetry regarding one or both of preliminarily determined bilateral symmetry and diphycercal symmetry.

Description

  The present invention relates to an image recognition apparatus, an image recognition method, and an image recognition program.

  In recent years, there are vehicle distance control devices (ACC: Adaptive Cruise Control), forward vehicle collision warning devices (FCW: Forward Collision Warning), etc. as vehicle driving support devices and preventive safety devices, and the spread of inexpensive devices using in-vehicle cameras. Is expected.

Pattern recognition is often used for object recognition using a vehicle-mounted camera.
The pattern recognition method learns the feature quantity of the object to be recognized in advance, creates a dictionary reflecting the learning result, and collates it with the contents of the dictionary, so that the object (object image) is captured. ) Is a technique for recognizing whether or not there is.
In pattern recognition, after a face recognition algorithm (for example, see Non-Patent Document 1) combining a Haar-like feature quantity and an AdaBoost classifier has been announced, in recent years, a technology applied to vehicle-mounted object recognition (See, for example, Patent Document 1).
In such an object recognition algorithm, when creating a Haar-like feature quantity, an integrated image (integral image) is created, and the AdaBoost discriminator is cascaded to speed up the arithmetic processing. That is also a feature.

Further, other frequently used feature values include HOG (Histograms of Oriented Gradients) feature values.
As another frequently used classifier, there is an SVM (Support Vector Machine) classifier.
In the pattern recognition as described above, in order to extract a target object (target object image) from the captured image, an object recognition area (window) is set in each size, and the above-mentioned is described for each window. The object recognition algorithm is processed.

In addition, a clustering process may be provided in the subsequent stage in order to narrow down an optimal target object window from a plurality of windows recognized by pattern recognition as described above.
Normally, the finer the raster scan step, the more windows are recognized for one target.

The pattern recognition process performed by the object recognition unit according to the background art (for example, the processing unit corresponding to the object recognition unit 13 illustrated in FIG. 1A) will be described with reference to FIGS. .
FIG. 11 is a flowchart illustrating an example of a procedure of processing performed by the object recognition unit according to the background art.
In this example, the recognition algorithm is configured by the HOG feature and the AdaBoost classifier.

  First, the object recognizing unit calculates the gradient direction and gradient intensity for all the acquired luminance images, performs a process of integrating the gradient intensity for each gradient direction, and obtains an integrated gradient image (integrator). (Histogram) is calculated (step S101). As a result, several integral gradient images in the gradient direction are calculated.

Next, the object recognition unit extracts an arbitrary area of each integral gradient image for each gradient direction by raster scanning while sliding an arbitrary coordinate area (window) set in advance (step S102).
Next, the object recognition unit calculates an HOG feature amount (vector) for the extracted coordinate area (window) (step S103). Thereby, a gradient histogram is created for each cell.

Next, the object recognition unit uses the calculated HOG feature value (vector) to perform identification by an AdaBoost classifier, and recognizes an object (an object image) that is a target (target) in advance (step S104).
Here, the object recognition unit checks whether or not a series of raster scans has been completed (step S105).
If the object recognition unit determines that the series of raster scans has been completed, the final target object window is specified by clustering the windows extracted by the classifier by a clustering by clustering ( Step S106), the process is terminated.

On the other hand, if the object recognizing unit determines that the series of raster scans has not ended, the object recognizing unit proceeds to the process of step S102, and shifts (slides) the window in the region of the integral gradient image to perform the next step. Perform a scan.
As a result, the object recognizing unit sequentially repeats the processing from step S102 to step S104 on the raster scan area until a series of raster scans is completed.

  Note that, in a series of raster scans, for example, sliding the windows of the same scale (size) sequentially for an arbitrary image area and repeating the above processing (processing for creating a gradient histogram) is completed. After that, the window scale and moving step (scan step) are changed, and the window is sequentially slid again and the above processing is repeated for a predetermined amount. Thereby, feature vectorization is performed in each scan.

  Further, as a method of processing to create a gradient histogram, there is a method that does not use an integral gradient image, but a method that uses an integral gradient image is more effective because the processing time can be shortened.

  Here, when using a luminance-based feature quantity such as a Haar-like feature quantity instead of a luminance gradient-based feature quantity such as an HOG feature quantity, a normal integrated luminance image is used instead of the above-described integral gradient image. . In this case, it is possible to reduce the processing because it is possible to create one integral image, but for example, only the feature of the luminance difference of a specific region can be captured, and the feature can be changed for each gradient direction. I can't catch it.

FIG. 12 is a flowchart illustrating another example of the processing procedure performed by the object recognition unit according to the background art.
In this example, a recognition algorithm is configured by the Haar-like feature quantity and the AdaBoost classifier.

First, the object recognizing unit performs a process of integrating the luminance image on the acquired luminance image, and calculates an integrated image (integral image) as a result (step S111).
Next, the object recognizing unit extracts the area of the integral image by an arbitrary coordinate area (window) set in advance by raster scanning (step S112).
Next, the object recognition unit calculates a Haar-like feature value (vector) for the extracted coordinate area (window) (step S113).

Next, the object recognition unit uses the calculated Haar-like feature value (vector) to perform identification by an AdaBoost classifier, and recognizes an object (object image) that is a target (target) in advance (step S114). ).
Here, the object recognition unit checks whether or not a series of raster scans has been completed (step S115).
If the object recognition unit determines that the series of raster scans has been completed, the final target object window is specified by clustering the windows extracted by the classifier by a clustering by clustering ( Step S116), this process is terminated.

On the other hand, if the object recognition unit determines that the series of raster scans has not ended, the object recognition unit proceeds to the process of step S112, and shifts (slides) the window in the region of the integral image to perform the next scan. I do.
As a result, the object recognizing unit sequentially repeats the processing from step S112 to step S114 on the raster scan area until a series of raster scans is completed.

  Note that in a series of raster scans, for example, a window with the same scale (size) is sequentially slid with respect to an arbitrary image area and the above processing is repeated, and then the scale and movement of the window are completed. Steps (scanning steps) are changed, and the window is sequentially slid again to repeat the above-described processing for a predetermined amount.

Here, as the feature amount, another feature amount may be used.
In addition to AdaBoost, any recognition algorithm such as SVM may be used.

JP 2007-310805 A

Paul Viola, Michael Jones, “Rapid Object Detection using a Boosted Cascade of Simple Features”, Accepted Conference On Computer Vision Recon

  When the recognition algorithm is implemented in the apparatus, not only the performance but also the processing load must be considered. For example, in an object recognition device for in-vehicle use, since various types of target objects such as vehicles, two wheels, bicycles, pedestrians, and road signs are recognized at one time, it is important to reduce the processing load even in the entire device. .

  Here, since the pattern recognition processing of the target object is repeated for the number of times of raster scanning, it is necessary to reduce the processing load per time. For this reason, the number of stages of the cascade type classifiers and the number of weak classifiers in the cascade type classifiers are reduced as much as possible. For example, in the vehicle recognition processing, a trade-off with performance degradation due to the reduction has been a problem.

Therefore, in order to compensate for the performance degradation caused by reducing the number of classifier stages or the number of weak classifiers as much as possible, it is conceivable to use a method of discriminating left and right symmetry in the case of vehicle recognition or the like.
However, if left / right symmetry determination is performed during pattern recognition, computation for the number of times of raster scanning is required, and conversely, the computation processing load may increase.

  The present invention has been made in view of such circumstances, and an image recognition apparatus and image recognition that can improve the recognition performance of a target object by left-right symmetry (or vertical symmetry) determination processing. It is an object to provide a method and an image recognition program.

  (1) In order to solve the above-described problem, an image recognition apparatus according to the present invention is predetermined for an image acquisition unit that acquires an image and a window that is set for the image acquired by the image acquisition unit. Symmetry determination that rejects a window that has been determined to be non-symmetrical with respect to one or both of the predetermined left-right symmetry and vertical symmetry, and the process of identifying to determine the window that captures the target object And a process for clustering the determined windows, and an object recognition unit for executing the processing.

  (2) The present invention provides the image recognition device according to (1) described above, wherein the object recognition unit is configured to determine the symmetry based on a distance between feature vectors in each of two symmetric regions in the window. And determining whether or not there is symmetry.

  (3) The present invention provides the image recognition apparatus according to (1) or (2) described above, in which the object recognition unit is configured to calculate feature vectors in each of two symmetry regions in the window in the symmetry determination. It is characterized in that it is determined whether or not there is symmetry based on the cross-correlation between them.

  (4) The present invention provides the image recognition device according to any one of (1) to (3) described above, wherein the object recognition unit is configured to determine two symmetry regions in the window in the symmetry determination. It is characterized in that the feature amount in each is identified, and whether or not there is symmetry is determined based on the result of determining these two identification results by AND logic.

  (5) In the image recognition device according to any one of (1) to (4) described above, the object recognition unit may use a gradient direction rectangular feature value or a HOG feature value as a feature value. And, as two symmetrical regions in the window, a region having a gradient amount in a symmetric direction is used.

  (6) In the image recognition device according to any one of (1) to (5) described above, the object recognition unit may have symmetry regarding left-right symmetry as the symmetry determination. A window that has been determined to be non-existent regardless of whether there is vertical symmetry or not, and a window that has been determined to be symmetrical with respect to vertical symmetry even if there is left-right symmetry is not adopted. Symmetry determination is performed using a window that is determined to be present and has no vertical symmetry.

  (7) In order to solve the above-described problem, in the image recognition method according to the present invention, an image acquisition unit acquires an image, and an object recognition unit sets a window set to the image acquired by the image acquisition unit. For a window that captures a predetermined target object, and a window that has been determined to have no symmetry for one or both of the predetermined left-right symmetry and vertical symmetry. And a process of clustering the determined windows, including symmetry determination to be adopted.

  (8) In order to solve the above-described problem, in the image recognition program according to the present invention, an image acquisition unit is set to a procedure for acquiring an image, and an object recognition unit is set to the image acquired by the image acquisition unit. A window that has been identified as having no symmetry with respect to one or both of a predetermined left-right symmetry and vertical symmetry, and a process for performing identification for determining a window that captures a predetermined target object A program for causing a computer to execute a process of clustering the determined windows, including a symmetry determination that does not adopt.

  As described above, according to the present invention, an image recognition apparatus, an image recognition method, and an image recognition program capable of improving the recognition performance of a target object by left / right symmetry (or vertical symmetry) determination processing are provided. It becomes possible to do.

(A) is a schematic block diagram which shows the structure (hardware structure) of the image processing system which concerns on one Embodiment of this invention, (B) shows the other structural example (3rd Embodiment) of an object recognition part. It is a schematic block diagram. It is a schematic block diagram which shows the structural example of the cascade type | mold discriminator of AdaBoost which concerns on one Embodiment of this invention. It is a flowchart figure which shows an example of the procedure of the process performed by the object recognition part which concerns on 1st Embodiment of this invention. It is a flowchart figure which shows an example of the procedure of the process of the clustering performed by the object recognition part which concerns on 1st Embodiment of this invention. It is a figure which shows the image of an example of the clustering process 1 performed by the object recognition part which concerns on 1st Embodiment of this invention. It is a figure which shows an example of the direction change of a gradient direction rectangular feature-value. It is a figure which shows the image of an example of the clustering process 2 performed by the object recognition part which concerns on 1st Embodiment of this invention. It is a figure which shows an example of the direction change of a HOG feature-value. It is a figure which shows the image of an example of the clustering process 2 performed by the object recognition part which concerns on 2nd Embodiment of this invention. It is a flowchart figure which shows an example of the procedure of the process performed by the object recognition part which concerns on 3rd Embodiment of this invention. It is a flowchart figure which shows an example of the procedure of the process performed by the object recognition part which concerns on background art. It is a flowchart figure which shows another example of the procedure of the process performed by the object recognition part which concerns on background art. It is a figure which shows a schematic example of the process from the image acquisition to object recognition in an image processing system. It is a figure which shows the gradient direction rectangular feature-value A which concerns on a 1st example. (A), (B), (C) is a figure which shows the gradient direction rectangular feature-value B which concerns on a 2nd example. It is a figure which shows the gradient direction rectangular feature-value C which concerns on a 3rd example. It is a figure which shows the gradient direction rectangular feature-value D which concerns on a 4th example. It is a figure which shows the gradient direction rectangular feature-value E which concerns on a 5th example. (A) is a figure which shows an example of a luminance image, (B) is a figure which shows an example of a gradient, (C) is a figure which shows an example of gradient strength, (D) is an example of a gradient direction. (E) is a figure which shows an example of the range of each gradient direction. (A)-(D) are figures which show the example of the combination of the direction for making left-right (or up-down) symmetry.

[Explanation of terms]
In the present application, the rectangle includes not only a rectangle but also a square.
In the present application, the gradient direction rectangular feature is a rectangular feature related to the gradient direction and gradient strength of the luminance of the image. In the present application, the gradient direction rectangular feature value (or ROG (Rectangle of Oriented Gradients) feature value) is a feature value of the gradient direction rectangular feature, and is a gradient direction feature value in a rectangular region. Here, the gradient direction feature amount is a feature amount related to the gradient direction and gradient strength of the luminance of the image. For example, a known feature can be used as the rectangular feature, and a known gradient frequency with a reduced amount of information can be used as the gradient strength. In this application, the gradient value in a broad sense including the gradient strength and the gradient frequency is defined as the gradient amount.
In the present application, an integral gradient image (integral histogram) is a result (integral gradient value) obtained by calculating a gradient direction and a gradient amount for the luminance of an image and integrating the gradient amount for each gradient direction. Here, this integration is performed, for example, for all the pixels (pixels) included in the region for which the integral gradient image is to be obtained.

The classifier includes various types such as an AdaBoost classifier and an SVM classifier.
As a subordinate concept of the classifier, there is a classifier that performs statistical learning. As a subordinate concept, there is a Boosting classifier. As a subordinate concept, there is an AdaBoost classifier.

[First Embodiment]
In the present embodiment, an in-vehicle image processing system that is mounted on a vehicle and performs object recognition processing will be described as an example.
FIG. 1A is a schematic block diagram showing a configuration (hardware configuration) of an image processing system according to an embodiment of the present invention.
The image processing system according to the present embodiment includes a camera ECU (Electronic Control Unit) 1, an alarm device 2, an ACC-ECU 3, and a collision avoidance control-ECU 4.
The camera ECU 1 is an example of an image recognition device.
The camera ECU 1 includes a camera 11, an image acquisition unit 12, an object recognition unit 13, and a control unit 14.

In this embodiment, the object recognition unit 13 is configured using the function of the FPGA 21, the function of a microcomputer 22 having a CPU (Central Processing Unit), and the function of a memory (an example of a storage unit) 23.
Note that a function of a DSP (Digital Signal Processor) may be used instead of the function of the FPGA 21 or together with the function of the FPGA 21.
In the present embodiment, the control unit 14 is configured using, for example, the function of the microcomputer 22 having a CPU and the function of the memory 23.
The memory 23 and peripheral circuits of the object recognition unit 13 and the control unit 14 may be those stored in an FPGA, DSP, or an integrated circuit in the microcomputer, or may be provided outside them.

  Here, in the object recognition unit 13 and the control unit 14, any number of devices such as FPGA, DSP, and microcomputer may be used, for example, all devices may be used, or Not all devices are necessarily required, and all the processing functions can be implemented in one device or two devices.

An overview of each processing unit provided in the image processing system according to the present embodiment will be described.
The camera 11 is provided, for example, at a vehicle position that captures an image in a predetermined direction such as the front, side, or rear of the vehicle. In the present embodiment, the camera 11 is provided at a vehicle position where a target object ahead of the vehicle can be imaged and recognized. In the present embodiment, as the recognition of the target object, the recognition of the rear surface of the vehicle will be described as an example for the vehicle that is the target object.
The camera 11 captures an image and outputs a signal of the captured image to the image acquisition unit 12.
The image acquisition unit 12 captures an image from the camera 11 as image data, and outputs the captured image data to the object recognition unit 13.

The object recognition unit 13 performs image processing and pattern recognition operations on the image data input from the image acquisition unit 12 by the FPGA 21 and the microcomputer 22 for object recognition, and obtains information on the result of the object recognition processing. Output to the control unit 14. As this image processing, for example, preprocessing filtering processing or the like is performed.
Further, as information on the result of pattern recognition, for example, information on a target object in an image of image data is used.

The memory 23 stores various information.
In this embodiment, for example, integral histogram (or integral image) data is created in the FPGA 21 (or in the DSP) or in the microcomputer 22, transferred to the memory 23, and stored (stored). The
In addition to the above, for example, information on image data from the image acquisition unit 12 and various pieces of information necessary in the apparatus (camera ECU 1) are stored in the FPGA (or DSP) 21 or the microcomputer 22 or It is stored (stored) in an external memory (memory 23 in this embodiment).

The control unit 14 performs a predetermined process based on the information on the result of the object recognition process input from the object recognition unit 13, and uses the information on the result as the alarm device 2, the ACC-ECU 3, and the collision avoidance control- It outputs to ECU4.
As a specific example, the control unit 14 performs TTC (with a target object) as a predetermined process based on information on a target object that is information on a result of object recognition input from the object recognition unit 13. Time to Collation), distance calculation processing, target object tracking processing, application functions of other devices and ECUs (in this embodiment, alarm device 2, ACC-ECU 3, and collision avoidance control-ECU 4) Perform processing to communicate.

The alarm device 2, the ACC-ECU 3, and the collision avoidance control-ECU 4 are provided to execute each application function via the control unit 14 of the camera ECU 1.
For example, the warning device 2 performs a forward vehicle collision warning when the target object is a forward vehicle, based on information input from the control unit 14.
The ACC-ECU 3 performs, for example, inter-vehicle distance control when the target object is a vehicle ahead, based on information input from the control unit 14.
The collision avoidance control-ECU 4 performs, for example, collision avoidance control when the target object is a vehicle ahead, based on information input from the control unit 14. The collision avoidance control-ECU 4 cooperates with, for example, brake control and steering control.

FIG. 2 is a schematic block diagram illustrating a configuration example of an AdaBoost cascade type identifier (cascade type AdaBoost classifier) according to an embodiment of the present invention.
In the present embodiment, such a cascade type discriminator is provided in the object recognition unit 13.
The cascade type classifier according to the present embodiment is a general one. In this embodiment, three-stage classifiers (AdaBoost classifiers) 101 to 103 are used.

The cascade discriminator according to the present embodiment includes a first discriminator 101 at the first stage, a second discriminator 102 at the second stage, and a third discriminator 103 at the third stage.
Note that each of the classifiers 101 to 103 includes a weak classifier in which a feature amount is set.
Here, in this embodiment, a cascade type discriminator is configured by a configuration in which the three stages of discriminators 101 to 103 are connected in series.

An example of operations performed in the cascade classifier according to the present embodiment will be described.
Window information in the raster scan is input to the first-stage discriminator 101.
Based on the input window information, the first-stage discriminator 101 indicates that a target object (target object image) exists inside the window in the image to be processed (true), or the target It is determined that the object does not exist (false). When the first-stage discriminator 101 determines that the target object exists in the window (true), the first-stage discriminator 101 outputs the window information to the second-stage discriminator 102, If it is determined that the target object does not exist (false), the information on the window is rejected.

Subsequently, similarly to the first-stage discriminator 101, the second-stage discriminator 102 is based on the input window information, and the inside of the window in the image to be processed is included in the target object in the window. Is determined to be present (true), the window information is output to the third-stage discriminator 103. On the other hand, if it is determined that the target object is not present in the window (false) , Dismiss the information in that window.
Subsequently, similarly to the first-stage discriminator 101, the third-stage discriminator 103, based on the input window information, has a target object in the window for the inside of the window in the processing target image. Is determined to be true (true), the information of the window is output as the information of the window (identification window) that is identified as the target object exists, while the target object does not exist in the window (false) ) Is rejected, the window information is rejected.

Here, in the cascade type classifier according to the present embodiment, the number of weak classifiers in each stage of the classifiers 101 to 103 is larger than that in the case where the clustering technique is not used by the clustering technique used in the present embodiment. , Can be less.
In the present embodiment, a cascade type classifier is used. However, as another configuration example, a classifier having a multi-class configuration (cascade type multi-class classifier) may be used.

With reference to FIG. 3, the process performed by the object recognition part 13 which concerns on this embodiment is demonstrated.
FIG. 3 is a flowchart illustrating an example of a procedure of processing performed by the object recognition unit 13 according to the present embodiment.
In this example, in the object recognition unit 13, a recognition algorithm is configured by the gradient direction rectangular feature amount and the cascade type AdaBoost classifier.
In the present embodiment, the cascade type AdaBoost classifier shown in FIG. 2 is used.

  First, the object recognition unit 13 performs a process of calculating the gradient direction and gradient intensity from the luminance image and integrating the gradient intensity for each gradient direction on the acquired luminance image. (Histogram) is calculated (step S1).

  Note that the process of calculating the integral gradient image (integral histogram) is for reducing the time of the subsequent calculation process using the integral gradient image, and is not necessarily performed. That is, in the subsequent processing, an operation that can obtain the same result without using the integral gradient image may be used.

Next, the object recognizing unit 13 determines whether or not a series of raster scans have been completed regarding the raster scan performed in the process of step S3 (step S2).
If the object recognition unit 13 determines that a series of raster scans has been completed, the object recognition unit 13 proceeds to the process of step S5.
On the other hand, if the object recognizing unit 13 determines that the series of raster scans has not ended, the window is shifted (slid) by raster scan in the region of the integral gradient image, and arbitrary coordinates set in advance. Extraction is performed in a region (window) (step S3).

Next, the object recognition unit 13 calculates a predetermined gradient direction rectangular feature (vector) for the extracted coordinate area (window). Then, the object recognition unit 13 performs identification by the cascade type AdaBoost classifier using the calculated gradient direction rectangular feature quantity (vector), and an object (an image of the object) that is set as a target in advance exists. One or more windows recognized (for example, guess) are extracted (step S4).
Here, in the discrimination by the cascade type AdaBoost discriminator in the process of step S4, when the window is rejected by each strong discriminator in the cascade type (the discriminators 101 to 103 in the present embodiment), the processing related to the raster scan ( The process returns to step S2.

  Thereby, the object recognition unit 13 sequentially repeats the processes of Step S2 to Step S4 for the raster scan area until a series of raster scans is completed.

  In a series of raster scans, for example, after sliding the windows of the same scale (size) sequentially and repeating the above processing, the window scale and scan step are changed, and then again, The predetermined process of repeatedly performing the above process by sequentially sliding the windows is executed.

In step S2, if the object recognition unit 13 determines that a series of raster scans has been completed, the object recognition unit 13 performs predetermined clustering processing (step S5).
Thereafter, the object recognition unit 13 ends this process.

Here, as the gradient direction rectangular feature amount, for example, those shown in FIGS. 14 to 18 can be used.
In the present embodiment, no particular limitation is imposed on the rectangular area in the window in each gradient direction with respect to the gradient direction rectangular feature value, and a classifier (in this embodiment, an AdaBoost classifier) regardless of the size of the target object. And others may be used) to select a weak classifier that indicates the characteristics of the target object.
Therefore, in the present embodiment, particularly effective portions can be carefully selected among the large and small gradient direction rectangular features of the target object, and the classifier can be configured with a small number of weak classifiers. The window in which the target object exists can be recognized in a short calculation processing time with a small number of memory accesses.

Using gradient-direction rectangular features, for example, a recognition algorithm that has a relatively low non-detection rate and a relatively low false detection rate even under conditions where the number of weak classifiers is as small as possible is used. Even it can be effective.
Regarding the discrimination by the discriminator, the non-detection rate is a rate at which the target object cannot be detected (detected) as the target object, and the false detection rate is recognized as a target object other than the target object. This is the rate of detection.

The clustering process will be described with reference to FIGS.
FIG. 4 is a flowchart illustrating an example of a clustering process performed by the object recognition unit 13 according to the present embodiment.
In this clustering process, a plurality of identified windows are clustered for information (window information) output from the cascade classifier shown in FIG. Specifically, based on the plurality of windows recognized in the processing of step S1 to step S4 shown in FIG. 3, the optimal window in which the target object exists (for example, the position and scale of the window are optimal with respect to the target object). To narrow down as a fixed window.

First, the object recognition unit 13 clusters similar windows as the clustering process 1 for a plurality of windows corresponding to information (identification results) output from the cascade classifier shown in FIG. 2 (step S11). .
Next, the object recognition unit 13 performs a left / right symmetry determination process as the clustering process 2 (step S12). Thereby, the object recognition unit 13 employs the window determined to have left-right symmetry, and rejects the window determined to have no left-right symmetry. In the present embodiment, an object having left / right symmetry is set as the target object. For this reason, the object recognition unit 13 captures the target object for a window determined to have left / right symmetry. On the other hand, the window determined to have no left-right symmetry is regarded as not capturing the target object and rejected.

Here, the clustering process (clustering process 1) in the process of step S11 will be described in detail.
Various algorithms may be used as the clustering algorithm. For example, a known algorithm such as Mean-Shift can be used.
In the present embodiment, clustering of similar windows is performed. As a condition for the subset to be clustered, a high similarity in window size (close window size) and a high similarity in distance between windows (close distance between windows), AND condition (both This is used as a condition that must be satisfied.

Specifically, as a condition that the window sizes are close, for example, the ratio of the sizes of two windows is compared with a predetermined threshold, and the ratio of the sizes of two windows is equal to or less than the threshold. In some cases, a condition for determining that the window size is close can be used. Other arbitrary conditions and determination methods may be used.
Specifically, as a condition that the distance between the windows is short, for example, between the two windows, the absolute value of the difference between the X coordinates in the XY coordinate system and the difference between the Y coordinates can be calculated. A condition for determining that the distance between the windows is close when the sum of the absolute values (Manhattan distance) or the Euclidean distance is obtained and compared with a predetermined threshold value and the distance is equal to or less than the threshold value. Can be used. Other arbitrary conditions and determination methods may be used.

As an example, an example of clustering using the Manhattan distance will be described with reference to Expression (1) and Expression (2).
In this example, it is assumed that the object recognition unit 13 obtains N (N is an integer of 2 or more) windows (referred to as temporary windows for convenience of explanation). In this example, it is assumed that the temporary window has a square shape and the size is specified by one window width.
In this example, a = 1, 2,..., N, W (a) represents the a-th temporary window, W (a + 1) represents the (a + 1) -th temporary window, and W (1) represents the first temporary window, W (N) represents the Nth (maximum) temporary window, Wb (a) represents the width of the ath temporary window, and Wb (a + 1) Represents the width of the (a + 1) th temporary window, Wx (a) represents the value of the x coordinate in the image coordinates (x, y) of the position of the ath temporary window, and Wx (a + 1) is (a + 1) ) Represents the value of the x coordinate in the image coordinates (x, y) of the position of the tentative temporary window, and Wy (a) is the value of the y coordinate in the image coordinates (x, y) of the position of the a tentative window. Wy (a + 1) is the image coordinates of the position of the (a + 1) th temporary window ( Represents the value of y-coordinate in y).

First, the object recognition unit 13 assigns W (1), W (2),..., W (N) for the N temporary windows in order from the smallest temporary window size.
Next, the object recognizing unit 13 sets the value of the variable a to a = 1 and sets it as the first allocation.
Next, the object recognizing unit 13 satisfies that both the expressions (1) and (2) are satisfied for the temporary window corresponding to W (a) and the temporary window corresponding to W (a + 1). Determine whether or not.

Here, Expression (1) shows an example of a condition related to the window width of the temporary window (in this embodiment, the window size). Various values may be set as the threshold value TH1 in the equation (1).
Specifically, in the expression (1), a ratio between the window width Wb (a) of the a-th temporary window and the window width Wb (a + 1) of the (a + 1) -th temporary window is obtained, and the ratio is calculated in advance. It is determined whether or not it is greater than a predetermined threshold value TH1. Here, since the window sizes of the temporary windows are in ascending order for a, Wb (a) <Wb (a + 1). That is, the condition of the expression (1) is a condition that the adjacent temporary windows are integrated when the window sizes are close.

Equation (2) shows an example of a condition regarding the position of the temporary window. In the formula (2), Abs (P) represents an absolute value of P (here, P = Wx (a) −Wx (a + 1) or P = Wy (a) −Wy (a + 1)). Various values may be set as the threshold value TH2 in the equation (2).
Specifically, in the expression (2), the absolute value of the difference between the x coordinate Wx (a) of the position of the ath temporary window and the x coordinate Wx (a + 1) of the position of the (a + 1) th temporary window. And the absolute value of the difference between the y-coordinate Wy (a) of the position of the a-th temporary window and the y-coordinate Wy (a + 1) of the position of the (a + 1) -th temporary window. It is determined whether or not the result is smaller than a predetermined threshold value TH2. That is, the condition of the expression (2) is a condition that integration is performed when adjacent temporary windows are close in position.

  Then, whether to integrate the two temporary windows is determined by the AND logic of the condition of Expression (1) and the condition of Expression (2).

As a result of such determination, the object recognizing unit 13 satisfies both Expression (1) and Expression (2) for the temporary window corresponding to W (a) and the temporary window corresponding to W (a + 1). If it is determined that this holds, the temporary window corresponding to W (a) and the temporary window corresponding to W (a + 1) are integrated.
Then, the object recognizing unit 13 increments the value of the variable a to add 1 to the value of a.
On the other hand, as a result of such determination, the object recognizing unit 13 determines that both Expression (1) and Expression (2) are satisfied for the temporary window corresponding to W (a) and the temporary window corresponding to W (a + 1). If it is determined that it is not satisfied (that is, at least one is not satisfied), 1 is added to the value of a by incrementing the value of the variable a without performing integration.

  Here, the value of a is incremented in order to shift to the next processing. That is, in the next processing, the current W (a + 1) becomes W (a), and the temporary window having the next larger size than the temporary window corresponding to the current W (a + 1) becomes W (a + 1).

Then, the object recognition unit 13 determines whether or not a = N.
As a result of this determination, if it is determined that a = N is not satisfied, the object recognition unit 13 performs the next process.
On the other hand, as a result of the determination, if the object recognition unit 13 determines that a = N, the process ends. In this way, the process is repeated until a = N.

Subsequently, the clustering process (clustering process 1) in the process of step S11 will be described.
As an example, for a class grouped by a subset condition (AND condition) in the process of step S11, a window with the highest response value of the discriminator can be selected.

Next, the clustering process (clustering process 2) in the process of step S12 will be described in detail.
In the present embodiment, the object recognizing unit 13 has an image in the window (in this embodiment, the back of the vehicle is the target object) having a predetermined left-right symmetry for the window determined by clustering. It is determined whether or not. And the object recognition part 13 employ | adopts about the window determined to have left-right symmetry, and rejects about the window determined not to have left-right symmetry on the other hand.

FIG. 5 is a diagram illustrating an example of the clustering process 1 performed by the object recognition unit 13 according to the present embodiment.
An image area 201 represents an image area of the entire screen to be recognized.
In this example, it is roughly classified into a group of four windows.
ClassWin_a, ClassWin_b, ClassWin_c, and ClassWin_d each represent a subset of windows.
In ClassWin_a, ClassWin_b, ClassWin_c, and ClassWin_d, one window 211, 221, 231, and 241 is determined (for example, selected) by the clustering process 1, respectively. Other windows are rejected.

In addition, in the example of FIG. 5, the example of the result in the case of using the structure which performs multiclass identification is also shown.
Specifically, class 1 classifiers learned from large truck images and class 2 classifiers learned from vehicles other than large trucks (mainly passenger cars) are used as multi-class classifiers.
In ClassWin_a, ClassWin_b, ClassWin_c, and ClassWin_d, the identification classes of the determined windows 211, 221, 231, and 241 are class 2 (for example, a distant passenger car) and class 1 (for example, a distant large truck), respectively. Class 2 (for example, a nearby passenger car) and Class 1 (for example, a nearby large truck).

  Here, when using a multi-class classifier, for example, a configuration in which clustering is performed on a plurality of windows of the same class can be used, and as another example, clustering is performed on a plurality of windows in which different classes are mixed. And a configuration employing a class corresponding to the window obtained by the clustering can be used.

With reference to FIG. 6 and FIG. 7, the left / right symmetry determination process using the gradient direction rectangular feature amount will be described.
FIG. 6 is a diagram illustrating an example of the direction change of the gradient direction rectangular feature value.
In this example, in order to simplify the description, a single rectangle feature amount (only) is used, and a rectangle on the side where the feature is present and the gradient appears is shown. On the other hand, it is also possible to use a rectangle that has no features and does not generate a gradient (for example, a rectangle that encloses an image with a gradient of true black).
Further, the feature amount is not limited to a single rectangular feature amount, and various feature amounts may be used.

FIG. 6 shows the feature vectors (1) to (10) of the region on the left side with respect to the center axis of the window (vertical axis passing through the center in the left-right direction). The state of dividing into feature vectors (1) ′ to (10) ′ in the right region is shown.
Here, feature vectors with the same number (1 to 10) correspond to the left and right.

Specifically, the average gradient image 301 shows a result of averaging a large number of learned gradient intensity images in all directions.
In addition, an image integrated in each of all eight directions 0 to 7 is shown as a window image 311-0 in the direction 0 and a window image 311-7 in the direction 7.
In the present embodiment, the same direction is used in the left and right regions for the direction 0 (vertical gradient) and the direction 4 (horizontal gradient). As for the gradient in the diagonal direction, different directions are used in the paired left and right regions. Specifically, the left side (or right side) of direction 1 is symmetrical with the right side (or left side) of direction 7. Yes, the left side (or right side) of direction 2 is symmetric with the right side (or left side) of direction 6, and the left side (or right side) of direction 3 is symmetric with the right side (or left side) of direction 5. Considering this, the gradient direction rectangular feature vectors in the left and right regions are aligned.

FIG. 7 is a diagram illustrating an example of clustering processing 2 performed by the object recognition unit 13 according to the present embodiment.
FIG. 7 shows an example of discriminating left-right symmetry with respect to a gradient direction rectangular feature vector in the left and right regions.

As the feature vector information 401 of the left region, feature quantities (for example, gradient strength) in each direction are obtained along the dimension of the feature quantity.
Similarly, feature amounts (for example, gradient strength) in each direction are obtained as feature vector information 402 in the right region along the feature amount dimension.
Here, the feature vector information 401 on the left region and the feature vector information 402 on the right region have a correspondence in a direction in which the left and right regions are symmetrical to each other (for example, the correspondence shown in FIG. 6). .

In the present embodiment, a first method (method a) for determining left-right symmetry will be described.
In method a, left-right symmetry is determined for the feature vector information 401 for the left region and the feature vector information 402 for the right region based on the proximity between the gradient-direction rectangular feature vectors.
In the first example (method a1) in method a, it is determined that there is left-right symmetry when the distance between feature vectors is short. Specifically, when the distance between two feature vectors is smaller than a predetermined threshold, it is determined that there is left-right symmetry.
Here, as a distance between feature vectors, for example, an arbitrary distance such as an Euclidean distance can be selected and calculated.

  In the second example (method a2) in method a, the left-right symmetry is determined based on the height of the cross-correlation value between feature vectors. Specifically, when the cross-correlation value between two feature vectors is higher than a predetermined threshold, it is determined that there is left-right symmetry.

Here, various methods may be used as a method for calculating the cross-correlation value.
As an example, the cross-correlation coefficient shown in Equation (3) can be used. Here, xl represents the left side feature vector (left region feature vector), and xr represents the right side feature vector (right region feature vector). i represents the dimension of the feature quantity, and i = 1, 2,..., n. Thereby, the cross correlation value _Cr is calculated.

  As another example, the cross-correlation function shown in Formula (4)-Formula (5) can be used. Here, xl represents the left side feature vector (left region feature vector), and xr represents the right side feature vector (right region feature vector). Further, τ represents a time interval.

In the present embodiment, a second method (method b) for determining left-right symmetry will be described.
In the method b, the left-right feature vector information 401 and the right-region feature vector information 402 are discriminated using the discriminators 411 to 412 and the AND circuit 421.
Here, as the discriminators 411 to 412, various types may be used. In the present embodiment, AdaBoost discriminators are used.

Specifically, the discriminator 411 discriminates the feature vector of the left region, and when determining that the target object exists (true), the discriminator 411 sets a value indicating true (for example, 1) to the AND circuit 421. On the other hand, if it is determined that the target object does not exist (false), a value indicating false (for example, 0) is output to the AND circuit 421.
Similarly, the discriminator 412 discriminates the feature vector in the right region and outputs a value (eg, 1) indicating true to the AND circuit 421 when determining that the target object exists (true). On the other hand, if it is determined that the target object does not exist (false), a value indicating false (for example, 0) is output to the AND circuit 421.

  When both the values input from the two discriminators 411 to 412 are values indicating true, the AND circuit 421 outputs a value indicating true (for example, 1) by AND logic. In this case, it is determined that the feature vector in the left region and the feature vector in the right region have left-right symmetry (true). On the other hand, when one or both of the values input from the two discriminators 411 to 412 are values indicating false, the AND circuit 421 generates a value indicating false (for example, 0) by AND logic. In this case, it is determined that there is no left-right symmetry between the feature vector in the left region and the feature vector in the right region (false).

As described above, in the present embodiment, by using the gradient-direction rectangular feature amount, a feature vector having a relatively small number of dimensions is obtained, so that it is possible to perform left-right symmetry determination processing with a light processing load.
Further, for example, when the gradient direction rectangular feature vector has already been calculated in the process such as pattern recognition, the calculation result can be used as it is for the determination process of the left / right symmetry, so that a new calculation is performed. Without being efficient. Note that a gradient direction rectangular feature vector may be newly calculated in the left / right symmetry determination processing.

  Further, in the present embodiment, a symmetry determination process (clustering process 2 in step S12 shown in FIG. 4) is performed on the windows integrated by the clustering process 1 (clustering process 1 in step S11 shown in FIG. 4). For this reason, it is possible to delete misidentified windows (in this embodiment, windows having no left-right symmetry) in a state where the number of windows is sufficiently narrowed down, and shorten the determination time.

  In the present embodiment, as shown in FIG. 4, a configuration in which the left / right symmetry determination processing (clustering processing 2) is performed after the clustering processing 1 is performed is shown. However, as another configuration example, left / right symmetry is shown. It is also possible to use a configuration in which the clustering process 1 is performed after the determination process (clustering process 2). In this configuration, for the windows selected by the classifier (in this embodiment, the cascade type AdaBoost classifier), first, the left / right symmetry determination process (clustering process 2) is performed, and only the windows having left / right symmetry are displayed. Next, the adopted window is integrated by clustering processing 1.

  Further, for the method a1, the method a2, and the method b, for example, two or more methods may be used in combination.

[Second Embodiment]
Regarding the image processing system according to the present embodiment, differences from the image processing system according to the first embodiment will be mainly described.
In the present embodiment, as another example of the process (clustering process 2) corresponding to the left / right symmetry determination process using the gradient direction rectangular feature value shown with reference to FIGS. 6 and 7 according to the first embodiment, The left / right symmetry determination process using the HOG feature amount will be described.

With reference to FIG. 8 and FIG. 9, the right / left symmetry determination process using the HOG feature amount will be described.
FIG. 8 is a diagram illustrating an example of the direction change of the HOG feature amount.
FIG. 8 shows the HOG feature vectors (1) to (8) in the region on the left side with respect to the center axis of the window (vertical axis passing through the center in the left-right direction) with respect to the window for which left / right symmetry is determined. The right side area HOG feature vectors (1) ′ to (8) ′ are shown.
Here, the HOG feature vectors having the same numbers (1 to 8) correspond to the left and right in combinations of directions that are paired with each other.

Specifically, the average gradient image 501 shows a result obtained by averaging a large number of learned gradient intensity images in all directions.
In addition, an image integrated in each of all eight directions 0 to 7 is shown as a window image 511-0 in the direction 0 and a window image 511-7 in the direction 7.
In the present embodiment, the same direction is used in the left and right regions for the direction 0 (vertical gradient) and the direction 4 (horizontal gradient). As for the gradient in the diagonal direction, different directions are used in the paired left and right regions. Specifically, the left side (or right side) of direction 1 is symmetrical with the right side (or left side) of direction 7. Yes, the left side (or right side) of direction 2 is symmetric with the right side (or left side) of direction 6, and the left side (or right side) of direction 3 is symmetric with the right side (or left side) of direction 5. Considering this, the HOG feature vectors in the left and right regions are aligned.

When the HOG feature value is used, the left and right areas of the window are equally divided by cells, so that (1) to (8) in the left area and (1) ′ to (8) ′ in the right area are Arranged symmetrically.
Note that normalization by the block of cells may or may not be performed.

FIG. 9 is a diagram illustrating an example of clustering processing 2 performed by the object recognition unit 13 according to the present embodiment.
FIG. 9 shows an example of discriminating left-right symmetry with respect to the HOG feature vectors in the left and right regions.

As the feature vector information 601 of the left region, feature quantities (for example, gradient strength) in the respective directions are obtained for the cell numbers (1) to (8) along the dimension of the feature quantity.
Similarly, as the feature vector information 602 in the right region, feature quantities (for example, gradient strengths) in the respective directions are obtained for the cell numbers (1) ′ to (8) ′ along the dimension of the feature quantity. ing.
Here, in the left region feature vector information 601 and the right region feature vector information 602, there is a correspondence in a direction in which the left and right regions are symmetrical to each other (for example, the correspondence shown in FIG. 8). .

In the present embodiment, as a method for determining the left-right symmetry, for example, the first method (method a1, method a2) similar to that described with reference to FIG. 7 according to the first embodiment, Two methods (method b) can be used.
In addition, about the method a1, the method a2, and the method b, for example, two or more methods may be used in combination.
Further, the discriminators 611 to 612 and the AND circuit 621 shown in FIG. 9 have the same functions as the discriminators 411 to 412 and the AND circuit 421 shown in FIG. 7 according to the first embodiment, respectively. Perform the action.

  In the present embodiment, in order to perform the symmetry determination process (clustering process 2 in step S12 shown in FIG. 4) on the windows integrated by the clustering process 1 (clustering process 1 in step S11 shown in FIG. 4). In the state where the number of windows is sufficiently narrowed down, misidentified windows (in this embodiment, windows without left-right symmetry) can be deleted, and the determination time can be shortened.

  In the present embodiment, as shown in FIG. 4, a configuration in which the left / right symmetry determination processing (clustering processing 2) is performed after the clustering processing 1 is performed is shown. However, as another configuration example, left / right symmetry is shown. It is also possible to use a configuration in which the clustering process 1 is performed after the determination process (clustering process 2). In this configuration, for the windows selected by the classifier (in this embodiment, the cascade type AdaBoost classifier), first, the left / right symmetry determination process (clustering process 2) is performed, and only the windows having left / right symmetry are displayed. Next, the adopted window is integrated by clustering processing 1.

  Here, with reference to FIGS. 6 and 7 according to the first embodiment, the left / right symmetry determination processing using the gradient direction rectangular feature amount, and with reference to FIGS. 8 and 9 according to the second embodiment. About the left-right symmetry discrimination processing using the HOG feature amount shown, for example, it is possible to use both of these in combination, and by narrowing down the windows having left-right symmetry in both of these, higher accuracy is achieved. It is possible to determine left-right symmetry.

[Third Embodiment]
Regarding the image processing system according to the present embodiment, differences from the image processing system according to the first embodiment will be mainly described.
FIG. 1B is a schematic block diagram showing the configuration of the object recognition unit 13a (another configuration example of the object recognition unit) according to another embodiment of the present invention.
The object recognition unit 13a according to the present embodiment includes a first recognition unit 31 and a second recognition unit 32.

In the object recognition unit 13a, the first recognition unit 31 performs object recognition processing on the image data input from the image acquisition unit 12 using the first object recognition method.
In the object recognition unit 13a, the second recognition unit 32 performs second object recognition on the image data input from the image acquisition unit 12 based on the result of the object recognition process performed by the first recognition unit 31. The object recognition process is performed by the technique.
The object recognition unit 13 a outputs information on the result of the object recognition process performed by the second recognition unit 32 to the control unit 14.

FIG. 10 is a flowchart illustrating an example of a procedure of processing performed by the object recognition unit 13a according to the present embodiment.
Here, in the process of the flowchart according to the present embodiment shown in FIG. 10, the process of step S21 to step S23 and the process of step S26 are respectively the process of step S1 to step S3 in the process of the flowchart shown in FIG. This is the same as the processing in step S5.
Schematically, in the process of the flowchart according to the present embodiment illustrated in FIG. 10, the process of step S24 and step S25 is performed instead of the process of step S4 in the process of the flowchart illustrated in FIG. 3.

In the present embodiment, in the raster scan, in the process of step S24, the first recognition unit 31 of the object recognition unit 13a uses a predetermined gradient direction rectangular feature quantity (vector) for the extracted coordinate area (window). calculate. Then, the first recognizing unit 31 of the object recognizing unit 13a performs identification by the cascade type AdaBoost classifier using the calculated gradient direction rectangular feature quantity (vector), and the object (target of the object) that is set as the target (target) in advance. One or more windows that are recognized (for example, estimated) as being present are extracted (step S24).
Here, in the discrimination by the cascade type AdaBoost discriminator in the process of step S24, if the window is rejected by each strong discriminator (in the present embodiment, the discriminators 101 to 103 in the cascade type), the process related to the raster scan ( The process returns to step S22.

Subsequently, in the process of step S25, the second recognition unit 32 of the object recognition unit 13a calculates a predetermined HOG feature amount (vector) for the coordinate area (window) extracted by the first recognition unit 31. To do. Then, the second recognizing unit 32 of the object recognizing unit 13a uses the calculated HOG feature value (vector) to perform identification using a cascade type AdaBoost classifier, and an object (an image of the object) that is set as a target in advance. One or more windows that are recognized (eg, guessed) are extracted (step S25). Then, the process proceeds to step S26 (clustering process).
Here, in the identification by the cascade type AdaBoost classifier in the process of step S25, when the window is rejected by each strong classifier in the cascade type, the process returns to the process related to the raster scan (the process of step S22).

In the present embodiment, the HOG feature amount is used in the process of step S25 by the second recognition unit 32. As described above, it is desirable that the second recognition unit 32 uses a feature amount other than the feature amount used in the first recognition unit 31 (in this embodiment, the gradient direction rectangular feature amount). As the feature amount other than the gradient direction rectangular feature amount, an arbitrary feature amount other than the HOG feature amount may be used. For example, in addition to the same gradient direction-based HOG feature amount, a luminance value-based feature amount is used. be able to.
In addition, in the process of step S25 by the second recognition unit 32, in addition to the configuration using the identification by the cascade type AdaBoost classifier learned by the HOG feature amount, an arbitrary method such as the identification by SVM is used. it can. Further, a configuration for performing multi-class identification may be used.
In the present embodiment, the recognition performance in a different view from the first recognition unit 31 is expected by the processing of the second recognition unit 32. Such a configuration can be used when further narrowing down the windows to be clustered in the processing of the next step S26 in advance.

  In this embodiment, since the HOG feature value (vector) is calculated by the second recognition unit 32, for example, the HOG feature value as shown in FIGS. 8 and 9 according to the second embodiment is used. When performing the symmetry determination process, the HOG feature amount (vector) calculated by the second recognition unit 32 can be used (for example, used as it is or converted and used), and the symmetry determination is performed. It is possible to omit or simplify the process of creating a new HOG feature (vector) for processing, which is efficient.

[Fourth Embodiment]
Regarding the image processing system according to the present embodiment, differences from the image processing system according to the first embodiment will be mainly described.
In the present embodiment, the discrimination process is performed using vertical symmetry instead of right / left symmetry. Alternatively, the discrimination processing is performed using the vertical symmetry as well as the left-right symmetry.
Here, the discriminating process using the vertical symmetry is the same as the discriminating process using the left / right symmetry, and the left / right symmetry also applies to combinations of directions in which the upper / lower symmetry is symmetric (directions 0 to 7 in this embodiment). This is the same as the combination of directions (in this embodiment, directions 0 to 7).

  Specifically, the object recognition unit 13 performs a vertical symmetry determination process as the clustering process 2 (step S12 illustrated in FIG. 4). Thereby, the object recognition unit 13 adopts the window determined to have vertical symmetry, and rejects the window determined to have no vertical symmetry. In the present embodiment, an object having vertical symmetry is set as the target object. For this reason, the object recognition unit 13 captures the target object for a window determined to have vertical symmetry. On the other hand, the window determined to have no vertical symmetry is regarded as not capturing the target object and rejected.

  In addition, when applying the process related to left-right symmetry to the process related to vertical symmetry, as an example, the left-side process (or right-side process) in the process related to left-right symmetry is changed to the upper process in the process related to vertical symmetry. At the same time, the processing on the right side (or the processing on the left side) in the processing related to left-right symmetry is replaced with the lower processing in the processing related to vertical symmetry.

[Fifth Embodiment]
The image processing system according to the present embodiment will be described mainly regarding differences from the image processing system according to the fourth embodiment.
In the present embodiment, the discrimination process is performed using the vertical symmetry as well as the left / right symmetry.

Specifically, the object recognition unit 13 performs a left-right symmetry determination process and a vertical symmetry determination process as the clustering process 2 (step S12 illustrated in FIG. 4). Thereby, the object recognition unit 13 adopts the window determined to have left-right symmetry and no vertical symmetry, and rejects the window determined to be other than that. That is, the object recognizing unit 13 adopts a window that is determined to have no left-right symmetry regardless of whether there is vertical symmetry or not, and even if there is left-right symmetry, there is symmetry regarding vertical symmetry. In this embodiment, the target object has left-right symmetry and up-down symmetry. For this reason, the object recognition unit 13 adopts a window that is determined to have a left-right symmetry and no vertical symmetry as a target object while adopting it. If the window is determined to be something other than, the target object is regarded as not being captured and is rejected.

Such processing can be applied, for example, to identification of a vehicle having left-right symmetry and no vertical symmetry.
As a specific example, distinguishing between a vehicle with left-right symmetry and no vertical symmetry and a signboard with left-right symmetry and vertical symmetry (for example, a signboard near a road), etc. In other words, a window that captures such a vehicle can be adopted, while a window that captures such a signboard can be rejected. Thereby, the discriminability of an object (in this example, a vehicle) can be improved.

Here, the combination of the presence / absence of left / right symmetry and the presence / absence of vertical symmetry is a combination of right / left symmetry and vertical symmetry other than the combination of right / left symmetry and no vertical symmetry. There is a combination that there is no left-right symmetry and vertical symmetry, and a combination that there is no left-right symmetry and no vertical symmetry.
In the present embodiment, the object recognizing unit 13 adopts a window determined to have left-right symmetry and no vertical symmetry, while rejecting windows determined to be other than that. However, a window is adopted for an arbitrary combination (any one or more and three or less combinations) among the four combinations as described above, while the other combinations (the remaining three or less are 1). For a combination of streets or more, a configuration of rejecting the window may be used.

[Description of Related Technology Regarding the Embodiments above]
With reference to FIGS. 13 to 19, techniques related to the above embodiments will be described.
FIG. 13 is a diagram showing a schematic example of processing from image acquisition to object recognition in an image processing system (for example, the image processing system shown in FIG. 1A).
Data of an image captured by the camera 11 is acquired as data of the input image 1201 by the image acquisition unit 12.
For example, this image shows a road and vehicles ahead.

The object recognizing unit 13 calculates the gradient strength for each of a plurality of gradient directions (eight directions from direction 0 to direction 7 in this embodiment) as shown in the gradient strength image 1202.
Then, the object recognizing unit 13 creates an integrated image of the gradient strength as the integrated gradient image 1203 for each gradient direction (the process of step S1 shown in FIG. 3).

Next, the object recognition unit 13 scans a window in the raster scan area 1204 (processing in step S3 shown in FIG. 3).
The object recognition unit 13 acquires an integral gradient image 1206 in each gradient direction with respect to the window image 1205 acquired by the raster scan.
For example, the image 1205 of this window is an image of a portion in which a vehicle ahead is shown.

  Then, the object recognition unit 13 calculates and acquires a predetermined rectangular feature amount (gradient direction rectangular feature amount) 1207 for the integrated gradient image 1206 for each gradient direction with respect to the same window (step shown in FIG. 3). Process of S4 (and step S24 shown in FIG. 10 is also the same)).

Next, the object recognition unit 13 performs identification using an AdaBoost classifier (in this embodiment, a cascade type AdaBoost classifier) (the process of step S4 shown in FIG. 3 (and also step S24 shown in FIG. 10). The same)).
In this embodiment, a weak classifier can be set for each gradient direction as the classifier.
Specifically, in the object recognition unit 13, the integral gradient image 1206 in each gradient direction (in this embodiment, the integral gradient image 1208-0 to the integral gradient image 1208-7 is provided for each of the eight gradient directions). Identification is performed by a weak classifier that has been learned in advance.

In the example of FIG. 13, the value of W _0−r1 = 419.4 / s0 is obtained as a result of identifying the integral gradient image 1208-0 in the direction 0 by the weak classifier. s0 is the area of the rectangular region from which this value was obtained. Then, the object recognition unit 13 performs processing 210-0 for calculating the response value h0 of the weak classifier based on this value.

In the example of FIG. 13, as a result of identifying the integral gradient image 1208-7 in the direction 7 by the weak classifier, a value of W _7−r1 = 198.1 / s0 is obtained. s0 is the area of the rectangular region from which this value was obtained. In the example of FIG. 13, the integrated gradient image 1208-7 in the direction 7 is identified by the weak classifier, and as a result, a value of W _7−r2 = 582.6 / s1 is obtained. s1 is the area of the rectangular region where this value is obtained. Subsequently, in this example, a subtracter 1209 obtains a subtraction result (difference) between these two values. Then, the object recognition unit 13 performs a process 1210-7 for calculating the response value h7 of the weak classifier based on the value of the subtraction result.

An example of the gradient direction rectangular feature value will be described with reference to FIGS.
In this embodiment, in order to extract a rectangular feature from the gradient intensity image divided in each gradient direction, the rectangular feature amount is defined as a gradient direction rectangular feature amount.

FIG. 14 is a diagram illustrating the gradient direction rectangular feature amount A according to the first example.
The gradient direction rectangular feature value A indicates the feature value (single rectangular feature value) of one rectangle (single rectangle) 1301.
The purpose of the gradient direction rectangular feature A is to extract a single rectangular feature from a gradient intensity image for each gradient direction and capture the gradient of a single region.
For example, when a part of the target object is concentrated in the same gradient direction, it can be captured by a single rectangular feature.

FIG. 15A, FIG. 15B, and FIG. 15C are diagrams showing gradient-direction rectangular feature quantity B according to the second example.
This gradient-direction rectangular feature value B uses a difference between rectangular features and the like, similar to the Haar-like feature value, for the gradient intensity image for each gradient direction. The gradient direction rectangular feature value B is used to obtain a feature difference or the like in the same direction using rectangles having the same area as adjacent rectangles.
FIG. 15A shows gradient intensity images 1311 and 1312 when the gradient direction rectangular feature amount B is obtained by dividing the rectangle into two rectangles. In this example, a feature difference or the like is obtained between adjacent white and black areas.
FIG. 15B shows gradient intensity images 1321 and 1322 when the gradient direction rectangular feature amount B is obtained by dividing the rectangle into three rectangles. In this example, a feature difference or the like is obtained between adjacent white and black areas.
FIG. 15C shows a gradient intensity image 1331 when the gradient direction rectangular feature amount B is obtained by dividing the rectangle into four rectangles. In this example, a feature difference or the like is obtained between adjacent white and black areas.
This gradient direction rectangular feature amount B is a technique for capturing the gradient relationship between adjacent regions. In this embodiment, since the gradient intensity image is extracted, the feature difference is more conspicuous than the region difference obtained from the normal luminance image. The effect that a site | part can be extracted is acquired.

FIG. 16 is a diagram illustrating a gradient direction rectangular feature value C according to the third example.
The gradient direction rectangular feature value C is obtained by applying a Haar-like feature value to the gradient intensity image for each gradient direction, and using a rectangular feature difference or the like.
This gradient-direction rectangular feature C is a technique for capturing the gradient relationship between adjacent regions, and the shape and area of the regions differ between two regions (white region and black region) that are adjacent regions. The gradient direction rectangular feature value C is used to obtain a feature difference in the same direction.
In the example of FIG. 16, for a plurality of (N) gradient directions, a window image 1341-0 in direction 0, a window image 1341-1 in direction 1,..., A window image in direction (N-1). 1341- (N-1) is shown. In this example, the shape and number of rectangles may be different for each gradient direction.
This gradient-direction rectangular feature value C is similar to the gradient-direction rectangular feature value B shown in FIGS. 15A, 15B, and 15C. For example, the gradient area ratio of adjacent regions Even when the values are different from each other, an effect that more appropriate extraction can be performed is obtained.

FIG. 17 is a diagram showing a gradient direction rectangular feature value D according to the fourth example.
The gradient direction rectangular feature value D uses a difference between a plurality of separated (separated) rectangular features in a gradient intensity image for each gradient direction.
This gradient direction rectangular feature amount D is a technique for capturing the gradient relationship between separated regions, and the two regions (white region and black region) that are separated regions may have the same shape or area, or May be different. The gradient direction rectangular feature value D is used to obtain a feature difference or the like in the same direction.
In the example of FIG. 17, for a plurality of (N) gradient directions, a window image 1351-0 in direction 0, a window image 1351-1 in direction 1,..., A window image in direction (N-1). 1351- (N-1) is shown. In this example, the shape and number of rectangles may be different for each gradient direction.
This gradient-direction rectangular feature value D is such that features can be extracted even between non-adjacent regions. This gradient direction rectangular feature amount D makes it easy to grasp that a gradient in a certain direction of the target object exists between separated parts.

FIG. 18 is a diagram showing a gradient direction rectangular feature quantity E according to the fifth example.
This gradient direction rectangular feature amount E uses differences between features of a plurality of rectangles across the gradient intensity images in different gradient directions.
This gradient direction rectangular feature amount E is a technique for capturing the gradient relationship of each region between gradient intensity images in different gradient directions, and the shape, area and position of the region in two regions (white region and black region). May be the same or different. The gradient direction rectangular feature value E is used to obtain a feature difference between different directions.
In the example of FIG. 18, for a plurality (N) of gradient directions, a window image 1361-0 in direction 0, a window image 1361-1 in direction 1,..., A window image in direction (N-1). 1361- (N-1) is shown. In this example, the shape and number of rectangles may be different for each gradient direction.
This gradient direction rectangular feature quantity E uses the feature relationship of a plurality of rectangles in the gradient intensity image across different gradient directions, so that the effect of being able to capture the relationship between the different gradient directions can be obtained.

Here, regarding the gradient direction rectangular feature values B to E, the difference between the features of a plurality of rectangles is not necessarily calculated in consideration of not only the differences between regions but also four arithmetic operations such as difference, sum, product, and quotient. Any one or more of these operations can be used to use (or select, etc.) the optimum feature amount. Also, as the difference (difference), an absolute value of the difference can be used.
In addition, the plurality of rectangles in the difference between features of the plurality of rectangles is not limited to two rectangles, for example, and three or more rectangles may be used.

  As described above, in the present embodiment, any one of a plurality of types of gradient direction rectangular feature values A to E, or any combination of two or more can be used. Features can be grasped by the number of discriminators (in the present embodiment, weak discriminators), and the configuration of the apparatus can be simplified.

Here, the effect obtained by using the gradient direction rectangular feature amount will be described in more detail.
In the present embodiment, in the process of step S4 shown in FIG. 3 (and also in step S24 shown in FIG. 10), the gradient direction rectangular feature quantity is identified by the classifier by AdaBoost.
In this case, when the gradient direction rectangular feature quantity and the AdaBoost classifier are combined, for example, even if the number of weak classifiers is as small as possible, the performance has a low non-detection rate and a low false detection rate. It is possible to construct an object recognition algorithm having

As a preferred specific example, with respect to the gradient direction rectangular feature amount, there is no particular limitation on the rectangular area (that is, the size) in the window in each gradient direction, and the discriminator (this embodiment) regardless of the size of each part of the target object. In the embodiment, learning is performed by an AdaBoost classifier), and a weak classifier indicating the characteristics of the target object is selected.
As a result, particularly effective portions can be carefully selected from the large and small gradient rectangular features of the target object, and a classifier can be configured with a small number of weak classifiers. For this reason, the number of calculations and the number of memory accesses are small, and the window in which the target object exists can be narrowed down or the target object can be identified with a short calculation processing time.

Regarding the discrimination by the discriminator, the non-detection rate is a rate at which the target object cannot be detected (detected) as the target object, and the false detection rate is recognized as a target object other than the target object. This is the rate of detection.
In this embodiment, AdaBoost has been described as an example of boosting. However, the same effect can be obtained even when applied to boosting in a broad sense.

A calculation example of the gradient direction and gradient intensity will be described with reference to FIGS.
FIG. 19A is a diagram illustrating an example of a luminance image.
In this example, the x-axis pixel position in the horizontal (horizontal) direction is represented by u, and the coordinate value increases toward the right. Further, the y-axis pixel position in the vertical (vertical) direction is represented by v, and the coordinate value increases as it goes downward.
The pixel position on the plane of the xy orthogonal coordinates is represented by coordinates (u, v).

FIG. 19B is a diagram illustrating an example of a gradient.
In this example, the luminance at the pixel position of coordinates (u, v) is represented by I (u, v).
The luminance gradient in the horizontal (horizontal) direction at the pixel position of coordinates (u, v) is represented by Ix, and Ix is represented by Expression (6). Further, the luminance gradient in the vertical (vertical) direction at the pixel position at coordinates (u, v) is represented by Iy, and Iy is represented by Expression (7).

FIG. 19C is a diagram illustrating an example of the gradient strength.
In this example, the gradient strength at the pixel position of coordinates (u, v) is represented by m (u, v), and m (u, v) is represented by equation (8).
FIG. 19D is a diagram illustrating an example of the gradient direction.
In this example, the gradient direction at the pixel position of coordinates (u, v) is represented by θ (u, v), and θ (u, v) is represented by equation (9).

FIG. 19E is a diagram illustrating an example of a range in each gradient direction.
In this example, 180 deg is divided into a range of 8 directions (direction 0 to direction 7). In this case, the range in one direction is 22.5 deg, and the range is ± 11.25 deg when the center angle of the range is used as a reference.
In this example, if direction 0 is a vertical gradient range, direction 4 is a horizontal gradient range.

Here, the gradient of luminance in the image corresponds to the differential image. Further, for example, it is possible to detect a connection between pixel positions whose luminance gradient is greater than a predetermined threshold as the gradient.
Further, as a method for calculating the gradient, for example, another method such as a method using a Sobel filter can be used.
The integral gradient image is often used, for example, when calculating the HOG feature amount. In this embodiment, the integral gradient image is used when calculating the rectangular feature amount.

FIGS. 20A to 20D are diagrams illustrating examples of combinations of directions for left-right (or vertical) symmetry.
In this example, directions 0 to 7 shown in FIG.

As shown in FIG. 20A, the direction 1 and the direction 7 form a symmetry pair (combination).
As shown in FIG. 20B, the direction 2 and the direction 6 are a pair of symmetry.
As shown in FIG. 20C, the direction 3 and the direction 5 form a symmetry pair.
As shown in FIG. 20D, direction 0 is itself a symmetric pair (ie, direction 0 and direction 0), and direction 4 is itself (ie, direction 4 and direction). 4) is a pair of symmetry.

[Description of configuration example related to the above-described embodiment]
About the image processing system which concerns on embodiment shown above (1st Embodiment-5th Embodiment), (the structural example 1 which concerns on embodiment)-(the structural example 14 which concerns on embodiment) is shown.

(Configuration example 1 according to the embodiment)
In the recognition of the target object, a plurality of windows obtained as a result of identification by the classifier (in the embodiment, a cascade type classifier) are subjected to clustering processing including determination of the symmetry of the target object.
(Configuration example 2 according to the embodiment)
The target object is a vehicle (more specifically, the rear surface of the vehicle) that runs ahead (runs ahead).

(Configuration example 3 according to the embodiment)
In the left / right symmetry determination, the gradient direction is matched symmetrically in each of the left and right regions of the window, and the symmetry is determined (determined) using the gradient direction rectangular feature amount.
(Configuration example 4 according to the embodiment)
In the left / right symmetry determination, symmetry is determined based on the distance between the gradient direction rectangular feature vectors in each of the left region and the right region of the window.
(Configuration example 5 according to the embodiment)
In the left / right symmetry determination, the symmetry is determined based on the cross-correlation between the gradient direction rectangular feature vectors in each of the left region and the right region of the window.
(Configuration example 6 according to the embodiment)
In the left / right symmetry discrimination, the symmetry is discriminated based on the result obtained by processing the result of identifying the gradient direction rectangular feature value in each of the left region and the right region of the window by the discriminator with AND logic.

(Configuration example 7 according to the embodiment)
In the left / right symmetry determination, the gradient directions are aligned symmetrically in the left and right regions of the window, and the symmetry is determined (determined) using the HOG feature value.
(Configuration Example 8 according to Embodiment)
In the left / right symmetry determination, the symmetry is determined based on the distance between the HOG feature vectors in each of the left region and the right region of the window.
(Configuration example 9 according to the embodiment)
In the left / right symmetry determination, the symmetry is determined based on the cross-correlation between the HOG feature vectors in each of the left region and the right region of the window.
(Configuration example 10 according to the embodiment)
In the left / right symmetry determination, the symmetry is determined based on the result of processing the HOG feature quantity in each of the left region and the right region of the window by the discriminator and processing with AND logic.

(Configuration Example 11 according to Embodiment)
In the determination of the left-right symmetry, the symmetry is determined by one or a combination of two or more of (Configuration Example 3 according to Embodiment) to (Configuration Example 10 according to Embodiment).
(Configuration example 12 according to the embodiment)
The discriminator is created by statistical learning such as Boosting.
(Configuration Example 13 according to Embodiment)
The discriminator uses AdaBoost or Real AdaBoost.
(Configuration Example 14 according to Embodiment)
In the above, the vertical symmetry is determined instead of the left / right symmetry or together with the left / right symmetry.

As described above, according to the object recognition units 13 and 13a of the image recognition apparatus (camera ECU 1 in the embodiment) in the image processing system shown in the embodiment (the first to fifth embodiments), left-right symmetry ( Alternatively, the recognition performance of the target object can be improved by the vertical symmetry) determination process.
Further, in the embodiment, in the clustering process after pattern recognition, it is possible to perform a left / right (or vertical) symmetry determination process that suppresses the load of the calculation process as much as possible.

According to the object recognition units 13 and 13a according to the present embodiment, for example, since the recognition performance is improved, a calculation processing speed sufficient to be mounted (embedded) in a vehicle-mounted device can be obtained. One device has an effect that the inter-vehicle distance control device (ACC), the forward vehicle collision warning device (FCW), the collision avoidance device, and the like can be applied.
In this embodiment, it is possible to provide an apparatus, a method, a computer program used for these, and the like suitable for recognizing a target object.

  Here, in the embodiment, a case where a vehicle is used as the target object has been described. However, as another configuration example, an object having left-right symmetry (or vertical symmetry) such as some road signs is used. It is also possible. Alternatively, an object having both left-right symmetry and vertical symmetry can be used as the target object.

[Description of Configuration Example According to the Present Invention]
Hereinafter, (Configuration Example 1 according to the present invention) to (Configuration Example 5 according to the present invention) will be described.
(Configuration example 1 according to the present invention)
An image acquisition unit (in the embodiment, the image acquisition unit 12) that acquires an image, and a window (in the embodiment, a window in raster scanning) that is set for the image acquired by the image acquisition unit, a predetermined target is set. And a process for performing identification (identification by a cascade classifier in the embodiment) for determining a window for capturing the object (vehicle in the embodiment) and one or both of predetermined left-right symmetry and vertical symmetry For the object recognition unit (in the embodiment, including a symmetry determination that rejects (for example, rejects) a window determined to have no symmetry, and performs clustering of the determined windows. An object recognition unit (13, 13a), and an image recognition device (in the embodiment, a camera ECU 1). It is.

  Here, in the configuration for determining both left-right symmetry and up-down symmetry, for example, when there is no left-right symmetry (regardless of the presence or absence of up-down symmetry), the window is not adopted, and the up-down symmetry is A mode in which a window is not adopted (regardless of whether there is left-right symmetry) or a mode in which a window is not adopted when there is no left-right symmetry and no vertical symmetry can be used.

(Configuration example 2 according to the present invention)
In the symmetry determination, the object recognition unit determines whether or not there is symmetry based on a distance between feature vectors in each of two symmetry regions in the window (the present invention). (In the embodiment, the processing of the method a1 shown in FIG. 7 or FIG. 9).

(Configuration example 3 according to the present invention)
In the symmetry determination, the object recognition unit determines whether or not there is symmetry based on a cross-correlation between feature vectors in each of two symmetry regions in a window (this book) The image recognition apparatus according to Configuration Example 1) or Configuration Example 2 according to the present invention (in the embodiment, the process of the method a2 illustrated in FIG. 7 or FIG. 9).

(Configuration example 4 according to the present invention)
In the symmetry determination, the object recognition unit identifies feature quantities in each of two symmetric areas in the window, and determines whether or not there is symmetry based on a result of determining these two identification results by AND logic. The image recognition apparatus according to any one of (Configuration Example 1 according to the present invention) to (Configuration Example 3 according to the present invention) is characterized in that (in the embodiment, FIG. 7 or FIG. Process of method b shown in 9).

(Configuration example 5 according to the present invention)
The object recognizing unit uses a gradient direction rectangular feature value or a HOG feature value as a feature value, and uses a gradient amount region in a symmetric direction as two symmetric regions in the window. The image recognition apparatus according to any one of (Configuration Example 1 according to the present invention) to (Configuration Example 4 according to the present invention) (in the embodiment, the objects shown in FIGS. 6, 8, and 20 are mutually targeted) Direction).

(Configuration example 6 according to the present invention)
The object recognizing unit, as the symmetry determination, rejects the window determined to have no symmetry with respect to the left-right symmetry regardless of the presence or absence of the vertical symmetry, and with respect to the vertical symmetry even if there is left-right symmetry. A window that is determined to be symmetric is not adopted, while a symmetry determination that uses a window that is determined to have left-right symmetry and no vertical symmetry is performed (configuration according to the present invention) The image recognition device according to any one of Example 1) to (Configuration Example 5 according to the present invention) (see the fifth embodiment).

[Summary of the above embodiments]
Here, in the above embodiment, the structure which provides the vehicle and the apparatus and system containing the object recognition part 13 shown by FIG. 1 (A) or the object recognition part 13a shown by FIG. 1 (B) was shown. However, as another example, it can be provided in any other moving body, or can be provided in other than the moving body.
In the above embodiment, the back of the vehicle has been described as a specific example of the target object. For example, the side of the vehicle, two-wheeled vehicle, pedestrian, etc. As described above, the present invention can be applied when recognizing other target objects, and can also be applied when recognizing related objects other than the vehicle.

  In addition, regarding the gradient direction rectangular feature amount, in the above embodiment, the method of creating the gradient direction rectangular feature amount using the integral gradient image (integral histogram) by the integral image for each gradient direction has been shown as an example. An integral gradient image (integral histogram) may not be used. As another configuration example, other methods such as a method of voting a direction for each pixel (1 pixel) in units of cells are available. May be used.

  It should be noted that a program for realizing the function of processing performed by an arbitrary constituent unit (for example, the object recognition unit 13, 13a, etc.) in FIG. 1A or FIG. Processing may be performed by recording, reading the program recorded on the recording medium into a computer system, and executing the program. Here, the “computer system” includes hardware such as an OS (Operating System) and peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible disk, a magneto-optical disk, a ROM (Read Only Memory), a portable medium such as a CD-ROM, and a hard disk incorporated in a computer system. say. Further, the “computer-readable recording medium” refers to a volatile memory (RAM (Random) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. (Access Memory)) and the like that hold a program for a certain period of time.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting a program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  Further, the present invention is not limited to the above, and the processing performed by an arbitrary component in FIGS. 1A and 1B is realized using a device such as an FPGA or a DSP in addition to the microcomputer. It is also possible.

As mentioned above, although each embodiment of this invention was explained in full detail with reference to drawings, the concrete structure is not restricted to this embodiment, The design etc. of the range which does not deviate from the summary of this invention are included.
Note that two or more of the embodiments described above (first to fifth embodiments) may be combined and implemented.

DESCRIPTION OF SYMBOLS 1 ... Camera ECU, 2 ... Alarm apparatus, 3 ... ACC-ECU, 4 ... Collision avoidance control-ECU, 11 ... Camera, 12 ... Image acquisition part, 13, 13a ... Object recognition part, 14 ... Control part, 21 ... FPGA (Or DSP), 22 ... microcomputer, 23 ... memory, 31 ... first recognition unit, 32 ... second recognition unit, 101-103, 411-412, 611-612 ... identifier, 201 ... image region , 211, 221, 231, 241, 311-313, 321... Window, 301, 501..., Average gradient image, 311-0 to 311-7, 511-0 to 511-7, 1201, 1205, 1341-0 to 1341. -(N-1), 1351-0 to 1351- (N-1), 1361-0 to 1361- (N-1), ... image, 401, 601 ... information on the feature vector of the left region, 02, 602... Right region feature vector information, 421, 621... AND circuit, 1202, 1311, 1312, 1321, 1322, 1331... Gradient intensity image, 1203, 1206, 1208-0 to 1208-7. 1204: Raster scan area, 1207 ... Rectangle direction rectangular feature, 1209 ... Subtractor, 1210-0 to 1210-7 ... Processing, 1301 ... Single rectangle

Claims (8)

An image acquisition unit for acquiring images;
For the window set in the image acquired by the image acquisition unit, a process for identifying to determine a window that captures a predetermined target object, and one of predetermined left-right symmetry or vertical symmetry Or, for both, the object recognition unit for performing the process of clustering the determined windows, including symmetry determination that rejects the window determined to have no symmetry,
An image recognition apparatus comprising: In the symmetry determination, the object recognition unit determines whether or not there is symmetry based on a distance between feature vectors in each of two symmetry regions in the window.
The image recognition apparatus according to claim 1. In the symmetry determination, the object recognition unit determines whether or not there is symmetry based on a cross-correlation between feature vectors in each of two symmetry regions in the window.
The image recognition apparatus according to claim 1, wherein the image recognition apparatus is an image recognition apparatus. In the symmetry determination, the object recognition unit identifies feature quantities in each of two symmetric areas in the window, and determines whether or not there is symmetry based on a result of determining these two identification results by AND logic. Determine
The image recognition apparatus according to claim 1, wherein the image recognition apparatus is an image recognition apparatus. The object recognition unit uses a gradient direction rectangular feature amount or a HOG feature amount as a feature amount,
As the two symmetric regions in the window, a region having a gradient amount in a symmetric direction is used.
The image recognition apparatus according to claim 1, wherein the image recognition apparatus is an image recognition apparatus. The object recognizing unit, as the symmetry determination, rejects the window determined to have no symmetry with respect to the left-right symmetry regardless of the presence or absence of the vertical symmetry, and with respect to the vertical symmetry even if there is left-right symmetry. Do not adopt the window determined to be symmetric, and perform symmetry determination that adopts the window determined to have left-right symmetry and no vertical symmetry.
The image recognition apparatus according to claim 1, wherein the image recognition apparatus is an image recognition apparatus. The image acquisition unit acquires the image,
A process in which the object recognition unit performs identification for determining a window that captures a predetermined target object for a window set in the image acquired by the image acquisition unit, and a predetermined left-right symmetry or vertical symmetry Performing a process of clustering the determined windows, including a symmetry determination that rejects a window determined to have no symmetry for one or both of the sexes,
An image recognition method characterized by the above. A procedure for an image acquisition unit to acquire an image;
A process in which the object recognition unit performs identification for determining a window that captures a predetermined target object for a window set in the image acquired by the image acquisition unit, and a predetermined left-right symmetry or vertical symmetry A process of clustering the determined windows, including a symmetry determination that rejects a window that has been determined to have no symmetry for one or both of the characteristics; and
Recognition program for causing a computer to execute.