JP2016071387A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus for detecting a specific object from an input image, configured to improve the accuracy of detecting the specific object.SOLUTION: An image processing apparatus 1 includes: an image acquisition section 11 which acquires an image captured by a camera; a feature quantity extraction section 12 which divides the image into a plurality of cells, to extract feature quantity of the cells; a storage section which stores background feature quantity 15 extracted from an image designated for a background image and dictionary data 14 including feature quantity of a specific object to be detected from an image; a collation feature quantity determination section 16 which determines collation feature quantity of cells in the image, on the basis of a differential value between the feature quantity extracted from each of the cells in the image to be collated with the dictionary data 14 with the background feature quantity; and a feature quantity collation section 17 which collates the collation feature quantity of each cell in the image with the feature quantity of the dictionary data 14.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for detecting a specific object in an image.

  Some image processing apparatuses detect a specific object such as a person from image or video data captured by a camera. As one method for detecting a specific object in this type of image processing apparatus, a detection method using Histograms of Oriented Gradients (HOG) features is known (see, for example, Non-Patent Document 1).

  The HOG feature value is a feature value obtained by making a histogram of luminance gradient directions in a local region in an image. In the detection method using the HOG feature amount, a plurality of images each including a specific object and non-existing images are prepared, and the HOG feature amount of the specific object and the non-specific object HOG feature amount regarded as the background object are learned and dictionary. Register as data. Then, when an image that appears to be a specific object (specific object candidate) exists in the input image, the HOG feature amount in the region (detection frame) including the specific object candidate is compared with the HOG feature amount of the dictionary data. As a result of the collation, when it is determined that the specific object candidate is the specific object, for example, the detection frame is combined with the input image, and the combined image is displayed on a display device such as a liquid crystal display.

  As a method for detecting a specific object, the input image is compared with a background image in which the specific object does not exist, and the specific object is detected from the contour shape of a region (region with change) different from the background image in the input image. There is also a detection method (see, for example, Patent Document 1).

  In the detection method that compares the input image and the background image, a difference image that is the difference between the input image and the background image is obtained, and the contour region of the moving object is extracted from the plurality of difference images arranged in time series. Detect an object.

  The above-described image processing apparatus is used for detecting a person or the like in a surveillance system for crime prevention.

  In recent years, utilization of the above-described image processing apparatus for observation of customer behavior in stores and the like has been studied. For customer behavior observation, grasp the needs such as what products the customer is interested in by using the movement path of the person detected from the image in the store or shopping street by the above-mentioned image processing device It becomes easy to do.

JP 2004-265292 A

Navneet Dalal, and Bill Triggs, "Histograms of oriented gradients for human detection," In IEEE Conf. On Computer Vision and Pattern Recognition, Vol.1, pp.886-893, 2005

  In the detection method using the HOG feature amount described above, when the HOG feature amount of the specific object candidate in the input image is more similar to the HOG feature amount of the specific object than the non-specific object HOG feature amount, the specific object A candidate is detected as a specific object. That is, even if the specific object candidate in the input image is not a specific object, if the HOG feature value of the specific object candidate is similar to the HOG feature value of the specific object, the specific object candidate is detected as the specific object.

  However, in the input image, there may be a non-specific object whose shape such as a flag can change. The HOG feature value of a non-specific object whose shape can change can be a value similar to the HOG feature value of a specific object such as a person. Therefore, an image processing apparatus that detects a person may erroneously detect a non-specific object as a person when a non-specific object such as a flag exists in the input image.

  Further, in the detection method using the difference image between the input image and the background image, when the specific object of the input image and the non-specific object of the background image overlap, the part of the specific object that overlaps the non-specific object The feature amount of the non-specific object is subtracted from the feature amount. For this reason, the contour area of the specific object in the difference image has a different shape from the contour area of the specific object in the input image, making it difficult to correctly detect the specific object.

  An object according to one aspect of the present invention is to improve the detection accuracy of a specific object in an image processing apparatus that detects the specific object from an input image.

  An image processing apparatus according to one aspect of the present invention includes an image acquisition unit that acquires an image captured by a camera, a feature amount extraction unit that extracts the feature amount of each cell by dividing the image into a plurality of cells, and an acquisition A storage unit that stores dictionary data including a background feature amount extracted from an image designated as a background image of the images and a feature amount of a specific object detected from the image; and A matching feature value determining unit that determines a matching feature value of each cell of the image based on a difference value between the feature value extracted from each cell and the background feature value; and a matching feature value of each cell of the image And a collation unit that collates the feature amount of the dictionary data.

  According to the above aspect, the detection accuracy of the specific object in the image processing apparatus that detects the specific object from the input image is improved.

1 is a functional block diagram illustrating a configuration of an image processing apparatus according to a first embodiment of the present invention. It is a schematic diagram for demonstrating a HOG feature-value. It is a flowchart which shows the procedure of prior learning. It is a flowchart which shows the procedure of the detection process of the specific object which concerns on 1st Embodiment. It is a figure which shows an example of an input image and a HOG feature-value. It is a flowchart which shows the processing content of the collation process of FIG. It is a flowchart which shows the processing content of the feature-value determination process for collation of FIG. It is a schematic diagram which shows an example of the input image of 2 sheets used by the feature-value determination process for collation. FIG. 9 is a schematic diagram illustrating an example of a feature value difference value ΔH in the input image of FIG. 8. It is a schematic diagram which shows the image after the characteristic value determination process for collation using the input image of FIG. 8, and the collation result. It is a schematic diagram which shows another example of two input images used by the feature-value determination process for collation. It is a schematic diagram which shows the example of difference value (DELTA) H of the feature-value in the input image of FIG. FIG. 12 is a schematic diagram showing images after matching feature value determination processing using the input image of FIG. 11 and images of matching results; It is a flowchart which shows the procedure of the detection process of the specific object which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows the processing content of the background feature-value setting process of FIG. It is a schematic diagram which shows the example of the time change of the presence or absence of a nonspecific object in an input image.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 is a functional block diagram showing the configuration of the image processing apparatus according to the first embodiment of the present invention.

  As shown in FIG. 1, the image processing apparatus 1 of the present embodiment includes an image acquisition unit 10, a feature amount extraction unit 12, a pre-learning unit 13, a matching feature amount determination unit 16, and a feature amount matching unit 17. And an output processing unit 18. The image processing apparatus 1 further includes an input image holding unit 11, feature quantity dictionary data 14, and a background feature quantity database 15.

  The image acquisition unit 10 acquires image or video data captured by the imaging device 2 and stores the acquired image or video data in the input image storage unit 11 as an input image.

  The feature amount extraction unit 12 extracts a feature amount used for detecting a specific object (detection target) such as a person from the input image. The feature amount extraction unit 12 divides one (one frame) input image into a plurality of cells, and extracts feature amounts for each cell. The feature amount extraction unit 12 in the image processing apparatus 1 of the present embodiment extracts a HOG feature amount that can be regarded as invariant to translation in the image as the feature amount.

  Further, the feature quantity extraction unit 12 registers the extracted feature quantity in the background feature quantity database 15 when the input image from which the feature quantity has been extracted is an image designated as the background image.

  The pre-learning unit 13 learns the feature amount of the specific object and the feature amount of the non-specific object in the image. The prior learning unit 13 uses machine learning to identify HOG feature values extracted from a plurality of images in which a specific object (for example, a person) exists and HOG feature values extracted from a plurality of images in which no specific object exists. Learn HOG features of objects and non-specific objects. In addition, the pre-learning unit 13 generates dictionary data 14 including the learned HOG feature value and stores it in the storage unit included in the image processing apparatus 1.

  The feature amount learning by the prior learning unit 13 is performed before the specific object detection process using the image processing device 1 is started. That is, while the specific object is detected using the image processing apparatus 1, the feature amount is not learned by the prior learning unit 13. As the dictionary data 14, data created by another image processing apparatus may be read into the image processing apparatus 1 through a portable recording medium or a network. When the dictionary data 14 created by another image processing apparatus is read and used, the image processing apparatus 1 may not have the pre-learning unit 13.

  The matching feature amount determination unit 16 uses the feature amount of the input image extracted by the feature amount extraction unit 12 and the background feature amount of the background feature amount database 15 to match the feature value for matching of the input image with the dictionary data 14. To decide. The matching feature amount determination unit 16 sets a detection frame including one or more cells in the input image, and obtains a difference value between the feature amount extracted for each cell in the detection frame and the background feature amount. Then, for each cell, the comparison feature value is determined by comparing the difference value with a predetermined threshold value. Specifically, for a cell having a difference value larger than a threshold value, the feature value extracted by the feature value extraction unit 12 is used as a matching feature value, and for a cell having a difference value equal to or less than the threshold value, the difference value is used as a matching feature value. That is, the matching feature value determination unit replaces the feature value of the cell whose difference value is equal to or less than the threshold value among the feature values of the cells in the detection frame in the input image with the difference value.

  The feature amount matching unit 17 checks the matching feature amount of each cell in the detection frame with the feature amount of the dictionary data 14, and determines whether or not a specific object exists in the detection frame. Then, when there is a detection frame determined that a specific object exists, information such as the position and size of the detection frame is sent to the output processing unit 18.

  The output processing unit 18 outputs input images to the display device 3 such as a liquid crystal display in time series order. When the output processing unit 18 receives information on the detection frame determined that the specific object is present from the feature amount matching unit 17, the output processing unit 18 outputs an image in which the detection frame is combined with the input image to the display device 3.

  The image processing apparatus 1 according to the present embodiment is realized by a computer and a program executed by the computer. At this time, the feature quantity extraction unit 12, the pre-learning unit 13, the matching feature quantity determination unit 16, the feature quantity matching unit 17, and the output processing unit 18 are a Central Processing Unit (CPU), a Digital Signal Processor (DSP), or the like. This is realized by causing a processor to execute a predetermined program. The storage unit for storing the input image holding unit 11 and the background feature quantity database 15 and the dictionary data 14 includes a hard disk drive connected to the processor via a bus, a storage device such as a semiconductor memory, or a cache memory inside the processor. Use.

<Pre-learning process>
When the specific object in the input image is detected by the image processing apparatus 1 of the present embodiment, the dictionary data 14 of the HOG feature amount used for collation with the specific object candidate in the input image is created and registered in advance. deep. Therefore, before explaining a method for detecting a specific object, a pre-learning processing procedure for registering the HOG feature value and the dictionary data 14 will be briefly explained.

FIG. 2 is a schematic diagram for explaining the HOG feature amount.
The HOG feature value is a histogram of the luminance gradient direction in a local region in the image, and is used as a feature value when detecting a specific object such as a person from the image.

  As shown in FIG. 2A, the HOG feature amount is extracted for each cell 41 by dividing one image 40 into a plurality of cells 41. One cell 41 is composed of a plurality of pixels (5 × 5 pixels in the example shown in FIG. 2A).

  The HOG feature amount of one cell 41 is a feature obtained by calculating the gradient direction and gradient intensity of the luminance in each pixel in the cell 41 and, for example, dividing the gradient direction into nine parts as shown in FIG. Amount. What is necessary is just to extract the HOG feature-value used by this embodiment with the well-known extraction method.

  Further, when the specific object is a person, in the pre-learning, a rectangular area 42 of n × m cells (n and m are arbitrary natural numbers) including the person 50 from the image 40 shown in FIG. Let the cut-out image be an image in which a specific object exists. Then, a plurality of images each having a specific object (person 50) such as the rectangular area 42 and images having no specific object are prepared as learning images, and the HOG feature amount in each image is extracted.

  Note that the HOG feature value is a feature value obtained by dividing the gradient direction into a histogram, but in the HOG feature value graph referred to in the following description, the gradient strength in each direction as shown in FIG. The HOG feature value is schematically represented by the curve PH based on it.

FIG. 3 is a flowchart showing a procedure for preliminary learning.
Prior learning in the image processing apparatus 1 is performed by a known learning method applied to a method for detecting a specific object using HOG feature values. When performing pre-learning, first, as shown in FIG. 3, a learning image is input to the image processing apparatus 1 (step S101). The learning image is input via the image acquisition unit 10 or an input interface (not shown), and is held in the input image holding unit 11. For the learning image, a plurality of images each including a specific object (person) and non-existing images are used. In addition, as the learning image, for example, an image obtained by cutting out a rectangular region of n × m cells from an image obtained by imaging the same range as the imaging range of the input image when performing the specific object detection process is used.

  Next, the feature amount extraction unit 12 extracts the HOG feature amount of each input learning image (step S102). Step S102 is performed by a known extraction method.

  Next, the pre-learning unit 13 learns the HOG feature quantity of the specific object and the HOG feature quantity of the non-specific object (background object), and creates dictionary data 14 (step S103). The learning of the HOG feature amount is performed by a known machine learning method such as Support Vector Machine (SVM) or Adaptive Boosting (AdaBoost).

  Then, the created dictionary data 14 is stored in a storage unit included in the image processing apparatus 1 (step S104).

<Specific object detection processing>
After the dictionary data 14 is created by the above prior learning, the image processing apparatus 1 starts a process for detecting a specific object in the input image.

  FIG. 4 is a flowchart illustrating the procedure of the specific object detection process according to the first embodiment. FIG. 5 is a diagram illustrating an example of the input image and the HOG feature amount.

  When the image processing device 1 starts the detection process of the specific object, the image acquisition unit 10 sequentially acquires images captured by the imaging device 2 and causes the input image storage unit 11 to store the images as input images. At this time, the image acquisition unit 10 sequentially assigns frame numbers (serial numbers) unique to the acquired input image in order from 1 in order.

  When the acquisition of the input image is started, next, as shown in FIG. 4, first, a variable i representing the frame number of the input image is initialized (step S2). Since the frame number of the input image is a serial number starting from 1, as described above, the variable i is set to 1 in step S2.

  Next, the feature quantity extraction unit 12 extracts the HOG feature quantity from the input image of frame number i (step S3).

  Next, the feature quantity extraction unit 12 determines whether the frame number i of the input image is greater than 1 (step S4). When i = 1 (step S4; No), the feature quantity extraction unit 12 registers the extracted HOG feature quantity in the background feature quantity database 15 as a background feature quantity (step S5).

  On the other hand, if i> 1 (step S4; Yes), the feature amount extraction unit 12 passes the extracted HOG feature amount to the matching feature amount determination unit 16, and the matching feature amount determination unit 16 and the feature amount matching unit 17 The collation process is performed (step S6). In step S6, a matching feature value of each cell in the input image is determined based on the difference value between the extracted HOG feature value and the background feature value, and input is performed by matching the matching feature value with the feature value of the dictionary data. It is determined whether or not a specific object exists in the image. The specific procedure of the collation process in step S6 will be described later. When the collation process in step S6 is completed, the collation result is output (step S7). Step S7 is performed by the output processing unit 18, and when a specific object is detected from the input image having the frame number i by the process of step S6, a frame surrounding the detected specific object is synthesized (added) and displayed on the display device 3. Output to. If no specific object is detected from the input image with the frame number i, the output processing unit 18 outputs the input image to the display device 3 as it is.

  After step S5 for registering the background feature amount or step S7 for outputting the collation result, it is determined whether to continue the detection process (step S8). When the detection process is continued (step S8; Yes), the variable i representing the frame number is incremented by 1 (i = i + 1), and the process from step S3 is repeated. If the detection process is not continued (step S8; No), the detection process is terminated.

  In step S3 in the above detection process, as shown in FIG. 5A, the input image 40 is divided into a plurality of cells 41, and HOG feature values are extracted for each cell 41. At this time, one cell 41 has w × h pixels. Further, the plurality of cells 41 are given a unique cell number, and are identified and identified by the xy coordinates of the upper left corner of each cell 41 in the input image 40. Then, the extracted HOG feature value of each cell 41 is associated with a unique cell number, cell position, and cell size, as shown in FIG.

Note that H _{1 to} H ₉ in the HOG feature amount shown in FIG. 5B are gradient intensities in the respective gradient directions when the gradient direction is divided into nine portions in the angle range of 0 to 180 degrees. The larger the numerical value of the subscript, the larger the angle representing the gradient direction. For example, the gradient strength H ₁ is a value obtained by integrating the gradient strengths of pixels whose gradient direction θ (degree) is 0 ≦ θ <20, and the gradient strength H ₂ is a pixel whose gradient direction θ (degree) is 20 ≦ θ <40. It is a value obtained by integrating the gradient intensities.

  In the specific object detection process according to the present embodiment, as described above, the input image with the frame number of 1, that is, the first input image (at the start of the detection process) is used as the background image, and the HOG feature amount extracted from the background image is used. Register in the background feature database 15. At this time, the input image 40 as the background image may include a non-specific object whose shape such as the flag 51 can change as shown in FIG. In the collation process in step S6, the collation feature quantity of each cell of the input image is determined based on the difference value between the HOG feature quantity and the background feature quantity of the input image, and the collation feature quantity and the dictionary data are collated. To do.

  FIG. 6 is a flowchart showing the contents of the collation process of FIG. FIG. 7 is a flowchart showing the processing contents of the matching feature amount determination processing of FIG.

  In the collation process, as shown in FIG. 6, first, a detection list representing the detection result of the specific object is initialized (step S601). The detection list is, for example, data describing the position and size of a frame surrounding an image detected as a specific object from the input image, and is held in a cache memory or the like provided in the image processing apparatus 1.

  Next, the matching feature amount determination unit 16 sets a detection frame in the input image (step S602). The detection frame is the outer periphery of a block composed of a plurality of cells that are continuous vertically and horizontally. A plurality of types of detection frame dimensions are prepared according to the dimensions of a specific object (person) in the input image. In step S602, the size of the detection frame is selected and applied to the input image, and the cells included in the detection frame are specified.

  Next, the matching feature value determining unit 16 performs a matching feature value determining process for determining the matching feature value of the cell in the detection frame (step S603). In step S603, processing as shown in FIG. 7 is performed.

  In step S603, first, one cell in the detection frame is selected (step S603a), and the HOG feature (background feature) of the cell in the background image corresponding to the selected cell is acquired from the background feature database ( Step S603b). In step S603b, a background feature amount is acquired based on the cell position shown in FIG.

  Next, a difference value ΔH between the HOG feature amount and the background feature amount of the selected cell is calculated (step S603c).

  Next, it is determined whether the difference value ΔH is greater than the threshold value TH1 (step S603d). When ΔH> TH1 (step S603d; Yes), the HOG feature value extracted in step S3 is set as the feature value for matching of the selected cell (step S603e). On the other hand, if ΔH ≦ TH1 (step S603d; No), the difference value ΔH is used as the matching feature amount of the selected cell (step S603f). The threshold value TH1 is set to a value such that a matching feature amount of a cell including at least a non-specific object existing in the background image becomes the difference value ΔH. For example, the threshold TH1 uses a plurality of images in which a non-specific object such as the flag 51 shown in FIG. 5A exists, and statistical processing is performed on the difference value ΔH of the HOG feature amount of a cell including the non-specific object. And decide. Further, since the HOG feature value obtained by dividing the gradient direction into nine parts is composed of nine values, the difference value ΔH is also composed of nine values. Therefore, the difference value ΔH is large when compared with all nine values (for example, the entire curve PH in FIG. 2), but decreases in several gradient directions when viewed individually. Sometimes. Therefore, for example, when 5 or more of the 9 values are larger than the threshold value TH1, ΔH> TH1 is set, and the determination criterion in step S603d may be determined as appropriate.

  After determining the matching feature amount of the selected cell, it is next determined whether or not there is a cell whose matching feature amount has not been determined in the detection frame (step S603g). When there is a cell for which the matching feature amount is not determined (step S603g; Yes), the processing from step S603a is repeated. When there is no cell whose matching feature value is not determined (step S603g; No), the matching feature value determination process (step S603) is terminated.

  As shown in FIG. 6, when the matching feature value determination process (step S603) is completed, the similarity between the image in the detection frame and the specific object is calculated from the matching feature value and the HOG feature value of the dictionary data. S is calculated (step S604). The feature amount matching unit 17 performs the process of step S604. The similarity S is calculated by a known calculation method in the detection method using the HOG feature amount.

  Next, the feature amount matching unit 17 determines whether the calculated similarity S is greater than the threshold value TH2 (step S605). The larger the value of the similarity S, the higher the similarity between the image in the detection frame and the specific object. Therefore, the feature amount matching unit 17 registers the set value (for example, position and size) of the detection frame in the detection list only when S> TH2 (step S605; Yes) (step S606). The threshold value TH2 is an arbitrary value, and any one of values adopted in the detection method using the HOG feature amount is used.

  Next, it is determined whether there is an unprocessed detection frame (step S607). In step S607, for example, it is first determined whether or not the processing in steps S602 to S606 with the selected detection frame of one dimension has been performed on all regions of the input image. If there is a region that has not been performed, it is determined that there is an unprocessed detection frame (step S607; Yes), and the processing of steps S602 to S606 is repeated for the region that has not been performed. In addition, when the processing of steps S602 to S606 using the detection frame of one size is performed on all regions of the input image, it is determined whether all the same processing using the detection frame of another size is performed. If there is a detection frame that has not been subjected to the same process, it is determined that there is an unprocessed detection frame (step S607; Yes), and steps S602 to S606 are performed until the same process is completed for the detection frames of all dimensions. Repeat the process.

  When the processes in steps S602 to S606 are repeated and it is determined that there is no unprocessed detection frame (step S607; No), the collation process ends.

  Note that when the processing of steps S602 to S606 with one detection frame is repeated in the collation processing of step S6, in step S602, for example, the setting position of the detection frame is shifted so as to raster scan the entire input image.

  In the specific object detection process in the image processing apparatus 1, when the collation process in step S6 ends, the collation result is output as shown in FIG. 4 (step S7).

  In step S7, for example, based on the position and size of the detection frame registered in the detection list in step S606, a frame line surrounding the detected specific object is synthesized (added) to the input image having the frame number i, and the display device 3 is added. Output to.

<Specific object detection example>
An example of detection of a specific object by the above-described detection process will be described with reference to FIGS.

  FIG. 8 is a schematic diagram illustrating an example of two input images used in the matching feature amount determination process. FIG. 9 is a schematic diagram illustrating an example of the feature value difference value ΔH in the input image of FIG. 8. FIG. 10 is a schematic diagram illustrating an image after the matching feature amount determination processing using the input image of FIG. 8 and the matching result image.

  In the specific object detection process of the present embodiment, the HOG feature value extracted from the first (frame number 1) input image is registered in the background feature value database 15 as the background feature value, separately from the previously learned HOG feature value. When a specific object is detected from the input image after frame number 2, the matching feature value of each cell in the input image is first determined by the matching feature value determination process (step S6).

  As shown in FIG. 8A, the input image (detection target image) for detecting a specific object includes an image 40b in which a flag 51 that is a non-specific object and a person 50 that is a specific object are present. On the other hand, as shown in FIG. 8B, only the flag 51 that is a non-specific object exists in the first input image (background image) 40a.

  When the matching process (step S6) is performed on the detection target image 40b, each cell in the detection frame 43 including the flag 51 of the detection target image 40b and the detection frame 44 including the person 50 are detected by the matching feature amount determination process. A matching feature amount for each cell or the like is determined.

  The matching feature amount of the leftmost cell 41a in the detection frame 43 of the detection target image 40b includes the HOG feature amount extracted from the cell 41a and the HOG feature amount (background) of the cell 41A of the background image 40a corresponding to the cell 41a. It is determined based on the difference value ΔH with respect to the feature amount. Similarly, the matching feature quantity of the right cell 41b in the third row from the top in the detection frame 43 of the detection target image 40a is the HOG feature quantity extracted from the cell 41b and the cell 41B of the reference image 40a corresponding to the cell 41b. It is determined based on the difference value ΔH from the HOG feature amount (background feature amount).

  When the flag 51 existing in the detection frame 43 is installed on the road surface or the like, the position of the flag 51 in both the images 40a and 40b is almost the same although there may be a difference in contour (shape). The HOG feature amount is a feature amount obtained by histogramating the luminance gradient direction in the local region, and is a feature amount that is invariant to the translation of the object in the image. Therefore, as shown in FIG. 9A, the HOG feature amount PH1 of the cell 41b of the detection target image 40b and the HOG feature amount PH2 of the cell 41B of the background image 40a have substantially the same value. Therefore, the difference value ΔH of the HOG feature amount between the cell 41b of the detection target image 40b and the cell 41B of the background image 40a is a value in which the gradient strengths in all gradient directions are very small as shown in FIG. become. In addition, the difference value ΔH of the HOG feature amount of other cells in the detection frame 43 is a very small value in the gradient strengths in all gradient directions, like the difference value ΔH shown in FIG. .

  On the other hand, the person 50 does not exist in the detection frame 44 of the background image 40a corresponding to the detection frame 44 of the detection target image 40b. Therefore, as shown in FIG. 9B, the HOG feature amount PH3 of the leftmost cell 41c in the detection frame 44 of the detection target image 40b and the HOG feature of the cell 41C of the background image 40a corresponding to the cell 41c. The value is different from the amount PH4. Therefore, the difference value ΔH of the HOG feature quantity between the cell 41c of the detection target image 40b and the cell 41C of the background image 40a has a large gradient strength in several gradient directions as shown in FIG. 9B. Become. Further, the difference value ΔH of the HOG feature amount of the other cells in the detection frame 44 has a large gradient strength in several gradient directions, like the difference value ΔH shown in FIG.

  As described above, the difference value ΔH of the HOG feature amount between the cell of the detection target image 40b and the cell of the background image 40a indicates whether an object existing in the cell of the detection target image 40b is also present in the cell of the background image 40a. But it is very different. In the present embodiment, using such a property of the HOG feature amount, when an object existing in the cell of the detection target image 40b is also present in the cell of the reference image 40a, the matching feature amount is set to the difference value ΔH. If there is no match, the HOG feature value is extracted from the matching feature value. Specifically, a threshold value TH1 as shown in FIGS. 9A and 9B is set, and for cells whose difference value ΔH is equal to or less than the threshold value TH1, the difference value ΔH is used as a matching feature amount. A cell larger than the threshold value TH1 uses the extracted HOG feature value as a matching feature value. For this reason, all the matching feature amounts of the cells in the detection frame 43 of the detection target image 40b are the difference value ΔH. On the other hand, the matching feature quantity of each cell in the detection frame 44 of the detection target image 40b is the HOG feature quantity that is all extracted.

  That is, when the matching feature amount determination process is performed on the detection target image 40b, the flag 51 does not exist in the detection frame 43 like the image 40c illustrated in FIG. Substantially equivalent to an image. Therefore, when the image 40c and the dictionary data 14 are collated, the similarity S between the feature amount in the detection frame 43 and the person (specific object) is low, and the similarity S between the feature amount in the detection frame 44 and the person is Get higher. Therefore, only the person 50 in the detection frame 44 is detected as a specific object from the detection target image 40b, and only the person 50 is surrounded by the frame 55 in the collation result image 40d as shown in FIG.

  As described above, in the method for detecting a specific object according to the present embodiment, the extracted feature value is replaced with the difference value ΔH for the cell whose feature value difference value ΔH is smaller than the threshold value TH1 among the cells of the detection target image 40b. The difference value ΔH is used as a matching feature amount. Therefore, even if the HOG feature value of the flag 51 existing in the detection frame 43 of the detection target image 40b is similar to the person's HOG feature value, the matching feature value to be matched with the dictionary data is the person's HOG feature value. Will have different values. Therefore, it is possible to prevent erroneous detection of a non-specific object whose HOG feature value is similar to the HOG feature value of a person (specific object) as a person.

  Next, another detection example of the specific object by the above-described detection processing will be described with reference to FIGS.

  FIG. 11 is a schematic diagram illustrating another example of two input images used in the matching feature amount determination process. FIG. 12 is a schematic diagram illustrating an example of the feature value difference value ΔH in the input image of FIG. 11. FIG. 13 is a schematic diagram illustrating an image of the matching result after the matching feature amount determination process using the input image of FIG.

  When the specific object detection process is performed by the image processing apparatus 1, the input image (detection target image) for detecting the specific object is captured more than the flag 51, which is a non-specific object, as shown in FIG. There is also an image 40e in which the person 50 located on the apparatus side overlaps the flag 51. On the other hand, as shown in FIG. 11B, only the flag 51 that is a non-specific object exists in the first input image (background image) 40a.

  When the matching process (step S6) is performed on the detection target image 40e, the matching feature quantity of each cell in the detection frame 45 including the flag 51 and the person 50 of the detection target image 40e is obtained by the matching feature quantity determination process. decide.

  The matching feature amount of the leftmost cell 41e in the detection frame 45 of the detection target image 40e includes the HOG feature amount extracted from the cell 41e and the HOG feature amount (background) of the cell 41E of the background image 40a corresponding to the cell 41e. It is determined based on the difference value ΔH with respect to the feature amount. Similarly, the matching feature quantity of the center cell 41f in the third row from the top in the detection frame 45 of the detection target image 40e is the HOG feature quantity extracted from the cell 41f and the cell 41F of the background image 40a corresponding to the cell 41f. It is determined based on the difference value ΔH from the HOG feature amount (background feature amount).

  When the flag 51 existing in the detection frame 43 is installed on the road surface or the like, the position of the flag 51 in both the images 40a and 40e is almost the same although there may be a difference in contour (shape). Therefore, as shown in FIG. 12A, the HOG feature amount PH5 of the cell 41e of the detection target image 40b in which only the flag 51 exists and the HOG feature amount PH6 of the cell 41E of the background image 40a have substantially the same value. Become. Therefore, the difference value ΔH of the HOG feature amount between the cell 41e of the detection target image 40e and the cell 41E of the background image 40a is such that the gradient strengths in all gradient directions are lower than the threshold value TH1 as shown in FIG. Small value. Therefore, the matching feature amount of the cell 41e of the detection target image 40e is the difference value ΔH.

  On the other hand, as shown in FIGS. 11A and 11B, only the flag 51 exists in the cell 41F of the background image 40a corresponding to the cell 41f of the detection target image 40e where the flag 51 and the person 50 exist. To do. Therefore, as shown in FIG. 12B, the HOG feature amount PH7 of the cell 41f of the detection target image 40e and the HOG feature amount PH8 of the cell 41F of the background image 40a have different values. Therefore, the difference value ΔH of the HOG feature amount between the cell 41f of the detection target image 40e and the cell 41F of the background image 40a is such that the gradient strength in several gradient directions is less than the threshold value TH1 as shown in FIG. Will also be a large value. Therefore, the matching feature amount of the cell 41f of the detection target image 40e is the HOG feature amount extracted from the cell 41f.

  That is, when the matching feature amount determination process is performed on the detection target image 40e, the processed image is a part of the flag 51 existing in the detection frame 45 as in the image 40f illustrated in FIG. Is substantially equivalent to a missing image. Thereby, the feature amount (shape) of the image existing in the detection frame 45 approaches the feature amount of the person 50. Therefore, when the image 40f and the dictionary data 14 are collated, the similarity S between the feature amount in the detection frame 45 and the person increases. Therefore, the person 50 in the detection frame 45 is detected as a specific object from the detection target image 40f, and the person 50 is surrounded by the frame 55 in the collation result image 40g as shown in FIG.

  As described above, in the specific object detection method of the present embodiment, even when the specific object in the input image overlaps the non-specific object, the non-specific object is suppressed while suppressing the change in the feature amount (contour region) of the specific object. The feature amount can be subtracted and a specific object can be detected correctly.

  As described above, in the specific object detection method according to the first embodiment, the difference between the HOG feature value extracted from the input image (detection target image) and the HOG feature value (background feature value) extracted from the background image. The feature value for collation of a cell having a small value ΔH is set to a difference value ΔH. Thereby, the feature amount of the non-specific object existing in the detection target image and the background image can be subtracted from the detection target image. Therefore, it is possible to prevent the non-specific object from being erroneously detected as the specific object when the feature amount of the non-specific object is similar to the feature amount of the specific object.

  In the specific object detection method according to the first embodiment, when the specific object in the input image overlaps the non-specific object, the non-specific object is suppressed while suppressing the change in the feature amount (contour region) of the specific object. The feature amount can be subtracted. Therefore, the specific object can be detected correctly even when the specific object and the non-specific object overlap.

  Therefore, the image processing apparatus 1 that realizes the above-described specific object detection method can improve the detection accuracy of the specific object.

  In the first embodiment, the detection method for detecting a specific object using the HOG feature amount has been described. However, the feature amount used for detecting the specific object is not limited to the HOG feature amount, and may be a feature amount that can be regarded as invariable or invariant with respect to translation of the object in the image. Therefore, the feature amount used for detecting the specific object may be a color invariant such as a hue, for example. Further, for example, when the feature values extracted from a plurality of images are statistically processed, other feature values may be used as long as the variance or standard deviation is small and can be regarded as invariant with respect to translation.

  In the matching feature amount determination process described in the first embodiment, as shown in FIG. 7, a detection frame is set, and a matching feature amount is determined for each cell in the detection frame. However, the matching feature amount is not limited to each cell in the detection frame, and may be determined by raster scanning the cell in the input image, for example. Therefore, for example, instead of step S603 illustrated in FIG. 6, a process for determining a matching feature amount may be provided between step S4 and step S6 in the process illustrated in FIG.

  Further, as shown in FIGS. 5A and 8B, the input image as the background image does not include a specific object such as a person, and the HOG feature value of the specific object is the HOG feature value. It is preferable that the image includes a non-specific object that can take a value similar to. However, in the method for detecting a specific object according to the first embodiment, the specific object may exist in the input image as the background image.

  When the specific object is a person, there is a high possibility that the person existing in the background image does not exist in the input image after a predetermined time has elapsed. Therefore, a region corresponding to a person existing in the background image in the input image is a background object, and the HOG feature value is different from the person's HOG feature value. Therefore, the difference value ΔH between the HOG feature value of the background object and the person's HOG feature value is larger than the threshold value TH1, and the HOG feature value of the background object becomes the matching feature value. Therefore, the specific object detection method according to the first embodiment can detect the specific object from the input image even if the specific object exists in the input image as the background image.

  As described above, in the method for detecting a specific object according to the first embodiment, a specific object (person) may exist in the input image as a background image. Therefore, when the detection process starts, the first input image is automatically set. Therefore, it can be used as a background image. Therefore, separately from the pre-learning process, before starting the detection process, it is not necessary to perform a process of acquiring an image without a specific object (person) and registering it in the background feature amount database. Therefore, it is possible to prevent an increase in labor until the detection process is started, and the detection process can be easily performed.

  The image processing apparatus 1 shown in FIG. 1 includes a pre-learning unit 13 for creating dictionary data 14. However, the dictionary data 14 may be created by another image processing apparatus and read into the image processing apparatus 1 through a portable recording medium or a network. Therefore, the image processing apparatus 1 according to the first embodiment may not have the prior learning unit 13.

(Second Embodiment)
FIG. 14 is a flowchart showing the procedure of the specific object detection process according to the second embodiment of the present invention.

  In the present embodiment, a specific object detection process procedure different from that of the first embodiment will be described. Note that the detection processing of this embodiment is performed by the image processing apparatus 1 shown in FIG.

  In the specific object detection process of the present embodiment, as shown in FIG. 14, first, a background image candidate condition is set (step S10). In step S10, the frame number of the image that is a candidate for the background image in the input image or the imaging time is set. When the frame number is used as a condition, in addition to the frame number 1, a condition such as when the frame number i is i = 1000 × n (n is a natural number) is set. When the imaging time is used as a condition, in addition to the detection processing start time (T = 0), a condition such as every u hours or every day s is set.

  After step S10, the image processing apparatus 1 starts acquiring an image captured by the imaging apparatus 2, and causes the input image holding unit 11 to hold the image as an input image. At this time, the image acquisition unit 10 sequentially assigns frame numbers (serial numbers) unique to the acquired input image in order from 1 in order.

  Once acquisition of the input image is started, next, a variable i representing the frame number of the input image is initialized (step S2). In step S2, the variable i is set to 1.

  Next, the feature quantity extraction unit 12 extracts the HOG feature quantity from the input image of frame number i (step S3).

  Next, the feature amount extraction unit 12 determines whether the input image with the frame number i is a background image candidate (step S11). In step S11, the condition set in step S10 is compared with the frame number i of the input image or the imaging time. If the input image is a background image candidate (step S4; Yes), the feature amount extraction unit 12 continues to perform background feature amount setting processing (step S12).

  On the other hand, when the input image is not a background image candidate (step S11; No), the collation process described in the first embodiment is performed (step S6), and the collation result is output (step S7).

  After the background feature amount setting process (step S5) or the process of outputting the collation result (step S7), it is determined whether to continue the detection process (step S8). When the detection process is continued (step S8; Yes), the variable i representing the frame number is incremented by 1 (i = i + 1), and the process from step S3 is repeated. If the detection process is not continued (step S8; No), the detection process is terminated.

FIG. 15 is a flowchart showing the processing content of the background feature amount setting processing of FIG.
In the background feature value setting process performed in the detection process of the present embodiment, as shown in FIG. 15, first, it is determined whether a background feature value is registered in the background feature value database 15 (step S1201). If the input image is the first (frame number 1) input image and no background feature value is registered (step S1201; No), the HOG feature value extracted in step S3 is registered in the database as a background feature value (step S1202). ), The background feature amount setting process is terminated.

  On the other hand, if the input image is not the first input image but a background feature amount is registered (step S1201; Yes), the background feature amount in the database is compared with the HOG feature amount extracted from the input image (step S1203). It is determined whether the difference between the feature amounts satisfies a predetermined update condition (step S1204). The update condition is set such that, for example, a predetermined number or more of cells having a feature value difference value ΔH larger than the threshold value TH1 exist.

  If the difference in feature quantity does not satisfy the predetermined update condition (step S1204; No), the background feature quantity setting process is terminated without updating the background feature quantity in the database. On the other hand, if the update condition is satisfied (step S1204; Yes), the background feature amount in the database is updated to the HOG feature amount extracted from the input image (step S1205), and the background feature amount setting process is terminated.

  The processes in steps S6 to S8 in the detection process of the present embodiment may be as described in the first embodiment.

  That is, in the detection process of the first embodiment, the HOG feature amount extracted from the first input image (at the start of the detection process) is continuously used as the background feature amount, whereas in the detection process of the present embodiment, the background feature amount is used as needed. Update.

FIG. 16 is a schematic diagram illustrating an example of a temporal change in the presence or absence of a non-specific object in the input image.
In the first embodiment, as an example of the background image, the HOG feature value can be a value similar to the HOG feature value of the specific object (person) as shown in FIG. 5A and FIG. 8B. An image in which a non-specific object (flag 51) is present is cited. However, the specific object detection process using the image processing apparatus 1 is started at an arbitrary timing. Therefore, the first input image (frame number 1) may be an image in which the flag 51 does not exist, as in the image 40h illustrated in FIG. In FIG. 16, time t0 is the start time of the detection process, and the time axis advances from top to bottom.

  In this case, the image processing apparatus 1 performs the matching process in step S6 using the HOG feature value extracted from the image 40h in which the flag 51 does not exist as the background feature value.

  Thereafter, when the flag 51 is set in the imaging region between the times t1 and t2, an image 40k in which the flag 51 is present is input to the image processing apparatus 1 as illustrated in FIG. In the detection process described in the first embodiment, the image 40k is a detection target image, and after extracting the HOG feature value, the matching feature value is determined based on the difference value from the background feature value of the image 40k. Therefore, in the detection process described in the first embodiment, the flag 51 of the image 40k may be erroneously detected as a specific object (person).

  On the other hand, in the detection method of the present embodiment, the background feature amount is updated as needed based on the condition set in step S10 and the update condition used in step S12. Therefore, after the time t2, when the image having the flag 51 such as the image 40k becomes a background image candidate, the background image is changed from the image 40h to the image 40k, and the HOG feature amount extracted from the image 40k is the background feature. Amount. Therefore, it is possible to prevent a situation in which the flag 51 is detected as a specific object by the matching process using the image 40k as an input image, and the frame 57 surrounding the flag 51 is displayed on the display device 3.

  Although detailed description is omitted, when the detection process is performed using the HOG feature value extracted from the image 40k as the background feature value, the flag 51 is withdrawn like the image 40h, or other non-specification is performed. When an object is installed, the background feature amount is updated at a predetermined timing.

  As described above, in the specific object detection method according to the second embodiment, the background feature amount is updated as needed according to a predetermined condition. Therefore, even when the presence or absence of the non-specific object in the input image changes over time, It prevents false detection of specific objects and maintains high detection accuracy. Therefore, for example, when observing a place where the presence or absence of a non-specific object changes depending on the business hours of a store such as a shopping street, it is difficult to be affected by the presence or absence of a non-specific object, and stable detection accuracy can be maintained.

  In the second embodiment, the feature amount extraction unit 12 of the image processing apparatus 1 illustrated in FIG. 1 performs background feature amount setting processing. However, the image processing apparatus that performs the detection process of the present embodiment is not limited to this, and may be an apparatus that includes a processing unit that performs a background feature amount setting process. In addition, the image processing apparatus, for example, causes the feature amount extraction unit 12 to register the HOG feature amount extracted from the first (frame number 1) input image in the background feature amount database 15 and updates the background feature amount to be performed thereafter. A processing unit different from the feature amount extraction unit 12 may perform the processing.

The following additional notes are further disclosed with respect to the embodiments including the examples described above.
(Appendix 1)
An image acquisition unit that acquires an image captured by the camera;
A feature amount extraction unit that divides the image into a plurality of cells and extracts a feature amount of each cell;
A storage unit for storing dictionary data including a background feature amount extracted from an image designated as a background image of the acquired images and a feature amount of a specific object detected from the image;
A matching feature value determining unit that determines a matching feature value of each cell of the image based on a difference value between the feature value extracted from each cell of the image to be checked against the dictionary data and the background feature value;
A feature amount matching unit for matching the feature amount for matching of each cell of the image with the feature amount of the dictionary data;
An image processing apparatus comprising:
(Appendix 2)
The matching feature amount determination unit determines, as the difference value, a matching feature amount of a cell whose difference value between the extracted feature amount and the background feature amount of the image cells is equal to or less than a predetermined threshold. Determining a feature quantity of a cell having the difference value larger than the threshold as a feature quantity extracted from the cell;
The image processing apparatus according to appendix 1, characterized by:
(Appendix 3)
A background feature amount updating unit that updates the background feature amount according to a predetermined condition;
The image processing apparatus according to appendix 1, characterized by:
(Appendix 4)
The feature extraction unit extracts a feature that is invariant to translation in the image;
The image processing apparatus according to appendix 1, characterized by:
(Appendix 5)
The dictionary data includes Histograms of Oriented Gradients (HOG) features of the specific object,
The feature quantity extraction unit extracts the HOG feature quantity;
The image processing apparatus according to appendix 1, characterized by:
(Appendix 6)
The feature amount matching unit matches a feature amount for matching in a detection frame including a plurality of cells with a feature amount of the dictionary data, and features of the matching feature amount in the detection frame and a specific object feature of the dictionary data The image processing apparatus according to appendix 1, wherein when the degree of similarity with the amount is high, it is determined that the specific object exists within the detection frame.
(Appendix 7)
The additional processing unit according to claim 6, further comprising: an output processing unit that synthesizes and outputs a frame that surrounds the specific object based on the position and size of the detection frame determined that the specific object exists. Image processing device.
(Appendix 8)
An image processing method for detecting a specific object from the image by collating an image captured by a camera with dictionary data including a feature amount of the specific object detected from the image,
A first process of dividing the image into a plurality of cells and extracting a feature amount of each cell;
A second process of registering a background feature amount extracted from an image designated as a background image of the images in a database;
A third process for determining a matching feature quantity of each cell of the image based on a difference value between the feature quantity extracted from each cell of the image to be matched with the dictionary data and the background feature quantity;
A fourth process of collating the image feature amount with the dictionary data feature amount;
An image processing method comprising:
(Appendix 9)
In the third process, the difference value between the extracted feature value and the background feature value of the cell of the image to be checked against the dictionary data is equal to or less than a predetermined threshold value. Determine the value,
An image processing method, wherein a matching feature amount of a cell having the difference value larger than the threshold value is determined as a feature amount extracted from the cell.
(Appendix 10)
The image processing method according to appendix 8, wherein the second process designates an image acquired first when the specific object detection process is started as the background image.
(Appendix 11)
In the second process, the background image candidate is designated according to a predetermined condition, and the difference between the feature quantity extracted from the background image candidate and the background feature quantity registered in the database is a predetermined condition. A process of updating the background feature value to a feature value extracted from the background image candidates when satisfied,
The image processing method according to appendix 8, wherein
(Appendix 12)
The image processing method according to appendix 11, wherein the second process specifies the background image candidate every time a predetermined time has elapsed since the start of the specific object detection process.
(Appendix 13)
The image processing method according to appendix 11, wherein the second process designates the background image candidate every time a predetermined time arrives.
(Appendix 14)
The first process extracts a feature quantity that is invariant to translation in the image;
The image processing method according to appendix 8, characterized by:
(Appendix 15)
The first processing includes extracting Histograms of Oriented Gradients (HOG) features,
The image processing method according to appendix 8, characterized by:
(Appendix 16)
Register in advance dictionary data including the feature quantity of the specific object detected from the image captured by the camera,
Dividing the image into a plurality of cells and extracting feature values of each cell;
The background feature amount extracted from the image designated as the background image among the images is registered in the database,
The feature value for matching of each cell of the image to be matched with the dictionary data is determined as a feature value extracted from the cell, or a difference value between the extracted feature value and the background feature value,
Collating the image feature amount with the dictionary data feature amount,
A program that causes a computer to execute processing.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 10 Image acquisition part 11 Input image holding | maintenance part 12 Feature-value extraction part 13 Prior learning part 14 Dictionary data 15 Background feature-value database 16 Matching feature-value determination part 17 Feature-value collation part 18 Output processing part 2 Imaging device 3 Display device 40, 40a-40k Image 41, 41a-41f, 41A-41F Cell 50 Person 51 Flag

Claims (8)

An image acquisition unit that acquires an image captured by the camera;
A feature amount extraction unit that divides the image into a plurality of cells and extracts a feature amount of each cell;
A storage unit for storing dictionary data including a background feature amount extracted from an image designated as a background image of the acquired images and a feature amount of a specific object detected from the image;
A matching feature value determining unit that determines a matching feature value of each cell of the image based on a difference value between the feature value extracted from each cell of the image to be checked against the dictionary data and the background feature value;
A feature amount matching unit for matching the feature amount for matching of each cell of the image with the feature amount of the dictionary data;
An image processing apparatus comprising: The matching feature amount determination unit determines, as the difference value, a matching feature amount of a cell whose difference value between the extracted feature amount and the background feature amount of the image cells is equal to or less than a predetermined threshold. Determining a matching feature amount of a cell having the difference value larger than the threshold value as a feature amount extracted from the cell;
The image processing apparatus according to claim 1. The feature extraction unit extracts a feature that is invariant to translation in the image;
The image processing apparatus according to claim 1. The dictionary data includes Histograms of Oriented Gradients (HOG) features of the specific object,
The feature quantity extraction unit extracts the HOG feature quantity;
The image processing apparatus according to claim 1. An image processing method for detecting a specific object from the image by collating an image captured by a camera with dictionary data including a feature amount of the specific object detected from the image,
A first process of dividing the image into a plurality of cells and extracting a feature amount of each cell;
A second process of registering a background feature amount extracted from an image designated as a background image of the images in a database;
A third process for determining a matching feature quantity of each cell of the image based on a difference value between the feature quantity extracted from each cell of the image to be matched with the dictionary data and the background feature quantity;
A fourth process of collating the image feature amount with the dictionary data feature amount;
An image processing method comprising: The third process determines, as the difference value, a matching feature amount of a cell whose difference value between the extracted feature amount and the background feature amount of the image cells is equal to or less than a predetermined threshold value,
The image processing method according to claim 5, wherein a matching feature amount of a cell having the difference value larger than the threshold is determined as a feature amount extracted from the cell. In the second process, the background image candidate is designated according to a predetermined condition, and the difference between the feature quantity extracted from the background image candidate and the background feature quantity registered in the database is a predetermined condition. A process of updating the background feature value to a feature value extracted from the background image candidates when satisfied,
The image processing method according to claim 5. Register in advance dictionary data including the feature quantity of the specific object detected from the image captured by the camera,
Dividing the image into a plurality of cells and extracting feature values of each cell;
The background feature amount extracted from the image designated as the background image among the images is registered in the database,
The feature value for matching of each cell of the image to be matched with the dictionary data is determined as a feature value extracted from the cell, or a difference value between the extracted feature value and the background feature value,
Collating the image feature amount with the dictionary data feature amount,
A program that causes a computer to execute processing.