JP2018165966A - Object detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately detect each object in a photographed image even if it is a photographed image in which a space where congestion may occur.SOLUTION: An object detection device that detects individual objects from a photographed image obtained by photographing a space in which congestion by a predetermined object may occur includes: density estimation means 50 that uses a density estimator that learns image features of each of density images obtained by photographing a space in which an object exists with predetermined density for each predetermined density, to estimate a distribution of density of an object photographed in the photographed image; and object position determination means 51 that sets a candidate position at which each object can exist in the photographed image, calculates an evaluation value indicating degree of appearance of an image feature of a single object in the photographed image at the candidate position, and determines a candidate position whose evaluation value is equal to or larger than the predetermined value as the position of the object; where the object position determination means 51 calculates the evaluation value by changing a part of importance which is emphasized among parts constituting the single object according to density at the candidate position.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an object detection device that detects individual objects from a captured image in which a space in which a predetermined object such as a person can exist is captured, and in particular, an individual object from a captured image in which a space in which congestion can occur is captured. The present invention relates to an object detection device for detecting the above.

  In an event venue or other space where congestion can occur, countermeasures such as placing a large number of guards in the crowded area are required to prevent accidents. Therefore, monitoring cameras can be arranged at various locations in the venue to estimate the distribution of people from the captured images and display the estimated distribution, thereby facilitating the understanding of the congestion situation by the monitoring staff.

  At that time, the position of each person is detected, and a model imitating the person's shape is displayed at each detected position, or / and the positional relationship of the person (for example, forming a matrix or surrounding) Further analysis efficiency can be expected by analyzing and notifying the analysis result.

  The method of detecting the position of each person from a photographed image taken by multiple people, combining multiple models that imitate people, and applying it to the photographed image, as well as feature values of images taken by a single person in advance There is a method of detecting a position where an image feature of a single person appears from a captured image using a previously prepared image feature of a single person, such as a method of scanning a captured image using a discriminator that has been learned.

  For example, in the moving object tracking device described in Patent Document 1, a moving object model imitating the shape of the moving object being tracked is extracted at the position where the changed pixel is extracted by comparing the monitoring image and the background image. The positions of individual moving objects are detected by combining and applying the number of moving objects. In this moving object tracking apparatus, the use of a moving object model that approximates the shape of a person's whole body is exemplified.

  Further, for example, the object detection device described in Patent Document 2 detects a person from an input image using a classifier previously learned using a large number of “human” image data and “non-human” image data. . It has been suggested that the classifier used by this object detection apparatus is learned using image data of the whole body of a person. In this object detection device, a circle is detected from an input image and used as a candidate region for the human head.

JP 2012-159958 A JP 2011-186633 A

  However, in a captured image in which a space where congestion can occur is captured, the concealment state of a person changes according to the congestion state. For this reason, there has been a problem that it is difficult to accurately detect individual persons if the image characteristics of the same part are always used as the image characteristics of a single person regardless of the congestion state.

  That is, for a captured image in which there are many people who are not crowded and the whole body has been photographed, both in the method using a model imitating a person and in the method using a discriminator that learns a human image, The person using the image feature can be detected with higher accuracy than using the image feature only near the head.

  On the other hand, for captured images that are congested and frequently concealed, both in the method using a model imitating a person and in the method using a discriminator that learns a human image, the image features of the whole body are used. The person can be detected with higher accuracy by using only the image feature near the head.

Therefore, for example, if image features only near the head are always used to increase detection accuracy during congestion, the detection accuracy when there is no congestion is reduced, and detection accuracy when there is no congestion is increased. For this reason, if the image features of the whole body are always used, the detection accuracy at the time of congestion decreases.
That is, there is a trade-off between the congestion state and the number of parts used for detecting individual objects due to a change in the concealment state.

  In addition, regions with different congestion states may be mixed in the captured image. This changes the detection accuracy for each region, and the problem becomes more complicated.

  In this way, in the captured image in which a space where congestion can occur is captured, the concealment state of the object to be detected changes according to the congestion state, and thus it is difficult to accurately detect individual objects from the captured image. there were.

  The present invention has been made in view of the above problems, and provides an object detection device that can accurately detect individual objects in a captured image even in a captured image of a space in which congestion can occur. The purpose is to do.

  In order to achieve such an object, the present invention provides an object detection device for detecting individual objects from a captured image in which a space in which congestion due to a predetermined object may occur is captured, and for each predetermined density Density estimating means for estimating the density distribution of the object photographed in the photographed image using a density estimator that has learned image features of the respective density images photographed in the space in which the object exists, and the photographed image A candidate position where each of the objects can exist is set, and an evaluation value representing the degree to which the image feature of the single object appears in the captured image at the candidate position is calculated, and the evaluation value is a predetermined value Object position determination means for determining the candidate position as described above as the position of the object, wherein the object position determination means is a part of a part constituting the single object according to the density at the candidate position. To provide an object detection apparatus and calculates the evaluation value by changing the portion to be emphasized.

  It is preferable that the object position determination unit calculates the evaluation value with an emphasis on image features of a small part of the parts constituting the single object as the density at the candidate position is higher.

  Further, the object position determination means represents the degree that the image features of a small part of the parts constituting the single object appear at the candidate position of the photographed image as the density at the candidate position is higher. It is preferable to calculate the evaluation value.

  Alternatively, the object position determination unit calculates a partial evaluation value indicating a degree of appearance of image features of a plurality of parts constituting the single object at the candidate position of the captured image, and the density at the candidate position is calculated. It is preferable that the higher the is, the higher the weight is given to the partial evaluation values of a small part of the parts constituting the object, and the evaluation value is calculated by summing the partial evaluation values.

  In addition, the object position determination unit is configured to generate a plurality of different arrangements each including one or more candidate positions, and to each candidate position for each of the plurality of arrangements. The model image generating means for generating a model image by drawing an object model simulating a small part of the parts constituting the single object, and the model for each of the plurality of arrangements, Evaluation value calculating means for calculating the evaluation value representing the degree of similarity of the image to the photographed image, and optimum arrangement determining means for determining the candidate position in the arrangement having the maximum evaluation value as the position of the object. Is preferred.

  In addition, the object position determination unit generates a plurality of different arrangements each including one or more candidate positions, and the object at each candidate position for each of the plurality of arrangements. Model image generation means for generating a model image by drawing an object model simulating an object model, and for each of the model images in the plurality of arrangements, the degree of similarity of the object model to the captured image for each part constituting the object And an evaluation value calculation means for calculating the evaluation value by performing weighting with a weight being applied to a smaller portion as the density at the candidate position is higher and summing up the similarities, and an arrangement in which the evaluation value is the maximum It is preferable that an optimum arrangement determining unit that determines the candidate position as the position of the object is included.

  In addition, the object position determination unit includes a candidate position setting unit that sets a plurality of candidate positions at predetermined intervals in the captured image, and for each of the candidate positions, the higher the density of the candidate positions, Evaluation value calculating means for calculating the evaluation value by inputting the image feature of the photographed image at the candidate position to the discriminator that has learned the image characteristics of a small part of the parts constituting the object, and a predetermined reference It is preferable that a position determination unit that determines the candidate position where the evaluation value satisfying the condition is calculated as the position of the object is included.

  In addition, the object position determination unit includes a candidate position setting unit that sets a plurality of candidate positions at predetermined intervals in the captured image, and a plurality of partial images that constitute the single object for each of the candidate positions. The image features of the photographed image at the candidate position are input to the classifier that has learned the features to obtain partial evaluation values of the plurality of portions, and weighting is applied to the smaller portions at the candidate positions with increasing weight. Evaluation value calculating means for calculating the evaluation value by summing the partial evaluation values, and position determining means for determining the candidate position where the evaluation value satisfying a predetermined criterion is calculated as the position of the object It is preferable to contain.

  According to the present invention, it is possible to accurately detect individual objects from a captured image in which a space where congestion can occur is captured.

It is a block diagram which shows the schematic structure of an image monitoring apparatus. It is a functional block diagram which shows the function of an image monitoring apparatus. It is a functional block diagram which shows the function of an image monitoring apparatus. It is the figure which represented typically the information of the object model which the object model memory | storage means has memorize | stored. It is the figure which showed typically the example of a process by a density estimation means, an arrangement | positioning production | generation means, and a model image generation means. It is the flowchart which showed operation | movement of the image monitoring apparatus. It is a flowchart of the object position determination process of an image monitoring apparatus. It is a flowchart of the object position determination process of an image monitoring apparatus. It is a functional block diagram which shows the function of an image monitoring apparatus. It is the figure which represented typically the information of the object model which the object model memory | storage means has memorize | stored, and the information of the weighting coefficient which the weighting coefficient memory | storage means has memorize | stored. It is the figure which showed typically the model image and weight image which the model image generation means produced | generated. It is a flowchart of the object position determination process of an image monitoring apparatus. It is a flowchart of the object position determination process of an image monitoring apparatus. It is a functional block diagram which shows the function of an image monitoring apparatus. It is the figure which represented typically the information of the single classifier which the single classifier memory | storage means has memorize | stored. It is the figure which showed typically the extraction window for identification which an evaluation value calculation means sets. It is a flowchart of the object position determination process of an image monitoring apparatus. It is a functional block diagram which shows the function of an image monitoring apparatus. It is the figure which represented typically the information of the single discriminator memorize | stored in the single discriminator memory | storage means, and the information of the weighting coefficient which the weighting coefficient memory | storage means has memorize | stored. It is the figure which showed typically a mode that an evaluation value calculation means calculates an identification score. It is a flowchart of the object position determination process of an image monitoring apparatus.

[First embodiment]
Hereinafter, as an embodiment of the present invention, an example of an image monitoring device 1 that includes an example of an object detection device that detects an individual person from a captured image obtained by shooting an event venue, and that displays a detection result to a monitor will be described. To do. In the image monitoring apparatus 1 according to this embodiment, in particular, an example in which the object detection device detects an individual person using an object model imitating a person, and the object detection apparatus switches the object model according to the density of the person at that time. Including.

<Configuration of Image Monitoring Device 1 according to First Embodiment>
FIG. 1 is a block diagram showing a schematic configuration of the image monitoring apparatus 1. The image monitoring apparatus 1 includes a photographing unit 2, a communication unit 3, a storage unit 4, an image processing unit 5, and a display unit 6.

  The photographing unit 2 is a surveillance camera and is connected to the image processing unit 5 via the communication unit 3. The photographing unit 2 shoots the monitoring space at a predetermined time interval to generate a photographed image, and sequentially captures the photographed image to the image processing unit 5. It is a photographing means to input. For example, the imaging unit 2 is installed on a pole installed at an event site with a view of the monitoring space. The visual field may be fixed, or may be changed according to a schedule in advance or an instruction from the outside via the communication unit 3. Further, for example, the imaging unit 2 captures the monitoring space with a frame period of 1 second and generates a color image. A monochrome image may be generated instead of the color image.

  The communication unit 3 is a communication circuit, one end of which is connected to the image processing unit 5 and the other end is connected to the photographing unit 2 and the display unit 6 via a communication network such as a coaxial cable, a LAN (Local Area Network), or the Internet. Connected. The communication unit 3 acquires a captured image from the imaging unit 2 and inputs the acquired image to the image processing unit 5, and outputs the detection result input from the image processing unit 5 to the display unit 6.

  The storage unit 4 is a memory device such as a ROM (Read Only Memory) or a RAM (Random Access Memory), and stores various programs and various data. The storage unit 4 is connected to the image processing unit 5 and inputs / outputs such information to / from the image processing unit 5.

  The image processing unit 5 is configured by an arithmetic device such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an MCU (Micro Control Unit). The image processing unit 5 is connected to the storage unit 4 and the display unit 6, operates as various processing units / control units by reading out and executing programs from the storage unit 4, and stores various types of data in the storage unit 4 for reading. . The image processing unit 5 is also connected to the imaging unit 2 and the display unit 6 via the communication unit 3 and detects individual persons by analyzing the captured image acquired from the imaging unit 2 via the communication unit 3. The detection result is displayed on the display unit 6 via the communication unit 3.

  The display unit 6 is a display device such as a liquid crystal display or a CRT (Cathode Ray Tube) display, and is a display unit that is connected to the image processing unit 5 via the communication unit 3 and displays a detection result by the image processing unit 5. . The monitor visually checks the displayed detection result to determine the occurrence of congestion, and takes measures such as changing the personnel arrangement as necessary.

  In the present embodiment, the image monitoring apparatus 1 in which the number of the photographing units 2 and the image processing units 5 is 1: 1 is illustrated, but in another embodiment, the number of the photographing units 2 and the image processing units 5 is illustrated. Can be many-to-one or many-to-many.

<Function of the image monitoring apparatus 1 according to the first embodiment>
2 and 3 are functional block diagrams showing functions of the image monitoring apparatus 1. The communication unit 3 functions as the image acquisition unit 30, the object position output unit 31, and the like, and the storage unit 4 functions as the density estimator storage unit 40, the single feature storage unit 41, and the like. The image processing unit 5 functions as a density estimation unit 50, an object position determination unit 51, and the like. The single feature storage unit 41 includes a function as an object model storage unit 410a, and the object position determination unit 51 functions as an arrangement generation unit 510a, a model image generation unit 512a, an evaluation value calculation unit 514a, and an optimum arrangement determination unit 516a. including.

  The image acquisition unit 30 sequentially acquires captured images from the imaging unit 2 that is an imaging unit, and sequentially outputs the acquired captured images to the density estimation unit 50 and the object position determination unit 51.

  The density estimator storage unit 40 is an estimated density calculation function that learns the image features of each density image obtained by photographing a space where an object (person) exists at a predetermined density for each predetermined density. When input, an estimated value (estimated density) of the density of an object photographed in the image is calculated, and information on an estimator (density estimator) that outputs the calculated estimated density is stored in advance. That is, parameters such as the coefficient of the estimated density calculation function are stored in advance as information on the density estimator.

  The density estimation unit 50 extracts a density estimation feature amount (estimation feature amount) from each part of the captured image input from the image acquisition unit 30 and reads the density estimator from the density estimator storage unit 40 to extract it. Each estimated feature amount is input to a density estimator to estimate an estimated density distribution (density distribution), and the estimated density distribution is output to the object position determination unit 51.

  The density estimation process and the density estimator will be specifically described.

  The density estimation means 50 sets a window (estimation extraction window) at the position of each pixel of the captured image, and extracts an estimation feature amount from the captured image in each estimation extraction window. The estimation feature amount is a GLCM (Gray Level Co-occurrence Matrix) feature.

  It is desirable that the area in the monitoring space photographed by each estimation extraction window is the same size. That is, preferably, the density estimation means 50 reads out the camera parameters of the photographing unit 2 stored in advance from a camera parameter storage means (not shown), and is photographed at an arbitrary pixel of the photographed image by homography conversion using the camera parameters. The estimation feature amount is extracted after the captured image is deformed so that the areas in the monitoring space have the same size.

The density estimator can be realized by a classifier that identifies multi-class images, and can be a discrimination function learned by a multi-class SVM (Support Vector Machine) method.
Density, for example, there is no human "Background" class is 0 people / m higher than ² is two / m ² or less "low density" class, higher than two / m ² 4 persons / m ² or less It can be defined as 4 classes of “medium density” class, “high density” class higher than 4 persons / m ² .

  The estimated density is a value given in advance to each class, and is a value output as a result of distribution estimation. In the present embodiment, values corresponding to each class are expressed as “background”, “low density”, “medium density”, and “high density”.

  That is, the density estimator applies the multi-class SVM method to the feature quantities of a large number of images (density images) belonging to the “background” class, “low density” class, “medium density” class, and “high density” class. This is an identification function for discriminating the images of each class from other classes. The parameters of the discriminant function derived by this learning are stored as a density estimator. The feature amount of the density image is the same type as the estimation feature amount and is a GLCM feature.

  The density estimation means 50 acquires the estimated density which is the output value by inputting each of the estimation feature quantities extracted corresponding to each pixel to the density estimator. In addition, when the estimated feature amount is extracted by deforming the captured image, the density estimating unit 50 deforms the density distribution into the original captured image shape by homography conversion using the camera parameter.

  A collection of estimated densities for each pixel of the captured image thus obtained is a density distribution.

From the density distribution output by the density estimation means 50, the density of people at various locations in the photographed image can be understood, but the position of each person cannot be determined from the density distribution.
On the other hand, the object position determination means 51 subsequent to the density estimation means 50 is a means for determining the position of each person appearing in the captured image.

  The object position determination means 51 detects an individual object by searching a captured image for a place where an image feature as a single object (person) appears, and determines the position of the individual object. That is, the object position determination unit 51 sets a candidate position where an individual object can exist in the photographed image, and represents the degree to which the image feature (single feature) of a single object appears in the photographed image at the candidate position. An evaluation value is calculated, and a candidate position whose evaluation value is greater than or equal to a predetermined value is determined as an object position. For example, the single feature is a human shape, and the single feature storage unit 41 stores the single feature in advance. For example, the evaluation value is the similarity between the edge of the captured image and a model representing the shape of a person.

  Here, in a captured image in which a space in which congestion can occur is captured, concealment that occurs between people due to congestion hides some of the single features, and if the evaluation value decreases thereby, individual persons cannot be detected. . When the photographing unit 2 is installed in an overhead view, the closer to the foot, the easier the concealment occurs, and the closer to the head, the less likely the concealment occurs. Considering this, if the single feature is only the head of a person so as to adapt to the congestion, the detection loss at the time of congestion is reduced. However, since a relatively high evaluation value is calculated for a single feature of the head only for the shoulder or the like, false detection when there is no congestion increases.

  The object position determination unit 51 refers to the density distribution to eliminate the trade-off between the number of parts to be evaluated and the detection accuracy of each object. That is, the object position determination unit 51 calculates an evaluation value by changing an important part of parts constituting a single object according to the density at the candidate position. In particular, the object position determination unit 51 calculates the evaluation value with an emphasis on the image features of a smaller part of the parts constituting the object as the density at the candidate position is higher. For example, the object position determination unit 51 calculates the evaluation value by equally evaluating the whole body if the estimated density of the candidate positions is low, and calculates the evaluation value by focusing on the upper body if the density is medium. In the case of density, the evaluation value is calculated with emphasis on the vicinity of the head.

  Hereinafter, detection of individual objects and single unit characteristics will be described.

  The single feature storage unit 41 functions as the object model storage unit 410a that stores in advance information on an object model that simulates the shape of a single person (object), and stores the information on the object model as a single feature.

  FIG. 4 is a diagram schematically showing the information of the single feature stored in the single feature storage unit 41, that is, the object model information stored in the object model storage unit 410a.

  The object model stored in the object model storage unit 410a is specifically a three-dimensional model 700 including three spheroids corresponding to the head, torso, and legs of a standing person. Note that the center of gravity of the head is the representative position of the person. Further, the object model storage unit 410a stores an evaluation range 702 for each density together with the stereo model 700, and stores the camera parameters 701 of the imaging unit 2 in order to project the stereo model 700 onto the coordinate system of the captured image. ing. The camera parameters 701 include external parameters such as the installation position and imaging direction of the imaging unit 2 in the actual monitoring space, internal parameters such as the focal length, angle of view, lens distortion and other lens characteristics of the imaging unit 2 and the number of pixels of the imaging device. It is information to include.

  The evaluation range 702 is a smaller portion of the portions constituting a single object as the density is higher. Specifically, the object model storage unit 410a associates the value representing the low density class with “entire”, the value representing the medium density class with “upper 2/3”, and the value representing the high density class. In addition, the setting “upper 1/3” is stored. Hereinafter, the low density object model 710 represented by a combination of the overall evaluation range and the three-dimensional model 700 is the whole body model, and the medium density medium model represented by the combination of the evaluation range of the upper 2/3 and the three-dimensional model 700 is used. The object model 711 is referred to as an upper body model, and the object model 712 for high density represented by a combination of the evaluation range of the upper ３ and the three-dimensional model 700 is referred to as a head vicinity model.

  As described above, the object model storage unit 410a associates the whole body model 710 with the low density class, the upper body model 711 with the medium density class, the head vicinity model 712 with the high density class, the camera It is stored together with the parameter 701 as object model information.

  The arrangement generation unit 510a generates a plurality of different arrangements each including one or more candidate positions, and outputs the generated arrangements to the model image generation unit 512a.

  For this purpose, the arrangement generation unit 510a uses one or more pixels (arrangement number) that are one or more and not more than the upper limit number of pixels whose estimated density is low density, medium density, or high density among pixels of the captured image based on random numbers. Are determined at random, and the determined position of each pixel is set as a candidate position to generate an arrangement. The arrangement generation unit 510a generates a plurality of different arrangements by repeating this generation by a predetermined number of times for each arrangement number while sequentially increasing the number of arrangements. Note that the upper limit of the number of arrangements can be the upper limit of the number of objects that can exist in the monitoring space. For example, it is possible to arrange a standing person's three-dimensional model in a virtual space imitating the monitoring space without overlapping. It can be calculated as a number.

  For each of a plurality of arrangements input from the arrangement generation unit 510a, the model image generation unit 512a simulates a smaller portion of the portions constituting a single object at each candidate position as the density at the candidate position is higher. The model object is drawn to generate a model image, and each generated model image is output to the evaluation value calculation unit 514a.

  For this purpose, the model image generation unit 512a reads the camera parameters from the object model storage unit 410a, and uses the camera parameters for each arrangement to set each candidate position to the height of the center of gravity of the head of the stereo model (for example, 1.5 m). The representative position in the virtual space imitating the monitoring space of the three-dimensional model projected at the candidate position is calculated by back-projecting to the horizontal plane.

  Further, the model image generation unit 512a reads out the head vicinity model from the object model storage unit 410a, arranges the head vicinity model at a representative position in the virtual space corresponding to each candidate position, and uses the camera parameters to The neighborhood model is projected onto the coordinate system of the captured image. Then, the model image generation unit 512a refers to the density distribution input from the density estimation unit 50 and totals the estimated density in the projection region of the head vicinity model corresponding to each candidate position, and the largest number at each candidate position. The estimated density (excluding the background class) is determined as the density of the candidate position.

In addition, the model image generation unit 512a reads an object model corresponding to the density of the candidate positions from the object model storage unit 410a for each candidate position. Specifically, the model image generation means 512a reads the whole body model if the density of candidate positions is low, reads the upper body model if the density is medium, and reads the head vicinity model if the density is high. Then, for each arrangement, the model image generating unit 512a arranges the object model read out corresponding to each candidate position at a representative position in the virtual space corresponding to the candidate position, and uses the camera parameters for each whole body model. By projecting the shape onto the coordinate system of the captured image, a model image for each arrangement is generated.
Note that the model image generation unit 512a sequentially projects from the object model arranged at the representative position that is far from the photographing unit 2, and generates a model image expressing the concealment between the object models by overwriting the projection area. .

In addition, the model image generation unit 512a calculates a concealment degree that represents the degree of overlap between the object models in the model image for each arrangement according to the following equation.
Concealment degree = area of overlapping area between models / area of sum area of model projection areas (1)

  Then, the model image generating unit 512a associates the arrangement, the model image, and the degree of concealment, and outputs them to the evaluation value calculating unit 514a.

FIG. 5 is a diagram schematically illustrating a processing example by the density estimation unit 50, the arrangement generation unit 510a, and the model image generation unit 512a according to the first embodiment.
The image 720 is an image of the density distribution estimated by the density estimation unit 50. In the density distribution, the white area is the area where the estimated density is the background, the horizontal line area is the area where the estimated density is low density, the shaded area is the area where the estimated density is medium density, and the lattice area is the high density estimated area Each region is shown.
The image 721 is obtained by plotting the eight candidate positions included in the arrangement generated by the arrangement generation unit 510a in the coordinate system of the photographed image with x marks.
The three-dimensional model 722 illustrates a state in which the three-dimensional model is arranged at the representative position in the virtual space corresponding to the eight candidate positions shown in the image 721 by the model image generating unit 512a.
The image 723 is created by the model image generation unit 512a specifying the density of each candidate position based on the density distribution shown in the image 720 and projecting a three-dimensional model in the evaluation range according to the density to each candidate position. A model image is shown.

  The evaluation value calculation unit 514a calculates an evaluation value representing the degree of similarity of the model image input from the model image generation unit 512a to the photographed image for each of a plurality of types of arrangement, and sets the evaluation value for each arrangement as an optimum arrangement determination unit. Output to 516a.

Specifically, the evaluation value calculation unit 514a calculates the similarity between each model image and the captured image according to the following equation.
Similarity = Shape conformity-W _Ha x Concealment (2)
However, W _Ha is a weighting coefficient larger than 0, and is preset based on a prior experiment. The degree of concealment subtracted from the shape matching degree is a penalty value for suppressing the overlap of excessive object models. In this way, by determining the optimum arrangement based on the similarity including the concealment degree, it is possible to prevent erroneous detection of the object position due to the application of an object model equal to or more than the original number of objects.

  The shape matching degree can be the similarity of the edge between the model image and the captured image. The evaluation value calculation means 514a extracts an edge from each model image and each photographed image, and for each model image, for each pixel from which a valid edge is extracted from the model image, the corresponding edge of the pixel of the photographed image. The absolute value of the difference is calculated and summed, and the value obtained by dividing the sum by the number of pixels from which the edge is extracted from the model image and inverting the sign is calculated as the shape suitability of the model image.

  Alternatively, the evaluation value calculation unit 514a generates an edge image from each model image and the captured image, and for each model image, chamfer matching between the edge image generated from the captured image and the edge image generated from the model image is performed. A value obtained by inverting the sign of the chamfer distance obtained by performing (Chamfer Matching) is calculated as the shape matching degree of the model image.

The optimum arrangement determining unit 516a refers to the evaluation value for each arrangement input from the evaluation value calculating unit 514a, determines the candidate position in the arrangement having the maximum evaluation value as the object position, and uses the information on the determined object position as the object Output to the position output means 31. That is, the optimum arrangement determining unit 516a determines each candidate position included in the arrangement where the maximum similarity is calculated as the position of each person photographed in the photographed image.
For example, the optimum arrangement determining unit 516a generates and outputs information on the object position by drawing an object model in each object position in a color-coded manner according to the density of the object position so that the observer can easily recognize it. Alternatively, the object position information may be the coordinate value of the object position itself, and the object position information may be a region of each drawn object model that does not overlap with other object models. Alternatively, the object position information may be data including two or more of the above-described data.

  The object position output unit 31 sequentially outputs the object position information input from the object position determination unit 51 to the display unit 6, and the display unit 6 displays the object position information input from the object position output unit 31. For example, the information on the object position is transmitted / received via the Internet and displayed on the display unit 6. The monitoring person grasps a point where the monitoring space is congested by viewing the displayed information, and takes measures such as dispatching or increasing the number of guards at the point.

<Operation of the image monitoring apparatus 1 according to the first embodiment>
The operation of the image monitoring apparatus 1 will be described with reference to the flowcharts of FIGS. 6, 7, and 8.

  When the image monitoring apparatus 1 starts operation, the image capturing unit 2 installed in the event venue captures the monitoring space every predetermined time and sequentially transmits the captured images to the image analysis center in which the image processing unit 5 is installed. To do. The image processing unit 5 repeats the operation according to the flowchart of FIG. 6 every time a captured image is received.

  First, the communication unit 3 operates as the image acquisition unit 30 and waits to receive a captured image from the imaging unit 2. The image acquisition means 30 that acquired the captured image outputs the captured image to the image processing unit 5 (step S1).

  The image processing unit 5 to which the photographed image is input operates as the density estimating means 50, and estimates the density distribution from the photographed image (step S2). The density estimation unit 50 extracts the estimation feature quantity at the position of each pixel of the captured image, reads the density estimator from the density estimator storage unit 40 of the storage unit 4, and uses each estimation feature quantity as the density estimator. The density distribution is estimated by inputting and obtaining the estimated density at each pixel of the captured image.

  The image processing unit 5 that has estimated the density distribution also operates as the object position determination unit 51. The captured image is input from the image acquisition unit 30 and the density distribution is input from the density estimation unit 50 to the object position determination unit 51. . The object position determination means 51 that has received these confirms whether the density distribution includes an estimated density other than the background class (step S3).

  If the estimated density other than the background class is included (YES in step S3), the object position determination unit 51 determines the position of each object from the captured image, assuming that at least one person is captured. A determination process is performed (step S4). On the other hand, in the case of only the background class (NO in step S3), the processes in steps S4 and S5 are omitted assuming that no person is photographed.

  The object position determination process in step S4 will be described with reference to the flowcharts of FIGS. The single feature storage unit 41 operates as the object model storage unit 410a, and the object position determination unit 51 operates as the arrangement generation unit 510a, the model image generation unit 512a, the evaluation value calculation unit 514a, and the optimal arrangement determination unit 516a. Judgment processing is executed.

  The arrangement generation unit 510a sequentially sets the number of arrangements within a range from 1 to the upper limit number (step S100), and controls the loop processing of steps S100 to S114.

  The arrangement generation unit 510a prepares a variable T for counting the number of iterations, initializes T to 0 (step S101), and starts the iterative process of steps S102 to S113.

  Next, the arrangement generation unit 510a sets the same number of candidate positions as the number of arrangements set in step S100 in the low density, medium density, or high density region in the density distribution input from the density estimation unit 50. By setting at random, a T-th arrangement in the arrangement number is generated and output to the model image generation unit 512a (step S102).

  The model image generation unit 512a reads camera parameters from the object model storage unit 410a, and converts each candidate position included in the arrangement generated in step S102 into three-dimensional coordinates in the virtual space using the camera parameters (step S103). .

  Next, the model image generation unit 512a prepares and initializes a model image having the same size as the captured image, calculates the distance to the imaging unit 2 of the three-dimensional coordinates of each candidate position, and the candidate position with a long distance Are sequentially set as processing targets (step S104), and the loop processing of steps S104 to S108 is executed.

  Subsequently, the model image generating unit 512a specifies the density of the candidate position to be processed with reference to the density distribution (step S105). The model image generation unit 512a reads out the head vicinity model from the object model storage unit 410a, arranges it in the three-dimensional coordinates of the candidate position, projects the head vicinity model on the coordinate system of the captured image using the camera parameters, The highest estimated density (except the background class) in the projection area is specified as the density of candidate positions.

  Subsequently, the model image generation unit 512a reads the object model corresponding to the density specified in step S105 from the object model storage unit 410a (step S106), arranges it at the three-dimensional coordinates of the candidate position to be processed, and sets the camera parameter. Is used to over-project the placed object model onto the model image (step S107). At this time, the model image generation means 512a records the projected area of the object model.

  Then, the model image generating means 512a checks whether or not all candidate positions included in the Tth arrangement in the current arrangement number have been processed (step S108), and if there is an unprocessed candidate position ( NO at step S108), the process returns to step S104 to process the next candidate position.

On the other hand, when all candidate positions have been processed (YES in step S108), the model image for the Tth arrangement in the current arrangement number is completed. The model image generating means 512a that completes the model image calculates the degree of concealment of the object model in the model image (step S109). That is, the model image generation unit 512a obtains the area of the projection area on the model image that is “the area of the sum area of the model projection areas” and sums up the projection areas for each object model recorded in step S107. Then, by subtracting the area of the sum area of the projected areas of the model from the total value, the “area of the overlapping area between the models” is obtained, and these are substituted into Equation (1) to calculate the degree of concealment.
The model image generation unit 512a that has calculated the concealment degree outputs the model image and the concealment degree to the evaluation value calculation unit 514a.

  The evaluation value calculation means 514a to which the model image and the concealment degree are input calculates the shape conformity between the model image and the captured image (step S110), and further, the model image and the photographed image are obtained from the shape conformity and the concealment degree. Is calculated as an evaluation value for the Tth arrangement in the current arrangement number (step S111). In other words, the evaluation value calculation unit 514a generates an edge image from each of the model image and the captured image input from the model image generation unit 512a, and calculates the similarity between these edge images as the shape matching degree. Then, the similarity is calculated by substituting the shape matching degree and the concealment degree into the equation (2).

When the evaluation value for the Tth arrangement in the current arrangement number is calculated, the evaluation value calculation means 514a records the arrangement and the evaluation value in association with each other, and the arrangement generation means 510a increases the number of iterations T by one. (Step S112), and compared with the specified number of times T _MAX (step S113). If T is less than T _MAX (NO in step S113), the process returns to step S102 to continue the iterative process for the current number of arrangements. .

When the number of iterations T has reached the specified number of times T _MAX (YES in step S113), the arrangement generation unit 510a ends the iterative process for the current number of arrangements and confirms whether all the arrangement numbers have been set. (Step S114). If there is an unset number of arrangements (NO in step S114), the process returns to step S100 to perform processing for the next number of arrangements.

  On the other hand, when all the arrangement numbers have been set (YES in step S114), evaluation value calculation means 514a inputs the arrangement and evaluation values recorded in step S112 to optimum arrangement determination means 516a, and optimum arrangement determination means 516a identifies the arrangement having the maximum evaluation value among them (step S115), and determines that the arrangement is information representing the position of each person photographed in the photographed image.

  The description will be continued with reference to FIG. The object position determination unit 51 outputs information on the position (object position) of each person determined in step S4 to the communication unit 3 (step S5). The communication unit 3 to which the object position information is input operates as the object position output unit 31 and transmits the object position information to the display unit 6.

  When the above process is completed, the process returns to step S1, and the process for the next captured image is performed.

[Second Embodiment]
Hereinafter, as a preferred embodiment of the present invention different from the first embodiment, an example of an image monitoring device 1 including an example of an object detection device that changes the weighting of an object model according to the density of a person will be described.

  The image monitoring apparatus according to the second embodiment differs from the image monitoring apparatus according to the first embodiment in the details of the single feature stored in the single feature storage unit 41 and the details of the processing performed by the object position determination unit 51. A part of the general configuration, the general function, and the operation is common. Therefore, a part of the schematic configuration, the schematic function, and the operation will be described with reference to the block diagram of FIG. 1, the functional block diagram of FIG. 2, and the flowchart of FIG. .

<Configuration of Image Monitoring Device 1 according to Second Embodiment>
The schematic configuration of the image monitoring apparatus 1 according to the second embodiment will be described with reference to the block diagram of FIG.
As in the first embodiment, the image monitoring apparatus 1 includes a photographing unit 2 that photographs a monitoring space at predetermined time intervals and outputs a photographed image, and a display unit 6 that receives information on an object position and displays the information. An image processing unit 5 that acquires a captured image, detects an individual person (object) from the captured image, and generates and outputs information on the position (object position) of the detected object. In addition to being connected to the communication unit 3 through which input and output of position information and the like are connected, a storage unit 4 that stores programs and various data and inputs and outputs them is connected to the image processing unit 5.

<Function of the image monitoring apparatus 1 according to the second embodiment>
The function of the image monitoring apparatus 1 according to the second embodiment will be described with reference to the functional block diagrams of FIGS. 2 and 9.

  Similar to the first embodiment, the communication unit 3 receives a captured image from the image capturing unit 2 and outputs the captured image to the density estimation unit 50 and the object position determination unit 51 and the object position determination unit 51. A function as an object position output means 31 or the like for outputting information on the object position to the display unit 6.

  In addition, as in the first embodiment, the storage unit 4 stores a density estimator that learns the image features of each density image obtained by photographing a space where an object exists at the predetermined density for each predetermined density. The single feature stored in the single feature storage unit 41 includes functions such as an estimator storage unit 40 and a single feature storage unit 41 that stores image features (single features) of a single object in advance. The higher the is, the more the evaluation can be made with an emphasis on the image features of a small part of the parts constituting the object.

  Further, as in the first embodiment, the image processing unit 5 estimates the density distribution of the object photographed in the photographed image by scanning the photographed image with the density estimator, and determines the estimated density distribution as the object position. The density estimation means 50 output to the means 51, and an evaluation value representing the degree to which the image feature of a single object appears in the photographed image at the candidate position by setting candidate positions where individual objects can exist in the photographed image And a candidate position having an evaluation value equal to or greater than a predetermined value is determined as an object position, and the object position determination unit 51 includes a function as an object position determination unit 51 that outputs information on the object position to the object position output unit 31. The means 51 uses the single feature according to the density of the candidate position, and calculates the evaluation value with an emphasis on the image feature of a smaller part of the parts constituting the object as the density at the candidate position is higher.

  However, as described above, the details of the processing performed by the object position determination unit 51 according to the second embodiment and the details of the single feature stored in the single feature storage unit 41 are the image monitoring apparatus according to the first embodiment. Different from 1. These points will be described with reference to the functional block diagram of FIG.

  The unit feature storage unit 41 according to the second embodiment includes an object model storage unit 410b that stores in advance information on an object model that imitates the shape of a single person (object), and a weighting factor that is used in calculating an evaluation value in advance. It functions as the stored weight coefficient storage means 412b, and stores object model information and weight coefficient information as single features.

  FIG. 10 shows the single feature stored in the single feature storage unit 41 according to the second embodiment, that is, the object model information stored in the object model storage unit 410b and the weight coefficient storage unit 412b. It is the figure which represented the information of the weighting coefficient typically.

  The object model stored in the object model storage unit 410b is a three-dimensional model 750 composed of three spheroids corresponding to the head, trunk, and legs of a standing person. This three-dimensional model 750 is an object model that represents the shape of a person's whole body, and is hereinafter referred to as a whole body model. Note that the center of gravity of the head is the representative position of the person. Further, the object model storage unit 410b stores the camera parameters 751 of the photographing unit 2 together with the whole body model in order to project the whole body model onto the coordinate system of the photographed image.

  The weighting factor is set such that the higher the density is, the more the weight is concentrated on the smaller part of the part constituting the single object. The weighting coefficient storage unit 412b associates with the value representing the low density class “weighting coefficient 0.333 applied to the upper 1/3” “weighting coefficient 0.333 applied to the middle 1/3” “lower 1/3. “Weighting coefficient 0.333” applied to the middle density class, “weighting coefficient 0.500 applied to the upper third”, “weighting coefficient 0.400 applied to the middle third”, “lower” “Weighting coefficient 0.100 applied to 1/3”, “weighting coefficient 0.700 applied to upper 1/3” “weighting coefficient 0.200 applied to middle 1/3” in association with a value representing a high-density class "" Weighting coefficient 0.100 applied to lower 1/3 "is stored. Hereinafter, a weight coefficient 760 for low density that is uniform throughout the whole body is expressed as a uniform weight coefficient for whole body, a weight coefficient 761 for medium density that emphasizes the upper body, an overweight weight coefficient for upper body, and a weight coefficient 762 for high density that emphasizes the vicinity of the head. This is referred to as the near-head weighting coefficient.

  As described above, the object model storage unit 410b stores the whole body model and camera parameters as object model information, and the weight coefficient storage unit 412b associates the whole body uniform weight coefficient 760 with the low density class and the medium density class. Are stored in association with the upper-body body weighting factor 761 and the high-density class in association with the head vicinity weighting factor 762.

  The arrangement generation unit 510b generates a plurality of different arrangements each including one or more candidate positions in the same manner as the arrangement generation unit 510a described in the first embodiment. And the arrangement | positioning production | generation means 510b outputs each produced | generated arrangement | positioning to the model image generation means 512b.

  The model image generating unit 512b generates a model image by drawing an object model imitating a single object at each candidate position for each of a plurality of arrangements input from the arrangement generating unit 510b, and generates each model image. Is output to the evaluation value calculation means 514b.

  For this purpose, the model image generation unit 512b reads the camera parameters from the object model storage unit 410b, and backprojects each candidate position onto the horizontal plane at the height of the center of gravity of the head of the stereoscopic model using the camera parameters for each arrangement. Thus, the representative position in the virtual space imitating the monitoring space of the three-dimensional model projected on the candidate position is calculated.

  The model image generation unit 512b reads the whole body model from the object model storage unit 410b, places the whole body model at a representative position in the virtual space corresponding to each candidate position, and uses the camera parameters to convert the whole body model to the captured image. Project to the coordinate system. Then, the model image generation unit 512b refers to the density distribution input from the density estimation unit 50 and totals the estimated densities in the upper third region in the projection area of the whole body model corresponding to each candidate position. The highest estimated density (except for the background class) at the candidate position is determined as the density of the candidate position.

  Further, the model image generation unit 512b reads out an object model corresponding to the density of the candidate positions from the weight coefficient storage unit 412b for each candidate position. That is, the model image generating unit 512b reads the whole body equal weighting coefficient if the density of the candidate positions is low, reads the upper body weighting coefficient if the density is medium, and calculates the head weight weighting coefficient if the density is high. read out.

In addition, the model image generation unit 512b arranges the whole body model at the representative position in the virtual space corresponding to each candidate position for each arrangement, and projects the shape of each whole body model on the coordinate system of the captured image using the camera parameters. By doing so, a model image for each arrangement is generated.
Note that the model image generation unit 512b sequentially projects from the object model arranged at the representative position far from the photographing unit 2, and overwrites the projection area to obtain a model image expressing the concealment between the object models.

  In addition, the model image generation unit 512b generates a weight image corresponding to each model image by setting a weighting factor according to the density of the candidate position in the projection area of the whole body model at each candidate position in the model image. That is, in the projection area at the candidate position where the density in the weighted image is low, the pixel in the upper １ ／ area is 0.333, the pixel in the middle ３ area is 0.333, and the lower ３. 0.333 is set for each pixel in the area. In the projection area at the candidate position where the density in the weighted image is medium density, 0.500 for the pixel in the upper 1/3 area, 0.400 for the pixel in the middle 1/3 area, and the lower 1/3 area Is set to 0.100. In the projection area of the candidate position where the density in the weighted image is high, 0.700 for the pixel in the upper 1/3 area, 0.200 for the pixel in the middle 1/3 area, and the lower 1/3 area Is set to 0.100.

  Then, the model image generation unit 512b associates the arrangement, the model image, and the weight image for each arrangement, and outputs them to the evaluation value calculation unit 514b.

  FIG. 11 is a diagram schematically illustrating a model image 770 and a weight image 771 generated by the model image generation unit 512b with respect to the density distribution and arrangement illustrated in FIG. In the weight image 771, the value of the weight coefficient is shown by one significant digit for convenience of space.

  The evaluation value calculation unit 514b obtains a similarity to the captured image of the object model for each portion constituting the object for each of the plurality of model images input from the model image generation unit 512b, and the density at the candidate position is high. An evaluation value is calculated by performing weighting with a biased number of parts as small as possible and summing up the similarities, and the evaluation value for each arrangement is output to the optimum arrangement determining means 516b.

  Specifically, the evaluation value calculation unit 514b calculates a weighted similarity between each model image and the photographed image according to a weight image corresponding to the model image.

  The weighted similarity can be the weighted similarity of the edge between the model image and the captured image. The evaluation value calculation means 514b extracts an edge from each model image and each captured image, and for each model image, for each pixel from which a valid edge is extracted from the model image, the edge of the corresponding captured image pixel. The absolute value of the difference is calculated and weighted with the weighting coefficient set for the pixel of the weighted image and summed, and the value obtained by dividing the sum by the number of pixels from which the edge is extracted from the model image is inverted. The weighted similarity of the model image is calculated.

  Alternatively, the evaluation value calculation unit 514b generates an edge image from each model image and the captured image, and for each model image, chamfer matching between the edge image generated from the captured image and the edge image generated from the model image is performed. And a value obtained by inverting the sign of the chamfer distance obtained by weighting the distance for each pixel calculated in the process according to the weighted image may be calculated as the weighted similarity of the model image. .

  The optimum arrangement determining unit 516b refers to the evaluation value for each arrangement input from the evaluation value calculating unit 514b, determines the candidate position in the arrangement having the maximum evaluation value as the object position, and uses the information on the determined object position as the object Output to the position output means 31. That is, the optimum arrangement determining unit 516b determines each candidate position included in the arrangement where the maximum similarity is calculated as the position of each person photographed in the photographed image.

<Operation of Image Monitoring Device 1 according to Second Embodiment>
Hereinafter, the operation of the image monitoring apparatus 1 according to the second embodiment will be described with reference to FIGS. 6, 12, and 13.

  When the image monitoring apparatus 1 starts operation, the imaging unit 2 sequentially transmits captured images, and the image processing unit 5 operates according to the flowchart of FIG. 6 every time it receives a captured image, as in the first embodiment. repeat.

  The communication unit 3 operates as the image acquisition unit 30, receives a captured image, and outputs it to the image processing unit 5 (step S1). The image processing unit 5 to which the photographed image is input operates as the density estimating unit 50, reads the density estimator from the density estimator storage unit 40 of the storage unit 4, and scans the photographed image with the density estimator, thereby density distribution. Is estimated (step S2).

  Next, the image processing unit 5 operates as the object position determination unit 51. The object position determination unit 51 receives the captured image from the image acquisition unit 30 and the density distribution from the density estimation unit 50, and the density distribution is not a background class. It is confirmed whether or not the estimated density is included (step S3).

  When the estimated density other than the background class is included (YES in step S3), the object position determination unit 51 performs a process of determining the position of each object from the captured image (step S4), and only the background class In the case of (NO in step S3), the processes in steps S4 and S5 are omitted.

  The object position determination process in step S4 will be described with reference to the flowcharts of FIGS. The single feature storage unit 41 operates as the object model storage unit 410b and the weight coefficient storage unit 412b, and the object position determination unit 51 operates as the arrangement generation unit 510b, the model image generation unit 512b, the evaluation value calculation unit 514b, and the optimum arrangement determination unit 516b. In operation, the object position determination process is executed.

  The placement generation unit 510b sequentially sets the number of placements within a range from 1 to the upper limit number (step S200), and controls the loop processing of steps S200 to S214.

  The arrangement generation unit 510b prepares a variable T for counting the number of iterations, initializes T to 0 (step S201), and starts the iterative process of steps S202 to S213.

  Next, the arrangement generation unit 510b sets the same number of candidate positions as the number of arrangements set in step S200 in an area where the estimated density is low density, medium density, or high density in the density distribution input from the density estimation unit 50. By setting at random, a T-th arrangement in the arrangement number is generated and output to the model image generation unit 512b (step S202).

  The model image generation unit 512b reads the camera parameters from the object model storage unit 410b, and converts each candidate position included in the arrangement generated in step S202 into three-dimensional coordinates in the virtual space using the camera parameters (step S203). .

  Next, the model image generation unit 512b prepares and initializes a model image and a weight image having the same size as the captured image, calculates the distance to the imaging unit 2 of the three-dimensional coordinates of each candidate position, and the distance is The processing target is set in order from a far candidate position (step S204), and the loop processing of steps S204 to S208 is executed.

  Subsequently, the model image generating unit 512b specifies the density of the candidate position to be processed with reference to the density distribution (step S205). The model image generation unit 512b reads out the whole body model from the object model storage unit 410b, arranges the whole body model at the three-dimensional coordinates of the candidate position, and projects the arranged whole body model on the coordinate system of the captured image using the camera parameters. Then, the model image generating unit 512b specifies the highest estimated density as the density of the candidate position in the upper third region of the projection area.

  Subsequently, the model image generating unit 512b reads out the weighting factor corresponding to the density specified in step S205 from the weighting factor storage unit 412b (step S206), and projects the whole body model and the weighting factor (step S207). That is, the model image generating unit 512b first projects the whole body model arranged in step S205 on the model image by overwriting using the camera parameters. At this time, the model image generating means 512b records the projected area of the object model. Further, the model image generation means 512b sets the read weighting factor in each part of the whole body model, and uses the camera parameter to project the whole body model with the weighting factor overwritten on the weight image.

  Then, the model image generating unit 512b checks whether or not all candidate positions included in the Tth arrangement in the current arrangement number have been processed (step S208), and if there is an unprocessed candidate position ( NO in step S208), the process returns to step S204 to process the next candidate position.

  On the other hand, when all candidate positions have been processed (YES in step S208), the model image and the weight image for the Tth arrangement in the current arrangement number are completed. The model image generating unit 512b that has completed the model image outputs the model image and the weight image to the evaluation value calculating unit 514b.

  The evaluation value calculation means 514b to which the model image and the weight image are input calculates the weighted similarity between the model image and the photographed image weighted according to the weight image as the evaluation value for the Tth arrangement in the current arrangement number ( Step S210). In other words, the evaluation value calculation unit 514b generates edge images from the model image and the photographed image input from the model image generation unit 512b, and weights the similarity of each of the edge images with the weight coefficient of the pixel. To calculate the total weighted similarity.

When the evaluation value for the Tth arrangement in the current arrangement number is calculated, the evaluation value calculation unit 514b records the arrangement and the evaluation value in association with each other, and the arrangement generation unit 510b increases the number of iterations T by one. (Step S212), and compared with the specified number of times T _MAX (step S213). If T is less than T _MAX (NO in step S213), the process returns to step S202 to continue the iterative process for the current number of arrangements. .

When the number of iterations T has reached the specified number of times T _MAX (YES in step S213), the arrangement generation unit 510b ends the iterative process for the current arrangement number and confirms whether all the arrangement numbers have been set. (Step S214). If there is an unset number of arrangements (NO in step S214), the process returns to step S200 to perform processing for the next number of arrangements.

  On the other hand, when all the arrangement numbers have been set (YES in step S214), evaluation value calculation means 514b inputs the arrangement and evaluation values recorded in step S212 to optimum arrangement determination means 516b, and optimum arrangement determination means 516b specifies an arrangement having the maximum evaluation value among them (step S215), and determines that the arrangement is information indicating the position of each person photographed in the photographed image.

  The description will be continued with reference to FIG. The object position determination unit 51 outputs the information on the object position determined in step S4 to the communication unit 3 (step S5), and the communication unit 3 operates as the object position output unit 31 to display the object position information on the display unit 6. Send.

  When the above process is completed, the process returns to step S1, and the process for the next captured image is performed.

[Third embodiment]
Hereinafter, as a preferred embodiment of the present invention different from the first and second embodiments, an example of an object detection device that detects an individual person using a discriminator that has learned an image feature of a single person is included. An example of the image monitoring apparatus 1 will be described. The image monitoring apparatus 1 according to this embodiment includes an example in which the object detection apparatus switches the discriminator depending on the human density.

  In the image monitoring apparatus according to the third embodiment, the details of the single feature stored in the single feature storage unit 41 and the details of the processing performed by the object position determination unit 51 are the image monitoring according to the first and second embodiments. Unlike the apparatus, the general configuration, the general function, and a part of the operation are common. Therefore, with respect to a schematic configuration, a general function, and a part of the operation, refer to the block diagram of FIG. 1, the functional block diagram of FIG. 2, and the flowchart of FIG. 6 referred to in the first and second embodiments, respectively. I will explain.

<Configuration of Image Monitoring Device 1 according to Third Embodiment>
The schematic configuration of the image monitoring apparatus 1 according to the third embodiment will be described with reference to the block diagram of FIG.
As in the first and second embodiments, the image monitoring apparatus 1 captures the monitoring space at predetermined time intervals and outputs a captured image, and receives information on the object position and displays the information. A display unit 6 and an image processing unit 5 that acquires a captured image, detects an individual person (object) from the captured image, generates information about the position (object position) of the detected object, and outputs the information. Connected to the communication unit 3, which is a communication circuit that intervenes input / output of image and object position information, etc., and a storage unit 4 that stores programs and various data and inputs / outputs them is connected to the image processing unit 5 Being done.

<Functions of Image Monitoring Device 1 according to Third Embodiment>
The function of the image monitoring apparatus 1 according to the third embodiment will be described with reference to the functional block diagrams of FIGS.

  As in the first and second embodiments, the communication unit 3 acquires a captured image from the imaging unit 2 and outputs the acquired image to the density estimation unit 50 and the object position determination unit 51, and the object position determination unit 51. The function as the object position output means 31 etc. which outputs the information of the object position input from the to the display part 6 is included.

  As in the first and second embodiments, the storage unit 4 stores a density estimator that learns the image features of each density image obtained by photographing a space where an object exists at the density for each predetermined density. The unit includes a function as a density estimator storage unit 40 and a unit feature storage unit 41 that stores an image feature (unit unit feature) of a single object generated by learning in advance. The single feature that can be evaluated can be evaluated with an emphasis on the image feature of a small part of the parts constituting the object as the density is high.

  Further, as in the first and second embodiments, the image processing unit 5 estimates the density distribution of the object photographed in the photographed image by scanning the photographed image with the density estimator, and calculates the estimated density distribution. The density estimation means 50 output to the object position determination means 51, and candidate positions where individual objects can exist in the captured image are set, and the degree to which the image feature of a single object appears in the captured image at the candidate position. Including a function as an object position determination unit 51 that calculates an evaluation value to be expressed, determines a candidate position whose evaluation value is equal to or greater than a predetermined value as an object position, and outputs information on the object position to the object position output unit 31; The object position determination unit 51 uses the single feature according to the density of the candidate positions, and calculates the evaluation value by placing importance on the image features of a smaller part of the parts constituting the object as the density at the candidate position is higher. To.

  However, as described above, the details of the processing performed by the object position determination unit 51 according to the third embodiment and the details of the single feature stored in the single feature storage unit 41 relate to the first and second embodiments. Different from the image monitoring apparatus 1. These points will be described with reference to the functional block diagram of FIG.

  The single feature storage unit 41 according to the third embodiment functions as a single discriminator storage unit 411c that stores in advance a discriminator (single discriminator) that has learned an image feature of a single person (object). Is stored as a single feature.

  FIG. 15 is a diagram schematically showing the single feature stored in the single feature storage unit 41 according to the third embodiment, that is, the single classifier information stored in the single classifier storage unit 411c. .

When a feature amount of an image is input, the single classifier calculates and outputs an evaluation value (discrimination score) indicating the likelihood that the image is an image of a single person (single image). It is represented by parameters such as a coefficient applied to the evaluation value calculation function and a threshold value applied to the identification score.
The single discriminator can be a discriminator that is learned by applying the linear SVM method to the feature amount of a learning image including a large number of single images and a large number of unmanned images in which only a person is captured.
When linear SVM is used as the learning algorithm, the coefficient of the evaluation value calculation function is a weight vector. This weight vector is a weight for each element of the feature quantity, and the value of the inner product of the feature quantity of the input image and the weight vector represents the identification score. In learning, when the inner product of the weight vector and the feature quantity is greater than 0, the person is identified. Therefore, the threshold for identifying whether or not the input image is a single image is 0 in principle, and the threshold can usually be set to 0. However, the threshold value may be set to a value smaller than 0 in order to reduce errors in identifying a single image as not being a single image.
Note that the feature amount of the learning image is a HOG (Histograms of Oriented Gradients) feature amount.

  The single discriminator stored in the single discriminator storage unit 411c is a discriminator that learns image features of a smaller part of the parts constituting a single object as the density increases. The single discriminator storage unit 411c is a single discriminator 800 that learns the image features of the whole body of a single person in association with a value that represents a low density class, and the upper part 2 of the single person in association with a value that represents a medium density class. A single discriminator 801 that has learned the image feature of / 3 and a single discriminator 802 that has learned the image feature of the upper third of a single person in association with the value representing the high-density class are stored. Hereinafter, the single classifiers 800, 801, and 802 are referred to as a whole body classifier, an upper body classifier, and a head vicinity classifier, respectively.

  The whole body discriminator 800 is a single discriminator learned using a single image obtained by photographing the whole body of a single person, and the upper body discriminator 801 is a single image obtained by photographing the upper 2/3 of a single person (a whole body of a person). Is a single discriminator that has been learned using an image obtained by cutting out the upper part 2/3 of the single image taken by the user, and the head vicinity discriminator 802 is a single image (upper third of a single person taken) This is a single discriminator that has been learned using an image obtained by cutting out the upper third of a single image obtained by photographing the whole body of a person.

  As described above, the single classifier storage unit 411c associates the whole body classifier 800 with the low density class, the upper body classifier 801 with the medium density class, and the head vicinity classifier with the high density class. 802 is stored.

  The candidate position setting unit 511c sets a plurality of candidate positions at predetermined intervals in the captured image, and outputs the set candidate positions to the evaluation value calculation unit 514c. Specifically, the predetermined interval is one pixel, and the candidate position setting unit 511c sequentially sets the position of each pixel of the captured image as a candidate position. The candidate position represents the human head center of gravity.

  For each candidate position input from the candidate position setting means 511c, the evaluation value calculating means 514c learns the image features of a smaller part of the parts constituting a single object as the density of the candidate positions is higher. The image feature of the photographed image at the candidate position is input to the device, the evaluation value is calculated, and the calculated evaluation value and accompanying information are output to the position determining means 517c.

  For this purpose, the evaluation value calculation means 514c sets a window defined in the shape of the upper third of a single person at each candidate position and refers to the density distribution input from the density estimation means 50, Aggregate the estimated density. The window is an identification extraction window described later. Then, the evaluation value calculating unit 514c determines the highest estimated density at each candidate position as the density of the candidate position.

  In addition, the evaluation value calculation unit 514c sets an extraction window for identification corresponding to the density of the candidate positions at each candidate position, and a feature quantity for identification (identification feature quantity) from the captured image in the identification extraction window. To extract. The identification extraction window has the shape of a single image (solid-line rectangle shown in FIG. 15) used for learning of a single classifier corresponding to each density, and is enlarged / reduced by a plurality of predetermined magnifications. The window. That is, the identification extraction window is a window defined in the shape of the whole body of a single person if the density of candidate positions is low, and is defined in the shape of the upper 2/3 of the single person if the density is medium. If it is high density, it is a window defined in the shape of the upper third of a single person.

  FIG. 16 is a diagram schematically illustrating an extraction window for identification set by the evaluation value calculation unit 514c at each candidate position illustrated in FIG. 5 when the density distribution illustrated in FIG. 5 is obtained.

  In addition, the evaluation value calculation unit 514c reads, from the single classifier storage unit 411c, a single classifier corresponding to the density of the candidate positions for each candidate position. That is, the evaluation value calculation unit 514c reads the whole body classifier if the density of the candidate positions is low, reads the upper body classifier if the density is medium, and reads the head vicinity classifier if the density is high. Then, the evaluation value calculation unit 514c inputs, for each candidate position, the identification feature amount extracted from the candidate position to the read single classifier, and acquires the identification score that is the output value as the evaluation value of the candidate position. To do.

  Then, the evaluation value calculating unit 514c outputs, for each candidate position, information that associates the candidate position, the density, the identification score, the threshold value of the used single classifier, and the used extraction window for identification, to the position determining unit 517c.

  The position determining unit 517c refers to the information input from the evaluation value calculating unit 514c, and determines a candidate position where an evaluation value satisfying a predetermined criterion is calculated as the position of the object.

  Specifically, the position determining unit 517c extracts candidate positions having an identification score equal to or greater than a corresponding threshold value, and selects a plurality of candidate positions that have the same density and are close to each other among the extracted candidate positions. The candidate positions are determined as the positions where the person is photographed.

  The process of grouping the candidate positions is performed in order to cope with the fact that a high identification score is calculated for the same person in the vicinity in addition to the position where the person is actually photographed. Specifically, for example, for each density, the position determination unit 517c sequentially sets candidate positions for which an identification score equal to or greater than the threshold is calculated as the attention position in descending order of the identification score and has a lower identification score than the attention position. Is set to the comparison position. Then, the position determining means 517c deletes information on the comparison position where the overlap between the identification extraction window set at the comparison position and the identification extraction window set at the target position is larger than a predetermined ratio among the comparison positions. By doing so, a plurality of candidate positions are combined into one.

  Then, the position determination unit 517c outputs the position where the person is photographed and the determined candidate position to the object position output unit 31 as information on the object position.

<Operation of Image Monitoring Device 1 according to Third Embodiment>
Hereinafter, the operation of the image monitoring apparatus 1 according to the third embodiment will be described with reference to FIGS. 6 and 17.

  When the image monitoring apparatus 1 starts its operation, as in the first and second embodiments, the imaging unit 2 sequentially transmits the captured images, and the image processing unit 5 receives the captured image in the flowchart of FIG. Repeat the action.

  The communication unit 3 operates as the image acquisition unit 30, receives a captured image, and outputs it to the image processing unit 5 (step S1). The image processing unit 5 to which the photographed image is input operates as the density estimating unit 50, reads the density estimator from the density estimator storage unit 40 of the storage unit 4, and scans the photographed image with the density estimator, thereby density distribution. Is estimated (step S2).

  Next, the image processing unit 5 operates as the object position determination unit 51. The object position determination unit 51 receives the captured image from the image acquisition unit 30 and the density distribution from the density estimation unit 50, and the density distribution is not a background class. It is confirmed whether or not the estimated density is included (step S3).

  When the estimated density other than the background class is included (YES in step S3), the object position determination unit 51 performs a process of determining the position of each object from the captured image (step S4), and only the background class In the case of (NO in step S3), the process of step S4 is omitted.

  The object position determination process in step S4 will be described with reference to the flowchart in FIG. The single feature storage unit 41 operates as the single classifier storage unit 411c, and the object position determination unit 51 operates as the candidate position setting unit 511c, the evaluation value calculation unit 514c, and the position determination unit 517c, and the object position determination process is executed. The

  The candidate position setting unit 511c sequentially sets the position of each pixel in the captured image as a candidate position and inputs it to the evaluation value calculation unit 514c (step S300), and controls the loop processing of steps S300 to S304.

  The evaluation value calculation means 514c that has received the candidate position specifies the density of the candidate position with reference to the density distribution (step S301). The evaluation value calculation means 514c sets a window defined in the shape of the upper third of a single person at the candidate position, and specifies the most estimated density as the candidate position density in the window.

  The evaluation value calculation means 514c that specifies the density reads the single classifier corresponding to the density from the single classifier storage means 411c, sets an extraction window for identification corresponding to the density, and takes a captured image in the extraction window for identification. The feature quantity for identification is extracted from (step S302), and the extracted feature quantity for identification is input to a single classifier corresponding to the density to calculate the identification score (evaluation value) (step S303).

  Then, the evaluation value calculation unit 514c records the candidate position, the extraction window for identification, the density, and the evaluation value in association with each other and confirms whether or not the positions of all the pixels of the photographed image have been set as the candidate positions. If there is an unset pixel (NO in step S304), the process returns to step S300 to process the position of the next pixel.

  On the other hand, when the positions of all the pixels have been set as the candidate positions (YES in step S304), the position determining unit 517c sets the candidate position, the identification extraction window, the density, and the evaluation value recorded in step S304. The pair whose evaluation value is less than the threshold value is deleted from the list (step S305), and for each remaining set, the identification extraction windows overlap each other by a larger amount than the predetermined ratio for each density. The groups that belong to the same person are grouped into one group (step S306). Then, the position determining unit 517c determines each set of candidate positions after the grouping as the position (object position) of each person photographed in the photographed image.

  The description will be continued with reference to FIG. The object position determination unit 51 outputs the information on the object position determined in step S4 to the communication unit 3 (step S5), and the communication unit 3 operates as the object position output unit 31 to display the object position information on the display unit 6. Send.

  When the above process is completed, the process returns to step S1, and the process for the next captured image is performed.

[Fourth embodiment]
Hereinafter, as a preferred embodiment of the present invention that is different from the first, second, and third embodiments, an example of an object detection device that changes a weight for a partial evaluation value output by a discriminator depending on a human density is included. An example of the image monitoring apparatus 1 will be described.

  In the image monitoring apparatus according to the fourth embodiment, the details of the single features stored in the single feature storage means 41 and the details of the processing performed by the object position determination means 51 are the same as those in the first, second and third embodiments. Unlike the image monitoring apparatus, the general configuration, the general function, and a part of the operation are common. Therefore, for the schematic configuration, a part of the schematic function and the operation, the block diagram of FIG. 1, the functional block diagram of FIG. 2, and the flowchart of FIG. 6 referred to in the first, second and third embodiments, respectively. The description will be given with reference again.

<Configuration of Image Monitoring Device 1 according to Fourth Embodiment>
A schematic configuration of the image monitoring apparatus 1 according to the fourth embodiment will be described with reference to the block diagram of FIG.
As in the first, second, and third embodiments, the image monitoring apparatus 1 captures the monitoring space every predetermined time and outputs a captured image, and receives the information on the object position and receives the information. A display unit 6 that displays the image, an image processing unit 5 that acquires a captured image, detects an individual person (object) from the captured image, and generates and outputs information on the position (object position) of the detected object Is connected to the communication unit 3 through which input / output of captured image and object position information and the like, and a storage unit 4 for storing programs and various data and inputting / outputting these are connected to the image processing unit 5. It becomes.

<Functions of the image monitoring apparatus 1 according to the fourth embodiment>
The function of the image monitoring apparatus 1 according to the fourth embodiment will be described with reference to the functional block diagrams of FIGS.

  As in the first, second, and third embodiments, the communication unit 3 acquires a captured image from the imaging unit 2 and outputs the captured image to the density estimation unit 50 and the object position determination unit 51, and the object position It includes a function as an object position output means 31 that outputs information on the object position input from the determination means 51 to the display unit 6.

  Further, as in the first, second, and third embodiments, the storage unit 4 is a density estimator that learns image features of each density image obtained by photographing a space where an object exists at the predetermined density for each predetermined density. Including a function as a density estimator storage means 40 that stores information, a single feature storage means 41 that stores image features (single features) of a single object generated by pre-learning, and the like. The unit features stored in the means 41 can be evaluated with an emphasis on the image features of a small part of the parts constituting the object as the density is higher.

  Further, the image processing unit 5 estimates and estimates the density distribution of the object photographed in the photographed image by scanning the photographed image with the density estimator as in the first, second, and third embodiments. A density estimation unit 50 that outputs the density distribution to the object position determination unit 51, and candidate positions where individual objects can exist in the captured image are set, and an image feature of a single object appears in the captured image at the candidate position. A function as an object position determination unit 51 or the like that calculates an evaluation value that represents the degree of movement, determines a candidate position whose evaluation value is equal to or greater than a predetermined value as an object position, and outputs information on the object position to the object position output unit 31 The object position determination unit 51 uses the single feature according to the density of the candidate positions, so that the higher the density at the candidate position, the more important the image features of the parts constituting the object are evaluated. It is calculated.

  However, as described above, the details of the processing performed by the object position determination unit 51 according to the fourth embodiment and the details of the single feature stored in the single feature storage unit 41 are the first, second, and third implementations. Different from the image monitoring apparatus 1 according to the embodiment. These points will be described with reference to the functional block diagram of FIG.

  The unit feature storage unit 41 according to the fourth embodiment includes a unit classifier storage unit 411d that stores in advance a classifier (unit classifier) that has learned the image features of a single person (object), and the evaluation value calculation. It functions as a weight coefficient storage means 412d that stores the weight coefficient to be used in advance, and stores information on the single discriminator and information on the weight coefficient as single features.

  FIG. 19 shows the unit features stored in the unit feature storage unit 41 according to the fourth embodiment, that is, the unit classifier information stored in the unit classifier storage unit 411d and the weight coefficient storage unit 412d. It is the figure which represented typically the information of the weighting coefficient.

  As described in the third embodiment, the single discriminator calculates and outputs an evaluation value (discrimination score) indicating the likelihood that the image is a single image when the image feature amount is input. Learning by applying the linear SVM method to the feature quantity of the learning image consisting of a large number of single images and a large number of unmanned images. Discriminator. The feature amount of the learning image can be a HOG feature amount.

  The single discriminator stored in the single discriminator storage unit 411d is a discriminator that has learned the image features of a plurality of portions that constitute a single object. Specifically, the single discriminator storage unit 411d stores three single discriminators that have learned image features of portions that are different from each other. That is, the single classifier storage unit 411d includes a single classifier 850 that has learned the image feature of the upper third of the person, a single classifier 851 that has learned the image feature of the middle third of the person, and the lower part 1 of the person. A single discriminator 852 that has learned the image feature of / 3 is stored. Hereinafter, the single discriminator 850 that identifies the upper 1/3 is the upper discriminator, the single discriminator 851 that identifies the middle 1/3 is the middle discriminator, and the single discriminator 852 that identifies the lower 1/3 is the lower discriminator. Called.

  The weighting factor is set such that the higher the density is, the more the weight is concentrated on the smaller part of the part constituting the single object. The weighting coefficient storage unit 412d associates the value representing the low density class with “weighting coefficient 0.333 applied to the upper 1/3” “weighting coefficient 0.333 applied to the middle 1/3” “lower 1/3”. “Weighting coefficient 0.333” applied to the middle density class, “weighting coefficient 0.500 applied to the upper third”, “weighting coefficient 0.400 applied to the middle third”, “lower” “Weighting coefficient 0.100 applied to 1/3”, “weighting coefficient 0.700 applied to upper 1/3” “weighting coefficient 0.200 applied to middle 1/3” in association with a value representing a high-density class "" Weighting coefficient 0.100 applied to lower 1/3 "is stored. Hereinafter, a weight coefficient 860 for low density that is equal to the whole body is represented as a weight coefficient for whole body, a weight coefficient 861 for medium density that emphasizes the upper body, an overweight weight coefficient for upper body, and a weight coefficient 862 for high density that emphasizes the vicinity of the head. This is referred to as the near-head weighting coefficient.

  As described above, the single discriminator storage unit 411d stores the upper discriminator 850, the middle discriminator 851 and the lower discriminator 852 as single discriminator information, and the weight coefficient storage unit 412d is associated with the low density class. The whole body equal weighting coefficient 860, the upper body weighting coefficient 861 associated with the medium density class, and the head vicinity weighting coefficient 862 associated with the high density class are stored.

  The candidate position setting unit 511d sets a plurality of candidate positions at predetermined intervals in the captured image, and outputs the set candidate positions to the evaluation value calculation unit 514d. Specifically, the predetermined interval is one pixel, and the candidate position setting unit 511d sequentially sets the position of each pixel of the captured image as the candidate position. The candidate position represents the human head center of gravity.

  The evaluation value calculation unit 514d uses, for each candidate position input from the candidate position setting unit 511d, an image feature of the captured image at the candidate position in a single discriminator that has learned the image features of a plurality of parts constituting a single object. To obtain partial evaluation values of a plurality of parts, and by calculating the evaluation value by summing the partial evaluation values by weighting the parts with a smaller weight as the density at the candidate position is higher, the calculated evaluation value and The accompanying information is output to the position determining means 517d.

  For this purpose, the evaluation value calculation means 514d sets an extraction window for identifying each part of the person at each candidate position, and extracts a feature quantity for identification (identification feature quantity) from the captured image in the identification extraction window. To do. The identification extraction window has a shape of a single image (solid-line rectangle shown in FIG. 19) used for learning of the single classifier of each part, and is a window whose size is enlarged or reduced at a plurality of predetermined magnifications. It is. That is, the identification extraction windows are three windows defined in the shape of the upper part 1/3, the middle part 1/3, and the lower part 1/3 of a single person.

  Further, the evaluation value calculation unit 514d reads out the unit classifiers of each part from the unit classifier storage unit 411d. That is, the evaluation value calculation unit 514d reads out the upper discriminator, the middle discriminator, and the lower discriminator. Then, for each candidate position, the evaluation value calculation unit 514d inputs, for each part, the identification feature amount extracted from the candidate position to the read single classifier and outputs the partial identification score that is the output value of the candidate position. Is obtained as a partial evaluation value of the part. That is, the evaluation value calculation unit 514d calculates a partial identification score by the upper classifier, a partial identification score by the middle classifier, and a partial identification score by the lower classifier for each candidate position.

  Further, the evaluation value calculation unit 514d refers to the density distribution input from the density estimation unit 50 and totals the estimated density in the upper third window set at each candidate position. Then, the evaluation value calculation unit 514d determines the highest estimated density at each candidate position as the density of the candidate position.

  In addition, the evaluation value calculation unit 514d reads the weighting coefficient corresponding to the density of the candidate position from the weighting coefficient storage unit 412d for each candidate position. That is, the evaluation value calculation means 514d reads the whole body equal weighting coefficient if the density of the candidate positions is low, reads the upper body weighting coefficient if it is medium density, and calculates the head weight weighting coefficient if it is high. read out. Then, the evaluation value calculation unit 514d calculates the evaluation value of the candidate position by weighting and adding the partial evaluation value of the corresponding portion with the read weighting coefficient for each candidate position.

That is, assuming that the partial discrimination score by the upper discriminator is S _U , the partial discrimination score by the middle discriminator is S _M , and the partial discrimination score by the lower discriminator is S _L , the evaluation value calculation means 514d has the candidate position of interest If the density is low, the identification score of the candidate position is calculated by the following equation.
Identification score = 0.333S _U + 0.333S _M + 0.333S _L (3)
Further, the evaluation value calculating unit 514d calculates the identification score of the candidate position according to the following equation if the density of the candidate position of interest is medium density.
Identification score = 0.500S _U + 0.400S _M + 0.100S _L (4)
In addition, the evaluation value calculation unit 514d calculates the identification score of the candidate position according to the following formula if the density of the candidate position of interest is high.
Identification score = 0.700S _U + 0.200S _M + 0.100S _L (5)

  FIG. 20 is a diagram schematically illustrating how the evaluation value calculation unit 514d calculates an identification score for each candidate position illustrated in FIG. 5 when the density distribution illustrated in FIG. 5 is obtained. The image 870 shows the relationship between each part and the weighting factor for three candidate positions having a low density among these candidate positions. An image 871 shows the relationship between each part and the weighting coefficient for three candidate positions having a medium density. The image 872 shows the relationship between each part and the weighting coefficient for two candidate positions having a high density. For reasons of space, the value of the weighting factor is indicated by one significant digit.

  Then, the evaluation value calculating unit 514d outputs information that associates the candidate position, the density, the identification score, and the used identification extraction window with each candidate position to the position determining unit 517d.

  The position determining unit 517d refers to the information input from the evaluation value calculating unit 514d and determines a candidate position where an evaluation value that satisfies a predetermined criterion is calculated as the position of the object.

  Specifically, the position determining unit 517d extracts candidate positions having an identification score of 0 or more, and among the extracted candidate positions, a plurality of candidate positions (identification extraction windows) that have the same corresponding density and are close to each other. (Candidate positions where the overlap between them is larger than a predetermined ratio) are combined into one, and the combined candidate positions are determined as positions where a person is photographed. The process of collecting candidate positions and the significance thereof are the same as the process performed by the position determining unit 517c according to the third embodiment and the significance thereof.

  Then, the position determination unit 517d outputs the position where the person is photographed and the determined candidate position to the object position output unit 31 as object position information.

<Operation of Image Monitoring Device 1 According to Fourth Embodiment>
Hereinafter, the operation of the image monitoring apparatus 1 according to the fourth embodiment will be described with reference to FIGS. 6 and 21.

  When the image monitoring apparatus 1 starts to operate, as in the first, second, and third embodiments, the imaging unit 2 sequentially transmits captured images, and the image processing unit 5 receives the captured image every time it receives the captured image. The operation according to the flowchart is repeated.

  The communication unit 3 operates as the image acquisition unit 30, receives a captured image, and outputs it to the image processing unit 5 (step S1). The image processing unit 5 to which the photographed image is input operates as the density estimating unit 50, reads the density estimator from the density estimator storage unit 40 of the storage unit 4, and scans the photographed image with the density estimator, thereby density distribution. Is estimated (step S2).

  Next, the image processing unit 5 operates as the object position determination unit 51. The object position determination unit 51 receives the captured image from the image acquisition unit 30 and the density distribution from the density estimation unit 50, and the density distribution is not a background class. It is confirmed whether or not the estimated density is included (step S3).

  When the estimated density other than the background class is included (YES in step S3), the object position determination unit 51 performs a process of determining the position of each object from the captured image (step S4), and only the background class In the case of (NO in step S3), the processes in steps S4 and S5 are omitted.

  The object position determination process in step S4 will be described with reference to the flowchart in FIG. The single feature storage unit 41 operates as the single classifier storage unit 411d and the weight coefficient storage unit 412d, and the object position determination unit 51 operates as the candidate position setting unit 511d, the evaluation value calculation unit 514d, and the position determination unit 517d. A position determination process is executed.

  The candidate position setting unit 511d sequentially sets the position of each pixel in the captured image as a candidate position and inputs it to the evaluation value calculation unit 514d (step S400), and controls the loop processing of steps S400 to S405.

  The evaluation value calculation means 514d to which the candidate position is inputted reads out the individual classifiers of each part (upper / middle / lower) from the single classifier storage means 411d, sets the extraction window for identification corresponding to each part, and sets each A feature value for identification is extracted from the captured image in the extraction window for identification (step S401), and each extracted feature value for identification is input to a corresponding unit single classifier to calculate a partial identification score (partial evaluation value). (Step S402).

  The evaluation value calculation means 514d that has calculated the partial evaluation value specifies the density at the candidate position with reference to the density distribution (step S403). The evaluation value calculation means 514d specifies the highest estimated density as the candidate position density in the upper third window set at the candidate position.

  The evaluation value calculation means 514d that has specified the density reads the weighting coefficient corresponding to the density from the weighting coefficient storage means 412d, and follows the expression according to the density among the expressions (3), (4), or (5). Then, the evaluation value of the candidate position is calculated by multiplying the read weight coefficient and the partial evaluation value (step S404).

  Then, the evaluation value calculation unit 514d records the candidate position, the extraction window for identification, the density, and the evaluation value in association with each other and confirms whether or not the positions of all the pixels of the captured image have been set as the candidate positions. If there is an unset pixel (NO in step S405), the process returns to step S400 to process the position of the next pixel.

  On the other hand, when the positions of all the pixels have been set as the candidate positions (YES in step S405), the position determining unit 517d sets the candidate position, the extraction window for identification, the density, and the evaluation value recorded in step S405. The pair whose evaluation value is less than the threshold value is deleted from among the combinations (step S406). Further, for each pair that remains without being deleted, the identification extraction windows overlap each other more than a predetermined ratio for each density. The existing groups are grouped into one group as those of the same person (step S407). Then, the position determination unit 517d determines each group of candidate positions after being collected as the position (object position) of each person photographed in the photographed image.

  The description will be continued with reference to FIG. The object position determination unit 51 outputs the information on the object position determined in step S4 to the communication unit 3 (step S5), and the communication unit 3 operates as the object position output unit 31 to display the object position information on the display unit 6. Send.

  When the above process is completed, the process returns to step S1, and the process for the next captured image is performed.

<Modification>
(1) In each of the above-described embodiments and modifications thereof, an example in which the object to be detected is a person has been shown. However, the present invention is not limited thereto, and the object to be detected is a vehicle, an animal such as a cow or a sheep, or the like. You can also.

  (2) In each of the above-described embodiments and modifications thereof, an example in which a single feature is set with a unit obtained by dividing an object by 1/3 has been shown, but the way of dividing is not limited to this. Depending on the difference in the characteristics of the detection target, the monitoring space to be photographed, the feature amount to be used, the type of evaluation value, etc., the single feature can be divided by a different ratio suitable for each. In addition, single features may be set by overlapping between densities.

  (3) The values of the weighting coefficients shown in the second and fourth embodiments and the modifications thereof are merely examples, such as the characteristics of the detection target, the monitoring space to be photographed, the feature quantity to be adopted, the type of evaluation value, etc. Depending on the difference, it can be set to a different value suitable for each.

(4) In each of the above-described embodiments and modifications thereof, the density estimator learned by the multi-class SVM method has been exemplified, but instead of the multi-class SVM method, a decision tree type random forest method, a multi-class Various density estimators such as a density estimator learned by the AdaBoost method or the multi-class logistic regression method can be used.
Alternatively, a density estimator using a discriminating CNN (Convolutional Neural Network) may be used.

(5) In each of the above-described embodiments and modifications thereof, the class of density other than the background estimated by the density estimator is set to three classes, but the class may be divided more finely.
In that case, instead of single-stage features in three levels (whole body, upper body, and the vicinity of the head), single-stage features of finer levels corresponding to classification are used, and classes and single-unit features are associated and stored in the single-unit feature storage unit 41. I can keep it. Alternatively, the class and the three-stage single feature can be associated in a many-to-one manner and stored in the single feature storage unit 41.

(6) In each of the above-described embodiments and modifications thereof, a density estimator that classifies into multiple classes is illustrated, but instead of this, a regression type density estimation that regresses a density value (estimated density) from a feature quantity It can also be a container. That is, a density estimator that has learned the parameters of the regression function for obtaining the estimated density from the features by ridge regression method, support vector regression method, regression tree-type random forest method or Gaussian Process Regression, etc. can do.
Alternatively, a density estimator using a regression type CNN may be used.
In these cases, the estimated density value range output as a continuous value instead of the density class value is stored in the single feature storage unit 41 in association with the single feature.

  (7) In the second and fourth embodiments and the modifications thereof, the example in which the weighting coefficient of each part is set to a constant value has been shown, but the weighting coefficient of each part may be a function. In that case, for example, the weight coefficient storage means 412b and 412d store a function that inputs the position of the pixel in each part and outputs a larger weight coefficient as the height in the part is higher, and the evaluation value calculation means 514b, 514d inputs the position of the pixel in each part to the said function, and performs weighting for every pixel.

  (8) In each of the above embodiments and the modifications thereof, the GLCM feature is exemplified as the feature amount learned by the density estimator and the estimation feature amount. However, instead of the GLCM feature, the local binary pattern (Local Binary Pattern (LBP) feature value, Haar-like feature value, HOG feature value, luminance pattern, and other various feature values, or a combination of GLCM features and a plurality of them You can also

  (9) In each of the above-described embodiments and modifications thereof, an example is shown in which the density estimation unit 50 and the object position determination unit 51 scan and process at intervals of one pixel. It is also possible to carry out at intervals.

(10) In each of the above-described embodiments and modifications thereof, an example has been shown in which candidate positions are selected and set from regions with low, medium, or high estimated density. Each of the means 510b, the candidate position setting means 511c, and the candidate position setting means 511d can set the candidate position only within the change region. In that case, the storage unit 4 includes background image storage means (not shown) for storing the background image of the monitoring space, and the image processing unit 5 performs a difference process between the captured image and the background image, and the difference value is a predetermined difference. A change area extraction unit that extracts a group of pixels that are equal to or greater than a threshold value as a change area, or performs a correlation process between a captured image and a background image to extract a group of pixels that have a correlation value equal to or less than a predetermined correlation threshold as a change area (Not shown), each of the arrangement generation unit 510a, the arrangement generation unit 510b, the candidate position setting unit 511c, and the candidate position setting unit 511d sets a candidate position with reference to the change area extracted by the change area extraction unit. .
In the case where the area where the candidate position is set is limited, each of the arrangement generation unit 510a and the arrangement generation unit 510b can change the upper limit number of arrangements according to the size of the limited area.
By limiting the region where the candidate positions are set, accidental similarity between the captured image and the model image or accidental calculation of a high identification score for the background can be prevented, and erroneous detection of the object position can be reduced.

  (11) In the first and second embodiments and the variations thereof, an example in which the arrangement generation unit 510a and the arrangement generation unit 510b randomly generate an arrangement for each iteration has been described. The arrangement may be generated by updating the candidate position slightly shifted from the previous candidate position to the MCMC (Markov chain by referring to the similarity to the previous arrangement after the second iteration. The arrangement may be generated by sequentially searching for candidate positions by the Monte Carlo method or by sequentially improving candidate positions by the hill-climbing method.

  (12) In each of the above-described embodiments and modifications thereof, a projection area of a model defined in the shape of the upper third of the person or a window defined in the shape is set at the candidate position of interest. The example in which the estimated density at the candidate position is determined by aggregating the estimated density in the area has been shown. However, in order to reduce the processing amount, the pixel at the candidate position and the 8 neighboring areas at the candidate position are replaced with the area. Alternatively, it may be a small area such as the 16 neighborhood area. Alternatively, in order to improve the accuracy, a projection area of a model defined in the shape of the upper 2/3 of a single person with the candidate position as a representative position instead of the area, a window defined in the shape, or a candidate position It can also be a projection area of a model defined in the shape of the whole body of a single person as a representative position or a large area such as a window defined in the shape.

  (13) The threshold value to be compared with the identification score shown in the third embodiment and the modifications thereof may be different for each single classifier.

  (14) In the third and fourth embodiments and the modifications thereof, the single classifier learned by the linear SVM method is exemplified, but conventionally known such as the AdaBoost method instead of the linear SVM method. It is also possible to use a single classifier that has been learned using various learning methods. In addition, a pattern matching device can be used in place of the discriminator. In this case, the discriminant score is an inner product of the average pattern of the feature amount extracted from the human learning image and the feature amount of the input image. The function can be a function having the score as an output value and the feature quantity of the input image as an input value. Further, an identification type CNN may be used as a single classifier.

  (15) In the third and fourth embodiments and their modifications, the HOG feature value is exemplified as the feature value learned by the single discriminator. However, these are local binary pattern features instead of the HOG feature value. Various feature amounts such as an amount, a Haar-like feature amount, and a luminance pattern can be used, or a HOG feature amount and a feature amount obtained by combining a plurality of these can be used.

  According to each of the above embodiments and the modifications thereof, the object detection device uses a single feature (image feature of a single object) suitable for the concealment state that can occur in the object due to the density according to the density for each candidate position. Since the position of each object is determined, the trade-off between the change in the concealment state of the object due to the change in the congestion state and the fluctuation in detection accuracy due to the number of parts used for the detection of each object is eliminated. It is possible to detect an object with high accuracy.

  The object detection apparatus according to the first embodiment and the modification thereof uses an object model representing a single feature, eliminates the trade-off by switching the object model according to the density for each candidate position, Enables high object detection.

  In addition, the object detection apparatus according to the second embodiment and the modification thereof uses an object model representing a single feature and a weighting factor when evaluating the similarity of the object model to a captured image, and uses the weight coefficient when evaluating the density for each candidate position. By switching the weighting factor accordingly, the trade-off is eliminated, and highly accurate object detection is possible.

  In addition, the object detection apparatus according to the third embodiment and the modification thereof uses a discriminator that has learned a single feature, and eliminates the trade-off by switching the discriminator according to the density for each candidate position, Enables highly accurate object detection.

  In addition, the object detection device according to the fourth embodiment and the modification thereof includes a classifier that learns a single feature for each part, and uses a weighting factor for summing the partial evaluation values for each part by the classifier. By switching the weighting coefficient according to the density for each position, the trade-off is eliminated, and highly accurate object detection is possible.

DESCRIPTION OF SYMBOLS 1 Image monitoring apparatus, 2 imaging | photography part, 3 communication part, 4 memory | storage part, 5 image processing part, 6 display part, 30 image acquisition means, 31 object position output means, 40 density estimator memory | storage means, 41 single-piece | unit characteristic memory | storage means, 410a, 410b Object model storage means, 411c, 411d Single classifier storage means, 412a, 412b, 412d Weight coefficient storage means, 50 Density estimation means, 51 Object position determination means, 510a, 510b Arrangement generation means, 511c, 511d Candidate positions Setting means 512a, 512b Model image generating means 514a, 514b, 514c, 514d Evaluation value calculating means 516a, 516b Optimal arrangement determining means 517c, 517d Position determining means

Claims (8)

An object detection device for detecting individual objects from a captured image in which a space in which congestion due to a predetermined object may occur is captured,
Estimate the distribution of the density of the object imaged in the captured image using a density estimator that learns the image characteristics of each density image captured in the space where the object exists at the density for each predetermined density Density estimation means to perform,
A candidate position where each of the objects can exist in the photographed image is set, and an evaluation value representing a degree of appearance of an image feature of the single object in the photographed image at the candidate position is calculated, and the evaluation value Object position determination means for determining a candidate position whose value is equal to or greater than a predetermined value as the position of the object;
With
The object position determination means calculates the evaluation value by changing a portion to be emphasized among portions constituting a single object according to the density at the candidate position.   The said object position determination means calculates the said evaluation value by attaching importance to the image feature of the few parts which comprise the said single object, so that the said density in the said candidate position is high. Object detection device.   The object position determination means indicates the degree that the image feature of a smaller part of the parts constituting the single object appears at the candidate position of the photographed image as the density at the candidate position is higher. The object detection apparatus according to claim 2, wherein a value is calculated.   The object position determination means calculates a partial evaluation value representing a degree of appearance of image features of a plurality of portions constituting a single object at the candidate position of the captured image, and the density at the candidate position is high. The object detection device according to claim 2, wherein the evaluation value is calculated by weighting the partial evaluation values of a small part of the parts constituting the object higher and summing the partial evaluation values. The object position determination means includes
Arrangement generation means for generating a plurality of different arrangements each including one or more candidate positions;
For each of the plurality of arrangements, a model image is generated by drawing an object model simulating a smaller part of the parts constituting the single object as the density at the candidate position is higher at each candidate position. Model image generation means for
Evaluation value calculating means for calculating the evaluation value representing the degree of similarity of the model image to the captured image for each of the plurality of arrangements;
An optimum arrangement determining means for determining the candidate position in the arrangement having the maximum evaluation value as the position of the object;
The object position detection apparatus according to claim 3, comprising: The object position determination means includes
Arrangement generation means for generating a plurality of different arrangements each including one or more candidate positions;
For each of the plurality of arrangements, model image generating means for generating a model image by drawing an object model imitating the single object at each candidate position;
For each of the model images in the plurality of arrangements, the partial evaluation value representing the degree of similarity of the object model with respect to the photographed image is obtained for each portion constituting the object, and the smaller the density at the candidate positions, the smaller An evaluation value calculating means for calculating the evaluation value by performing weighting with partial weighting and summing the partial evaluation values;
An optimum arrangement determining means for determining the candidate position in the arrangement having the maximum evaluation value as the position of the object;
The object position detecting device according to claim 4 including: The object position determination means includes
Candidate position setting means for setting a plurality of candidate positions in the captured image;
For each of the candidate positions, the image feature of the photographed image at the candidate position is input to the discriminator that has learned the image feature of the smaller part of the parts constituting the single object as the density of the candidate position is higher. Evaluation value calculating means for calculating the evaluation value,
Position determining means for determining, as the position of the object, the candidate position from which the evaluation value that satisfies a predetermined criterion is calculated;
The object position detection apparatus according to claim 3, comprising: The object position determination means includes
Candidate position setting means for setting a plurality of candidate positions in the captured image;
For each of the candidate positions, an image feature of the captured image at the candidate position is input to a discriminator that has learned the image features of a plurality of parts constituting the single object, and partial evaluation values of the plurality of parts are obtained. An evaluation value calculating means for calculating the evaluation value by performing weighting that is biased toward a smaller portion as the density at the candidate position is higher, and summing the partial evaluation values;
Position determining means for determining, as the position of the object, the candidate position from which the evaluation value that satisfies a predetermined criterion is calculated;
The object position detecting device according to claim 4 including:

Images