JP2018156236A - Object position estimating apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object position estimating apparatus capable of accurately estimating an individual object position from a photographed image obtained by photographing a space where congestion may occur.SOLUTION: The object position estimating apparatus of the present invention includes: density estimating means 50 for estimating a density distribution of objects in a photographed image by using a density estimator which has previously learned features of a density image obtained by photographing a space where the objects exist at a predetermined density for each predetermined density; object model storing means 41 for storing object models imitating the object; model image generating means 51 for setting an arrangement region in the photographed image and assigning positions in the arrangement region to the object models the number of which corresponds to the distribution to set a plurality of arrangements and drawing the object models in the plurality of arrangements to generate model images; and optimum arrangement estimation means 52 for calculating similarities between the respective model images in the plurality of arrangements and the photographed images to output the arrangement having the highest similarity out of the plurality of arrangements.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an object position estimation apparatus that estimates the position of a predetermined object such as a person from an image, and more particularly to an object position estimation apparatus that estimates the position of an individual object from an image in which a space where congestion can occur is captured.

  In an event venue or other space where congestion can occur, countermeasures such as placing a large number of guards in the crowded area are required to prevent accidents. Therefore, monitoring cameras can be arranged at various locations in the venue to estimate the distribution of people from the captured images and display the estimated distribution, thereby facilitating the understanding of the congestion situation by the monitoring staff.

  In doing so, by estimating the position of the individual person, a model imitating the person's shape is displayed at the estimated individual position, and / or the positional relationship of the person (for example, forming a matrix, surrounding) ) And reporting the analysis result, further improvement in monitoring efficiency can be expected.

  One of the methods for estimating the position of each person from a photographed image obtained by photographing a plurality of people is a method of applying a plurality of models imitating people to the photographed image.

  In the moving object tracking device described in Patent Document 1, the moving object that is tracking the moving object model that imitates the shape of the moving object that is being tracked at the position where the changed pixel is extracted by comparing the monitoring image and the background image. The position of each moving object is estimated by combining and applying the same number. At that time, the estimated object area is synthesized for each object position, and the change pixels outside the synthesis area among the change pixels are detected and labeled, and if the label is large enough to be regarded as a moving object, the label It describes that the position is the position of a newly appearing moving object.

JP 2012-159958 A

  However, it is difficult to estimate the number of models to be applied to a photographed image obtained by photographing a space at the time of congestion, and there is a problem that the position of a person cannot be estimated with high accuracy.

  In other words, in the method of applying a combination of models to a change area, if a combination with a large area overlap ratio is included, more models than the original number are applied, so it is necessary to limit the number of models to be combined to a reasonable number. There is.

  However, in a photographed image obtained by photographing a crowded space, a large number of new appearances can occur simultaneously with the occlusion between people. This means that depending on the degree of occlusion, the relationship between the area of the changed area (label outside the combined area) due to the new appearance and the number of newly appearing people can vary. For this reason, it is difficult to estimate the number of models to be applied to the label from the area of the label outside the synthesis region.

  In addition, disappearance of a large number of people may occur at the same time as the emergence of a large number of people in a captured image obtained by photographing a space at the time of congestion. For this reason, it is difficult to estimate the number of models to be applied not only to a change area due to a new appearance but also to a change area including other portions.

  For this reason, the number of models to be applied to a photographed image obtained by capturing a crowded space cannot be limited to a reasonable number, and the position of a person cannot be estimated with high accuracy.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an object position estimation apparatus that can accurately estimate the position of each object from a captured image in which a space in which congestion may occur is captured.

  In order to solve such a problem, the present invention provides an object position estimation device that estimates the position of each object from a captured image obtained by capturing an estimation target space in which congestion due to the predetermined object may occur. The density estimation means for estimating the density distribution of the object photographed in the photographed image using the density estimator that has previously learned the features of the density image photographed in the space where the object exists at the object, and the object model imitating the object The object model storage means for storing the image and the arrangement area are set in the photographed image, and a plurality of arrangements are set by assigning positions in the arrangement area to the number of object models corresponding to the distribution. The model image generating means for drawing the object model and generating the model image at the same time, and calculating the similarity between the model image and the captured image of each of the plurality of arrangements, Similarity score to provide an object position estimation device characterized by comprising a, and optimal placement estimating means for outputting the highest placement.

  In such an object position estimation device, the model image generation means assigns a plurality of types of object models within a range of an estimation error assumed in the distribution and sets a plurality of kinds of arrangements, and the optimum arrangement estimation means determines the similarity for each arrangement. It is preferable to calculate the degree as the degree of estimation error corresponding to the number in the arrangement increases.

  In such an object position estimation device, the model image generation means calculates the degree of overlap between the object models in the model image for each arrangement, and the optimum arrangement estimation means decreases the similarity for each arrangement as the degree of overlap in the arrangement increases. It is preferable to calculate.

  The object position estimation apparatus further includes a size conversion unit that converts the size of the local region in the captured image into an actual size in the estimation target space, and the density estimation unit estimates the density for each local region and generates a model image. Preferably, the means determines the number to be allocated to the arrangement area from the actual size and density of the local area included in the arrangement area.

  The object position estimation apparatus further includes a change area extraction unit that compares the captured image with the background image of the estimation target space and extracts a change area that differs from the background image by a predetermined reference or more in the captured image, and generates a model image. It is preferable that the means sets the change area as the arrangement area.

  In such an object position estimation apparatus, it is preferable that the model image generation unit sets an area where a density greater than 0 is estimated in the distribution as the arrangement area.

In such an object position estimation device, the object model storage means stores an object model that imitates the shape of the object, and the optimum arrangement estimation means has a high degree of fitness of the shape of the object model drawn on the model image with respect to the arrangement area. It is preferable to calculate the degree of similarity according to.

According to the present invention, since the density distribution of an object is estimated from the photographed image and the position of each object is estimated using the number of object models corresponding to the density distribution, the photographed image in which a space in which congestion can occur is photographed. Thus, the position of each object can be estimated with high accuracy.

1 is a block diagram showing a schematic configuration of an image monitoring device 1. FIG. 2 is a functional block diagram of the image monitoring apparatus 1. FIG. FIG. 4 is a schematic diagram showing the contents stored in an object model storage means 41. 4 is a schematic diagram showing components of a size conversion means 54. FIG. It is a figure explaining the example of a process of the real size calculation means 540. FIG. It is the figure which showed typically an example of the calculation process of a shape fitting degree. It is the figure which showed a mode that the concealment degree was calculated with respect to the model image. 6 is a flowchart for explaining the operation of the image processing unit 5. FIG. It is the 1st figure explaining optimal arrangement | positioning estimation of an attention change area | region. It is the 2nd figure explaining optimal arrangement | positioning estimation of an attention change area | region.

  Hereinafter, as an example of a preferred embodiment including the object position estimation apparatus of the present invention, an image that estimates the position of each person from a captured image obtained by photographing an event venue by the object position estimation apparatus and displays information on the estimated position The monitoring device 1 will be described.

<Configuration of Image Monitoring Device 1>
FIG. 1 is a block diagram showing a schematic configuration of the image monitoring apparatus 1. The image monitoring apparatus 1 includes a photographing unit 2, a communication unit 3, a storage unit 4, an image processing unit 5, and an output unit 6.

  The photographing unit 2 is a surveillance camera and is connected to the image processing unit 5 via the communication unit 3. The photographing unit 2 shoots the monitoring space at a predetermined time interval to generate a photographed image, and sequentially captures the photographed image to the image processing unit 5. It is a photographing means to input. For example, the imaging unit 2 is installed on a pole installed at an event venue with a predetermined fixed visual field overlooking the monitoring space, and images the monitoring space with a frame period of 1 second to generate a color image. A monochrome image may be generated instead of the color image.

  The communication unit 3 is a communication circuit, one end of which is connected to the image processing unit 5, and the other end is connected to the photographing unit 2 and the output unit 6 via a communication network such as a coaxial cable, a LAN (Local Area Network), or the Internet. Connected. The communication unit 3 acquires a captured image from the imaging unit 2 and inputs the acquired image to the image processing unit 5, and outputs an estimation result input from the image processing unit 5 to the output unit 6.

  The storage unit 4 is a memory device such as a ROM (Read Only Memory) or a RAM (Random Access Memory), and stores various programs and various data. The storage unit 4 is connected to the image processing unit 5 and inputs / outputs such information to / from the image processing unit 5.

  The image processing unit 5 is configured by an arithmetic device such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an MCU (Micro Control Unit). The image processing unit 5 is connected to the storage unit 4 and the output unit 6, operates as various processing units / control units by reading out and executing programs from the storage unit 4, and stores various types of data in the storage unit 4 for reading. . The image processing unit 5 is also connected to the imaging unit 2 and the output unit 6 via the communication unit 3, and analyzes a captured image acquired from the imaging unit 2 via the communication unit 3 to analyze the person existing in the monitoring space. The position and the person area are displayed on the output unit 6 via the communication unit 3.

  The output unit 6 is a display device such as a liquid crystal display or a CRT (Cathode Ray Tube) display, and is a display unit that is connected to the image processing unit 5 via the communication unit 3 and displays an estimation result by the image processing unit 5. . The monitor visually recognizes the displayed estimation result to determine the occurrence of congestion, and takes measures such as changing the personnel assignment as necessary.

  In the present embodiment, the image monitoring apparatus 1 in which the number of the photographing units 2 and the image processing units 5 is 1: 1 is illustrated, but in another embodiment, the number of the photographing units 2 and the image processing units 5 is illustrated. Can be many-to-one or many-to-many.

<Function of the image monitoring apparatus 1>
FIG. 2 is a functional block diagram of the image monitoring apparatus 1. The communication unit 3 functions as the image acquisition unit 30 and the object position output unit 31, and the storage unit 4 functions as the background image storage unit 40, the object model storage unit 41, and the like. The image processing unit 5 functions as a density estimation unit 50, a model image generation unit 51, an optimum arrangement estimation unit 52, a change area extraction unit 53, a size conversion unit 54, and the like.

  Hereinafter, each means will be described with reference to FIGS.

  The image acquisition unit 30 sequentially acquires captured images from the imaging unit 2 that is an imaging unit, and sequentially outputs the acquired captured images to the density estimation unit 50 and the change area extraction unit 53.

  The density estimation unit 50 extracts the feature quantity for density estimation (estimation feature quantity) from the captured image input from the image acquisition unit 30 and acquires the extracted estimation feature quantity into the density estimator. A human density distribution (density distribution) is estimated using the output value obtained, and the estimated density distribution is output to the model image generation means 51.

Specifically, the density estimation means 50 sets a window (estimation extraction window) at the position of each pixel of the captured image, and calculates a feature amount for estimation of the captured image in each estimation extraction window. The feature value for estimation is extracted every time. The estimation feature amount is a GLCM (Gray Level Co-occurrence Matrix) feature.
It is desirable that the area in the monitoring space photographed by each estimation extraction window is the same size. That is, preferably, the density estimation means 50 reads out the camera parameters of the photographing unit 2 stored in advance from a camera parameter storage means (not shown), and is photographed in an arbitrary region of the photographed image by homography conversion using the camera parameters. The estimation feature amount is extracted after the captured image is deformed so that the areas in the monitoring space have the same size. The camera parameter storage means can also be shared with camera parameter storage means 411 or / and 420 described later.

  And the density estimation means 50 acquires the estimated density which is the output value for every pixel by inputting the estimation feature-value corresponding to the said pixel to a density estimator. The estimated density for each pixel is a density distribution. In addition, when the estimated feature amount is extracted by deforming the captured image, the density estimating unit 50 deforms the density distribution into the original captured image shape by homography conversion using the camera parameter.

  The density estimator is a function that, when a feature amount of an image is input, estimates the density of a person photographed in the image and outputs an estimated value (estimated density). The function including parameters such as the coefficient is stored in advance as part of the program of the density estimation means 50. The density estimator can be realized by a discriminator that identifies multi-class images, and can be a discrimination function learned by the multi-class SVM method.

Density, for example, there is no human "Background" class is 0 people / m higher than ² is two / m ² or less "low density" class, higher than two / m ² 4 persons / m ² or less It can be defined as 4 classes of “medium density” class, “high density” class higher than 4 persons / m ² .

  The estimated density is a value given in advance to each class, and is a value output as a result of distribution estimation. In the present embodiment, values corresponding to each class are expressed as “background”, “low density”, “medium density”, and “high density”.

  That is, the density estimator applies the multi-class SVM method to the feature quantities of a large number of images (density images) belonging to the “background” class, “low density” class, “medium density” class, and “high density” class. This is an identification function for discriminating the images of each class from other classes. The parameters of the discriminant function derived by this learning are stored as a density estimator. The feature amount of the density image is the same type as the estimation feature amount and is a GLCM feature.

From the density distribution output by the density estimation means 50, the density of people at various locations in the photographed image can be understood, but the position of each person cannot be determined from the density distribution.
The model image generation means 51 and the optimum arrangement estimation means 52 in the subsequent stage of the density estimation means 50 estimate the position of each person by applying a model imitating a person to the captured image. The model to be applied is stored in the object model storage means 41, and this model imitates the shape of a person. The optimal arrangement estimation means 52 evaluates the degree of model fit based on the similarity between the shape of the arranged model and the shape feature amount appearing in the captured image.

  Specifically, the model image generation unit 51 refers to the density distribution input from the density estimation unit 50, reads the object model from the object model storage unit 41, sets an arrangement area in the captured image, and sets the density. Assign multiple positions according to the distribution (number of placements) to the object model, set multiple placements, draw the object model with the set placements, generate a model image, and generate The model image information including the model image and the number of arrangements is output to the optimum arrangement estimation means 52. The arrangement area is an area where an object is estimated to be photographed, such as a change area.

  Then, the optimum arrangement estimation unit 52 calculates the similarity between the model image of each of the plurality of arrangements input from the model image generation unit 51 and the photographed image, and the similarity is the highest among the plurality of arrangements. Information on the object position is generated from the position of the object model indicated by the arrangement (optimum arrangement) and is output to the object position output means 31.

  By arranging the number of object models according to the density distribution in this way, it is possible to accurately limit the number of models to be applied and estimate the position of each object. For this reason, it is possible to prevent the object models exceeding the original number of objects from being applied, and it is possible to accurately estimate the position of each object from a captured image in which a space where congestion may occur is captured.

  Generation of an object model and a model image will be described.

  As shown in FIG. 3A, the object model storage unit 41 includes a three-dimensional model storage unit 410 that stores a three-dimensional model of an object to be estimated in advance, and a camera that stores a camera parameter of the photographing unit 2 in advance. A parameter storage unit 411 is provided, and the model image generation unit 51 reads the stereo model and camera parameters.

  The three-dimensional model is data describing a partial model representing a three-dimensional shape for each of a plurality of constituent parts constituting an object to be estimated, and an arrangement relationship between the partial models. The object to be estimated by the image monitoring apparatus 1 is a standing person, and a three-dimensional model in which spheroids approximating the three-dimensional shape of the human head, torso, and legs are stacked in the Z-axis direction. Set. In this embodiment, in order to simplify the description, the height and width of the three-dimensional model are set to a standard human size and are common to all. The center of the head is the representative position of the person. Note that the three-dimensional model may be further simplified and approximated by one spheroid.

  The camera parameters include information regarding projection conditions when the imaging unit 2 captures a captured image in which the monitoring space is projected. For example, camera parameters include external parameters such as the installation position and imaging direction of the imaging unit 2 in an actual monitoring space, internal parameters such as the focal length, angle of view, lens distortion, and other lens characteristics of the imaging unit 2 and the number of pixels of the imaging device. It is information including. By rendering the stereoscopic model using the camera parameters, a virtual image (model image) imitating the photographing of a person by the photographing unit 2 can be generated. In addition, by using this camera parameter to back-project any pixel on the captured image onto the horizontal plane at the center of the head of the stereo model, the monitoring space of the stereo model projected at the position of the pixel is simulated. The representative position (center of the head) in the virtual space can be calculated.

  The model image generating means 51 assigns positions on the captured image to the object models as many as the number of arrangements, obtains a position in the virtual space corresponding to each position using the camera parameters, and puts a three-dimensional model at each obtained position. Deploy. Then, the model image generating unit 51 generates a model image by rendering the virtual space in which the three-dimensional model is arranged on the shooting surface of the shooting unit 2 using the camera parameters. Concealment between objects is also expressed in the model image generated by rendering.

  As shown in FIG. 3B, the object model storage means 41 stores a two-dimensional model image corresponding to the position of each pixel of the photographed image in advance, instead of the configuration shown in FIG. The image storage unit 412 may be used. These model images are generated by performing a projection of a three-dimensional model using camera parameters in advance. In this case, when the model image generation unit 51 assigns a position on the captured image to each object model, the model image generation unit 51 draws a model image corresponding to the position. At this time, the model image generation unit 51 expresses concealment between objects by overwriting and drawing in order from a position far from the photographing unit 2.

  Alternatively, the object model storage unit 41 may be configured by a model image storage unit 412 that stores in advance a number of representative model images smaller than the number of pixels of the captured image. In this case, a model image at an arbitrary position is generated by performing deformation processing such as enlargement / reduction on these representative model images.

  As described above, the object model storage unit 41 stores an object model imitating an object to be estimated, and particularly stores an object model imitating its shape. When the model image generation unit 51 assigns a position on the captured image to each object model, the model image generation unit 51 reads the object model from the object model storage unit 41 and draws the object model at each assigned position to generate a model image.

  Next, the area (arrangement area) where the object model is arranged and the number of arrangements will be described.

  In a simple example, the model image generation unit 51 sets an area estimated to be a low density class, a medium density class, or a high density class as an arrangement area, and sets each of one or more object models in the arrangement area. While assigning random positions, changing the assignment, and increasing the number of arrangements up to the upper limit number (for example, the upper limit number that can be arranged without overlapping the three-dimensional model in the virtual space), a plurality of different positions and / or arrangement numbers are mutually different Generate a model image.

However, in the above simple example, since the combination of the position and the number of arrangements is excessive, the amount of processing is increased, and an erroneous estimation is likely to occur due to the application of an object model exceeding the original number of objects.
Therefore, the model image generation unit 51 sets a more strict arrangement area by setting the change area in the captured image as the arrangement area, and determines a more strict arrangement number from the actual size and density of the arrangement area. Thus, the amount of processing is reduced, and erroneous estimation due to the application of an object model exceeding the original number of objects is more accurately prevented.

  For this purpose, the background image storage unit 40 stores the background image of the monitoring space, and the change area extraction unit 53 compares the captured image input from the image acquisition unit 30 with the background image read from the background image storage unit 40. Then, a change area that differs from the background image by a predetermined reference or more in the captured image is extracted, and information on the extracted change area is output to the model image generation means 51 and the optimum arrangement estimation means 52.

Specifically, the change area extraction unit 53 stores, in the background image storage unit 40, a photographed image when no person is present as a background image prior to estimation. In addition, the change area extraction unit 53 appropriately updates the background image by adding and averaging the partial images other than the change area in the new captured image to the background image.
Alternatively, the change area extraction unit 53 can generate a background image by rendering an environmental model that imitates an unattended monitoring space. In this case, the change area extraction unit 53 reads the environment model stored in advance from the environment model storage unit (not shown) and reads the camera parameters of the photographing unit 2 stored in advance from the camera parameter storage unit (not shown). Render the environment model using camera parameters. The camera parameter storage means can be shared with the camera parameter storage means 411 or / and 420 described later. The environmental model includes the three-dimensional shape, material characteristics, and the like of each of the objects constituting the monitoring space, and the illumination parameters of the light source that illuminates the monitoring space, and the change area extraction unit 53 estimates the illumination conditions from the captured image or the like, The background image is appropriately updated by changing and rendering the illumination parameter according to the estimated illumination condition.

The change area extraction unit 53 extracts the change area by background difference processing. That is, the change area extraction unit 53 calculates the absolute value (difference value) of the luminance difference between the captured image and the background image for each pixel and compares it with a predetermined threshold value to detect a pixel whose difference value is equal to or greater than the threshold value. Then, a block in which the detected pixels are spatially connected is extracted as a change area. A lump that is smaller than the estimated size of the estimation target is not extracted.
Alternatively, the change area extraction unit 53 can extract the change area by background correlation processing.
Note that the change region extraction unit 53 may include the difference value or the correlation value in each pixel in the change region information output to the model image generation unit 51.

  As described above, the background image storage unit 40 stores the background image of the estimation target space, and the change area extraction unit 53 compares the captured image with the background image of the estimation target space and differs from the background image in the captured image by more than a predetermined reference. The change area to be extracted is extracted. Then, the model image generation means 51 sets the change area input from the change area extraction means 53 as the arrangement area.

The size conversion means 54 designates an arbitrary pixel in the captured image from the model image generation means 51, and outputs the area (actual size) of the monitoring space projected on the designated pixel to the model image generation means 51.
Then, the model image generation unit 51 acquires the actual size of each pixel by designating each pixel in the change region to the size conversion unit 54, and the actual size of each pixel in the change region is input from the density estimation unit 50. Further, the number of models arranged in the change area is determined by multiplying the density of the pixels and summing up the products.
That is, if the set of pixels included in the change region is R, an arbitrary pixel in the set R is i, the actual size of the pixel i is a _i , and the density at the pixel i is d _i , the number of arrangements P is given by Is derived by

  Specifically, as shown in FIG. 4A, the size conversion means 54 is a pixel specified by using the camera parameter storage means 420 that stores the camera parameters of the photographing unit 2 in advance and the camera parameters. Can be configured by an actual size calculation unit 540 that converts the area on the captured image represented by the above into an area in the monitoring space and calculates the area of the area as the actual size of the designated pixel. The camera parameter storage unit 420 and the camera parameter storage unit 411 may be shared.

With reference to FIG. 5, a processing example of the actual size calculation unit 540 will be described. An XYZ space 100 in FIG. 5 is a virtual space that imitates a monitoring space. The XY plane represents a horizontal plane such as a ground surface or a floor surface in the monitoring space. FIG. 5 shows a horizontal plane 101 having a height (for example, 1.5 m) at the center of the head of the object model 102 simulating a person having a standard height.
The size conversion means 54 calculates an intersection point P ₀ (X ₀ , Y ₀ , Z ₀ ) at which the line-of-sight vector connecting the optical center of the photographing unit 2 and the designated pixel intersects the plane 101 using the camera parameters. . Similarly, the size conversion means 54 uses the camera parameters to determine the intersections P _r (X _r , Y _r , Z _r ) and P _b (X _b ) of the pixel adjacent to the right of the designated pixel and the lower pixel, respectively. , Y _b , Z _b ). Actual size corresponding to the designated pixel can be approximated by a parallelogram area vector is made from the vector and intersection point P ₀ from the intersection P ₀ to the intersection P _r to the intersection point P _b. The size conversion unit _{54, | (X r -X 0} ) (Y b -Y 0) - (Y r -Y 0) (X b -X 0) | and outputs them.

  As shown in FIG. 4B, the size conversion means 54 is associated with each pixel of the photographed image by an actual size storage means 421 that stores in advance the actual size of the pixel calculated in advance. It may be configured. In this case, the actual size storage unit 421 outputs the actual size stored corresponding to the pixel designated from the model image generation unit 51.

Here, the density indicated by the density distribution is an estimated value and includes an error. For this reason, it is preferable to estimate the position of the object by generating model images with a plurality of arrangement numbers with a wide range of arrangement numbers and selecting a model image most similar to the photographed image from among them.
Therefore, the model image generation means 51 assigns a plurality of types of object models and sets a plurality of types of arrangement within a range of estimation errors assumed in the density distribution.

The density distribution output from the above-described density estimation means 50 is represented by an estimated density having a width, and is an expression including an estimation error. That is, the estimation error ranges of the low density, medium density, and high density classes are each higher than 0 person / m ² and 2 persons / m ² or less, but higher than 2 persons / m ² and 4 persons / m ² or less, 4 persons / m ^2. It can be higher than 8 persons / m ² . However, the upper limit of the high density class is 8 people / m ² . Model image generating means 51, d _i lower place number P when applied to, P when applied to d _i of equation (1) the upper limit of each class of the formula (1) the lower limit of each class Can be set to the upper limit arrangement number.

For example, the model image generating means 51 has a lower limit number of arrangements of 4 (= 0.01 × 2 × 100 + 0.01 × 4 × 50) for a change area composed of 100 pixels of the medium density class and 50 pixels of the high density class. The upper limit arrangement number is calculated as 8 (= 0.01 × 4 × 100 + 0.01 × 8 × 50), a model image in which four object models are arranged, a model image in which five object models are arranged,..., 8 A model image in which individual object models are arranged is generated. However, in this example, for simplicity, it is set to 0.01 m ² per pixel.

In addition, the estimation error is estimated to be large, and the range of the estimation error of the low density, medium density, and high density classes is 0 person / m ² or more and 3 persons / m ² or less, 2 persons / m ² or more and 5 persons / m ^{2, respectively.} Hereinafter, 4 people / m ² or more and 8 people / m ² or less may be used.

As described above, the size conversion unit 54 converts the size of the local region in the captured image into the actual size in the estimation target space. Then, the model image generation unit 51 sets an arrangement area including a plurality of local areas on the photographed image, and determines the number to be allocated to the arrangement area from the actual size and density distribution of the local areas included in the arrangement area. . For this reason, it is possible to reduce the amount of processing and to prevent erroneous estimation due to the application of object models exceeding the original number of objects.
At that time, the model image generating means 51 assigns a plurality of object models and sets a plurality of arrangements within a range of estimation errors assumed in the density distribution. For this reason, even if an estimation error is included in the density distribution, a plurality of arrangements including the arrangement with the original number of objects can be set, and erroneous estimation of the position of the object can be prevented.

  Next, the similarity will be described.

The optimal arrangement estimation means 52 calculates the similarity according to the height of the degree of matching (shape fitting degree) of the shape of the object model drawn on the model image with respect to the arrangement area according to the following equation.
Similarity = shape conformity-( _WH x concealment degree + W _N x number of placement estimation errors) (2)
However, W _H and W _N are weighting factors larger than 0, and are set in advance based on prior experiments. The value subtracted from the shape conformity corresponds to the penalty value.

The shape suitability can be, for example, a degree of overlap between a change area extracted from a captured image and an area (model area) in which an object model is drawn in a model image. In that case, by reducing the non-overlap degree between the arrangement area and the model area from the overlap degree, it is possible to obtain a more reliable shape conformity degree that takes into account the protrusion of each other area. That is, the optimal arrangement estimation means 52 calculates the shape suitability according to the following equation.
Shape suitability = overlap-non-overlap
= (Overlapping area-non-overlapping area)
/ (Area of change area + area of model area−area of overlap area) (3)

FIG. 6 is a diagram schematically showing an example of the shape matching degree calculation process.
The optimal arrangement estimation means 52 receives the difference image 200 including the information on the change area 201 from the change area extraction means 53 and the model image 210 on which six model images 211 are drawn from the model image generation means 51. Has been. The sum area of the model image 211 is the model area. The image 220 shows a state in which the optimum arrangement estimation means 52 overlaps the change area 201 and the model area. In the figure, the white part is the overlapping area 221, the horizontal line is the non-overlapping area 222 on the model area side, and the shaded part is the non-overlapping area 223 on the change area side. The sum of the non-overlapping area 222 and the non-overlapping area 223 is the non-overlapping area of Equation (3).
The optimum arrangement estimation means 52 counts the number of pixels of the overlapping region 221, the non-overlapping region, the change region 201, and the model region to obtain the area of each region, and applies these areas to Equation (3) to determine the shape fitness. calculate.

More preferably, the shape matching degree is calculated by weighting the overlapping degree and the non-overlapping degree with a difference value in each pixel. That is, the optimum arrangement estimation unit 52 can also calculate the shape suitability according to the following equation.
Shape suitability = overlap-non-overlap
= Sum of difference values in overlapping area-Sum of difference values in non-overlapping area
/ (Area of change area + area of model area−area of overlap area) (4)
When the change area extraction unit 53 extracts the change area by the background correlation process, a value obtained by subtracting the correlation value from 1.0 is used instead of the difference value.

  Alternatively, the shape matching degree can be the edge similarity between the captured image and the model image. In this case, the optimum arrangement estimation unit 52 receives the captured image from the image acquisition unit 30 and extracts an edge from each of the captured image and the model image. Then, the optimal arrangement estimation unit 52 calculates, for example, the absolute value of the difference from the edge extraction result of the corresponding captured image pixel for each pixel from which a valid edge has been extracted from the model image, and calculates the total value. The edge similarity is calculated by dividing by the number of pixels used.

  Furthermore, the difference between the overlapping degree and the non-overlapping degree and the weighted sum of the edge similarity degrees can be used as the shape matching degree.

  Next, the degree of concealment included in the penalty term for similarity will be described.

  If the overlap between the object models in the model image becomes too large, the change in the shape conformity with the increase / decrease in the number of arrangements becomes small, which causes a false estimation of the position of the object. The degree of concealment is the degree of overlap between object models in a model image, and is a measure for suppressing excessive overlap of object models.

The model image generation means 51 calculates a concealment degree representing the degree of overlap between object models in the model image for each arrangement according to the following equation, and the optimum arrangement estimation means 52 calculates the similarity for each arrangement as shown in equation (2). The calculation is performed with a lower concealment degree in the arrangement.
Concealment level = area of overlapping area between models / area of sum area of model areas (5)

FIG. 7 is a diagram showing how the concealment degree is calculated for the model image 210 of FIG.
An image 300 is an image representing an overlap of six model regions. A region obtained by combining regions 301, 302, and 303 indicated by hatching is an overlapping region between models.
An image 310 is an image representing a logical sum of six model areas. A region obtained by combining regions 311, 312, and 313 indicated by hatching is a sum region of model regions.
The model image generation means 51 counts the number of pixels in the regions 301, 302, and 303 to determine the area of the overlap region between the models, and counts the number of pixels in the regions 311, 312 and 313 to calculate the sum of the model regions. The area is applied, and the area is applied to Equation (5) to calculate the concealment degree.

  By doing in this way, it is possible to estimate the optimal arrangement based on the similarity including the concealment degree, and it is possible to more accurately prevent erroneous estimation due to the application of object models that are more than the original number of objects.

  Next, the estimation error of the number of arrangements included in the penalty term of similarity will be described.

  The number of arrangements in each model image is a number set based on the estimated value of density, and includes a density estimation error. The estimation error of the number of arrangement is a scale representing the degree of the estimation error of the density distribution included in the number of arrangement in each model image.

  The optimum arrangement estimation means 52 calculates the similarity for each arrangement by lowering the estimation error corresponding to the number of arrangements in the arrangement.

ただし、変化領域に含まれる画素の集合をＲ、集合Ｒの中の任意の一画素をｉ、画素ｉの実サイズをａ_ｉ、画素ｉにおける密度の代表値をｃ_ｉとしている。低密度クラスの密度の代表値は１人／ｍ^２、中密度クラスの密度の代表値は３人／ｍ^２、高密度は６人／ｍ^２などとすることができる。αは０よりも大きな定数であり、事前実験に基づいて予め定められる。 Specifically, the optimum arrangement estimation unit 52 calculates an estimation error of the arrangement number according to the following equation, for example.
Arrangement number estimation error = −exp {−α (representative value of arrangement number−number of arrangement) ² } (6)

However, a set of pixels included in the change region is R, an arbitrary pixel in the set R is i, an actual size of the pixel i is a _i , and a representative value of the density in the pixel i is c _i . The representative value of the density of the low density class may be 1 person / m ² , the representative value of the density of the medium density class may be 3 persons / m ² , the high density may be 6 persons / m ² , and the like. α is a constant larger than 0 and is determined in advance based on a prior experiment.

  In this way, the optimal arrangement can be estimated based on the similarity including the estimation error of the number of arrangements, so that an object model more than the original number of objects will be applied with a deviation from the density distribution. It is possible to prevent erroneous estimation of the position of each object more accurately.

As described above, the optimum arrangement estimation means 52 outputs the object position information from the position of the object model indicated by the arrangement (optimum arrangement) having the highest similarity among the plurality of arrangements.
For example, the optimum arrangement estimation unit 52 generates and outputs a distribution image in which each of the object models drawn on the model image of the optimum arrangement is color-coded according to the density class as information on the object position that is easy for the observer to visually recognize. .

  The object position information may be the object position itself indicated by the optimum arrangement. Alternatively, the object position information may be a region of each object model in the optimum arrangement that does not overlap with other object models. Alternatively, the object position information may be data including two or more of the data described above.

  The object position output means 31 sequentially outputs the information of the object position input from the optimum arrangement estimation means 52 to the output unit 6, and the output unit 6 displays the information input from the object position output means 31. For example, the information on the object position is transmitted / received via the Internet and displayed on the output unit 6. The monitor can quickly grasp the location where the monitoring space is congested and the state of the location by visually checking the displayed distribution image, and take measures such as dispatching or increasing the number of guards at the location. Do.

<Operation of Image Monitoring Device 1>
The operation of the image monitoring apparatus 1 will be described with reference to the flowcharts of FIGS.

  The image monitoring device 1 is activated when the event venue is unmanned, and when the image monitoring device 1 starts operating, the imaging unit 2 installed in the event venue captures the monitoring space and captures the captured image at predetermined intervals. The images are sequentially transmitted to the image analysis center where the image processing unit 5 is installed. The image processing unit 5 repeats the operation according to the flowchart of FIG. 8 every time a captured image is received.

  First, the communication unit 3 operates as the image acquisition unit 30 and waits to receive a captured image from the imaging unit 2. And the image acquisition means 30 which acquired the picked-up image outputs the said picked-up image to the image process part 5 (step S1).

  The image processing unit 5 to which the photographed image is input operates as the change area extracting unit 53, reads the background image from the background image storage unit 40 of the storage unit 4, compares it with the photographed image, and extracts the change region (step S2). . However, immediately after activation, the change area extraction unit 53 omits the extraction of the change area and stores the captured image in the background image storage unit 40 as a background image.

  When the change area is not extracted (NO in step S3), the processes in steps S4 to S8 are omitted. At this time, the change area extraction unit 53 replaces the background image in the background image storage unit 40 with a captured image from which no change area has been extracted.

  When the change region is extracted (YES in step S3), the image processing unit 5 also operates as the model image generation unit 51 and the optimum arrangement estimation unit 52, and the change region is transferred from the change region extraction unit 53 to the model image generation unit 51. And the difference value information is input from the change region extraction unit 53 to the optimum arrangement estimation unit 52. The model image generation means 51 and the optimum arrangement estimation means 52 hold these pieces of information, and the process proceeds to step S4. Further, the image processing unit 5 operates as the density estimation unit 50, and the captured image is input to the density estimation unit 50. The change area extraction unit 53 updates the background image using the captured image.

  The density estimation means 50 to which the photographed image is input scans the photographed image with the density estimator and estimates the density distribution (step S4). The image processing unit 5 that has estimated the density distribution also operates as the model image generation unit 51, and the density distribution is input from the density estimation unit 50 to the model image generation unit 51.

  The model image generation means 51 to which the density distribution has been input sequentially sets the change area held as the attention change area (step S5), and controls the optimum arrangement estimation of the attention change area (step S6).

  With reference to the flowchart of FIG. 9, the optimum arrangement estimation of the attention change area will be described.

  First, the image processing unit 5 also operates as the size conversion unit 54, and the model image generation unit 51 designates each pixel of the target change region as the size conversion unit 54 and calculates the actual size of each pixel (step S60). .

Next, the model image generating unit 51 determines the range of the number of arrangements with respect to the attention changing region (step S61). The model image generating means 51 calculates the lower limit arrangement number by substituting the actual size of each pixel in the attention change region and the lower limit value of the density class of each pixel into a _i and d _i in the equation (1), respectively. Further, the model image generating means 51 calculates the upper limit arrangement number by substituting the actual size of each pixel in the attention change region and the upper limit value of the density class of each pixel into a _i and d _i in Expression (1), respectively. .

Further, the model image generating unit 51 substitutes the actual size of each pixel in the target change region and the representative value of the density class of each pixel into a _i and c _i in Expression (7), respectively, and thereby represents the representative value of the number of arrangements. Is calculated (step S62).

  Subsequently, the model image generating unit 51 sequentially sets the integer value within the range set in step S61 as the number of arrangement (step S63), and performs the loop processing of steps S63 to S73.

  The model image generating means 51 prepares a counter T for counting the number of iterations, initializes it to 0 (step S64), and starts control of the iteration process.

  The model image generation unit 51 assigns positions to the object model by setting positions at random in the attention change area by the same number as the number of arrangements set in step S63 (step S65).

  The model image generation means 51 prepares a model image having the same size as the captured image, and draws an object model at each position set in step S65 of the model image (step S66). The model image generation means 51 reads the camera parameters from the camera parameter storage means 411 and the stereo model from the stereo model storage means 410 with reference to the object model storage means 41 of the storage unit 4. The model image generation means 51 converts each position into a position in the virtual space using the camera parameter, arranges the stereo model at each position converted, and uses the camera space as the model image in the virtual space where the stereo model is arranged. The object model is drawn by rendering to.

  Further, the model image generation means 51 obtains the area of the overlapping area between the drawn object models and the area of the sum area of the drawn object model, substitutes these into equation (5), and generates the model image generated in step S66. The degree of concealment at is calculated (step S67). The image processing unit 5 also operates as the optimum arrangement estimation means 52, and the model image generation means 51 sends the optimum arrangement estimation means 52 to the model image, the overlapping area between the object models, and the non-overlapping that excludes the overlapping area from the sum area of the object model. The area, the number of arrangements, the position of each object model, the number of representative arrangements, and the degree of concealment are input.

  With reference to the flowchart of FIG. 10, the optimum arrangement estimation of the attention change area will be described.

  The optimal arrangement estimation means 52 calculates the estimation error of the arrangement number by substituting the input arrangement number and the representative arrangement number into Expression (6) (step S68).

  Further, the optimum arrangement estimating means 52 obtains the sum of the difference values in the overlap region and the sum of the difference values in the non-overlap region using the input overlap region, non-overlap region, and difference value, and these are expressed in equation (4). Substituting and calculating the shape adaptability (step S69).

  Then, the optimum arrangement estimation unit 52 substitutes the shape matching degree calculated in step S69, the estimation error calculated in step S68, and the input concealment degree into the equation (2), thereby generating the model image generated in step S66. The degree of similarity is calculated, and the model image, the position of each object model, and the degree of similarity are associated with each other and stored in the storage unit 4 (step S70).

When the model image similarity is calculated in this way, the model image generating means 51 increases the number of iterations T by 1 (step S71) and compares the value with a predetermined number of times T _MAX (step S72). .

If the number of iterations T is less than the specified number of times T _MAX (NO in step S72), model image generating means 51 returns the process to step S65 and repeats the iterative process.

On the other hand, when the number of iterations T has reached the specified number of times T _MAX (YES in step S72), the model image generating means 51 checks whether or not all the arrangement numbers in the range set in step S63 have been set (step S63). S73).

  If there is an arrangement number that has not yet been set (NO in step S73), the model image generating unit 51 returns the process to step S63 and performs the process with the next arrangement number.

  On the other hand, when the setting of the total number of arrangements has been completed (YES in step S73), the model image generation unit 51 notifies the optimum arrangement estimation unit 52 that generation of a plurality of model images for the attention change region has been completed. .

  Upon receiving this notification, the optimum arrangement estimating means 52 determines an arrangement having the maximum similarity (step S74). The optimum arrangement estimation means 52 selects the maximum value from the similarities recorded in step S70, and stores the model image associated with the selected similarity and the position of each object model as the object position in the attention change region. Store in part 4. The optimum arrangement estimation means 52 clears the information recorded in step S70 and advances the process to step S7 in FIG.

  A description will be given with reference to FIG. 8 again. The model image generation means 51 confirms whether or not the processing has been completed for all of the change areas that are held (step S7). If there is a change area that has not yet been processed (NO in step S7), the model image generating means 51 returns the process to step S5 and performs the process of the next change area. When the next change area is processed, information on the object position in the change area is added in step S74.

  On the other hand, when all the changed areas have been processed (YES in step S7), the optimum arrangement estimating means 52 outputs the object position information recorded in step S74 to the communication unit 3 (step S8). The communication unit 3 that has received the object position information operates as the object position output unit 31 and transmits the object position information to the output unit 6. The output unit 6 displays information on the object position and transmits it to the monitor.

  When the above process is completed, the process returns to step S1, and the process for the next photographed image is performed.

<Modification>
(1) In the above-described embodiment, an example in which the object to be estimated is a person has been shown. However, the present invention is not limited to this, and the object to be estimated may be a vehicle, an animal such as a cow or a sheep, or the like.

  (2) In the above-described embodiment and its modification, an example in which the photographing unit 2 is a single camera has been described. However, the photographing unit 2 can be configured by a plurality of cameras having a common field of view. In that case, the background image storage means 40 stores the background image of each camera, the camera parameter storage means (not shown) referred to by the change area extraction means 52 stores the camera parameters of each camera, and the change area extraction means. 52 extracts a change area for each camera. The camera parameter storage means (not shown) to which the density estimation means 50 refers is stored with camera parameters of each camera, and the density estimation means 50 estimates the density distribution for each camera. The camera parameter storage means 411 and 420 store the camera parameters of each camera, the size conversion means 54 calculates the actual size for each camera, and the model image generation means 51 calculates the range of the number of arrangements for each camera. . Then, the model image generation means 51 determines the range of the number of arrangements with the minimum lower limit arrangement number and the maximum upper limit arrangement number, and arranges as many object models as the number of arrangements in the virtual space. The model image generation means 51 generates a model image for each camera by rendering it using the camera parameters of each camera, and calculates a concealment degree for each camera. Then, the optimum arrangement estimation means 52 calculates the shape similarity, the estimation error of the number of arrangements, and the similarity for each camera, and determines the optimum arrangement from the model image having the maximum sum total value of the similarities of all cameras. In this way, an optimal arrangement can be determined comprehensively based on the similarity from a plurality of viewpoints with different object concealment states, and the object position estimation accuracy is improved.

  (3) In the above-described embodiment and the modifications thereof, an example using the fixed-field imaging unit 2 has been described, but a variable-field imaging unit 2 may be used. In that case, the imaging unit 2 outputs the camera parameters after the field of view change to the image processing unit 5 via the communication unit 3, and the image processing unit 5 uses a camera parameter storage unit (not shown) referred to by the density estimation unit 50, the change The camera parameter storage means (not shown) and the camera parameter storage means 411 and 420 referred to by the area extraction means 53 are changed to the input camera parameters. In this case, the change area extraction unit 53 generates a background image by rendering the environmental model.

  (4) In the above-described embodiment and its modification, the model image generation unit 51 sets the change area as the arrangement area. However, the model image generation unit 51 sets the area other than the background class as the arrangement area. You can also. That is, the model image generating unit 51 sets an area where a density greater than 0 is estimated in the density distribution as an arrangement area. Specifically, the model image generating unit 51 sets an area including pixels whose estimated densities are estimated to be a “low density” class, a “medium density” class, and a “high density” class as an arrangement area. When an area other than the background class is set as the arrangement area, the background image storage unit 40 and the change area extraction unit 53 can be omitted.

ただし、撮影画像の総画素数をＮ、撮影画像中の任意の一画素をｉ、画素ｉにて算出した窓内配置数をｍｉ、画素ｉにおける密度の代表値をｃｉとしている。背景クラスの密度の代表値は０人／ｍ^２、低密度クラスの密度の代表値は１人／ｍ^２、中密度クラスの密度の代表値は３人／ｍ^２、高密度は６人／ｍ^２などとすることができる。
或いは、最適配置推定手段５２は、画素ごとに窓内配置数と対応する密度クラスの値を求めて配置数の推定誤差を算出してもよい。例えば、或る配置において或る画素に対応して設定した推定用抽出窓内の物体モデルの位置が２個であれば当該画素の値は「低密度」クラスを示す値となる。最適配置推定手段５２は、こうして求めた画素ごと密度クラスの値を密度分布において対応する画素の値と比較し、同一値でない画素数を総画素数で除した値を配置数の推定誤差として算出する。 (5) In the above-described embodiment and each modification thereof, an example in which the estimation error of the number of arrangements is calculated according to the equations (6) and (7) is shown. It can also be calculated as an error. In that case, the density estimation means 50 also outputs the density distribution to the optimum arrangement estimation means 52. In each arrangement, the optimum arrangement estimation means 52 sets an extraction window for estimation for each pixel, counts the position of the object model in the extraction window for estimation, and obtains the number of arrangements in the window. The optimum arrangement estimation means 52 arranges the value obtained by dividing the sum of the differences between the number of arrangements in the window for each pixel and the representative value of the density of the corresponding pixel in the density distribution by the total number of pixels for each arrangement according to the following equation. Calculated as number estimation error.

However, the total number of pixels of the photographed image is N, one arbitrary pixel in the photographed image is i, the number of arrangement in the window calculated by the pixel i is mi, and the representative value of the density at the pixel i is ci. The representative value of the density of the background class is 0 person / m ² , the representative value of the density of the low density class is 1 person / m ² , the representative value of the density of the medium density class is 3 persons / m ² , and the high density is 6 persons / m ² . m ² or the like.
Alternatively, the optimal arrangement estimation means 52 may calculate the estimation error of the arrangement number by obtaining the density class value corresponding to the arrangement number in the window for each pixel. For example, if there are two positions of the object model in the estimation extraction window set corresponding to a certain pixel in a certain arrangement, the value of the pixel is a value indicating the “low density” class. The optimum arrangement estimation means 52 compares the density class value obtained for each pixel with the corresponding pixel value in the density distribution, and calculates a value obtained by dividing the number of non-identical pixels by the total number of pixels as an arrangement number estimation error. To do.

(6) In the above-described embodiment and each modification thereof, the example in which the number of arrangements is set based on the distribution density and the actual size has been shown, but the number of arrangements is set in a search based on the above-described estimation error of the number of arrangements. You can also For example, the model image generating unit 51 performs a temporary arrangement number loop process from 1 to a predetermined upper limit number (for example, the number of stereoscopic models that can be arranged without overlapping in the virtual space corresponding to the captured image). set the temporary placement of T _MAX street assigns a random position of the placement area each arrangement number of the object model, for each temporary arrangement of T _MAX as the formula (6) or formula (8), such as An estimation error is calculated and compared with a predetermined threshold value ε, and a model image is generated with a temporary arrangement in which the estimation error is equal to or less than the threshold value ε. Also by doing this, the model image generating means 51 assigns positions in the arrangement area to the number of object models corresponding to the density distribution, sets a plurality of arrangements, and draws the object models in the plurality of arrangements. Model images can be generated.

  (7) In the above-described embodiment and each modification thereof, the GLCM feature is exemplified as the feature amount learned by the density estimator and the estimation feature amount extracted by the density estimation means 50. However, these are replaced with the GLCM feature. Various feature quantities such as local binary pattern (LBP) feature quantities, Haar-like feature quantities, luminance patterns, HOG (Histograms of Oriented Gradients) feature quantities, or GLCM features And a feature amount combining a plurality of them.

(8) In the above-described embodiment and each modification thereof, the density estimator learned by the multi-class SVM method is exemplified, but instead of the multi-class SVM method, a decision tree type random forest method, a multi-class Adaboost Various density estimators such as a density estimator learned by (AdaBoost) method or multi-class logistic regression method can be used.
Alternatively, the density estimation means 50 can be realized by using a method such as CNN (Convolutional Neural Network) in which the feature amount extraction processing and the estimation processing by the density estimator are expressed by one network.

(9) In the above embodiment and the modifications thereof, the model image generation unit 51 calculates an example of the representative value c _{i of the} number of arrangements based on the density estimation value of each pixel output from the density estimation unit 50. However, the density estimation means 50 outputs the score of each class calculated in the estimation process in addition to the density estimation value, and the model image generation means 51 corrects the representative value c _{i of the} number of arrangements based on these scores. You can also
The score of the class is the background score indicating the likelihood of being the “background” class of the “background” class and other classes of the image from which the estimation feature amount is extracted, and the “low density” class and other classes. A low density score representing the likelihood of being a “low density” class, a medium density score representing the likelihood of being a “medium density” class of the “medium density” class and other classes, “ It is a high density score representing the likelihood of being a “high density” class and a “high density” class among other classes. Incidentally, the class showing the highest score among these is the density estimation value.
The model image generating unit 51 corrects the representative value c _{i of the} number of arrangements as the class score indicated by the density estimation value is higher and lower as the class score is lower.

  (10) In the above-described embodiment and its modifications, the density classes estimated by the density estimator are four classes, but the classes may be divided more finely.

(11) In the above-described embodiment and its modifications, the density estimation means 50 uses an example of a density estimator that classifies into multiple classes, but instead of this, regression that regresses the density value from the feature quantity It can also be a mold density estimator. That is, the density estimator can learn the parameters of the regression function for obtaining the density from the feature quantity by the ridge regression method, the support vector regression method, or the regression tree-type random forest method.
In that case, the model image generating means 51 uses the density value output from the density estimator as the representative value c _i of the number of arrangements. In addition, the model image generation unit 51 outputs a density estimator such that the lower limit arrangement number is (c _i -2) (however, 0 if it is 0 or less), and the upper limit arrangement number is (c _i +2). The lower limit arrangement number and upper limit arrangement number obtained by adding or subtracting a predetermined value to the density value are used.

(12) In the above-described embodiment and its modifications, the model image generation unit 51 randomly assigns a position to the object model every time it is repeated. However, the position slightly shifted from the previous position. May be assigned, a method of searching for a position by the MCMC (Markov chain Monte Carlo) method with reference to the similarity to the previous arrangement, or an assignment to improve the position sequentially by a hill-climbing method, etc. Also good.

DESCRIPTION OF SYMBOLS 1 ... Image monitoring apparatus 2 ... Imaging | photography part 3 ... Communication part 4 ... Memory | storage part 5 ... Image processing part 6 ... Output part 30 ... Image acquisition means 31 ... Object Position output means 40 ... background image storage means 41 ... object model storage means 50 ... density estimation means 51 ... model image generation means 52 ... optimum arrangement estimation means 53 ... change area extraction means 54 ... Size conversion means 410 ... Stereo model storage means 411 ... Camera parameter storage means 412 ... Model image storage means 420 ... Camera parameter storage means 421 ... Actual size storage means 540・ Actual size calculation means

Claims (7)

An object position estimation device that estimates the position of each object from a captured image obtained by capturing an estimation target space in which congestion due to a predetermined object may occur,
The density distribution of the object imaged in the captured image is estimated using a density estimator that has previously learned the characteristics of the density image obtained by capturing the space where the object exists at the density for each predetermined density. Density estimation means;
Object model storage means storing an object model imitating the object;
In addition to setting an arrangement area in the captured image, assigning positions in the arrangement area to the number of the object models according to the distribution, setting a plurality of arrangements, and setting the object models in the plurality of arrangements. Model image generating means for generating a model image by drawing;
Calculating the degree of similarity between the model image of each of the plurality of arrangements and the captured image, and outputting an arrangement having the highest degree of similarity among the plurality of arrangements;
An object position estimation apparatus comprising: The model image generating means assigns a plurality of the object models in the range of estimation errors assumed in the distribution and sets the plurality of arrangements;
The optimum arrangement estimating means calculates the similarity for each arrangement by lowering the estimation error corresponding to the number in the arrangement,
The object position estimation apparatus according to claim 1. The model image generation means calculates the degree of overlap between the object models in the model image for each arrangement,
The optimum arrangement estimation means calculates the similarity for each arrangement by lowering the degree of overlap in the arrangement,
The object position estimation apparatus according to claim 1 or 2. Size conversion means for converting the size of the local region in the captured image into the actual size in the estimation target space,
The density estimation means estimates the density for each local region,
The model image generation means determines the number to be allocated to the arrangement area from the actual size and the density of the local area included in the arrangement area.
The object position estimation apparatus according to claim 1. A change area extracting unit that compares the captured image with a background image of the estimation target space and extracts a change area that differs from the background image by a predetermined reference or more in the captured image;
The model image generating means sets the change area to the arrangement area;
The object position estimation apparatus according to any one of claims 1 to 4. The model image generating means sets an area where the density greater than 0 in the distribution is estimated as the arrangement area;
The object position estimation apparatus according to any one of claims 1 to 5. The object model storage means stores the object model imitating the shape of the object,
The optimum arrangement estimating means calculates the similarity according to the degree of fitness of the shape of the object model drawn on the model image with respect to the arrangement area.
The object position estimation apparatus according to any one of claims 1 to 6.

Images