JP6851233B2 - Object position estimation device - Google Patents

Description

The present invention relates to an object position estimation device that estimates the position of a predetermined object such as a person from an image, and more particularly to an object position estimation device that estimates the position of an individual object from an image in which a space where congestion may occur is photographed.

In spaces where congestion may occur, such as event venues, it is necessary to take measures such as allocating a large number of security guards in areas where congestion is occurring in order to prevent accidents. Therefore, by arranging surveillance cameras at various places in the venue, estimating the distribution of people from the captured image, and displaying the estimated distribution, it is possible to facilitate the grasp of the congestion situation by the observer.

At that time, by estimating the position of each person, a model imitating the shape of the person is displayed at the estimated individual position, and / and the positional relationship of the person (for example, forming a matrix, surrounding). ) Is analyzed and the analysis result is notified, which can be expected to further improve the monitoring efficiency.

One of the methods for estimating the position of each person from the images taken by a plurality of people is to combine a plurality of models imitating a person and apply them to the photographed image.

In the moving object tracking device described in Patent Document 1, a moving object that is tracking a moving object model that imitates the shape of the moving object being tracked at a position where change pixels are extracted by comparing a surveillance image and a background image. The positions of individual moving objects are estimated by combining and applying as many as the number of. At that time, the estimated object area is synthesized for each object position, the changed pixels outside the synthesized area are detected among the changed pixels and labeled, and if the label is large enough to be regarded as a moving object, the label is displayed. It is described that the position is the position of a newly appearing moving object.

Japanese Unexamined Patent Publication No. 2012-159985

However, it is difficult to estimate the number of models to be applied to the captured image of the space at the time of congestion, and there is a problem that the position of a person cannot be estimated with high accuracy.

In other words, in the method of applying a combination of models to a changing region, if a combination with a large region overlap ratio is included, more models than the original number will be applied, so it is necessary to limit the number of models to be combined to a plausible number. There is.

However, in a photographed image of a crowded space, new appearances of a large number of people may occur at the same time with occlusion between people. This means that the relationship between the area of the change area due to new appearance (label outside the synthetic area) and the number of newly appearing people can change variously depending on the degree of occlusion. Therefore, it is difficult to estimate the number of models to be applied to the label from the area of the label outside the composite area.

In addition, in a photographed image of a crowded space, the disappearance of a large number of people may occur at the same time as the new appearance of a large number of people. Therefore, it is difficult to estimate the number of models that should be applied not only to the change region due to new appearance but also to the change region including other parts.

Therefore, it is not possible to limit the number of models to be applied to a photographed image of a crowded space to a plausible number, and it is not possible to estimate the position of a person with high accuracy.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an object position estimation device capable of accurately estimating the position of an individual object from a photographed image in which a space where congestion may occur is photographed.

In order to solve such a problem, the present invention is an object position estimation device that estimates the position of each object from a captured image in which an estimation target space where congestion due to a predetermined object may occur is captured, and the density is set for each predetermined density. Using a density estimator that has learned the characteristics of the density image of the space in which the object exists in advance, a density estimation means that estimates the distribution of the density of the object captured in the captured image, and an object model that imitates the object. The object model storage means that stores the above and the arrangement area are set in the captured image, and the positions in the arrangement area are assigned to the number of object models according to the distribution to set multiple arrangements, and multiple arrangements are made. The model image generation means for drawing the object model and generating the model image and the similarity between the model image and the photographed image of each of the multiple arrangements are calculated, and the similarity among the multiple arrangements is the highest. Provided is an object position estimation device provided with an optimum arrangement estimation means for outputting a high arrangement.

In such an object position estimation device, the model image generation means allocates a plurality of types of object models within the range of estimation error assumed in the distribution and sets a plurality of arrangements, and the optimum arrangement estimation means is similar for each arrangement. It is preferable to calculate the degree lower as the estimation error corresponding to the number in the arrangement becomes larger.

In such an object position estimation device, the model image generation means calculates the degree of overlap between the object models in the model image for each arrangement, and the optimum arrangement estimation means lowers the degree of similarity for each arrangement as the degree of overlap in the arrangement increases. It is preferable to calculate.

Such an object position estimation device further includes a size conversion means for converting the size of a local region in a captured image into an actual size in an estimation target space, and the density estimation means estimates the density for each local region and generates a model image. As a means, it is preferable to determine the number to be allocated to the arrangement area from the actual size and density of the local area included in the arrangement area.

Such an object position estimation device further includes a change area extraction means for comparing a photographed image with a background image in an estimation target space and extracting a change area in the photographed image that differs from the background image by a predetermined reference or more, and generates a model image. As the means, it is preferable to set the change region as the arrangement region.

In such an object position estimation device, it is preferable that the model image generation means sets a region in which a density larger than 0 is estimated in the distribution as the arrangement region.

In such an object position estimation device, the object model storage means stores an object model that imitates the shape of the object, and the optimum placement estimation means has a high degree of suitability of the shape of the object model drawn on the model image with respect to the placement area. It is preferable to calculate the degree of similarity according to the above.

According to the present invention, since the density distribution of an object is estimated from the captured image and the position of each object is estimated using a number of object models corresponding to the density distribution, the captured image in which a space where congestion may occur is photographed. The position of each object can be estimated accurately from.

It is a block diagram which shows the schematic structure of the image monitoring apparatus 1. It is a functional block diagram of the image monitoring device 1. It is a schematic diagram which shows the storage content of the object model storage means 41. It is a schematic diagram which shows the component of the size conversion means 54. It is a figure explaining the processing example of the actual size calculation means 540. It is a figure which showed typically an example of the calculation process of the goodness of fit. It is a figure which showed the state of calculating the concealment degree with respect to a model image 210. It is a flow chart explaining the operation of the image processing unit 5. It is the first figure explaining the optimum arrangement estimation of the attention change area. It is a second figure explaining the optimum arrangement estimation of the attention change area.

Hereinafter, as an example of a preferred embodiment including the object position estimation device of the present invention, an image in which the position of an individual person is estimated from a photographed image of the event venue taken by the object position estimation device and information on the estimated position is displayed. The monitoring device 1 will be described.

<Configuration of image monitoring device 1>
FIG. 1 is a block diagram showing a schematic configuration of the image monitoring device 1. The image monitoring device 1 includes a photographing unit 2, a communication unit 3, a storage unit 4, an image processing unit 5, and an output unit 6.

The photographing unit 2 is a surveillance camera, is connected to the image processing unit 5 via the communication unit 3, photographs the monitoring space at predetermined time intervals to generate a photographed image, and sequentially transfers the photographed images to the image processing unit 5. It is a shooting means to input. For example, the photographing unit 2 is installed on a pole installed at the event venue with a predetermined fixed field of view overlooking the monitoring space, and photographs the monitoring space with a frame period of 1 second to generate a color image. A monochrome image may be generated instead of the color image.

The communication unit 3 is a communication circuit, one end of which is connected to the image processing unit 5, and the other end of the communication unit 3 is connected to the photographing unit 2 and the output unit 6 via a communication network such as a coaxial cable, LAN (Local Area Network), or the Internet. Be connected. The communication unit 3 acquires a photographed image from the photographing unit 2 and inputs it to the image processing unit 5, and outputs the estimation result input from the image processing unit 5 to the output unit 6.

The storage unit 4 is a memory device such as a ROM (Read Only Memory) and a RAM (Random Access Memory), and stores various programs and various data. The storage unit 4 is connected to the image processing unit 5 and inputs / outputs these information to and from the image processing unit 5.

The image processing unit 5 is composed of arithmetic units such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and an MCU (Micro Control Unit). The image processing unit 5 is connected to the storage unit 4 and the output unit 6, operates as various processing means / control means by reading and executing a program from the storage unit 4, and stores and reads various data in the storage unit 4. .. Further, the image processing unit 5 is also connected to the photographing unit 2 and the output unit 6 via the communication unit 3, and analyzes the photographed image acquired from the photographing unit 2 via the communication unit 3 to analyze a photographed image of a person existing in the monitoring space. The position and the person area are displayed on the output unit 6 via the communication unit 3.

The output unit 6 is a display device such as a liquid crystal display or a CRT (Cathode Ray Tube) display, and is a display means connected to the image processing unit 5 via the communication unit 3 and displaying the estimation result by the image processing unit 5. .. The observer visually recognizes the displayed estimation result, judges the occurrence of congestion, etc., and takes measures such as changing the staffing as necessary.

In this embodiment, the image monitoring device 1 in which the number of the photographing unit 2 and the image processing unit 5 is 1: 1 is illustrated, but in another embodiment, the number of the photographing unit 2 and the image processing unit 5 is illustrated. Can be many-to-one or many-to-many.

<Function of image monitoring device 1>
FIG. 2 is a functional block diagram of the image monitoring device 1. The communication unit 3 functions as the image acquisition means 30, the object position output means 31, and the like, and the storage unit 4 functions as the background image storage means 40, the object model storage means 41, and the like. Further, the image processing unit 5 functions as a density estimation means 50, a model image generation means 51, an optimum arrangement estimation means 52, a change area extraction means 53, a size conversion means 54, and the like.

Hereinafter, each means will be described with reference to FIGS. 2 to 7.

The image acquisition means 30 sequentially acquires captured images from the photographing unit 2 which is the photographing means, and sequentially outputs the acquired captured images to the density estimation means 50 and the change area extraction means 53.

The density estimation means 50 extracts the feature amount for density estimation (feature amount for estimation) from the captured image input from the image acquisition means 30, and inputs the extracted feature amount for estimation into the density estimator to acquire the feature amount. The human density distribution (density distribution) is estimated using the output value, and the estimated density distribution is output to the model image generation means 51.

Specifically, the density estimation means 50 sets a window (estimation extraction window) at the position of each pixel of the captured image, and calculates the estimation feature amount of the captured image in each estimation extraction window to obtain pixels. The feature amount for estimation is extracted for each. The estimation feature quantity is a GLCM (Gray Level Co-occurrence Matrix) feature.
It is desirable that the areas in the monitoring space photographed by each estimation window have the same size. That is, preferably, the density estimation means 50 reads out the camera parameters of the photographing unit 2 stored in advance from the camera parameter storage means (not shown), and is photographed in an arbitrary region of the captured image by homography conversion using the camera parameters. The captured image is deformed so that the areas in the monitoring space are the same size, and then the estimation feature amount is extracted. The camera parameter storage means can also be shared with the camera parameter storage means 411 and / and 420 described later.

Then, the density estimation means 50 acquires the estimated density, which is the output value, by inputting the estimation feature amount extracted corresponding to the pixel into the density estimator for each pixel. The estimated density for each pixel is the density distribution. When the captured image is deformed to extract the feature amount for estimation, the density estimation means 50 transforms the density distribution into the shape of the original captured image by homography transformation using camera parameters.

The density estimator is a function that estimates the density of the person photographed in the image and outputs an estimated value (estimated density) when the feature amount of the image is input. The function, including parameters such as its coefficient, is stored in advance as part of the program of the density estimation means 50. The density estimator can be realized by a discriminator that discriminates multi-class images, and can be a discriminant function learned by the multi-class SVM method.

The density is, for example, a "background" class with no people, a "low density" class ^{higher than 0 people / m 2} and 2 people / m ^{2 or less, higher than 2 people / m 2} and 4 people / m ² or less. It can be defined as 4 classes of "medium density" class and ^{"high density" class higher than 4 people / m 2.}

The estimated density is a value given in advance to each class and is a value output as a result of distribution estimation. In this embodiment, the values corresponding to each class are described as "background", "low density", "medium density", and "high density".

That is, the density estimator applies the multi-class SVM method to the features of a large number of images (density images) belonging to each of the "background" class, "low density" class, "medium density" class, and "high density" class. It is an identification function for distinguishing the image of each class from other classes. The parameters of the discriminant function derived by this learning are stored as a density estimator. The feature amount of the density image is the same as the feature amount for estimation, and is a GLCM feature.

From the density distribution output by the density estimation means 50, the density of people in various parts of the photographed image can be known, but from the density distribution, the position of each person cannot be known.
The model image generation means 51 and the optimum placement estimation means 52, which are the latter stages of the density estimation means 50, estimate the positions of individual people by applying a model imitating a person to a captured image. The model to be applied is stored in the object model storage means 41, and this model imitates the shape of a person. The optimum placement estimation means 52 evaluates the degree of fit of the model based on the similarity between the shape of the placed model and the shape feature amount appearing in the captured image.

Specifically, the model image generation means 51 refers to the density distribution input from the density estimation means 50, reads the object model from the object model storage means 41, sets an arrangement area in the captured image, and sets the density. Assign positions in the placement area to the number of object models (number of placements) according to the distribution, set multiple placements, draw the object model with the set multiple placements, generate a model image, and generate The model image information including the model image and the number of arrangements is output to the optimum arrangement estimation means 52. The arrangement area is an area where an object is presumed to be photographed, such as a change area.

Then, the optimum arrangement estimating means 52 calculates the similarity between the model image and the captured image of each of the plurality of arrangements input from the model image generation means 51, and the degree of similarity among the plurality of arrangements is the highest. Information on the object position is generated from the position of the object model indicated by the arrangement (optimal arrangement) and output to the object position output means 31.

By arranging the number of object models according to the density distribution in this way, the number of models to be applied can be accurately limited and the positions of individual objects can be estimated. Therefore, it is possible to prevent an object model having more than the original number of objects from being applied, and it is possible to accurately estimate the position of each object from a photographed image in which a space where congestion may occur is photographed.

The generation of the object model and the model image will be described.

As shown in FIG. 3A, the object model storage means 41 includes a three-dimensional model storage means 410 that stores a three-dimensional model of the object to be estimated in advance, and a camera that stores the camera parameters of the photographing unit 2 in advance. The parameter storage means 411 is provided, and the model image generation means 51 reads out the stereoscopic model and the camera parameters.

The three-dimensional model is data that describes a partial model that represents the three-dimensional shape of each of a plurality of constituent parts constituting the object to be estimated, and the arrangement relationship between the partial models. The object to be estimated by the image monitoring device 1 is a standing person, and a three-dimensional model in which spheroids that approximate the three-dimensional shapes of the human head, body, and legs are stacked in the Z-axis direction is created. Set. In this embodiment, for the sake of simplicity, the height and width of the three-dimensional model are standard human sizes and are common to all. In addition, the center of the head is the representative position of the person. The three-dimensional model may be simplified and approximated by one spheroid.

The camera parameters include information regarding projection conditions when the photographing unit 2 captures a captured image projected on the monitoring space. For example, the camera parameters include external parameters such as the installation position and imaging direction of the photographing unit 2 in the actual monitoring space, internal parameters such as the focal length, angle of view, lens distortion and other lens characteristics of the photographing unit 2 and the number of pixels of the imaging element. Information including. By rendering a three-dimensional model using this camera parameter, it is possible to generate a virtual image (model image) that imitates the shooting of a person by the shooting unit 2. In addition, by using this camera parameter to back-project any pixel on the captured image onto the horizontal plane at the height of the center of the head of the 3D model, the surveillance space of the 3D model projected at the position of the pixel is imitated. The representative position (center of the head) in the virtual space can be calculated.

The model image generation means 51 assigns positions on the captured image to as many object models as the number of arrangements, obtains positions in the virtual space corresponding to each position using camera parameters, and creates a three-dimensional model at each of the obtained positions. Deploy. Then, the model image generation means 51 generates a model image by rendering the virtual space in which the three-dimensional model is arranged on the shooting surface of the shooting unit 2 using the camera parameters. Concealment between objects is also expressed in the model image generated by rendering.

As shown in FIG. 3B, the object model storage means 41 stores a two-dimensional model image corresponding to the position of each pixel of the captured image in advance instead of the configuration of FIG. 3A. It can also be configured by the image storage means 412. These model images are generated by projecting a three-dimensional model using camera parameters in advance. In that case, when the model image generation means 51 assigns a position on the captured image to each object model, the model image generation means 51 draws a model image corresponding to the position. At that time, the model image generation means 51 expresses the concealment between the objects by overwriting and drawing in order from the position far from the photographing unit 2.

Alternatively, the object model storage means 41 may be configured by the model image storage means 412 that stores in advance a number of representative model images that is smaller than the number of pixels of the captured image. In this case, a model image at an arbitrary position is generated by performing deformation processing such as enlargement / reduction on these typical model images.

As described above, the object model storage means 41 stores an object model that imitates the object to be estimated, and particularly stores an object model that imitates the shape. Then, when the model image generation means 51 assigns a position on the captured image to each object model, the model image generation means 51 reads the object model from the object model storage means 41, draws the object model at each assigned position, and generates a model image.

Next, the area (arrangement area) in which the object model is arranged and the number of arrangements will be described.

In a simple example, the model image generating means 51 sets a region estimated to be a low-density class, a medium-density class, or a high-density class as a placement region, and sets each of the one or more object models in the placement region. Multiple ways of assigning random positions, changing the allocation, and increasing the number of placements up to the maximum number (for example, the maximum number of 3D models that can be placed without overlapping in virtual space), with different positions and / or numbers of placements. Generate a model image.

However, in the above simple example, since the combination of the position and the number of arrangements is excessive, the amount of processing is large and an object model larger than the original number of objects is applied, which makes it easy to cause an erroneous estimation.
Therefore, the model image generation means 51 sets a more strict arrangement area by setting the change area in the captured image as the arrangement area, and determines the more strict arrangement number from the actual size and density of the arrangement area. This reduces the amount of processing and more accurately prevents erroneous estimation due to the application of an object model larger than the original number of objects.

Therefore, the background image storage means 40 stores the background image of the monitoring space, and the change area extraction means 53 compares the captured image input from the image acquisition means 30 with the background image read from the background image storage means 40. Then, a change region different from the background image by a predetermined reference or more is extracted from the captured image, and the information of the extracted change region is output to the model image generation means 51 and the optimum arrangement estimation means 52.

Specifically, the change region extraction means 53 stores a captured image when no person is present in the background image storage means 40 as a background image prior to estimation. Further, the change area extraction means 53 appropriately updates the background image by adding, averaging, or replacing the partial image other than the change area of the new captured image with the background image.
Alternatively, the change area extraction means 53 can also generate a background image by rendering an environment model that imitates an unmanned surveillance space. In this case, the change area extraction means 53 reads the environment model stored in advance from the environment model storage means (not shown) and reads the camera parameters of the photographing unit 2 stored in advance from the camera parameter storage means (not shown). Render the environment model using camera parameters. The camera parameter storage means can also be shared with the camera parameter storage means 411 and / and 420 described later. The environment model includes the three-dimensional shape, material characteristics, and the lighting parameters of the light source that illuminates the monitoring space for each of the constituent objects of the monitoring space, and the change area extraction means 53 estimates the lighting conditions from the captured image and the like. The background image is updated as appropriate by rendering by changing the lighting parameters according to the estimated lighting conditions.

The change area extraction means 53 extracts the change area by background subtraction processing. That is, the change region extraction means 53 calculates the absolute value (difference value) of the brightness difference between the captured image and the background image for each pixel, compares it with a predetermined threshold value, and detects the pixel for which the difference value equal to or greater than the threshold value is calculated. Then, the mass in which the detected pixels are spatially connected is extracted as a change region. In addition, lumps smaller than the estimated size of the estimation target are excluded from the extraction target.
Alternatively, the change region extraction means 53 can also extract the change region by background correlation processing.
The change area extraction means 53 may include a difference value or a correlation value in each pixel in the information of the change area output to the model image generation means 51.

In this way, the background image storage means 40 stores the background image of the estimation target space, and the change area extraction means 53 compares the captured image with the background image of the estimation target space and differs from the background image by a predetermined reference or more in the captured image. Extract the change area to be used. Then, the model image generation means 51 sets the change area input from the change area extraction means 53 as the arrangement area.

The size conversion means 54 specifies an arbitrary pixel in the captured image from the model image generation means 51, and outputs the area (actual size) of the monitoring space projected on the designated pixel to the model image generation means 51.
Then, the model image generation means 51 designates each pixel in the change region as the size conversion means 54, acquires the actual size of each pixel, and inputs the actual size of each pixel in the change region from the density estimation means 50. The number of models arranged with respect to the change region is determined by multiplying the density of the pixels and summing the products.
That is, the set of pixels included in the change region R, any one pixel i in the set R, the actual size a _i of the pixel _i, when the density at pixel i and d _i, arrangement number P is expressed by the following equation Derived by.

Specifically, as shown in FIG. 4A, the size conversion means 54 uses the camera parameter storage means 420 that stores the camera parameters of the photographing unit 2 in advance and the pixels designated by using the camera parameters. It can be configured by the actual size calculation means 540 that converts the area on the captured image represented by the above into an area in the monitoring space and calculates the area of the area as the actual size of the designated pixel. The camera parameter storage means 420 and the camera parameter storage means 411 may be shared.

A processing example of the actual size calculation means 540 will be described with reference to FIG. The XYZ space 100 in FIG. 5 is a virtual space that imitates a monitoring space. The XY plane represents a horizontal plane such as the ground and floor in the monitoring space. Further, FIG. 5 shows a horizontal plane 101 at the height of the center of the head (for example, 1.5 m) of the object model 102 imitating a person of standard height.
_{The size conversion means 54 uses camera parameters to calculate the intersection P 0} (X ₀ , Y ₀ , Z ₀ ) at which the line-of-sight vector connecting the optical center of the photographing unit 2 and the designated pixel intersects the plane 101. .. _{Similarly, the size conversion means 54 uses the camera parameters to make the intersections Pr} (X _r , Y _r , Z _r ) and P _b (X _b ) of the pixel to the right of the specified pixel and the pixel below it, respectively. , Y _b , Z _b ) are calculated. Actual size corresponding to the designated pixel can be approximated by a parallelogram area vector is made from the vector and intersection point P ₀ from the intersection P ₀ to the intersection P _r to the intersection point P _b. The size conversion means 54 calculates and outputs | (X _r −X ₀ ) (Y _b −Y ₀ ) − (Y _r −Y ₀ ) (X _b −X ₀ ) |.

As shown in FIG. 4B, the size conversion means 54 is provided by the actual size storage means 421 that stores the actual size of the pixel calculated in advance in association with each pixel of the captured image. It may be configured. In this case, the actual size storage means 421 outputs the actual size stored corresponding to the pixels designated by the model image generation means 51.

Here, the density indicated by the density distribution is an estimated value and includes an error. Therefore, it is preferable to estimate the position of the object by generating a model image with a plurality of arrangements with a width in the arrangement number and selecting a model image most similar to the photographed image from them.
Therefore, the model image generation means 51 allocates a plurality of different number of object models within a range of estimation error assumed in the density distribution and sets a plurality of different arrangements.

The density distribution output by the density estimation means 50 described above is represented by an estimated density having a width in itself, and is an expression including an estimation error. That is, the range of estimation error of the low density, medium density, and high density classes ^{is higher than 0 person / m 2} and 2 people / m ² or less, and ^{higher than 2 people / m 2} and 4 people / m ² or less, 4 people / m ^2. It can be higher, 8 people / m ² or less. However, the upper limit of the high-density class is 8 people / m ² . Model image generating means 51, d _i lower place number P when applied to, P when applied to d _i of equation (1) the upper limit of each class of the formula (1) the lower limit of each class Can be set to the maximum number of placements.

For example, the model image generation means 51 has a lower limit arrangement number of 4 (= 0.01 × 2 × 100 + 0.01 × 4 × 50) for a change region consisting of 100 pixels in the medium density class and 50 pixels in the high density class. The upper limit number of arrangements is calculated to be 8 (= 0.01 × 4 × 100 + 0.01 × 8 × 50), and a model image in which 4 object models are arranged, a model image in which 5 object models are arranged, ..., 8 Generate a model image in which individual object models are arranged. However, in this example, for the sake of simplicity, it is uniformly set to 0.01 m ² per pixel.

In addition, the estimation error is overestimated, and the range of estimation error of low density, medium density, and high density class is 0 person / m ² or more and 3 person / m ² or less, 2 person / m ² or more and 5 person / m ^{2 respectively.} Hereinafter, it may be 4 people / m ² or more and 8 people / m ^{2 or less.}

As described above, the size conversion means 54 converts the size of the local region in the captured image into the actual size in the estimation target space. Then, the model image generation means 51 sets an arrangement area composed of a plurality of local areas on the captured image, and determines the number to be allocated to the arrangement area from the actual size and density distribution of the local area included in the arrangement area. .. Therefore, it is possible to reduce the amount of processing and more accurately prevent erroneous estimation due to the application of an object model larger than the original number of objects.
At that time, the model image generation means 51 allocates a plurality of different number of object models within the range of the estimation error assumed in the density distribution and sets a plurality of different arrangements. Therefore, even if the density distribution includes an estimation error, it is possible to set a plurality of arrangements including the arrangement with the original number of objects, and it is possible to prevent erroneous estimation of the position of the object.

Next, the similarity will be described.

The optimum placement estimation means 52 calculates the similarity of the shape of the object model drawn on the model image according to the high degree of fit (shape fit) with respect to the placement region according to the following equation.
Similarity = shape goodness of fit- ( _WH x concealment + W _N x number of placement estimation error) (2)
However, _WH and W _N are weighting factors greater than 0 and are preset based on prior experiments. The value subtracted from the shape conformity corresponds to the penalty value.

The shape conformity can be, for example, the degree of overlap between the change region extracted from the captured image and the region (model region) in which the object model is drawn in the model image. In that case, by subtracting the non-overlapping degree of the arrangement area and the model area from the overlap degree, it is possible to obtain a more reliable shape conformity in consideration of the protrusion of each area. That is, the optimum arrangement estimating means 52 calculates the shape conformity according to the following equation.
Shape Goodness of Fit = Multiplicity-Non-Multiplicity
= (Area of overlapping area-Area of non-overlapping area)
/ (Area of change area + Area of model area-Area of overlapping area) (3)

FIG. 6 is a diagram schematically showing an example of the shape conformity calculation process.
The difference image 200 including the information of the change region 201 is input from the change region extraction means 53 to the optimum arrangement estimation means 52, and the model image 210 in which the images 211 of the six models are drawn from the model image generation means 51 is input. Has been done. The sum region of the model image 211 is the model region. Image 220 shows that the optimum arrangement estimation means 52 superimposes the change region 201 and the model region. The white part in the figure is the overlapping area 221, the part marked with a horizontal line is the non-overlapping area 222 on the model area side, and the part marked with diagonal lines is the non-overlapping area 223 on the changing area side. The sum of the non-overlapping region 222 and the non-overlapping region 223 is the non-overlapping region of the equation (3).
The optimum arrangement estimating means 52 counts the number of pixels in the overlapping area 221 and the non-overlapping area, the changing area 201, and the model area to obtain the area of each area, and applies these areas to the equation (3) to obtain the shape goodness of fit. calculate.

More preferably, the goodness of fit is calculated by weighting the multiplicity and the non-overlapping by the difference value in each pixel. That is, the optimum arrangement estimating means 52 can also calculate the shape goodness of fit according to the following equation.
Shape Goodness of Fit = Multiplicity-Non-Multiplicity
= Sum of difference values in overlapping areas-Sum of difference values in non-overlapping areas
/ (Area of change area + Area of model area-Area of overlapping area) (4)
When the change area extraction means 53 extracts the change area by the background correlation process, a value obtained by subtracting the correlation value from 1.0 is used instead of the difference value.

Alternatively, the shape conformity can be the edge similarity between the captured image and the model image. In that case, the optimum arrangement estimation means 52 inputs a captured image from the image acquisition means 30, and extracts edges from each of the captured image and the model image. Then, the optimum arrangement estimating means 52 calculates, for example, the absolute value of the difference between the pixel edge extraction result of the pixel of the captured image corresponding to each pixel from which the effective edge is extracted from the model image, and sums them up to calculate the total value. The edge similarity is calculated by dividing by the number of pixels used in.

Further, the difference between the degree of overlap and the degree of non-overlap and the weighted sum of the edge similarity can be used as the shape goodness of fit.

Next, the degree of concealment included in the penalty term of similarity will be described.

If the overlap between the object models in the model image becomes too large, the change in the shape suitability with respect to the increase or decrease in the number of arrangements becomes small, which causes an erroneous estimation of the position of the object. The degree of concealment is the degree of overlap between object models in a model image, and is a measure for suppressing excessive overlap of object models.

The model image generation means 51 calculates the degree of concealment representing the degree of overlap between the object models in the model image for each arrangement according to the following equation, and the optimum arrangement estimation means 52 calculates the similarity for each arrangement as shown in the equation (2). The larger the degree of concealment in the arrangement, the lower the calculation.
Concealment = Area of overlapping area between models / Area of sum area of model area (5)

FIG. 7 is a diagram showing how the degree of concealment is calculated for the model image 210 of FIG.
The image 300 is an image showing the overlap of the six model regions. The area in which the areas 301, 302, and 303 shown by diagonal lines are combined is the overlapping area between the models.
Further, the image 310 is an image representing the logical sum of the six model regions. The area in which the areas 311, 312, and 313 shown by diagonal lines are combined is the sum area of the model area.
The model image generation means 51 counts the number of pixels in the areas 301, 302 and 303 to obtain the area of the overlapping area between the models, and counts the number of pixels in the areas 311, 312 and 313 to obtain the sum of the model areas. Let the area be an area, and apply those areas to the equation (5) to calculate the degree of concealment.

By doing so, the optimum arrangement can be estimated based on the degree of similarity including the degree of concealment, and erroneous estimation due to the application of an object model larger than the original number of objects can be prevented more accurately.

Next, the estimation error of the number of arrangements included in the penalty term of similarity will be described.

The number of arrangements in each model image is a number set based on the estimated value of the density, and includes an estimation error of the density. The estimated error of the number of arrangements is a measure of the degree of estimation error of the density distribution included in the number of arrangements in each model image.

The optimum arrangement estimating means 52 calculates the similarity for each arrangement to be lower as the estimation error corresponding to the number of arrangements in the arrangement is larger.

ただし、変化領域に含まれる画素の集合をＲ、集合Ｒの中の任意の一画素をｉ、画素ｉの実サイズをａ_ｉ、画素ｉにおける密度の代表値をｃ_ｉとしている。低密度クラスの密度の代表値は１人／ｍ^２、中密度クラスの密度の代表値は３人／ｍ^２、高密度は６人／ｍ^２などとすることができる。αは０よりも大きな定数であり、事前実験に基づいて予め定められる。 Specifically, the optimum placement estimation means 52 calculates, for example, an estimation error of the number of placements according to the following equation.
Estimation error of the number of placements = -exp {-α (representative value of the number of placements-number of placements) ² } (6)

However, the set of pixels included in the change region R, is an arbitrary one pixel i, the actual size a _i of the pixel _i, the representative value of the density at pixel i c _i in the set R. The representative value of the density of the low density class can be 1 person / m ² , the representative value of the density of the medium density class can be 3 people / m ² , and the high density can be 6 people / m ² . α is a constant greater than 0 and is predetermined based on prior experiments.

By doing so, the optimum placement can be estimated based on the similarity including the estimation error of the number of placements, so that the object model that deviates from the density distribution and exceeds the original number of objects is applied. It can be prevented, and misestimation of the position of each object can be prevented more accurately.

As described above, the optimum arrangement estimating means 52 outputs the object position information from the position of the object model indicated by the arrangement (optimum arrangement) having the highest degree of similarity among the plurality of arrangements.
For example, the optimum placement estimation means 52 generates and outputs a distribution image in which each of the object models drawn on the model image of the optimum placement is color-coded according to the density class as information on the object position that is easily visible to the observer. ..

The information on the object position may be the object position itself indicated by the optimum arrangement. Alternatively, the object position information may be a region of each object model in the optimum arrangement in the optimum arrangement that does not overlap with other object models. Alternatively, the object position information may be data including two or more of the above-mentioned data.

The object position output means 31 sequentially outputs the object position information input from the optimum placement estimation means 52 to the output unit 6, and the output unit 6 displays the information input from the object position output means 31. For example, the information on the position of the object is transmitted and received via the Internet and displayed on the output unit 6. By visually recognizing the displayed distribution image, the observer can quickly grasp the point where congestion is occurring in the monitoring space and the state of that point, and take measures such as dispatching or increasing the number of guards to that point. Do.

<Operation of image monitoring device 1>
The operation of the image monitoring device 1 will be described with reference to the flowcharts of FIGS. 8 to 10.

The image monitoring device 1 is activated when the event venue is unmanned, and when the image monitoring device 1 starts operating, the photographing unit 2 installed at the event venue photographs the monitoring space at predetermined time intervals and captures captured images. The images are sequentially transmitted to the image analysis center in which the image processing unit 5 is installed. Each time the image processing unit 5 receives the captured image, the image processing unit 5 repeats the operation according to the flowchart of FIG.

First, the communication unit 3 operates as the image acquisition means 30, and is in a state of waiting for reception of the captured image from the photographing unit 2. Then, the image acquisition means 30 that has acquired the captured image outputs the captured image to the image processing unit 5 (step S1).

The image processing unit 5 to which the captured image is input operates as the change area extraction means 53, reads the background image from the background image storage means 40 of the storage unit 4, compares it with the photographed image, and extracts the change area (step S2). .. However, immediately after the start-up, the change area extraction means 53 omits the extraction of the change area, and stores the captured image as the background image in the background image storage means 40.

If the change region is not extracted (NO in step S3), the processes of steps S4 to S8 are omitted. At this time, the change area extraction means 53 replaces the background image of the background image storage means 40 with a captured image in which the change area has not been extracted.

When the change region is extracted (YES in step S3), the image processing unit 5 also operates as the model image generation means 51 and the optimum arrangement estimation means 52, and the change region extraction means 53 changes to the model image generation means 51. Information is input, and the difference value information is input from the change area extraction means 53 to the optimum arrangement estimation means 52. The model image generation means 51 and the optimum arrangement estimation means 52 hold the information, and the process proceeds to step S4. Further, the image processing unit 5 operates as the density estimation means 50, and the captured image is input to the density estimation means 50. Further, the change area extraction means 53 updates the background image using the captured image.

The density estimation means 50 in which the captured image is input scans the captured image with the density estimator and estimates the density distribution (step S4). The image processing unit 5 that estimates the density distribution also operates as the model image generation means 51, and the density distribution is input from the density estimation means 50 to the model image generation means 51.

The model image generation means 51 in which the density distribution is input sequentially sets the holding change region to the attention change region (step S5), and controls the optimum arrangement estimation (step S6) of the attention change region.

The optimum arrangement estimation of the attention change region will be described with reference to the flowchart of FIG.

First, the image processing unit 5 also operates as the size conversion means 54, and the model image generation means 51 designates each pixel in the attention changing region as the size conversion means 54 and calculates the actual size of each pixel (step S60). ..

Next, the model image generation means 51 determines the range of the number of arrangements with respect to the attention change region (step S61). Model image generating unit 51 calculates the number of lower limit placement by substituting a _i and d _i in the formula the lower limit of the density class of the actual size and each pixel of each pixel in the target change area, respectively (1). Further, the model image generating unit 51 calculates the maximum number of placement by substituting a _i and d _i, respectively and the upper limit value of the density class of the actual size and each pixel of each pixel in the target change area (1) ..

Further, the model image generating unit 51 substitutes the representative value of the density class of the actual size and each pixel of each pixel in the target change area to a _i and c _i of each formula (7), the arrangement number of the representative value Is calculated (step S62).

Subsequently, the model image generation means 51 sequentially sets an integer value within the range set in step S61 to the number of arrangements (step S63), and performs the loop processing of steps S63 to S73.

The model image generation means 51 prepares a counter T for counting the number of iterations, initializes it to 0 (step S64), and starts controlling the iteration process.

The model image generation means 51 assigns positions to the object model by randomly setting the same number of positions in the attention change region as the number of arrangements set in step S63 (step S65).

The model image generation means 51 prepares a model image having the same size as the captured image, and draws an object model at each position of the model image set in step S65 (step S66). The model image generation means 51 reads out the camera parameters from the camera parameter storage means 411 and the three-dimensional model from the three-dimensional model storage means 410 with reference to the object model storage means 41 of the storage unit 4. The model image generation means 51 converts each position into a position in the virtual space using camera parameters, arranges a three-dimensional model at each converted position, and uses camera parameters to arrange a three-dimensional model in a virtual space as a model image. The object model is drawn by rendering to.

Further, the model image generation means 51 obtains the area of the overlapping area between the drawn object models and the area of the sum area of the drawn object models, substitutes them into the equation (5), and generates the model image in step S66. The degree of concealment in is calculated (step S67). The image processing unit 5 also operates as the optimum placement estimation means 52, and the model image generation means 51 transfers the model image, the overlapping area between the object models, and the non-overlapping area excluding the overlapping area from the sum area of the object model to the optimum placement estimation means 52. The area, the number of arrangements, the position of each object model, the number of representative arrangements, and the degree of concealment are input.

The optimum arrangement estimation of the attention change region will be described continuously with reference to the flowchart of FIG.

The optimum arrangement estimation means 52 substitutes the input number of arrangements and the representative number of arrangements into the equation (6) to calculate the estimation error of the number of arrangements (step S68).

Further, the optimal placement estimation means 52 uses the input overlapping area, non-overlapping area, and difference value to obtain the total sum of the difference values in the overlapping area and the total difference value in the non-overlapping area, and formulates these in the equation (4). Substituting and calculating the shape conformity (step S69).

Then, the optimum placement estimation means 52 substitutes the shape goodness of fit calculated in step S69, the estimation error calculated in step S68, and the input concealment degree into the equation (2), so that the model image generated in step S66 is generated. The similarity with respect to the model image is calculated, and the model image, the position of each object model, and the similarity are associated with each other and stored in the storage unit 4 (step S70).

Thus the similarity of the model image is calculated, the model image generating means 51 increases the number of iterations T by 1 (step S71), comparing the predetermined number of times T _MAX which defines its value in advance (step S72) ..

If the number of iterations T is less than a specified number _{T MAX} (NO in step S72), the model image generation means 51 repeats the iterative process returns the process to step S65.

On the other hand, if the number of iterations T has reached a predetermined number T _MAX (YES in step S72), the model image generation means 51 checks whether the set of all the arrangement number in the range set in step S63 (step S73).

If there is an arrangement number that has not yet been set (NO in step S73), the model image generation means 51 returns the process to step S63 and performs processing with the next arrangement number.

On the other hand, when the total number of arrangements has been set (YES in step S73), the model image generation means 51 notifies the optimum arrangement estimation means 52 that the generation of a plurality of types of model images for the change region of interest has been completed. ..

Upon receiving this notification, the optimum placement estimation means 52 determines the placement with the maximum similarity (step S74). The optimum arrangement estimating means 52 selects the maximum value from the similarity recorded in step S70, and stores the model image associated with the selected similarity and the position of each object model as the object position in the attention change region. Store in part 4. The optimum arrangement estimating means 52 clears the information recorded in step S70, and proceeds to the process in step S7 of FIG.

This will be described again with reference to FIG. The model image generation means 51 confirms whether or not all the changed regions held by the model image generation means 51 have been processed (step S7). When there is a change region that has not been processed yet (NO in step S7), the model image generation means 51 returns the processing to step S5 and performs the processing of the next change region. When the processing of the next change area is performed, the information of the object position in the change area is added in step S74.

On the other hand, when the processing of all the change regions is completed (YES in step S7), the optimum placement estimation means 52 outputs the object position information recorded in step S74 to the communication unit 3 (step S8). The communication unit 3 to which the object position information is input operates as the object position output means 31, and transmits the object position information to the output unit 6. The output unit 6 transmits information to the observer by displaying information on the position of the object.

When the above processing is completed, the processing is returned to step S1, and the processing for the next captured image is performed.

<Modification example>
(1) In the above embodiment, an example in which the object to be estimated is a human is shown, but the object is not limited to this, and the object to be estimated can be a vehicle, an animal such as a cow or a sheep, or the like.

(2) In the above-described embodiment and its modification, the photographing unit 2 shows an example of one camera, but the photographing unit 2 can be configured by a plurality of cameras having a common field of view. In that case, the background image of each camera is stored in the background image storage means 40, the camera parameters of each camera are stored in the camera parameter storage means (not shown) referred to by the change area extraction means 52, and the change area extraction means 52 extracts a change area for each camera. Further, the camera parameters of each camera are stored in the camera parameter storage means (not shown) referred to by the density estimation means 50, and the density estimation means 50 estimates the density distribution for each camera. Further, the camera parameter storage means 411 and 420 store the camera parameters of each camera, the size conversion means 54 calculates the actual size for each camera, and the model image generation means 51 calculates the range of the number of arrangements for each camera. .. Then, the model image generation means 51 determines the range of the number of arrangements based on the minimum number of arrangements and the maximum number of arrangements, and arranges the object models for each arrangement in the virtual space. The model image generation means 51 generates a model image for each camera by rendering this using the camera parameters of each camera, and calculates the degree of concealment for each camera. Then, the optimum placement estimation means 52 calculates the shape similarity, the estimation error of the number of placements, and the similarity for each camera, and determines the optimum placement from the model image having the maximum sum of the similarities of all the cameras. By doing so, it is possible to comprehensively determine the optimum arrangement based on the degree of similarity from a plurality of viewpoints in which the concealment state of the object is different, and the estimation accuracy of the object position is improved.

(3) In the above-described embodiment and each modification thereof, an example in which the fixed-field photographing unit 2 is used is shown, but the variable-field photographing unit 2 can also be used. In that case, the photographing unit 2 outputs the camera parameters after the field change to the image processing unit 5 via the communication unit 3, and the image processing unit 5 changes the camera parameter storage means (not shown) referred to by the density estimation means 50. The camera parameters of the camera parameter storage means (not shown) and the camera parameter storage means 411,420 referred to by the area extraction means 53 are changed to the input camera parameters. In that case, the change area extraction means 53 generates a background image by rendering the environment model.

(4) In the above-described embodiment and its modification, the model image generation means 51 sets the change area as the arrangement area, but the model image generation means 51 sets the area other than the background class as the arrangement area. You can also do it. That is, the model image generation means 51 sets a region in which the density larger than 0 is estimated in the density distribution as the arrangement region. Specifically, the model image generation means 51 sets a region consisting of pixels whose estimated densities are estimated to be "low density" class, "medium density" class, and "high density" class as the arrangement region. When an area other than the background class is set as the arrangement area, the background image storage means 40 and the change area extraction means 53 can be omitted.

ただし、撮影画像の総画素数をＮ、撮影画像中の任意の一画素をｉ、画素ｉにて算出した窓内配置数をｍｉ、画素ｉにおける密度の代表値をｃｉとしている。背景クラスの密度の代表値は０人／ｍ^２、低密度クラスの密度の代表値は１人／ｍ^２、中密度クラスの密度の代表値は３人／ｍ^２、高密度は６人／ｍ^２などとすることができる。
或いは、最適配置推定手段５２は、画素ごとに窓内配置数と対応する密度クラスの値を求めて配置数の推定誤差を算出してもよい。例えば、或る配置において或る画素に対応して設定した推定用抽出窓内の物体モデルの位置が２個であれば当該画素の値は「低密度」クラスを示す値となる。最適配置推定手段５２は、こうして求めた画素ごと密度クラスの値を密度分布において対応する画素の値と比較し、同一値でない画素数を総画素数で除した値を配置数の推定誤差として算出する。 (5) In the above-described embodiment and each modification thereof, an example of calculating the estimation error of the number of arrangements according to the equations (6) and (7) is shown, but the difference between the density distribution and the arrangement is estimated by the number of arrangements. It can also be calculated as an error. In that case, the density estimation means 50 also outputs the density distribution to the optimum arrangement estimation means 52. The optimum arrangement estimation means 52 sets an estimation window for each pixel in each arrangement, counts the positions of the object models in the estimation extraction window, and obtains the number of arrangements in the window. The optimum arrangement estimation means 52 allocates a value obtained by dividing the sum of the differences between the number of arrangements in the window for each pixel and the representative value of the density of the corresponding pixels in the density distribution by the total number of pixels for each arrangement according to the following equation. Calculated as a number estimation error.

However, the total number of pixels in the captured image is N, any one pixel in the captured image is i, the number of arrangements in the window calculated by pixel i is mi, and the representative value of the density in pixel i is ci. The representative value of the density of the background class is 0 person / m ² , the representative value of the density of the low density class is 1 person / m ² , the representative value of the density of the medium density class is 3 people / m ² , and the representative value of the density is 6 people / m 2. It can be m ² or the like.
Alternatively, the optimum arrangement estimation means 52 may calculate the estimation error of the arrangement number by obtaining the value of the density class corresponding to the number of arrangements in the window for each pixel. For example, if there are two positions of the object model in the estimation window set corresponding to a certain pixel in a certain arrangement, the value of the pixel becomes a value indicating the "low density" class. The optimum placement estimation means 52 compares the value of the density class for each pixel thus obtained with the value of the corresponding pixel in the density distribution, and calculates the value obtained by dividing the number of pixels that are not the same value by the total number of pixels as the estimation error of the number of placements. To do.

(6) In the above-described embodiment and each modification thereof, an example in which the number of arrangements is set based on the distribution density and the actual size is shown, but the number of arrangements is exploratoryly set based on the above-mentioned estimation error of the number of arrangements. You can also do it. For example, the model image generation means 51 performs loop processing of the number of temporary arrangements from 1 to a predetermined upper limit number (for example, the number of three-dimensional models that can be arranged without overlapping in the virtual space corresponding to the captured image). set the temporary placement of T _MAX street assigns a random position of the placement area each arrangement number of the object model, for each temporary arrangement of T _MAX as the formula (6) or formula (8), such as The estimation error is calculated and compared with a predetermined threshold value ε, and a model image is generated in a temporary arrangement in which the estimation error is equal to or less than the threshold value ε. By doing so, the model image generation means 51 assigns positions in the arrangement area to the number of object models according to the density distribution, sets a plurality of arrangements, and draws the object model in the plurality of arrangements. Can generate a model image.

(7) In the above-described embodiment and each modification thereof, the GLCM features are exemplified as the feature amount learned by the density estimator and the estimation feature amount extracted by the density estimation means 50, but these are replaced with the GLCM features. It can be a variety of features such as Local Binary Pattern (LBP) features, Haar-like features, brightness patterns, HOG (Histograms of Oriented Gradients) features, or GLCM features. And a plurality of these can be combined into a feature quantity.

(8) In the above embodiment and each modification thereof, the density estimator learned by the multi-class SVM method is illustrated, but instead of the multi-class SVM method, a decision tree type random forest method and multi-class AdaBoost are used. Various density estimators such as density estimators learned by the (AdaBoost) method or the multiclass logistic regression method can be used.
Alternatively, the density estimation means 50 can be realized by using a method such as CNN (Convolutional Neural Network) that expresses the feature quantity extraction process and the estimation process by the density estimator in one network.

(9) In the above embodiment and modifications, the model image generating means 51, an example of calculating a representative value c _i of the arrangement number based on the density estimates of each pixel output from the density estimating means 50 was, but outputs a score for each class density estimation unit 50 calculates in the course of the estimated in addition to the density estimate model image generating unit 51 corrects the representative value c _i of the number of arranged based on these scores You can also do it.
The class score is the background score, which indicates the plausibility of being the "background" class of the image from which the estimation features are extracted and the "background" class of the other classes, the "low density" class and other classes. Low density score, which indicates the plausibility of being in the "low density" class, medium density score, which indicates the plausibility of being in the "medium density" class and other classes. It is a high-density score that represents the plausibility of being a "high-density" class among the "high-density" class and other classes. By the way, the class showing the highest score among these is the density estimate.
Model image generating means 51 is high the higher the height of the score of the class indicated by the density estimate, correcting the representative value c _i of the lower the low number of arrangement.

(10) In the above-described embodiment and each modification thereof, the density class estimated by the density estimator is set to 4 classes, but the classes may be further divided.

(11) In the above embodiment and each modification thereof, an example is shown in which the density estimation means 50 uses a density estimator that classifies into multiple classes, but instead, a regression that returns the density value from the feature amount. It can also be a mold density estimator. That is, it can be a density estimator that learns the parameters of the regression function for obtaining the density from the features by the ridge regression method, the support vector regression method, the random forest method of the regression tree type, or the like.
In that case, the model image generating unit 51 uses the value of the density of the output density estimators as a representative value c _i of the arrangement number. Further, the model image generating section 51, a number of lower and placement (c i _-2) (if the proviso below 0 0), such as the maximum number and placement (c i _+2), density estimator outputs The lower limit arrangement number and the upper limit arrangement number obtained by adding or subtracting a predetermined value to the density value are used.

(12) In the above-described embodiment and each modification thereof, an example in which the model image generating means 51 randomly assigns a position to the object model each time it is repeated is shown, but the position is slightly deviated from the previous position. May be assigned, or the position may be probabilistically searched by the MCMC (Markov chain Monte Carlo) method with reference to the similarity to the previous arrangement, or the position may be sequentially improved by the mountain climbing method. May be good.

1 ... Image monitoring device 2 ... Imaging unit 3 ... Communication unit 4 ... Storage unit 5 ... Image processing unit 6 ... Output unit 30 ... Image acquisition means 31 ... Object Position output means 40 ... Background image storage means 41 ... Object model storage means 50 ... Density estimation means 51 ... Model image generation means 52 ... Optimal placement estimation means 53 ... Change area extraction means 54 ... Size conversion means 410 ... Three-dimensional model storage means 411 ... Camera parameter storage means 412 ... Model image storage means 420 ... Camera parameter storage means 421 ... Actual size storage means 540 ...・ Actual size calculation method

Claims (7)

It is an object position estimation device that estimates the position of each of the objects from the captured image in which the estimation target space where congestion may occur due to a predetermined object is captured.
The distribution of the density of the object photographed in the photographed image is estimated by using a density estimator that has learned in advance the characteristics of the density image obtained by photographing the space in which the object exists at the predetermined density for each predetermined density. Density estimation means and
An object model storage means that stores an object model that imitates the object,
The arrangement area is set in the captured image, and the positions in the arrangement area are assigned to the number of the object models according to the distribution to set a plurality of arrangements, and the object model is arranged in the plurality of arrangements. A model image generation means that draws and generates a model image,
Optimal placement estimation means that calculates the similarity between the model image and the captured image of each of the plurality of arrangements and outputs the arrangement having the highest degree of similarity among the plurality of arrangements.
An object position estimation device characterized by being equipped with. The model image generation means allocates a plurality of the number of the object models within the range of the estimation error assumed in the distribution and sets the plurality of arrangements.
The optimum arrangement estimation means calculates the similarity for each arrangement by lowering it as the estimation error corresponding to the number in the arrangement becomes larger.
The object position estimation device according to claim 1. The model image generation means calculates the degree of overlap between the object models in the model image for each arrangement.
The optimum arrangement estimating means calculates the similarity for each arrangement by lowering the degree of overlap in the arrangement.
The object position estimation device according to claim 1 or 2. Further provided with a size conversion means for converting the size of the local region in the captured image into the actual size in the estimation target space.
The density estimation means estimates the density for each local region and estimates the density.
The model image generation means determines the number to be allocated to the arrangement region from the actual size and the density of the local region included in the arrangement region.
The object position estimation device according to any one of claims 1 to 3. A change area extraction means for comparing the photographed image with the background image of the estimation target space and extracting a change area in the photographed image that differs from the background image by a predetermined reference or more is further provided.
The model image generation means sets the change area in the arrangement area.
The object position estimation device according to any one of claims 1 to 4. The model image generation means sets a region in which the density is estimated to be greater than 0 in the distribution as the arrangement region.
The object position estimation device according to any one of claims 1 to 5. The object model storage means stores the object model that imitates the shape of the object, and stores the object model.
The optimum placement estimation means calculates the similarity of the shape of the object model drawn on the model image according to the high degree of conformity with respect to the placement region.
The object position estimation device according to any one of claims 1 to 6.

Images