JP4675368B2 - Object position estimation apparatus, object position estimation method, object position estimation program, and recording medium recording the program - Google Patents

Description

  The present invention can be used for a human walking photographed image captured by a fixed camera such as a surveillance camera, a time-series image captured of a moving object such as a vehicle or a pedestrian, and the target from a time-series image acquired by the camera. The present invention relates to an object position estimation device, an object position estimation method, an object position estimation program, and a recording medium on which the program is recorded.

  As a technique to measure the crowded state from video captured by a surveillance camera, a person area is conventionally extracted using image processing such as background difference, and correction according to the number of pixels constituting the person and the depth of the space There is a method for measuring the degree of congestion in the monitoring area by performing the above. On the other hand, there is a method of improving the accuracy of calculating the degree of congestion by three-dimensionally measuring the approximate position of a person using a stereo camera. Stereo cameras facilitate the search between images by setting the optical axes almost in parallel, and the depth of the point can be determined by the principle of triangulation if the correspondence between pixels is processed accurately. Furthermore, if the depth from the camera to each object is known, it can be expected to solve the occlusion problem. Alternatively, a method of detecting a position of an object by photographing a monitoring area using a plurality of cameras, associating a target image or the target area with a three-dimensional space, and the like is disclosed (see Patent Documents 1 and 2).

  On the other hand, the usefulness of a particle filter (see Non-Patent Document 1) has been reported as a new application for person tracking in surveillance video. The particle filter expresses the tracking target as a discrete probability density by a large number of hypotheses with state quantities and likelihoods, and is robust against movement fluctuations and observation noise by propagating using a state transition model. Used to realize tracking.

As a technique related to the present invention, extracting an object region from an image using a Gaussian mixture distribution is described in Non-Patent Document 2, for example.
JP 2003-331263 A JP 2004-46464 A Tomoyuki Higuchi, “Particle Filter”, Journal of IEICE, Vol. 88, no. 12, pp. 989-994, 2005. C. Stauffer & W. E. L Grimson, “Adaptive background mix models for real-time tracking”, Proceeding of International Conferencing on Computer Vision and Pattern Recognition. 2, pp. 246-252, 1999.

  Since the monocular camera has indefiniteness in the depth direction, there is no guarantee that the measurement of the degree of congestion according to the number of pixels is accurate. Multi-camera shooting such as a stereo camera can be considered to eliminate the depth ambiguity, but the area captured by surveillance cameras such as public plazas, stations, department stores, etc. is wide-area shooting on many people monitoring There are many. In other words, in a situation where a slightly distant area is monitored at a low angle, a person is imaged small on the screen, so it is unclear whether or not a feature point with sufficient accuracy for stereo vision can be observed. The depth value estimated that the corresponding point is not accurate is low in reliability. In order to detect an object position by associating a camera image with a three-dimensional space, a back projection method (Patent Document 2) of an image (2D) onto a voxel space (3D) has been proposed, but the calculation cost is high. In addition, in a plurality of cameras, the object position may be dispersed in the voxel space.

  On the other hand, 3D tracking using a particle filter is considered a promising method for monitoring video over a wide area, but there are many processes that depend on computer power, and the load is high when the resources of the computer that processes the monitoring video are scarce. Is big.

  The object of the present invention is to solve the above-mentioned problems, and an object of the present invention is to estimate an object position from a monitoring image at a high speed and efficiently even in a computer having a small memory and a low processing speed. The object is to provide an apparatus, an object position estimation method, an object position estimation program, and a recording medium recording the program.

The object position estimation apparatus according to claim 1 for solving the above-described problem is to extract a target area from an image obtained by using one or a plurality of image input devices such as a camera, and to obtain a space from the target area. An object position estimation device for estimating a position in an object, wherein the object region extraction unit extracts the target region from each time-series image acquired by each of the image input devices in an image processing manner, and the object region extraction unit Which pixel of the image obtained from the image input device is arranged in which spatial position, which projection of the three-dimensional model prepared for each pixel corresponding to each image input device extracted by By using a reference table that indicates whether the projected 3D model is projected, and if there is a projected 3D model, vote on the spatial position of the model. A space map generating means for generating a map indicating an increased likelihood of the presence of an object in place vote value is large, the vote value of maps generated by the spatial map generating means detects the position at which the maximum point A circular area having a radius R centered on the position of the object as an exclusive area, and a circular area having a radius R in an area of the map where the exclusive area is not set. Is set as another exclusive area, the vote values in the exclusive area are totaled for each exclusive area of the other exclusive area, and when the total value is equal to or greater than a predetermined value, And an object position estimating means for setting the center of another exclusive area as the position of the object.

  According to a second aspect of the present invention, there is provided the object position estimation apparatus according to the first aspect, wherein the reference table includes a plurality of three-dimensional models arranged at spatial positions on a certain reference coordinate system, and the position and orientation of the image input apparatus. And a table indicating the relationship between the image coordinates of the three-dimensional perspective projection image generated using the internal parameters unique to the image input apparatus and the arrangement position of the three-dimensional model. .

The object position estimation method according to claim 6 extracts a target area from an image obtained by using one or a plurality of image input devices such as a camera, and estimates a position in the space from the target area. An object position estimation method, wherein an object area extraction unit extracts an object area from each time-series image acquired by each of the image input devices in an image processing manner, and a reference table A reference table construction step in which the construction means constructs a reference table indicating in which spatial position each pixel of the image obtained from each image input device is projected; and a spatial map generation means Which projection extracted from the three-dimensional model prepared for each pixel corresponding to each image input device extracted by the object region extraction step, If there is a projected 3D model using the reference table, and voting for each model's spatial position, it is highly possible that the object exists in a place with a high vote value. A spatial map generating step for generating a map representing the object, and an object position estimating unit for estimating the position of the object based on a vote value of the map generated by the spatial map generating step , The position estimation step includes a first sub-step in which a position where the vote value of the map generated by the spatial map generation step is a maximum point is detected as a position of the object, and a radius centered on the position of the object A second sub-step for setting an R circle area as an exclusive area, and a circular area with a radius R in another area of the map where the exclusive area is not set; A third sub-step for setting the target area as a target area and totaling the vote values in the other exclusive area; and if the total value is equal to or greater than a predetermined value, the center of the other exclusive area is targeted And a fourth sub-step as an object position, and the second to fourth sub-steps are repeated for the entire map .

The object position estimating method according to claim 7 according to claim 6, wherein the lookup table construction step, a plurality of three-dimensional model located in spaces located on one reference coordinate system, the position of the image input device A perspective projection image of the three-dimensional shape is generated using the posture and internal parameters specific to the image input apparatus, and each image coordinate of the perspective projection image is associated with the arrangement position of the three-dimensional model. Yes.

  According to the above configuration, the position of a moving object such as a vehicle or a person is estimated from all time-series images acquired using an image input device, for example, a camera, without performing depth correction according to the number of pixels as in the past. can do.

  Further, when a plurality of cameras are used, the spatial map distribution indicating the object position becomes more accurate, the influence of noise can be reduced, and the object position estimation accuracy can be improved.

  Further, since an object region is extracted from each image and only voting processing by referring to the reference table is required, real-time processing by a commercially available personal computer is possible and processing cost can be reduced.

The object position estimation device according to claim 3 is the object position estimation device according to claim 1, wherein the object position estimation unit quantizes the generated space map in a lattice shape, and obtains each vote value in the quantized range. When is equal to or greater than a predetermined value, the quantized range is set as a place where the object exists.

The object position estimation method according to claim 8 is the object position estimation method according to claim 6 or 7 , wherein the object position estimation step quantizes the generated spatial map in a lattice shape, and obtains each vote value in the quantized range. When is equal to or greater than a predetermined value, the quantized range is set as a place where the object exists.

  According to the above configuration, it is possible to reduce the influence due to the error (noise) of the processing for obtaining the difference from the background image, which occurs at the time of object region extraction, for example.

According to a fourth aspect of the present invention, there is provided the object position estimation apparatus according to the third aspect , wherein the grid-like quantization in the space map is performed by a predetermined angle between a direction in which each image input device faces and an installation surface of the device. When the angle is smaller than the angle, the quantization in the direction in which the image input device faces is set coarsely.

The object position estimation method according to claim 9 is the object position estimation method according to claim 8 , wherein the grid-like quantization in the space map is such that an angle formed between a direction in which each image input device faces and an installation surface of the device is predetermined. When the angle is smaller than the angle, the quantization in the direction in which the image input device faces is set coarsely.

  According to the above configuration, when the angle with respect to the installation surface of the camera is small, the distribution of the spatial map may flow in the depth direction, but in this case, the number of estimated positions is prevented from increasing unnecessarily. Processing costs can be reduced.

According to a fifth aspect of the present invention , in the object position estimation device according to any one of the first to fourth aspects, with respect to a projection result on the image of the prepared three-dimensional model, an area projected on the image is used as a projection effective pixel. Effective pixel setting means for setting, the reference table is constructed using the projection effective pixels set by the effective pixel setting means, and the object region extraction means is set by the effective pixel setting means It is characterized by extracting only the area of effective projection pixels.

An object position estimation method according to a tenth aspect of the present invention is the object position estimation method according to any one of the sixth to ninth aspects, wherein the projection result on the image of the prepared three-dimensional model is defined as a projection effective pixel. An effective pixel setting step for setting; the reference table construction step is constructed using the projection effective pixels set by the effective pixel setting step; and the object region extraction step is set by the effective pixel setting step. Only the projected effective pixel area is extracted.

  According to the above configuration, a pixel region that is not projected can be excluded from the processing target, and a series of processing can be executed efficiently.

An object position estimation program according to an eleventh aspect is a program for causing a computer to execute each step according to any one of the sixth to tenth aspects.

A recording medium according to claim 12 is a computer-readable recording medium in which the object position estimation program according to claim 11 is recorded.

(1) According to the first to twelfth aspects of the present invention, the position of a moving object such as a vehicle or a person can be estimated from all time-series images acquired using an image input device, for example, a camera. In the case of using a monocular camera, the depth direction is indefinite, so the prior art had to correct the depth according to the number of pixels, but the present invention does not require any such correction and the position of the target object. It can be estimated as a spatial map representing.

  In the case of a plurality of cameras, a stereotype with a fixed optical axis and a base length are known and a multi-lens camera has been used in many cases. However, in the present invention, it can be easily applied to a plurality of cameras arranged appropriately. Can do.

  Conventionally, a back projection method to a voxel space has been used for a plurality of cameras. However, this method has a problem in that the calculation cost is high and the voxel space is diverged by handling many cameras. On the other hand, in the multi-view camera voting according to the present invention, the spatial map distribution indicating the object position becomes more accurate and the influence of noise can be reduced, so that the object position estimation accuracy can be improved.

In addition, 3D tracking using a conventional particle filter is computationally expensive and relies on computer power. However, the present invention extracts the object region from each image by creating and providing a model table in advance. Thus, the spatial map can be generated only by voting by referring to the table. In other words, even a commercially available personal computer can perform real-time processing at a video rate, and even when computer resources are insufficient, the processing cost in collation processing can be reduced and object position estimation processing can be performed. .
(2) According to the third and eighth aspects of the invention, it is possible to reduce the influence caused by, for example, processing error (noise) that takes a difference from the background image, which occurs at the time of object region extraction.
(3) According to the inventions described in claims 4 and 9 , when the angle with respect to the installation surface of the camera is small, there is a possibility that the distribution of the spatial map flows in the depth direction. Can be prevented from increasing, and the processing cost is reduced.
(4) According to the inventions described in claims 5 and 10 , a pixel region that is not projected can be excluded from the processing target, and a series of processing can be executed efficiently.

Hereinafter, embodiments of the present invention will be described with reference to the drawings, but the present invention is not limited to the following embodiments.
Example 1
First, mathematical expressions and notations used with a camera (image input device), a coordinate system in which a three-dimensional model is arranged, will be described. FIG. 11 shows the positional relationship between the camera and the three-dimensional model. It is assumed that the camera is calibrated in advance, and the position (X _i , Y _i , Z _i ) and orientation (rotation around the X axis Rx, around the Y axis) from the reference coordinate system XYZ with the point O in the space as the origin And the internal parameters (effective focal length f, image center (Cx, Cy), aspect ratio α, skew γ, etc.) are known. It is assumed that the XY plane is positioned as the bottom side (floor, ground, etc.) of the monitoring area, and that a person or an object moves on this plane. The monitoring area is Wx [m] × Wy [m], and the center is the origin O. This Wx × Wy area is divided into M equal parts in the X-axis direction at intervals of ΔX and N equal parts in the Y-axis direction at intervals of ΔY. In this plane, the three-dimensional model is positioned at positions (X _m , Y _n ), m = 1, 2,..., M, n = 1, 2,. Placed at an odd number) position.

The three-dimensional model is a three-dimensional ellipsoid having a center (X _m , Y _n , C) and radii A, B, C according to equation (2).

Here, for example, C is ½ of the average height of a person. On the other hand, the radii A and B are equal to or smaller than ΔX and ΔY, and need not necessarily be the size of a person. In the present invention, for example, the quantization numbers M and N are sufficiently increased to set A and B of the three-dimensional model to be small, and a fine three-dimensional model at a finely quantized spatial position (X _m , Y _n ). Handle.

  FIG. 1 is a basic configuration diagram in the case of a single image input apparatus according to the first and second aspects of the present invention. In FIG. 1, 1 is a time series image database in which time series images are stored, and 2 is an object area extraction unit (object area extraction means) for extracting an object or person area in each image in the database 1. Reference numeral 3 denotes a camera information data input unit that gives internal and external parameters of the camera. Reference numeral 4 denotes a perspective projection of a three-dimensional model that is arranged from the camera information data at a spatial position according to the equation (1) in the XYZ coordinate system. It is a model table construction unit that constructs a model table (reference table) for calculating and referring to an image. Reference numeral 5 denotes a spatial map generation unit (spatial map generation means) that obtains a spatial position of the object by performing collation processing from each pixel of the object region using the table, and 6 is an object position estimation that estimates the object position from the spatial map. Part (object position estimating means). Each unit in FIG. 1 is configured by a computer, for example.

  In this configuration, the time-series image database 1 may be in the form of using a recording medium such as a hard disk, a RAID device, a CD-ROM, or a remote data resource via a network.

  Further, FIG. 2 is a processing configuration diagram in the case of processing in real time, and the present invention does not necessarily require a storage device such as each database unit (time-series image database 1). 2 is different from FIG. 1 in that an image input unit 7 for capturing an image from a camera is provided instead of the time-series image database 1, and the other configuration is the same as that in FIG.

  FIG. 3 is a processing flow of this embodiment, and the operation of the apparatus of FIGS. 1 and 2 will be described with reference to this figure. First, generation of a model table in advance by giving camera information data will be described. Since the camera is calibrated in advance, the camera information data input unit 3 in FIGS. 1 and 2 uses external parameters (position and orientation) with respect to the reference coordinate system and camera-specific internal parameters (focal length, lens distortion, image center, etc.) ) And parameters A, B, and C are given to the three-dimensional model input (steps S1 and S2).

Next, in step S3, the indices m and n are incremented from 1 respectively, and the three-dimensional ellipse of equation (2) is obtained as a three-dimensional model at the position (X _m , Y _n ) according to equation (1). The sample points (X _j , Y _j , Z _j ), j = 1, 2,..., Q on the body surface are calculated (steps S2, S3). The number of sample points Q is set to a sufficiently large value (for example, set so densely that no space is generated in the ellipsoid when converted into a perspective projection image).

Next, in the perspective projection image generation in step S4, the image coordinates (x _j , y) are calculated by the equations (3) to (5) using the camera information data from each sample point (X _j , Y _j , Z _j ). _j ), j = 1, 2,..., Q are generated. Note that the image coordinates may be calculated by perspective projection calculation in consideration of lens distortion.

  Next, in step S5, a model table shown in FIG. 4 for associating the image coordinates with the model is constructed. FIG. 4 is an example of a model table when the image size is 640 × 480 pixels. Columns corresponding to pixels (1, 1) to (640, 480) pixels are prepared in order in the vertical direction, and columns corresponding to quantization numbers m in the X direction from 1 to M are sequentially prepared in the horizontal direction. The size of the model table is (640 × 480) × M.

First, all the array columns are initialized to 0 before the table construction. Next, the image coordinates (x _j , y _j ) obtained by the calculation according to the equations (3) to (5) are written in this table in the order of j = 1, 2,. That is, with respect to the perspective projection image (x _j , y _j ) from the point on the three-dimensional model placed at the position (X _m , Y _n ), the vertical row number (x _j , y _j ) and the horizontal column Write “n” in the column designated by the number m. This is performed for all the pixels (x _j , y _j ), j = 1, 2,..., Q, and after that, a three-dimensional model is placed at the next position (X _m , Y _n ). The image coordinates (x _j , y _j ), j = 1, 2,..., Q are obtained by the same perspective projection calculation and written in the same table. The table thus obtained is a model table indicating which model each pixel is associated with.

In the example of FIG. 4, the pixel (1, 1) is projected from a certain point on the three-dimensional model placed at (X ₂ , Y ₁ ), and the pixel (1, 2) is projected to (X ₃ , Y ₁₀ ) and ( X _m , Y ₂₅ ) is projected from a certain point on the three-dimensional model. Each pixel is not associated with only one model, and a plurality of models may form the same pixel. When the value of the array is 0, it means that the point on the three-dimensional model placed at the position (X _m , Y _n ) is not projected at all, and when it is all 0, the quantization number m for the pixel. , N means that nothing is projected onto the pixel from a point on the three-dimensional model prepared using n. The above table construction is performed at all positions (X _m , Y _n ) to complete the model table construction.

  In the present invention, the arrangement form of FIG. 4 is not necessary, and the number of models projected on each pixel (that is, the number of object positions) and the position of the model on the XY plane are shown as shown in FIG. The index (m, n) may be sequentially written in the horizontal direction. In this format, only the number of projected models is written in the horizontal direction, and it is not necessary to write any value in the case of a pixel on which nothing is projected, so this is suitable for effective use of memory resources. Furthermore, if these model tables are constructed in advance by another processing device and the table is also held by another processing device, the object position can be estimated in the same manner only by the subsequent processing.

  On the other hand, in FIG. 1, the temporal movement of the object is photographed by the camera, and the time series image is taken into the time series image database 1 (step S6). 1 sequentially extracts images from the time-series image database 1, and extracts a moving object region or a person region (generally referred to as an object region in the following description) from the image (step S7).

  In FIG. 2, the image taken by the camera in the same manner as described above is captured from the image input unit 7 (step S6) and input to the object region extraction unit 2.

Here, as shown in FIG. 6, if a background image is obtained in advance, an object region can be extracted from each image by differential processing (background difference) with the background image. Alternatively, if a method using a Gaussian mixture distribution described in Non-Patent Document 2 is used, a moving object region can be extracted only by inter-frame differences without requiring a background image. Through the above processing, each pixel (x _j , y _j ), j = 1, 2,..., P (the total number of pixels in the extracted region is P) forming the object region is obtained.

Next, in step S8, each pixel (x _j , y _j ), j = 1, 2,..., P obtained from the object region extraction unit 2 is collated with the model table. In the collation process, first, an M × N two-dimensional buffer having a size corresponding to the quantization numbers M and N is prepared in the XY space, and the column corresponding to each pixel (x _j , y _j ) is referred to in the horizontal direction. If there is a value other than 0, one vote is put in the buffer position (m, n) determined by m and n (step S9). This is performed for all the extracted pixels (x _j , y _j ), j = 1, 2,..., P (step S11). As a result, pattern matching between the extracted object region and the perspective projection image of the three-dimensional model is performed, and a spatial map is obtained in which the position with a larger number of votes is more likely to be the object position (step S10). .

This map is a buffer of size M × N, but since it is associated with the real world scale by equation (1), the array (m, n) corresponds to the spatial position (X _m , Y _n ). is doing. The spatial map generated by the above voting results has (X _m , Y _n ) as coordinates, and the values of the coordinates correspond to the vote values. The three-dimensional distribution has a mountain shape centering on the place where the object exists, and the map is expressed as a contour map as shown in FIG.

In the example of FIG. 7, there is a camera in the Y _n axis direction, and there is a feature that the camera is distributed so as to face the direction. In addition, the front distributions (A and B) are relatively grouped, but the rear distributions (C and D) tend to slightly increase. This indicates that the accuracy of voting deteriorates as the distance from the camera increases, and this tendency becomes a distribution that extends longer as the camera tilts with respect to the floor surface.

  Next, in step S12, the object position estimation unit 6 in FIGS. 1 and 2 estimates the position of the object from the spatial map in FIG. The processing flow is shown in FIG. First, since there may be isolated points due to the influence of noise in object region extraction, all the lattice points (m = 1, 2,..., M, n = 1, 2,... , N), a voting value is entered using a two-dimensional Gaussian with an appropriate window standard deviation σ (steps S21 and S22). Next, a maximum point is detected as indicated by x in FIG. 9 in the spatial map using the wrinkled value as a vote value, and that point is set as the position of the object (steps S23 and S24).

  However, depending on the arrangement of the cameras and the density of the objects, there is not always a single object at the maximum point. In particular, when the distribution located on the far side (C or D in FIG. 9) extends long in the depth direction, it is conceivable that a plurality of overlapping objects are formed as a single distribution. The processing flow of FIG. 8 calculates a plurality of object positions in consideration of such overlapping objects.

  That is, after detecting the local maximum point from each distribution in step S23, the position is surrounded by a circle with a radius R that includes the object at the center (the thick solid line circle in FIG. 10), and the inside of the circle is an exclusive region (first As an exclusive area) (step S25). That is, there is one object in the exclusive area. Further, a circular area having a radius R (the circular area is not shared with the exclusive area) is set as another exclusive area (second to n-th exclusive area) at a place where the set exclusive area is not shared ( The broken line circle in FIG. 10), the vote values in the range are totaled (step S26), and if the total value exceeds a predetermined value (step S27), it is assumed that there is an object at that location. The center of the region (the second to nth exclusive regions) is set as the position of the object (step S24). The first, second to n-th exclusive regions may be a figure such as a rectangle or a polygon in addition to a circle with a radius R. By performing this process on the entire space map, the object position is obtained from the space map, and the object position is output (step S13).

  The above series of processing is repeated for all the target images, and the position of the object in each image can be estimated according to the embodiment of the present invention. In addition, it is possible to estimate the object position in a long time by lengthening the image series.

(Example 2)
12 is a basic configuration diagram in the case of a plurality of image input devices according to the first and second aspects of the invention. Compared with the configuration of FIG. 1, the models corresponding to the camera information data input units 3a to 3n corresponding to the number of cameras. Table construction units 4a to 4n are added. FIG. 13 is a processing configuration diagram in the case of processing in real time, and the present invention does not necessarily require a storage device such as each database unit (time-series image database 1).

  13 is different from FIG. 12 in that an image input unit 7 for capturing images from a plurality of cameras is provided instead of the time-series image database 1, and the other configuration is the same as that in FIG.

  In FIGS. 12 and 13, the description of the same parts as those of the first embodiment is omitted, and the following description will be focused on the points different from the first embodiment.

  In the second embodiment, a case where a plurality of C (C ≧ 2) cameras are used will be described. Each camera does not necessarily have to be the same type of camera. It is only necessary to obtain the internal and external parameters of each camera by camera calibration work in advance, and the camera information data input units 3a to 3n in FIG. Keep it. All cameras are synchronized in time, and images in each frame are acquired. In the case of FIG. 12, the following description will be made on the assumption that the images are stored in the time-series image database 1.

In the perspective projection image generation (step S4 in FIG. 3) performed by the model table construction units 4a to 4n, the values of the indices m and n are sequentially increased, and the spatial position (X _m , Y _n ) and a point (X _j , Y _j , Z _j ) on the three-dimensional model of the expression (2) at that position using the camera parameters given from the respective camera i information data input units 3a to 3n Image coordinates (x _ij , y _ij ), i = 1, 2,..., C, j = 1, 2,.

The only difference from the first embodiment is that the image coordinates (x _ij , y _ij ) are obtained for each camera, and the spatial position (X _m , Y _n ) and the arrangement of the three-dimensional model at that position are as described above. Same as Example 1. In the model table construction units 4a to 4n, C types of model tables are generated according to each camera by the same procedure as in the first embodiment (steps S1 to S5 in FIG. 3).

Next, the object region extraction unit 2 extracts an image from the time-series image database 1, and extracts the object region based on the background difference and the inter-frame difference as in the first embodiment (steps S6 and S7 in FIG. 3). If an object region is obtained, each pixel (x _ij , y _ij ), i = 1, 2,..., C, j = 1, 2,. Vote for the 2D map space. At this time, since each camera is geometrically associated with the XY plane, that is, the position (X _m , Y _n ) indicated by the indices m and n by prior camera calibration, the collating process by each table is sequentially executed. Then, it is possible to vote for the same M × N two-dimensional map space. As a result, pattern matching between the extracted object region and the perspective projection image obtained by all cameras of the three-dimensional model is performed (steps S8 to S11 in FIG. 3).

  Since the second embodiment uses a plurality of cameras as compared with the monocular camera of the first embodiment, a highly reliable spatial map in which the ambiguity associated with the monocular camera is greatly reduced can be obtained. In the first embodiment, there may be a case where it extends long in the depth direction as shown in the distribution C or the distribution D in FIG. On the other hand, in the voting based on a plurality of cameras in the second embodiment, each distribution based on the plurality of cameras can be crossed in the same spatial map even if the distributions of the respective cameras extend long. It is possible to obtain a highly reliable spatial map that reflects the location of the object as the object position. Thereafter, the object position is estimated from the space map as in the first embodiment (steps S12 and S13 in FIG. 3).

(Example 3)
Next, an embodiment relating to claims 3 and 4 will be described. The third embodiment has the same processing configuration as that shown in FIGS. 1 and 2, and the description of the same parts is omitted. In the third embodiment, since the processing flow in the object position estimation unit 6 is different from that in FIG. 8, only this portion will be described below.

FIG. 14 is a processing flow in the object position estimation unit 6 in the third embodiment. The spatial position map obtained from the object region extraction is affected by noise due to background processing, and the distribution may flow depending on the tilt of the camera. Therefore, the spatial position map does not always provide a faithful maximum point corresponding to the number of objects. Therefore, first, in step S31 in FIG. 14, quantization (cellization) is performed in a lattice shape in the (X _m , Y _n ) space (the broken line portion in FIG. 15). The size of this cell may be determined in consideration of the camera installation status. For example, when the angle between the camera and the floor is large, it is assumed that the person occupies one person as shown in FIG. 15, and when the angle is smaller than a predetermined angle, for example, the distribution may flow in the depth direction. Therefore, a cell having coarse quantization in the Y _n direction as shown in FIG. 16 is set.

Next, in steps S32 and S33, each vote value is normalized in advance so that the maximum vote value on the space map is 1, and the value (voting rate) at (X _m , Y _n ) at that time is If it is less than 0.5, a process of deleting (steps S34 and S35) is performed. This is because the spatial map is generated including errors during background difference processing, so that the votes obtained from this noise are removed as much as possible, and those with a vote ratio of more than half are left as object position candidates. .

Subsequently, a new vote rate in the set cell is totaled to generate a vote rate map (step S36). If the vote rate in the cell coordinate space is larger than a predetermined value, the cell position is set as the object position (steps S37 to S39). A cell painted in black in FIG. 17 is an example of processing determined as an object position. If the cell quantization in the _Yn direction is the same as the cell quantization in the _Xm direction as shown in FIG. 15, the _number of cells detected as the back object position may increase as shown in FIG. In that case, by detecting the object position in the cell space of FIG. 16 which slightly rougher quantization of Y _n direction, as shown in FIG. 18, it is possible to suppress the number of cells detected as a position as much as possible.

Example 4
Next, an embodiment relating to claim 5 will be described. A basic configuration diagram of the fourth embodiment is as shown in FIG. 19 and can be easily extended to a plurality of cameras, and therefore description and illustration thereof are omitted.

  19 differs from FIG. 1 in that an effective pixel setting unit 8 is further provided for setting a region projected on an image as a projection effective pixel for a projection result on an image of a prepared three-dimensional model. The rest of the configuration is the same as in FIG.

  FIG. 20 is a processing configuration diagram in the case of processing in real time, and the present invention does not necessarily require a storage device such as each database unit (time-series image database 1).

  20 is different from FIG. 19 in that an image input unit 7 for capturing images from a plurality of cameras is provided instead of the time-series image database 1, and the other configuration is the same as that in FIG.

In the case of a normal monitoring camera, for example, as shown in FIG. 21, when the monitoring area is photographed, the camera is often tilted so as to face the floor (or the ground). In the model table construction of Examples 1 to 3 described above, the position (X _m , Y _n ) is changed to form a three-dimensional model, and the table is constructed based on the projected coordinates. However, if the angle of view of the camera is Φ in FIG. 21, the three-dimensional model is arranged only in a finite area designated by the position (X _m , Y _n ). It is clear that nothing is projected, and the image is projected as shown in FIG. 22 (pixels on which the white part is projected, pixels on which the black part is not projected).

  That is, even if a model table is constructed for all pixels, a “sparse” table (a table with many blanks in FIGS. 4 and 5) is constructed depending on the angle of view and the camera arrangement. Therefore, in this embodiment, a three-dimensional model to be arranged is grasped in advance, and processing is performed only on the projected pixel. In the following, since the processing of the effective pixel setting unit 8 in FIG. 19 is different from those in the first to third embodiments, this processing will be mainly described.

As described in step S4 (perspective projection image generation step) in FIG. 3 of the first embodiment, from the point (X _j , Y _j , Z _j ) on the three-dimensional model at the position (X _m , Y _n ). Obtain image coordinates (x _j , y _j ), j = 1, 2,..., Q. Next, as shown in FIG. 22, since there is a pixel on which nothing is projected, the corresponding column of the pixel area not projected in the model table construction is deleted. Pixels remaining after this deletion processing are projection effective pixels, and these projection effective pixels are used in the object region extraction unit 2 and the space map generation unit 5. There are roughly two types of pixel deletion processing. One is a method of deleting a column of image coordinates that is found not to be projected. In the example of FIG. 4, a horizontal column of a certain pixel (x _j , y _j ) If all are 0, delete all the rows of (x _j , y _j ).

  Alternatively, in the example of FIG. 5, when the number of projections is 0, the row is deleted. The other is processing for filtering a pixel area that is not projected as shown in FIG. 23 with a rectangle or the like, and deleting a column of image coordinates corresponding to the filtered pixel area from the model table. The following is for the latter deletion process.

When constructing the model table, the projected image area is set in the effective pixel setting unit 8 as a projection effective pixel. Subsequently, when the object region extraction unit 2 extracts the object region from the time-series image by the background difference or the inter-frame difference as in the first embodiment, only the pixel region set as the effective projection pixel (the pixel of the object region ( x _j , y _j ). Next, a voting process is performed for each extracted pixel with reference to the model table to generate a space map. Subsequent processing is the same as in the first embodiment. In the present embodiment, since a pixel area that is not projected is excluded from the processing target, a series of processes of object area extraction, model table construction, and spatial map generation can be executed efficiently.

  Note that the object position estimation method of the present invention executes each process described in the first to fourth embodiments.

    A program for causing a computer to execute the object position estimation method is constructed.

  Further, the present invention can be realized by configuring some or all of the functions of each means in the object position estimation apparatus of the present embodiment by a computer program and executing the program using the computer. It goes without saying that the procedure in the object position estimation method of the above can be configured by a computer program and the program can be executed by the computer, and the program for realizing the function by the computer can be read by the computer, For example, FD (Floppy (registered trademark) Disk), MO (Magneto-Optical disk), ROM (Read Only Memory), memory card, CD (Compact Disk) -ROM, DVD (Digital Versati) e Disk) -ROM, CD-R ,, CD-RW, HDD, and recorded in a removable disk, or stored, it is possible or distribute. It is also possible to provide the above program through a network such as the Internet or electronic mail.

It is a basic composition figure of an image storage type of an object position estimating device of an example of an embodiment of the present invention. 1 is a basic configuration diagram of a real-time processing type of an object position estimation apparatus according to an embodiment of the present invention. It is a processing flow figure of Example 1 of the present invention. It is explanatory drawing which shows an example of the model table of this invention. It is explanatory drawing which shows the other example of the model table of this invention. It is explanatory drawing which shows the example which extracts an object area | region by the background difference of a background image and each frame image in this invention. It is explanatory drawing which shows the example of the space map in this invention. It is a processing flowchart which shows an example of the object position estimation in this invention. It is explanatory drawing which shows the example of the process result of the object position estimation by the processing flow of FIG. It is explanatory drawing which shows the example of the process result of the object position estimation by the processing flow of FIG. It is explanatory drawing of the lattice map which arrange | positions the positional relationship of a camera and an object in this invention, and a three-dimensional model. It is an image storage type | mold basic block diagram of the object position estimation apparatus of the other embodiment of this invention. It is a basic-structure figure of the real-time processing type of the object position estimation apparatus of the other embodiment of this invention. It is a processing flowchart which shows the other example of the object position estimation in this invention. It is explanatory drawing which shows the state of the object position estimation process of Example 3 of this invention. It is explanatory drawing which shows the state of the object position estimation process of Example 3 of this invention. It is explanatory drawing which shows the process result of FIG. It is explanatory drawing which shows the process result of FIG. It is an image storage type | mold basic block diagram of the object position estimation apparatus of Example 4 of this invention. It is a basic block diagram of the real-time processing type | mold of the object position estimation apparatus of Example 4 of this invention. Explanatory drawing which shows the example of arrangement | positioning of a camera and a three-dimensional model. It is explanatory drawing which shows the example of the projection result of a three-dimensional model. It is explanatory drawing which shows the example extracted from the projection effective pixel in the object area | region by the background difference of Example 4 of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Time series image database, 2 ... Object area extraction part 3, 3a-3n ... Camera information data input part, 4, 4a-4n ... Model table construction part, 5 ... Spatial map generation part, 6 ... Object position estimation part , 7... Image input unit, 8... Effective pixel setting unit.

Claims (12)

An object position estimation device that extracts a target area from a video acquired by using one or a plurality of image input devices such as a camera, and estimates a position in the space from the target area,
From each time-series image acquired by each of the image input devices, an object region extraction unit that extracts a target region in an image processing manner,
Each pixel of the image obtained from the image input device indicates which projection of the three-dimensional model prepared by each pixel corresponding to each image input device extracted by the object region extraction means corresponds. A reference table indicating which spatial position is projected from which 3D model is projected is used for collation. If there is a projected 3D model, the vote value is obtained by voting for each model spatial position. A spatial map generating means for generating a map indicating that there is a high possibility of the presence of an object in a place where there are many
The position where the vote value of the map generated by the spatial map generation means becomes the maximum point is detected as the position of the object, and the circular area of radius R around the position of the object is set as the exclusive area Then, a circular area having a radius R is set as another exclusive area in the area where the exclusive area is not set in the map, and the exclusive area is set for each exclusive area of the other exclusive area. Object position estimation means comprising: an object position estimating means for summing up the vote values obtained at the position, and when the summed value is equal to or greater than a predetermined value, the center of the other exclusive area is set as the position of the target object. Estimating device. The reference table is
Each image of a three-dimensional perspective projection image generated by arranging a plurality of three-dimensional models at spatial positions on a certain reference coordinate system and using the position and orientation of the image input device and internal parameters unique to the image input device. The object position estimation apparatus according to claim 1, wherein the object position estimation apparatus is constructed as a table indicating a relationship between coordinates and an arrangement position of the three-dimensional model. The object position estimating means includes
When the generated spatial map is quantized in a lattice shape, and each vote value in the quantized range is equal to or greater than a predetermined value, the quantized range is set as a place where the object exists. The object position estimation apparatus according to claim 1 or 2, characterized in that The grid-like quantization in the spatial map is rough when the angle between the direction in which each image input device faces and the installation surface of the device is smaller than a predetermined angle, in the direction in which the image input device faces. The object position estimation apparatus according to claim 3 , wherein the object position estimation apparatus is set. The projection result on the image of the prepared three-dimensional model further includes effective pixel setting means for setting an area projected on the image as a projection effective pixel,
The reference table is constructed using the projection effective pixels set by the effective pixel setting means,
The object area extracting means, the effective pixel object position estimation apparatus according to any one of claims 1 to 4, characterized in that to extract only region of the projection effective pixels set by the setting means. An object position estimation method for extracting a target area from a video acquired by using one or a plurality of image input devices such as a camera, and estimating a position in the space from the target area,
An object area extracting step in which the object area extracting means extracts the area of the object in an image processing manner from each time-series image acquired by each of the image input devices;
A reference table construction step, wherein the reference table construction means constructs a reference table indicating which spatial position each pixel of the image obtained from each image input device is projected from;
The spatial map generation means uses the reference table to determine which projection of the three-dimensional model prepared for each pixel corresponding to each image input device extracted in the object region extraction step. If there is a three-dimensional model to be collated and voted, the space for generating a map indicating that there is a high possibility of the presence of an object at a place with a large vote value by voting on the spatial position of the model. A map generation step;
An object position estimating means comprising: an object position estimating step for estimating a position of an object based on a vote value of the map generated by the spatial map generating step ;
The object position estimating step includes:
A first sub-step of detecting a position where the vote value of the map generated by the spatial map generating step is a maximum point and setting the position of the object; and a circular region having a radius R around the position of the object A second sub-step that is set as an exclusive area, and a circular area having a radius R is set as another exclusive area in the area where the exclusive area is not set in the map, And a fourth sub-step in which the center of the other exclusive area is set as the position of the object when the calculated value is equal to or greater than a predetermined value. And the object position estimation method characterized by repeating said 2nd sub-step-4th sub-step with respect to the whole map . The reference table construction step includes
A plurality of three-dimensional models are arranged at spatial positions on a reference coordinate system, and a perspective projection image of the three-dimensional shape is generated using the position and orientation of the image input device and internal parameters unique to the image input device, The object position estimation method according to claim 6 , wherein each image coordinate of the perspective projection image is associated with an arrangement position of the three-dimensional model. The object position estimating step includes:
When the generated spatial map is quantized in a lattice shape, and each vote value in the quantized range is equal to or greater than a predetermined value, the quantized range is set as a place where the object exists. The object position estimation method according to claim 6 or 7 , wherein the object position is estimated. The grid-like quantization in the spatial map is rough when the angle between the direction in which each image input device faces and the installation surface of the device is smaller than a predetermined angle, in the direction in which the image input device faces. The object position estimation method according to claim 8 , wherein the object position estimation method is set. The projection result on the image of the prepared three-dimensional model further includes an effective pixel setting step for setting an area projected on the image as a projection effective pixel,
The reference table construction step is constructed using the projection effective pixels set by the effective pixel setting step,
The object position estimation method according to any one of claims 6 to 9 , wherein the object region extraction step extracts only a region of the projection effective pixel set by the effective pixel setting step. An object position estimation program for causing a computer to execute the steps according to any one of claims 6 to 10 . The computer-readable recording medium which recorded the object position estimation program of Claim 11 .

Images