JP2009210286A - Apparatus and program for image processing - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus enabling estimation of an object with high accuracy from an image of a single-lens camera by simple setting. <P>SOLUTION: Regarding a rectangular area (tracking start (new) object and tracking object) extracted by a change area extraction processing part 151, an object position-size estimation processing part 153 performs estimation of the position and the size of the object and various object processings annexed thereto, by using reference tables (real distance Y table, real distance X table and real distance Z table) calculated by an object model table calculating part 154. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing program for processing an image captured by a monocular camera (one camera).

  As a method for obtaining the world coordinates on the image captured by the camera, the three-dimensional coordinates are obtained on the two-dimensional screen by the camera information, the three-dimensional measurement using the camera installation conditions, and the position on the screen is obtained. There are methods such as seeking.

  As this type of system, there is a camera system that estimates the position of a person by photographing the same person with two or more asynchronous cameras and estimating the three-dimensional top position of the person (see Patent Document 1).

However, the camera technique based on the above method has a problem that it is necessary to accurately measure camera information and an installation position, and depending on the installation position of the camera, there are cases where measurement is not possible. Also, in image processing, when identifying an object from a change area that appears on the screen, how far the object is located from the camera, and how large the object's width and height are actually It is necessary to use information such as whether or not, extracting the change area from the camera video and making a judgment only by the size on the image, but the size of the target object such as a small animal in front of the screen and a person at the back of the screen In contrast, there is a problem that the position of the existence cannot be calculated correctly.
JP 2007-233523 A

  As described above, the conventional image processing technology for camera images has a problem that it takes a lot of time for installation and setting, and there is also a problem in terms of recognition accuracy.

  It is an object of the present invention to provide an image processing apparatus and an image processing program that can solve the above-described problems and can estimate an object position with high accuracy from an image of a monocular camera with simple settings.

  The present invention relates to the depth direction on the screen based on the information on the four vertices on the screen indicating the monitoring range of the monocular camera and the screen information and measured information on three or more sample images taken by the monocular camera. Real distance table acquisition means for obtaining an actual distance table for each pixel in the vertical direction on the screen by obtaining an actual distance to the pixel by an exponential approximation formula, information on the actual distance table, information on the four vertices, A lateral weight table for obtaining a weight of an actual distance with respect to a pixel in the horizontal direction on the screen by a linear approximation formula based on the screen information and the actual measurement information of the sample image, and creating a horizontal weight table on the screen Based on the acquisition means, the information on the actual distance table, the information on the four vertices, the screen information and the actual measurement information on the sample image, the actual distance to the pixels in the vertical direction on the screen The weight is obtained by a linear approximation formula, and a vertical weight table acquisition means for creating a vertical weight table on the screen, and each table is held as a reference table, and at least one reference table of the reference tables is stored. And an image processing unit that executes predetermined image processing on an image captured by the monocular camera as a processing target.

  Further, the present invention is an image processing program for causing a computer to function as an image processing apparatus for a monocular camera, and information on four vertices on a screen indicating a monitoring range of the monocular camera and three locations photographed by the monocular camera. Based on the screen information and the actual measurement information of the sample image described above, an actual distance table for the pixels in the depth direction on the screen is obtained by an exponential approximation formula, and an actual distance table for each pixel in the vertical direction on the screen is created. Based on the processing function, the information on the real distance table, the information on the four vertices, the screen information on the sample image, and the actual measurement information, a linear approximation of the weight of the real distance on the horizontal pixel on the screen A processing function for creating a horizontal weight table on the screen, information on the real distance table, information on the four vertices, and screen information on the sample image. Based on the actual measurement information, the weight of the actual distance to the vertical pixel on the screen is obtained by a linear approximation formula, and a processing function for creating the vertical weight table on the screen and the respective tables are referred to. And a processing function for executing predetermined image processing on an image captured by the monocular camera using the at least one reference table among the reference tables as a processing target. It is characterized by that.

  By this image processing, three or more monitoring ranges (four vertices) and position information on the captured image can be input without using camera information or camera installation conditions in monocular (one camera) image processing. The position on the shooting screen can be estimated. In addition, it is possible to calculate the actual distance with respect to the position on the image, and the position of the object on the real world coordinates can be grasped with high accuracy. Further, when an object on the captured video is extracted (region), the actual distance of the size (width, height) can be estimated, and further, the actual distance of the position can be estimated. In addition, by setting a monitoring range on the captured video, an object outside the zone can be excluded and noise can be removed.

  According to the present invention, it is possible to estimate an object position with high accuracy from an image of a monocular camera with a simple setting.

  Embodiments of the present invention will be described below with reference to the drawings. In this embodiment, an example in which the present invention is applied to a tracking processing technique of a monocular camera is shown, but the present invention can be widely applied to various image processing techniques using a monocular camera.

  An image processing apparatus according to an embodiment of the present invention is an object that obtains an actual position (real world coordinates) of an object detected (extracted) or tracked from an image captured by a monocular camera (one camera) from a position on the image. The position estimation function is realized.

In the image processing of the monocular camera, the present invention is a component that realizes a function for estimating the position on the world coordinates based on the position on the image of the object extracted from the input image captured by the monocular camera.
Based on the information of the four vertices on the screen indicating the monitoring range of the monocular camera and the screen information and measured information of the three or more sample images taken by the monocular camera, the actual pixels for the depth direction on the screen are measured. An actual distance graph acquisition unit that obtains an exponential approximate expression of the distance and acquires an actual distance graph in pixel units in the vertical direction on the screen; information acquired by the actual distance graph acquisition unit; information on the four vertices; and the sample A lateral weight graph acquisition unit that obtains a weight of an actual distance to a pixel in the horizontal direction on the screen based on the screen information and the actual measurement information of the image by a linear approximation formula and acquires the horizontal weight graph on the screen On the basis of the information acquired by the actual distance graph acquisition means, the information on the four vertices, the screen information and the actual measurement information of the sample image, the actual distance to the vertical pixel on the screen A vertical weight graph acquisition means for acquiring a vertical weight graph on the screen and each of the graphs as a reference table, and at least one of the reference tables is stored as a reference table. And image processing means for executing predetermined image processing on an image captured by the monocular camera as a processing target.

  An outline of the operation of the object position estimation process in the image processing apparatus according to the embodiment of the present invention will be described with reference to FIG. Here, it is assumed that an image captured by the monocular camera 11 is output as a camera image by a QVGA of 320 × 240 pixels per frame.

  The estimation of the object position according to the embodiment of the present invention is performed based on an image obtained by photographing a real object (target object) serving as a sample model with a monocular camera. The object as the sample model is, for example, an object in which an interval in the width direction (lateral direction) can be set, such as a person model in which two people are arranged as shown in the figure.

  First, in process A1 shown in FIG. 1, three or more images in which an object to be the sample model is photographed are prepared from the monitoring video of the monocular camera 11. At this time, these images are the same object (same sample model) and are taken at different locations on the screen. Note that the angle of view of the monocular camera 11 is set based on installation conditions for imaging a monitoring target from obliquely above, for example, in a monitoring area.

  Next, an object region is enclosed by a rectangle (rectangle) from the prepared image, and object information at three or more places enclosed by the rectangle is acquired. Here, as shown in FIG. 5, three images (P1, P2, P3) having different rectangular sizes (different shooting positions) are prepared, and the objects of the three images are rectangular. The process enclosed by is performed.

  In process A2 shown in FIG. 1, the monitoring range (four vertices) monitored by the monocular camera 11 is designated on the screen. Here, as shown in FIG. 6, the monitoring range is set with one of the prepared images. The line at the left end of the screen is the horizontal reference line (0m line), and the position is estimated within this monitoring range.

  For each of the three images (P1, P2, P3) shown in FIG. 5 described above, the object enclosed by a rectangle is shown in FIG. ) Distances Pyi (pix), Wyi (m), the distance between two persons ΔPxi (pix), ΔWx (pix), and the height ΔPhi (pix), ΔWh (m) of one of the objects (I is the number of images, and ΔWx and ΔWh are common to each image).

  Next, the actual distance graph acquisition process (vertical distance model calculation process), the horizontal weight graph acquisition process (horizontal distance weight model calculation process), and the vertical weight graph acquisition process ( By the vertical distance weight model calculation process), the actual distance graph (foot position-distance graph), the horizontal weight graph (distance-horizontal weight graph), and the vertical (height) direction weight graph (distance-high). Directional weight graph).

  In the acquisition (calculation) processing of the actual distance graph (foot position-distance graph), information on the distances Pyi (pix) and Wyi (m) from the lower side of the above three (or three or more) images to the foot position. Is used to obtain an exponential approximate expression of the actual distance Wy with respect to the foot position Py on the image by the method of least squares.

Distance model: Wy = Ae ^BPy (A and B are parameters determined by the method of least squares) (1)
An actual distance graph with respect to the position on the image is acquired from the distance model approximate expression calculated by the expression (1) (Note that the distance from the position immediately below the camera to the position on the lower side of the image (distance that becomes a blind spot) is set to the value of Wy. By adding it, it is also possible to find the actual distance from directly under the camera). An example of the actual distance graph (foot position-distance graph) is shown in FIG.

  In the acquisition (calculation) process of the horizontal weight graph (distance-horizontal weight graph), the vertical distance model acquired in the acquisition process of the actual distance graph and the interval between three (or three or more) two persons From ΔPxi (pix) and ΔWx (pix), a weight (ωxi = ΔWx / ΔPxi) per pixel in the horizontal direction at the position of the actual distance Wy is obtained.

  Using three (or three or more) actual distances Wy and lateral weights ωxi per pixel, a linear approximation formula of lateral weights ωxi per pixel with respect to actual distances Wy is obtained by the least square method. To do.

Weight model per pixel in the horizontal direction: ωx = CWy + D (C and D are parameters determined by the method of least squares) (2)
A horizontal weight graph with respect to the actual distance Wy is acquired from the horizontal weight approximate expression calculated by the expression (2). An example of this horizontal weight graph (distance-horizontal weight graph) is shown in FIG.

  The horizontal position of each pixel on the screen is calculated by multiplying the distance on the image from the reference line to the pixel by the calculated weight.

  In the acquisition (calculation) process of the vertical (height) direction weight graph (distance-height direction weight graph), the vertical distance model acquired in the actual distance graph acquisition process and the heights of three or more objects From ΔPhi (pix) and ΔWh (pix), the weight per pixel in the height direction at the position of the actual distance Wy (ωzi = ΔWh / ΔPhi) is obtained.

Using three (or three or more) actual distances Wy and the weight ωzi per pixel in the height direction, a linear approximation formula of the weight ωzi per pixel in the height direction with respect to the actual distance Wy by the least square method To get.

Weight model per pixel in the height direction: ωz = EWy + F (E and F are parameters determined by the method of least squares) (3)
A height direction weight graph with respect to the actual distance Wy is acquired from the height direction weight approximate expression calculated by the expression (3). An example of this vertical weight graph (distance-height weight graph) is shown in FIG.

  The height direction position of each pixel on the screen is calculated by multiplying the distance on the image from the foot position on the image to the designated pixel in the height direction by the calculated weight.

  In process A3 shown in FIG. 1, an actual distance table (actual distance Y table shown in FIG. 3) is created based on the information (parameter values) of the above-described actual distance graph (foot position-distance graph), and the horizontal direction Based on the information (parameter value) of the weight graph (distance-horizontal weight graph), a horizontal weight table (actual distance X table shown in FIG. 3) is created, and a vertical (height) direction weight graph (distance− Based on the information (parameter values) of the height direction weight graph), a vertical direction weight table (actual distance Z table shown in FIG. 3) is created.

  These real distance table (real distance Y table), horizontal weight table (real distance X table), and vertical weight table (real distance Z table) are used as reference tables for the position and size of the object photographed by the monocular camera 11. Estimation and image processing based on this estimation are performed.

  It is possible to obtain the actual distance at the position on the image, the distance between two points, or the height based on the estimation coefficient obtained by creating each graph. Also, it is possible to calculate how many meters a position is from a certain point.

When calculating the actual distance of a certain position (i, j) on the image, the actual distance in the vertical direction is obtained by substituting j for Py from the above formula (1) (vertical distance model approximate expression Wy = Ae ^BPy). Wy is calculated. Ωx is calculated by substituting the calculated Wy into the above formula (2) (lateral distance model weight approximation formula ωx = CWy + D).

  The number of pixels DPx in the horizontal direction from the horizontal reference line in the monitoring range to a certain position (i, j) is calculated. The actual distance Wx in the horizontal direction is calculated by ωx × DPx.

  Thereby, the actual distance (Wx, Wy) of a certain position (i, j) on the image is obtained.

When obtaining the actual distance of the height of the number of pixels DPz on the image from a certain position (i, j) on the image, from the above formula (1) (vertical distance model approximation formula Wy = Ae ^BPy) , By substituting j for Py, the actual vertical distance Wy is calculated. Ωz is calculated by substituting the calculated Wy into the above formula (3) (distance model weight approximation formula in the height direction ωz = EWy + F). The actual distance in the height direction is calculated by ωz × DPz.

  Using the above-described actual distance table (actual distance Y table), horizontal weight table (actual distance X table), and vertical weight table (actual distance Z table) as a reference table, an image captured by the monocular camera 11 is used. Estimation of the object position and various image processing associated therewith are performed.

  In this embodiment, in the process A11 shown in FIG. 1, the image of the monocular camera 11 is input at a predetermined cycle, the input current image on the screen and the past image are subjected to differential processing, and a rectangular area including changed pixels To extract.

  In the process A12 shown in FIG. 1, for the extracted rectangular area, the newly extracted rectangular area is used as a tracking start object, the object is tracked, the tracked object information is held, and a tracking object appears. Update object information.

  In the process A13 shown in FIG. 1, for the extracted rectangular area (tracking start object and tracking object), the object position is estimated and various object processes based on the estimation are performed using the reference table acquired by the preprocessing. Do. In this object process, for example, a process of recognizing the actual position of an image captured by the monocular camera 11 can be executed using an actual distance table (actual distance Y table). Moreover, the process which excludes the image image | photographed with the monocular camera 11 from a process target can be performed using an actual distance table (actual distance Y table). In addition, an actual image (for example, an object) captured by the monocular camera 11 using an actual distance table (actual distance Y table) and a horizontal weight table (actual distance X table) or a vertical weight table (actual distance Z table). For example, a sorting process based on the actual object size can be executed. Further, the monitoring range is determined using the information on the four vertices of the monitoring range, the actual distance table (actual distance Y table), and the horizontal weight table (actual distance X table) or the vertical weight table (actual distance Z table). In the real space, various processes such as object discrimination, selection (extraction), image deletion, space monitoring, and the like can be performed for the real space monitoring range. Each processing using these monocular camera images is possible only by the graph processing using the exponential approximation model and each linear approximation model.

  The configuration of the image processing apparatus according to the embodiment using the above-described actual distance table (actual distance Y table), horizontal weight table (actual distance X table), and vertical weight table (actual distance Z table) as a reference table is illustrated. 2 shows the configuration of the main part of the image processing apparatus.

  As shown in FIG. 2, the image processing apparatus according to the embodiment of the present invention includes a camera (monocular camera) 11, a capture unit 12, an image processing storage unit 13, an image processing unit 15, and a display unit 16. It is provided and configured.

  The camera 11 includes a lens unit and an image pickup device (for example, a CCD solid-state image pickup device or a CMOS image sensor) provided at an image forming position of the lens unit, so that a subject (animal body) that moves outdoors or indoors can be used. An image for one screen imaged at a certain angle of view is output to the target in a predetermined pixel unit (for example, one frame 320 × 240 pixels (pix) = QVGA).

  The capture unit 12 has a processing function of capturing an image in frame units captured by the camera 11 as a processing target image (input image) of the image processing unit 15 and holding it in the image buffer 131 in the image processing storage unit 13. The input image on the screen captured in the image buffer 131 is an original image here, but may be an edge image.

  The image processing storage unit 13 stores various data including the input image captured by the capture unit 12 and each image being processed, which is used for the processing of the image processing unit 15. In this image processing storage unit 13, under the control of the image processing unit 15, among the frame-unit images captured by the capture unit 12, an image for one screen captured this time (current image) and one previously captured image are stored. An area constituting the image buffer 131 that holds images for the screen (past images) as processing target images is secured, and an image area used for processing by the image processing unit 15 is secured. Further, the image processing storage unit 13 has a storage area for storing various parameters and control data used for the processing of the image processing unit 15, a storage area for various information generated or obtained in the tracking process, and the like. The

  The position estimation coefficient parameter calculation unit 14 uses the information on the four vertices described above, the screen information of three or more sample images captured by the monocular camera 11, the actual measurement information, and the like as input information. A position estimation coefficient parameter value of the model is calculated, and the position estimation coefficient parameter value is provided in the image processing unit 15 as element data for creating a reference table (actual distance Y table, actual distance X table, actual distance Z table). This is supplied to the position estimation table calculation unit 154.

  The image processing unit 15 generates a difference binarized image obtained by binarizing the past image and the current image held in the image buffer 131 of the image processing storage unit 13 and includes the difference image. The change area extraction processing unit 151 that extracts a rectangular area including the change pixel from the difference binarized image and the rectangular area extracted by the change area extraction processing unit 151 are processed. The tracking processing unit 152 that tracks the area and the reference tables calculated by the position estimation table calculation unit 154 (actual distance Y table, actual distance X table, actual distance Z table) are used for various types of position estimation as described above. And an object position / size estimation processing unit 153 that executes processing.

  The display unit (display device) 16 displays and outputs the moving object region image-processed by the image processing unit 15.

  FIG. 4 shows a processing procedure in the above-described image processing apparatus. In the process of step B3 shown in FIG. 4, the object position / size estimation process using the reference table (actual distance Y table, actual distance X table, actual distance Z table) is performed.

  The operation of the image processing apparatus shown in FIGS. 2 and 3 will be described according to the processing procedure shown in FIG.

  The change area extraction processing unit 151 provided in the image processing unit 15 generates a difference binarized image obtained by binarizing the past image and the current image stored in the image buffer 131 of the image processing storage unit 13 by performing a difference process. Then, noise included in the difference binarized image is removed, and the image information is held in the change area buffer 132. Further, a rectangular area including the change pixel is extracted from the information of the difference binarized image, and the information of the rectangular area is held in the tracking information buffer 133 (step B1 in FIG. 4).

  The tracking processing unit 152 performs tracking processing on the rectangular area extracted by the change area extraction processing unit 151 and held in the tracking information buffer 133 as a tracking target object. In this tracking process, since there is no object to be tracked in the initial operation, this object is registered as a new object in the tracking information buffer 133 and is set as a target for the subsequent tracking process. For example, a region tracking process combining region determination based on a rectangular shape and overlap and region determination based on a prediction position according to a prediction direction obtained by least square approximation can be applied to this tracking processing (step B2 in FIG. 4).

  The object position / size estimation processing unit 153 uses the reference table (actual distance Y) calculated by the position estimation table calculation unit 154 for the rectangular regions (tracking start (new) object and tracking object) extracted by the change region extraction processing unit 151. The object is estimated using the table, the actual distance X table, and the actual distance Z table (step B3 in FIG. 4).

  Image processing according to the processing procedure shown in FIG. 4 is performed using this reference table.

  The change area extraction processing unit 151 provided in the image processing unit 15 generates a difference binarized image obtained by binarizing the past image and the current image stored in the image buffer 131 of the image processing storage unit 13 by performing a difference process. Then, noise included in the difference binarized image is removed, and the image information is held in the change area buffer 132. Further, a rectangular area including the change pixel is extracted from the information of the difference binarized image, and the information of the rectangular area is held in the tracking information buffer 133 (step B1 in FIG. 4).

  The tracking processing unit 152 performs tracking processing on the rectangular area extracted by the change area extraction processing unit 151 and held in the tracking information buffer 133 as a tracking target object. In this tracking process, since there is no object to be tracked in the initial operation, this object is registered as a new object in the tracking information buffer 133 and is set as a target for the subsequent tracking process. For example, a region tracking process combining region determination based on a rectangular shape and overlap and region determination based on a prediction position according to a prediction direction obtained by least square approximation can be applied to this tracking processing (step B2 in FIG. 4).

  The object position / size estimation processing unit 153 uses the reference table (actual distance Y) calculated by the object model table calculation unit 154 for the rectangular regions (tracking start (new) object and tracking object) extracted by the change region extraction processing unit 151. Table, actual distance X table, and actual distance Z table) are used to estimate the object position and size and perform various object processing (step B3 in FIG. 4). Here, an object position estimation function for obtaining the position (real world coordinates) of the detected or tracked object from the position on the image is realized. For example, the user can specify a monitoring range (two-dimensional plane), and can estimate the feet and top positions of objects existing in the monitoring range with relatively high position accuracy. In addition, as described above, the processing for recognizing the actual position of an image captured by the monocular camera 11 using the actual distance Y table, and the image captured by the monocular camera 11 using the actual distance Y table are excluded from the processing target. Processing, real distance Y table, real distance X table, or real distance Z table to recognize the substance of an image photographed by the monocular camera 11 (for example, the actual size of the object), the actual object size Sorting process based on the information, information on the four vertices of the monitoring range, real distance Y table and real distance X table or real distance Z table, the monitoring range is set in the real space, the object recognition processing, the real space monitoring range Various processes such as object discrimination, selection (extraction), image deletion, space monitoring, and the like can be performed.

  The object position and size estimation function according to the embodiment of the present invention will be further described with reference to FIGS.

(1). Concept of monitoring range On the screen imaged by the monocular camera 11 shown in FIG. 11, four points on the screen, p0 (x0, y0), p1 (x1, y1), p2 (x2, y1), p3 (x3) , Y0), a quadrilateral connecting the four points is regarded as a plane area, and the plane area is set as a monitoring range. A monitoring range designation image is shown in FIG. 11 by a straight line connecting the four points.

The y-coordinates of the front line in the depth direction (p0-p3 line) and the back line (p1-p2 line) are the same and are parallel (the planar area does not have to be an exact quadrangle, To do). It should be noted that points outside the screen can also be input (however, p1 must be set within the screen). x1 <x2, x0 <x3, y0> y1 are set. In addition, by setting the height direction (the height of the world coordinates), the monitoring range in the height direction is set, and it is possible to delete the unnecessary area (the monitoring is performed at the end when X, Y, and Z are obtained). Set the range space). Since p0 and p1 are reference lines, it is assumed that p1 exists in the screen (2). Position estimation concept and reference position (reference position)
The coordinates (x, y) and the corresponding world coordinates (X, Y, Z) of a plurality of positions (a plurality of equally spaced points) serving as a reference (reference) within the monitoring range are obtained, and the distance (Y) − A screen position (y) graph is acquired.

  Using the acquired distance (Y), a graph of distance per pixel (ΔX) −distance (Y), height per pixel (ΔZ) −distance (Y) is obtained. FIG. 12 shows an example of a distance-image coordinate relationship graph (Yy graph) from the camera. An example of a distance-image coordinate relationship graph (ΔX-Y graph) per pixel is shown in FIG.

  The reference position is as follows.

  As shown in the screen of FIG. 14, three reference positions (y0, y1, y2) at the front, center, and back of the monitoring range are determined, and their respective intervals ΔY0 [m] (can be set by eye measurement), ΔY1 [M] and ΔY2 [m] are measured to obtain Y0 [m], Y1 [m], and Y2 [m]. The reference position of this ΔY [m] interval may be made common with the position for obtaining the object model in the previous section. In addition, pixel widths (Δx [pix]) of certain equal intervals ΔX1 [m], X2 [m], and X3 [m] are obtained at each reference position. At this time, the y coordinate on the image is set to be substantially the same on the left and right (in parallel).

  Obtained Y0 [m], Y1 [m], Y2 [m], y0 [pix], y1 [pix], y2 [pix], ΔX1 [m], X2 [m], X3 [m], Δx1 [pix ], Δx2 [pix], and Δx3 [pix] are used to obtain an exponential approximate expression of distance (Y) −image coordinates (y) first by the least square method. Next, using the acquired distance (Y), a linear approximate expression of distance (X) per pixel−distance (Y) is obtained (in the same way of thinking, height (Z) per pixel−distance (Y) is obtained.

  A reference table (actual distance graph shown in FIG. 8; foot position-distance graph shown in FIG. 8) is created from the obtained approximate expression.

  A Y coordinate reference table (actual distance table; actual distance Y table) as shown in FIG. 16 is created from the exponential approximation formula.

  An X coordinate reference table (lateral weight table; actual distance X table) as shown in FIG. 17 is created from the linear approximation formula.

  A Z coordinate reference table (vertical weight table; actual distance Z table) as shown in FIG. 18 is created from the linear approximation formula.

(3). Object position estimation method (Concept of monitoring range and 3D position map)
When the monitoring range is expanded into a three-dimensional map, it becomes as shown in FIG.

  Using the foot position (xp, yp) of the object as input, Yp coordinates corresponding to the yp coordinates are obtained from the Y coordinate reference table.

  Using the foot position (xp, yp) of the object as an input, the Xp coordinate corresponding to the (xp, yp) coordinate based on the left side of the monitoring range is obtained from the X coordinate reference table.

  From the Z coordinate reference table, the height Z is obtained using the difference (yt−yp).

  The (X, Y, Z) coordinates obtained by this method are used as object positions (plotting is possible on a three-dimensional map).

(4). Deletion of Unnecessary Area Using Monitoring Range As an example, when it is determined that an area of an object to be monitored appears outside the monitoring range indicated by a straight line in FIG. 20, the area (object) is deleted. For example, an object that is determined to have appeared outside the monitoring range is deleted only when the space setting (world coordinate height information) is performed for the monitoring range (the height direction setting is set by the monitoring range space setting). If not, do not delete). Further, for example, as shown by a rectangle in FIG. 21, when the foot does not exist on the monitoring plane or the head position exists outside the monitoring area, the region is deleted.

  According to the above-described embodiment of the present invention, in the image processing of a single eye (one camera), the monitoring range (four vertices) and the position information on the captured image can be obtained without using camera information or camera installation conditions. The position on the shooting screen can be estimated by inputting three or more locations. In addition, it is possible to calculate the actual distance with respect to the position on the image, and the position of the object on the real world coordinates can be grasped with high accuracy. Further, when an object on the captured video is extracted (region), the actual distance of the size (width, height) can be estimated, and further, the actual distance of the position can be estimated. In addition, by setting a monitoring range on the captured video, an object outside the zone can be excluded and noise can be removed.

The figure which shows the operation | movement outline | summary of the image processing apparatus which concerns on embodiment which concerns on embodiment of this invention. The block diagram which shows the structure of the image processing apparatus which concerns on the said embodiment. The figure which shows the flow of the information between the components of the image processing apparatus which concerns on the said embodiment. 5 is a flowchart showing a processing procedure of the image processing apparatus according to the embodiment. The figure which shows the picked-up image for creating the reference table of the image processing apparatus which concerns on the said embodiment. The figure which shows the example of the monitoring range setting of the image processing apparatus which concerns on the said embodiment. The figure for demonstrating the positional information setting method of the image processing apparatus which concerns on the said embodiment. The figure which shows the example of an acquisition graph of the image processing apparatus which concerns on the said embodiment. The figure which shows the example of an acquisition graph of the image processing apparatus which concerns on the said embodiment. The figure which shows the example of an acquisition graph of the image processing apparatus which concerns on the said embodiment. The figure for demonstrating the monitoring range setting method of the image processing apparatus which concerns on the said embodiment. The figure which shows the example of an acquisition graph of the image processing apparatus which concerns on the said embodiment. The figure which shows the example of an acquisition graph of the image processing apparatus which concerns on the said embodiment. The figure for demonstrating the position coordinate of the image processing apparatus which concerns on the said embodiment. The figure which shows the example of an acquisition graph of the image processing apparatus which concerns on the said embodiment. The figure which shows the reference table structural example of the image processing apparatus which concerns on the said embodiment. The figure which shows the reference table structural example of the image processing apparatus which concerns on the said embodiment. The figure which shows the reference table structural example of the image processing apparatus which concerns on the said embodiment. The figure for demonstrating the object position estimation method of the image processing apparatus which concerns on the said embodiment. The figure which shows the processing operation example of the image processing apparatus which concerns on the said embodiment. The figure which shows the processing operation example of the image processing apparatus which concerns on the said embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11 ... Monocular camera, 12 ... Capture part, 13 ... Image processing memory | storage part, 14 ... Position estimation coefficient parameter calculation part, 15 ... Image processing part, 16 ... Display part (display device), 151 ... Change area extraction process part, 152 ... Tracking processing unit, 153... Object position / size estimation processing unit, 154.

Claims (9)

Based on the information of the four vertices on the screen indicating the monitoring range of the monocular camera and the screen information and measured information of three or more sample images taken by the monocular camera, the actual pixels for the depth direction on the screen are measured. An actual distance table obtaining means for obtaining a distance by an exponential approximation formula and creating an actual distance table in units of vertical pixels on the screen;
Based on the information of the real distance table, the information of the four vertices, the screen information and the actual measurement information of the sample image, the weight of the real distance for the pixels in the horizontal direction on the screen is obtained by a linear approximation formula, A horizontal weight table obtaining means for creating a horizontal weight table on the screen;
Based on the information of the real distance table, the information of the four vertices, the screen information and the actual measurement information of the sample image, the weight of the real distance for the pixels in the vertical direction on the screen is obtained by a linear approximation formula, Vertical weight table acquisition means for creating a vertical weight table on the screen;
Image processing means for holding each table as a reference table and executing predetermined image processing on an image captured by the monocular camera using at least one of the reference tables as a processing target When,
An image processing apparatus comprising:   The screen information and the actual measurement information of the sample image each include information indicating a distance from a reference position to a grounding position of a sample model in which the interval can be set, and information indicating a height and an interval of the sample model. The image processing apparatus according to claim 1.   The image processing apparatus according to claim 2, wherein the image processing unit executes a process of recognizing an actual position of an image captured by the monocular camera using the actual distance table.   The image processing apparatus according to claim 2, wherein the image processing unit executes a process of excluding an image captured by the monocular camera from a processing target using the real distance table.   3. The image according to claim 2, wherein the image processing unit executes processing for recognizing an entity of an image captured by the monocular camera using the actual distance table and the horizontal weight table or the vertical weight table. Processing equipment.   The image processing means includes processing means for setting the monitoring range in real space using the information on the four vertices, the real distance table, and the horizontal weight table or the vertical weight table. The image processing apparatus according to claim 2.   The image processing means estimates or specifies an image captured by the monocular camera using the actual distance table and the horizontal weight table or the vertical weight table. Image processing apparatus.   The sample image is shown as a rectangular area on the screen, and screen information and actual measurement information of the sample image are acquired based on the position and size of the rectangular area. Image processing apparatus. An image processing program for causing a computer to function as an image processing apparatus for a monocular camera,
Based on the information of the four vertices on the screen indicating the monitoring range of the monocular camera and the screen information and measured information of three or more sample images taken by the monocular camera, the actual pixels for the depth direction on the screen are measured. A processing function for obtaining a distance by an exponential approximation formula and creating an actual distance table in units of pixels in the vertical direction on the screen;
Based on the information of the real distance table, the information of the four vertices, the screen information and the actual measurement information of the sample image, the weight of the real distance for the pixels in the horizontal direction on the screen is obtained by a linear approximation formula, A processing function for creating a horizontal weight table on the screen;
Based on the information of the real distance table, the information of the four vertices, the screen information and the actual measurement information of the sample image, the weight of the real distance for the pixels in the vertical direction on the screen is obtained by a linear approximation formula, A processing function for creating a vertical weight table on the screen;
A processing function for holding each of the tables as a reference table and executing predetermined image processing on an image captured by the monocular camera using at least one of the reference tables as a processing target; ,
An image processing program for realizing a computer.

Images