JPH11266450A - Object area tracking system and object area tracking method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain the object area racking system that stably extracts an image area of a tracked object against a size change of the object in the image by utilizing it that the object moves on a plane fixed to a camera in the case of tracking the object through the use of a reference image segmented from the image. SOLUTION: A collation processing control section 13 allows an image size conversion section 14 to estimate a proper image size of a reference image at a collation position based on a past image or more and information related to the image stored in a passage information storage section 16, based on the assumption that a tracked object moves on a plane fixed to an installed camera, in the case of allowing an image collation section 15 to process the collation of an image entered by an image entry section 11 and stored in an input image storage section 12 with the reference image to extract a photographed area of the tracked object. The control section 13 allows the conversion section 14 to convert the size of the reference image into the estimated size. Thus, the collation of the image collation section 15 copes with a change in the size of the tracked object in its image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing technology of a monitoring system for automatically measuring a traffic flow and automatically detecting an unexpected event from an image captured by a monitoring camera mounted on a road, for example, and more particularly to such a technology. The present invention relates to an object area tracking device and an object area tracking method for tracking, for example, an area of a car running on a road by image processing.

[0002] Further, an object area tracking device by image processing which can be widely applied to, for example, track an object moving on a plane when a range captured as an image is wide, such as a monitoring camera in a station yard, and the like. The present invention relates to an object area tracking method.

[0003]

2. Description of the Related Art In recent years, there has been a demand for the development of a monitoring system that automatically measures a traffic flow from an image captured by a monitoring camera installed on a highway, a highway, or the like, and automatically detects an unexpected event.

[0004] The measurement of the traffic flow includes, for example, the measurement of the traveling speed of each vehicle. For this purpose, it is necessary to precisely extract the position of each vehicle at each time in an image. Examples of the detection of an unexpected event include detection of a flow different from the normal flow such as traffic jam or sudden stop, and detection of frequent lane changes due to an accident or a stopped vehicle. Extracting the running locus of each vehicle is also an important factor for detection.

As a conventional method for extracting a traveling trajectory of a vehicle in an image, an image position of the vehicle is first extracted by calculating a difference between a background image and an input image, and the position is temporally accumulated. For example, there are many methods for extracting a traveling locus (e.g., JP-A-9-91439 (object monitoring device) and JP-A-6-337938 (image processing device and distance measuring device)). As another method, there is a method of extracting a driving route by extracting a feature point of an object in an image, extracting an image position of a vehicle, and accumulating the extracted image position over time (for example, see Japanese Patent Application Laid-Open No. HEI 9-163572). 6
No. 195644 (detection device for target object) ").

However, in the method using a background image, it is necessary to prepare an image in which the vehicle is not shown in the image as the background image. Since the brightness and color of the background image fluctuate drastically depending on the weather, time, lighting conditions, and the like, frequent updating is required. In addition, on a road with a lot of traffic, the timing at which the vehicle is not reflected is very rare, so that it is difficult to generate a background image. Further, when a plurality of vehicles overlap, there remains a difficult problem that the changed region that has become one must be divided into a plurality of regions.

On the other hand, the method of extracting the feature points of the object from the image has a problem that the processing is stalled when the feature points are hidden by overlapping vehicles. The feature points are weather,
It is difficult to extract stably because the appearance greatly changes depending on the time or lighting conditions.

Therefore, when it is necessary to extract the movement trajectory of each vehicle, for example, in traffic flow monitoring or automatic detection of an unexpected event, a method using a background image as an image processing method or a feature point of an object in the image are used. Should not be used.

There are several methods for detecting the timing at which a vehicle enters a surveillance camera installed on a road. For example, in the case of a camera that captures an image of a traveling route from behind, the vehicle is imaged so as to enter from below the screen and exit upward. At this time, the approach of the vehicle can be detected by focusing on only the scanning line at the lower end of the screen and observing the change in luminance. This is generally known as a method using a spatiotemporal image.

[0010] When the timing at which the vehicle has entered is known, the region of the vehicle at the time of entry is cut out from the image and used as a reference image, and an image region corresponding to the reference image for each time image is subjected to pattern matching between the images. Extract by technique,
If the process of cutting out the image area is continued, the traveling locus of the vehicle can be extracted. In other words, this is a method of extracting a traveling locus by tracking an image area (window) in an image sequence.

The method of tracking an image area is more resistant to environmental changes such as lighting conditions than the method using a background image in that only the image at that time is used. Also,
There are roughly two methods for tracking a reference image in an image sequence. The first is a method of setting an initial reference image area including a vehicle when entering a vehicle and performing correlation matching with the image at each time using the template as a template, and the second is a method of matching at each time in a time series. Is a method of cutting out the image area obtained as a result of the above and updating it as a template for the next time.

However, for example, in an image input by a camera that shoots from behind in the traveling direction of the vehicle, the size of the image in each vehicle decreases with the passage of time.
Therefore, a method of using the initial reference image as a template and continuously using the template from the time of entry of the vehicle to the end of passing the vehicle has a great risk of failure in tracking as the vehicle moves. On the other hand, even in the method of updating the template over time, if the size of the image cut out as the reference image is constant, the size of the vehicle becomes smaller than the size of the template in the same manner. Therefore, as the image pattern of the reference image, the background pattern and the shading pattern become dominant over time as compared with the shading pattern of the vehicle itself, so that the tracking also ultimately fails.

If the imaging range of the surveillance camera is widened, images are taken in a wide range in the depth direction, and the problem that the size of the vehicle changes in the image inevitably occurs.
Further, a method of performing tracking with a variable template size has also been proposed (for example, “Gregor”).
yD. Hager and Peter. Belhu
meur "Real-Time Tracking o
f ImageRegions with Chang
es in Geometry and Illumi
nation "" Proceedings Compu
terVision and Pattern Rec
opinion '96 "pp. 403-410"),
These methods only expand the search space in consideration of the size change, and thus only unilaterally increase the processing amount.

That is, in the conventional method of tracking a reference image in an image sequence, no method has been proposed which focuses on a change in the size of a vehicle in such an image and improves tracking performance.

[0015]

As described above, in the conventional method of extracting the vehicle trajectory by extracting the vehicle location from the image and accumulating the same, the vehicle position at the first stage is extracted. There has been a problem that the extraction process is unstable with respect to vehicle overlap and environmental changes.

Also, the method of tracking the reference image in each image sequence has a problem that it is unstable with respect to a change in the size of the target object that occurs when the motion of the object occurs in the depth direction.

The present invention has been made in view of the above circumstances, and in tracking an object using a reference image cut out from the image, the object moves on a plane fixed to a camera. The object of the present invention is to provide an object area tracking apparatus and an object area tracking method that can stably extract an image area of a tracking target object even when the size of the object in the image changes.

[0018]

In order to achieve the above-mentioned object, the present invention provides an image input means for inputting a two-dimensional image of an object in time series, and an image input means for inputting the two-dimensional image. Input image storage means for storing an image, progress information storage means for storing at least one set of an image including a tracked object at a past time point and information on the image, and The position and size of the object to be tracked are estimated, and the position and size of the object to be tracked and the image including the object to be tracked stored in the progress information storage means and information on the image are used to determine the tracking object. Image size conversion means for generating a reference image of an appropriate size for extracting an imaging region of an object; and a reference image generated by the image size conversion means. Image matching means for comparing an image stored in the input image storage means with an image stored in the input image storage means, and an output for outputting position information of the tracking target object extracted by the image matching means Means.

That is, according to the present invention, when a reference image for extracting an imaging region of a tracking target object is collated with an input image, the tracking target object moves on a plane fixed with respect to a camera installed. Based on this assumption, an appropriate image size of the reference image at the collation position is estimated and converted, and then collation processing is performed to cope with a change in the size of the tracking target object.

This makes it possible to stably extract an image area even when the size of the tracking target object changes when the movement of the object occurs in the depth direction. The way is open to realization.

[0021]

Embodiments of the present invention will be described below with reference to the drawings. (First Embodiment) FIG. 1 is a block diagram showing a configuration of an object area tracking device according to a first embodiment of the present invention.

In the following, the first
The operation of the object area tracking device according to the embodiment will be described. The image captured by the image input unit 11 is A / D converted and stored in the input image storage unit 12. Initial stage when the object to be tracked has just entered the screen (time t = 0)
Then, the progress information storage unit 1 stores the initial position of the object on the screen and the information on the shape of the initial area from the entry detection unit (not shown).
6 is sent.

In the object entry stage, the area at the position is cut out from the input image in accordance with the position information and the area information of the object transmitted from the entry detection section, and stored in the progress information storage section 16 together with the object identification number. . That is, a partial image including the target object is cut out from the input image and stored in the progress information storage unit 16.

In the following description, the image area of the object to be extracted is also referred to as a window, such as a "tracking window" or an "image feature inside the window". Also,
A window used as a reference image for the matching processing is expressed as a template. The template extracted at this initial stage (time t = 0) is
(0; i, j).

In the object area tracking device according to the first embodiment, the entry detecting section operates only in the object entry stage, and is unnecessary at the stage of performing the tracking process which is the main object of the present invention. is there.

Before explaining the operation at a general time other than the initial stage, first, the basic functions of the collation processing control unit 13, image size conversion unit 14, and image collation unit 15 will be described.

Collation processing control unit 13 and image collation unit 15
Is based on the partial image T (t; i, j) in which the position information and the area information have been specified at the time t, and from the image I at the time t + 1, the image area I (u) corresponding to the template. , V; i, j).

(I, j) of the template T (t; i, j)
j) is generally an expression that the template has size and shape. The image is a two-dimensional space spanned by two axes of UV. For example, if the template T (t; i, j) is an m × n rectangle,

[0029]

(Equation 1) Means that

The partial image area serving as a template may be of any shape including the object to be tracked, and may be, for example, a rectangle, a circle, a parallelogram, or a combination of (i, j) in the area. Any complex shape can be handled. In the first embodiment, for simplicity of explanation, a case where the template has a rectangular shape will be described as the most obvious example.

The image area at time t (template T
(T; i, j)) at time t + 1 at the corresponding position (u,
To obtain v), (2) from the image at time t + 1
It is sufficient to find a position where the residual R (t; u, v) (subscript t means the calculation result with the template T (t; i, j)) expressed by the formula is minimized. This function is realized by the image matching unit 15.

[0032]

(Equation 2)

Here, m and n are templates T (t;
i, j) in the image, which is m pixels wide and n pixels high. In addition, as a value indicating the goodness of correspondence between the template and the corresponding position, a correlation coefficient C (t;
An image area in which u, v) is maximized may be obtained.

[0034]

(Equation 3)

In addition, minimization of the sum of the absolute values of the differences in image luminance can be applied. In any case, pattern matching between regions based on brightness correlation between partial images is performed. In the following description, these “indexes expressing good correlation between images” are collectively expressed as a correlation value.

Considering the situation where a vehicle is tracked by an image of a surveillance camera installed on a road, as shown in FIG. 2, as the vehicle moves on the road in the depth direction, the image of the vehicle There is a problem that the size of the image changes (reduces).

For this reason, a correct corresponding position cannot be obtained only by performing pattern matching using the equation (2) or the equations (3) to (7) while keeping the template size fixed.

Therefore, in the first embodiment, an image size conversion unit 14 is prepared in order to cope with a change in the size in the image accompanying the movement of the vehicle. In particular,
Pattern matching is performed by equation (8) instead of equation (2).

[0039]

(Equation 4) The equation corresponding to equation (3) is equation (9).

[0040]

(Equation 5)

Here, M (u, v) is the template T
The width of the template Ts (t; u, v; i, j) when (t; i, j) moves to the image position (u, v),
Similarly, N (u, v) is the template Ts when the template T (t; i, j) moves to the image position (u, v).
(T; u, v; i, j). The template Ts (t; u, v; i, j) is obtained by reducing or expanding T (t; i, j) to a size of M (u, v) × N (u, v). Then, M (u, v) and N (u, v)
Can be calculated from the template image position (u, v) and the template size (m and n) at time t.

To calculate the size of the template at the image position (u, v), the image size conversion unit 14
As shown in FIG. 3, a plane information storage unit 141 is prepared.
In the plane information storage unit 141, parameters for calculating the size of a vehicle traveling on a road in an image are measured and stored in advance.

For example, as shown in FIG. 4, when the horizontal level of the camera is set horizontally and the road surface can be assumed to be a horizontal surface, the inclination angle ψ of the optical axis of the camera, the height Hc of the focal position of the camera, , And the camera-level focal length f is stored as a camera parameter. At this time, the height of the back of the vehicle is H
h, when the width of the vehicle is W, the following formula (10) is provided between the vertical length h of the back surface in the image and the width w of the vehicle in the image and the coordinate value (u, v) in the image. The relationship of Expression (11) holds.

[0044]

(Equation 6)

That is, when the object to be tracked moves on a horizontal plane, the size of the object to be tracked in the image is represented by the coordinate value (v coordinate value) and the equations (10) and (11). Has a linear relationship. This relationship is shown in FIG.
The same holds true for curved roads as in 1) and (b-2).

Alternatively, there is also a method in which the relational expression is obtained by measuring the relational expression from a sample image using the knowledge that the size of the tracking object in the image and the coordinate value have a linear relation.

More specifically, a sample image is sampled from the input image storage unit 12 in advance, and the v-coordinate and the width of the road on the image on the v-coordinate are measured using the paint edge of the road or the like. Do the work of storing in the table. If the width of the road in the real space is constant (sampling from such a section is possible), the rate of change of the road width in the image with respect to the v coordinate can be calculated, and the rate of change is the same as the width of the vehicle. It is.

In order to take into account the distortion of the image and the inclination of the road, it is only necessary to sample several reference points of the v coordinate and the road width, and then approximate by interpolation using a linear expression or a higher-order function. Alternatively, there is a method of storing the v coordinate and the size of the object in an image in a table, instead of storing them as camera parameters or linear expressions. This method is simpler when the road surface has a slight inclination, or when the camera parameters cannot be measured when the camera is installed.

Under more general conditions, the moving direction of the object is not limited to a fixed plane, but can be considered to be on a surface having irregularities or inclination. In any case, as long as the object to be tracked does not fly in the air such as an airplane or an insect, the object to be tracked moves on a fixed surface fixed to the camera. Therefore, when it is assumed that the image moves on this plane, the size of the image at each position on the screen should be checked in advance and stored as a table. It is not limited to monitoring only a moving object on a plane.

As described above, when the position (u, v) of the template in the image is received, the image size conversion unit 14 determines the optimal template size M (u, v) × N for the position.
(U, v) can be calculated. Since the size of the template T (t; i, j) at time t is m × n, if the calculated optimal template size is different, the template size may be converted by geometric conversion of various images. .

As described above, the image size conversion unit 1
The function of No. 4 stores the relationship between the height of the back of the car and the image position (u, v) by a parameter or table based on the geometrical positional relationship, and stores a partial image T including the target object.
Given (t; i, j), a template of an appropriate size is created for each image position (u, v) in consideration of the size conversion accompanying the movement of the vehicle, and image matching is performed. Part 15
Is to send to.

The operation at the general time (time t + 1) of the object region extracting apparatus having the above-described collation processing control unit 13, image size conversion unit 14, and image collation unit 15 is as follows. First, a new image is input by the image input unit 11 and stored in the input image storage unit 12. Collation processing control unit 1
3 determines the search range using the trajectory information of the target object from the progress information storage unit 16 to time t. The easiest way to determine this is to determine, for example, to search within a rectangle of ΔU × ΔV around the position (U, V) at time t. In addition, another method of determining the search range is described in a second method described later.
The embodiment will be described.

The collation processing control unit 13 further transmits a control signal to the progress information storage unit 16, and the progress information storage unit 16
Generates a template to be a reference image and generates an image matching unit 15
Send to The progress information storage unit 16 stores the input image at time t and the position (u,
v) and the size m × n. When receiving the control signal from the collation processing control unit 13, the progress information storage unit 16 cuts out the image of the area (the image cut out in advance may be stored) and stores it in the template T.
(T; i, j).

The matching processing control unit 13 performs matching processing with the input image while appropriately converting the size of the template within the rectangle of ΔU × ΔV. FIG. 5 schematically shows the matching operation.

Assume that the control points of the template are set as shown in FIG. The matching processing control unit 13 scans the control points around ΔU × ΔV around the control points at the time t + 1, and the image size conversion unit 14 determines the template Ts (t; u, v; i, j) size M
(U, v) × N (u, v) is calculated to generate an image converted to an appropriate size, and the image matching unit 15 generates the size-converted template TS (t; u, v).
v; i, j) and the correlation between the input image and the input image are calculated by the equations (2) and (3), and the position (umax (t + 1), vmax) where the correlation value is the largest or the square error is the smallest.
(T + 1)) is extracted.

By the above processing, the image position (umax (t + 1), vmax (t +
1)) and the image area size M (umax (t + 1), vma
x (t + 1)) × N (umax (t + 1), vmax (t +
1)) are obtained, and the result is output from the output unit 17 as numerical or graphical information. Further, it is stored in the progress information storage unit 16 as the position and the area at the time t + 1.

Thus, the flow of the process at the time t + 1 is completed, and the process proceeds to the next time. FIG. 7 shows a basic processing procedure of the object area tracking device of the first embodiment, and FIG. 8 shows a detailed processing procedure of the matching processing.

(Second Embodiment) Next, a second embodiment of the present invention will be described. In the second embodiment, a setting method different from the setting method of the rectangular search area described in the first embodiment will be described.

In the first embodiment, if the actual size of the vehicle is constant, a linear expression such as Expressions (10) and (11) is provided between the coordinate values in the image and the size in the image. Utilizing the relationship, the vehicle was tracked by actively changing the size of the template for tracking.

Similarly, if the moving distance of the vehicle is within a certain range, the linear relationship also exists between the coordinate value in the image and the moving distance (the number of moving pixels of the template) in the image of the vehicle. Holds. That is, as the vehicle moves deeper, the moving distance in the image also decreases in a proportional relationship, and as a result, the range in which the template matching process is performed can be reduced.

More specifically, for example, it is assumed that the maximum speed of the vehicle is 120 km, and that the image input is performed at the frame rate of the television signal (1/30 second). At this time, since the vehicle travels about 1 m per frame, the maximum number of pixels moving on the image can be estimated using the above-described parameters of FIG.

More simply, first, a collation area ΔU (0) × ΔV (0) is set using some image samples so as to sufficiently include the moving amount in the image of the traveling vehicle when the vehicle enters. . If there is enough processing time, the collation area may be set to be relatively large.

Similarly to the equations (10) and (11), the amount of movement in the image at the time of vehicle entry and the amount of movement after the vehicle moves and the position of the vehicle in the image changes, as in equations (10) and (11). Since there is a relation that can be approximated by the equation, the size of the collation area is ΔU (0) × ΔV according to the relation stored in the image size conversion unit 14.
(0) is the size ΔU (t) × ΔV corresponding to the position
It is sufficient to change to (t) and perform the collation processing.

(Third Embodiment) Next, a third embodiment of the present invention will be described. In the first embodiment, a partial image is extracted from the image at the time t and the template T (t; i,
j), and the corresponding position and region were obtained from the image at time t + 1. This method corresponds to sequentially updating the template.

The sequential updating of the template is performed by changing the lighting conditions depending on the road surface position (for example, when the front is bright and the back is dark).
Or changes in lighting conditions during the transit time of the traveling vehicle (such as when a cloud suddenly hits), or changes in the appearance of the vehicle due to a change in the angle at which the vehicle looks down (at the point directly below the camera, Even if the part looks large, only the rear of the vehicle becomes visible as the camera moves away from the camera, etc.). However, on the other hand, there is a characteristic that the tracking ability deteriorates when the background becomes dominant in the window.

Assume, for example, that a tree is planted beside the road as shown in FIG. In FIG. 9A, since the vehicle occupies most of the template, the tracking window tracks the vehicle as the vehicle advances. Then, when proceeding to the stairs in FIG. 9B, the texture of the tree enters the window.

For example, when the color of the vehicle is dark and close to the road,
If the vehicle has poor image characteristics (texture),
With respect to the values indicating the good correlation between the images, such as Expressions (2) and (3), the contribution of the texture of the tree to the calculated value may be greater than the contribution of the vehicle texture to the calculated value. is there. In this case, the texture of the tree becomes dominant inside the window.

Therefore, if the tracing process is continued by successively updating the template as it is, as shown in FIG.
As shown in (d), the tracking window remains attached to the tree. In addition, assume that a high-speed vehicle approaches a low-speed vehicle from behind as shown in FIG. 10 and overlaps on an image. In this case, FIG.
In FIG. 10B, there is no problem because each is covered with a separate rectangle that does not overlap. However, in the state shown in FIGS. 10C and 10D, the two areas overlap, and in particular, FIG.
In the state of 0 (d), a part of the vehicle behind enters the tracking window for the vehicle traveling ahead.
As time goes on, the vehicle behind will become dominant in the tracking window of the vehicle in front, and the tracking window for the vehicle in front will track the vehicle behind.

Between the times shown in FIGS. 9 and 10,
The correlation value between the tracking windows is high, and it cannot be determined that such a phenomenon has occurred. Also, if the template becomes smaller as the vehicle moves further away, sufficient texture for tracking may be insufficient. For example,
In the case of FIG. 11, the size of the template is sufficiently large (for example, about 100 × 100 pixels) at the time of FIG. 11A, so that the time J in FIG. 11A and the time J in FIG.
Both +1 windows have sufficient texture. However, when the time advances and the state as shown in FIGS. 11C and 11D is reached, the size of the template becomes smaller (for example, about 10 × 10 pixels), and the time N in FIG.
To calculate the correlation value between the templates at time N + 1 in FIG. 11D, the texture may be insufficient.

In this case, there is a method of stopping the sequential updating of the template and using the template T (0; i, j) at the time of vehicle entry from the vehicle entry until the vehicle passes through (by converting the size) consistently. However, this method is unstable with respect to a change in the lighting conditions in terms of position or time and a change in the appearance of the vehicle.

In the example shown in FIG. 10, when the tracking of the vehicle in front can be ended when a plurality of tracking windows overlap on the image, the window tracking that has occurred in time is ended and the old window is deleted. You can address this problem by leaving only the new window.

In the case where the tracking window sticks to the background as shown in FIG. 9, if the background area can be set to be ignored in advance, this problem can be dealt with by not calculating the correlation with the background area. .

However, there are many cases where such measures cannot be taken. Therefore, in the third embodiment, a method for uniformly solving many of such problems, including a case where the size of the vehicle is reduced, and enabling robust tracking will be described.

In order to solve the above-described problems, the object area tracking device according to the third embodiment uses the information of the tracking result of the vehicle stored in the progress information storage unit 16 so far in the collation processing. Use.

FIG. 12 is a diagram showing the internal configuration of the progress information storage unit 16.
61 and the position sequence storage unit 12. The position series storage unit 162 stores, for each object identification number, an image number (frame number of the input image), a position in the image of the object (control point of the template), and a size in the image of the object (size of the template). Is stored.

The image sequence storage unit 161 stores, for each object identification number, a partial image (template) extracted according to the image number (frame number) and the position and area of the object in the image number as an image sequence. Is done.
Alternatively, the input image itself may be stored together with the image number as a series, and cut out when necessary.

Therefore, at time t + 1, the progress information storage unit 16 stores the partial image sequences T (0; i, j), T
(1; i, j), T (2; i, j),..., T (t; i,
j) is stored.

The matching processing at time t + 1 is performed by using ΔU centered on the position (u (t), v (t)) at time t.
From the collation search area of (t) × ΔV (t), the following (1)
2) A process for finding a position where the residual SR represented by the equation is minimum.

[0079]

(Equation 7)

Here, w (k) is a weight coefficient for the k-th template. The weighting factor may be determined based on the specifications of the tracking device (eg, how much tracking is continued when templates overlap).

Normally, in order to be robust to the spatial or temporal change of the lighting environment, the weight of the template cut out from the image at the previous time is reduced by half, and the weight of the template cut out from the image at an earlier time is reduced. Weight is 1 in total
/ 2 may be set.

[0082]

(Equation 8)

In the third embodiment, the linear sum of the voting values of the correlation values is calculated for a plurality of templates, and the position (u, v) at which the voting value is maximized is extracted. I used the information.

However, the method of using a plurality of templates is not limited to this, and it should be added that there are various methods of using. For example, each template T
A voting value is obtained for each (t; i, j), a position (u (T (t)), v (T (t)) where the voting value is maximum is obtained, and the position is used as a new voting value. The method may be used to extract an optimum position using the method, or a weighted barycentric position of a position (u (T (t)), v (T (t)) obtained for each template may be calculated. .

In practice, it is sometimes difficult to store the image areas at all times and calculate the equations (12) and (13) due to the limitation of the memory capacity and the calculation time. In this case, for example, an image area is stored at a fixed time interval (for example, an image area is stored every 5 frames or every 1 second), or only a template at a fixed time interval (12
If the calculation of the expression 2) or the expression (13) is performed, the load on the storage area and the amount of calculation is reduced.

Although the sampling of the storage or calculation object can be controlled at time intervals, there is also a method of sampling at intervals of positions in the template image. For example, when the size of the input image is 300 pixels high, a sampling line may be set every 10 pixels, and a sample may be extracted every time the tracking window crosses the sampling line. An advantage of sampling based on the position interval is that the number of template image sequences that must be stored for each vehicle is constant regardless of the passing time of the vehicle, so that the capacity of the storage device can be kept constant (for example, during traffic jams). Since the transit time becomes longer, sampling by time intervals requires a large storage device).

On the other hand, sampling at time intervals is equivalent to successively updating the template sequence, so that there is an advantage that stable tracking can be performed even when a change occurs in the illumination environment due to a time change.

As described above, by using the sequence image of the tracking result provided with the progress information storage unit 16 having such a configuration, it is possible to perform tracking that is robust against changes in the background and the appearance of the vehicle.

(Fourth Embodiment) Next, a fourth embodiment of the present invention will be described. The application of the present invention is not limited to the vehicle monitoring system, and can be applied to any tracking device in an environment in which the moving direction of the moving object moves on a plane fixed to the camera. Therefore, in the fourth embodiment, a surveillance camera in a station premises will be described as an example. FIG.
3 is a diagram schematically showing the installation environment of the monitoring camera,
FIG. 14 is a diagram schematically showing an input image of the monitoring camera.

In the case of a surveillance camera in a station premises, a person to be monitored moves on a plane fixed to the camera. Although the background is omitted in the schematic diagram of FIG. 14, there are many objects having complicated shapes and patterns within the angle of view of the surveillance camera, and the background is extremely complicated.

In such an environment, stable tracking is difficult unless the present invention is applied to perform tracking processing in consideration of a change in image size due to movement of a tracking target. In the application of the present invention, once the initial area is set when the tracking target enters, the subsequent tracking processing can be used in exactly the same manner.

In this case, the difference from the vehicle monitoring on the road is that the painting of the road cannot be used at the stage of storing the position information of the plane in the image size conversion unit 14.
As described in the first embodiment, the position information of the plane may be calculated from parameters such as the installation angle and the installation position of the camera.

Further, since the present invention is based on the sequential update of the template (the third embodiment has described a method more stable than the simple sequential update), even when the object to be tracked is a non-rigid body like a human. Applicable. Obviously, tracking non-rigid bodies is difficult with a method that uses the initial template consistently from vehicle entry to vehicle exit.

[0094]

As described above in detail, according to the present invention, in order to automatically track a tracking target object in an image by image processing, the object moves on a plane fixed to the camera. This makes it possible to stably extract the image area of the tracking target object even when the size of the object changes in the image. As a result, it is possible to play an important role in the automation of the surveillance camera installed on the road or in the station premises, and this practical effect is very large.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an object area tracking device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing an event in which the size of an image of the vehicle changes (reduces) as the vehicle according to the first embodiment moves on a road in a depth direction.

FIG. 3 is a diagram showing an internal configuration of an image size conversion unit according to the first embodiment.

FIG. 4 is an exemplary view for explaining parameters stored in a plane information storage unit of the image size conversion unit according to the first embodiment;

FIG. 5 is a diagram schematically showing a matching operation of the first embodiment.

FIG. 6 is a diagram illustrating control points of the template according to the first embodiment;

FIG. 7 is a flowchart showing a basic processing procedure of the object area tracking device according to the first embodiment;

FIG. 8 is a flowchart showing a detailed processing procedure of a matching process in FIG. 7;

FIG. 9 is a view for explaining an event in which the tracking window of the third embodiment is stuck to the background.

FIG. 10 is a view for explaining an event in which tracking windows overlap in the third embodiment.

FIG. 11 is an exemplary view for explaining an event that the size of the tracking window according to the third embodiment becomes extremely small.

FIG. 12 is a diagram showing an internal configuration of a progress information storage unit according to the third embodiment.

FIG. 13 is a diagram schematically showing an installation environment of the monitoring camera according to the fourth embodiment.

FIG. 14 is a diagram schematically showing an input image of the monitoring camera according to the fourth embodiment.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 ... Image input part 12 ... Input image storage part 13 ... Collation processing control part 14 ... Image size conversion part 15 ... Image collation part 16 ... Progress information storage part 17 ... Output part 141 ... Plane information storage part 142 ... Optimal size calculation part 143: size-converted image generation unit 161: image sequence storage unit 162: position sequence storage unit

Claims (10)

[Claims] 1. An image input unit for inputting a two-dimensional image of an object in a time series, an input image storage unit for storing an image input by the image input unit, and a tracking target object at a past time point And a progress information storage means for storing at least one set of images and information related to the images, and a position and a size of the tracking target object in the image stored in the input image storage means, and the estimated tracking target Generating a reference image of an appropriate size for extracting an imaging region of the tracking target object from the position and size of the object, an image including the tracking target object stored in the progress information storage unit, and information on the image Image size conversion means for comparing the reference image generated by the image size conversion means with the image stored in the input image storage means, An image matching means for extracting the image capturing area of the tracking target object, the image matching unit object region tracking apparatus characterized by comprising an output means for outputting the position information of the tracking target object extracted by. 2. The image size conversion means calculates a position of a tracking target object in the image and its size according to a positional relationship between a surface on which the tracking target object moves and a surveillance camera. Object area tracking device. 3. An image including a tracking target object at a past time point stored in the progress information storage means is used as an individual reference image, and a collation process with an image stored in the input image storage means is comprehensive. 2. The object area tracking apparatus according to claim 1, further comprising a matching processing control means for controlling the image matching means and the image size converting means matching processing so as to be applied. 4. An image including a tracking target object at a past time point stored in said progress information storage means is used as an individual reference image, and a matching process with an image stored in said input image storage means is individually performed. And a collation processing control means for controlling the image collating means and the image size converting means such that a position and an area at which the linear sum of the collation values obtained by applying the optimal value shows the optimum value are extracted. 2. The object area tracking device according to claim 1, wherein: 5. An input image storage means for calculating an image area in the image which needs to be subjected to a matching process in accordance with a positional relationship between a surface on which a monitoring target moves and a monitoring camera, and the calculated image area. 2. The object area tracking according to claim 1, further comprising a matching processing control means for controlling the image matching means and the image size converting means so as to perform a matching processing between the image stored in the memory and the reference image. apparatus. 6. A two-dimensional image in which an object is captured is input as a time series, the input image is stored, and at least one set of an image including an object to be tracked at a past time and information on the image is stored. An image including at least one set of the position of the tracked object in the stored image and the information related to the image, and including the estimated position and size of the tracked object and the stored tracked object. And generating a reference image of an appropriate size for extracting an imaging area of the tracking target object from the information on the image and the image, and comparing the generated reference image with the stored image to obtain the tracking target object. An object area tracking method, comprising: extracting an imaging area of a target object; and outputting position information of the extracted tracking target object. 7. The calculation of the position and size of the tracking target object in the image is performed according to the positional relationship between the surveillance camera and the surface on which the tracking target object moves.
The tracking method of the object area according to the description. 8. The collation between the generated reference image and the stored image is performed comprehensively by using each of the stored images including the tracking target object at a past time point as an individual reference image. 7. The object area tracking method according to claim 6, wherein the object area tracking method is performed. 9. A method of comparing the generated reference image with the stored image, wherein each of the stored images including the tracking target object at the past time is used as an individual reference image, and the input image storage is performed. 7. The object area tracking method according to claim 6, wherein a position and an area in which a linear sum of collation values obtained by individually performing collation processing with the image stored in the means shows an optimum value are extracted. . 10. The collation between the generated reference image and the stored image needs to perform a collation process in the image according to a positional relationship between a surveillance target moving surface and a surveillance camera. 7. The object area tracking method according to claim 6, wherein the method is performed on an image area calculated as an image area.