JPH09231363A - Object monitoring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To quickly and highly accurately extract each of plural objects. SOLUTION: An changed area inter-label distance calculating means 6 calculates a distance between a changed area labels or horizontal and vertical distances between the lables based upon the gradation picture of a changed area extracted by an image pickup means 1, an A/D conversion means 2, a changed area extracting means 4, etc. A changed area label integrating means 12 integrates changed area labels in which the inter-label distance or inter-label horizontal and vertical distances are less than a fixed value as one object based upon thresholds obtained from setting means 10A, 10B, so that the object can be quickly and highly accurately extracted. There are various deformed examples capable of furthermore accelerating operation by reducing the number of label combinations.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention detects the presence or absence of a specific object by extracting a target object from a picked-up image obtained through an image pickup means such as a television camera and tracking the object over time. Or count the number,
The present invention relates to an object monitoring device that uses image processing and has a function of measuring the moving speed of an object.

[0002]

2. Description of the Related Art Conventionally, as an object tracking processing apparatus using image processing, for example, one shown in FIG. 9 is known (see Japanese Patent Laid-Open No. 4-36014: Conventional method 1).
This is an A / D that converts an analog image input from the image pickup means 31 such as a television camera or a camera into a digital image.
A conversion unit 32, a background image memory 33, a difference processing unit 34 for extracting changes between the input image and the background image, a binarization processing unit 35,
The feature amount extraction unit 36, the feature amount association processing unit 37, the motion amount extraction processing unit 38, the determination processing unit 39, the region-based feature amount setting unit 40, and the alarm output unit 41 are included. And the difference binarized image
The area and the center of gravity are associated as a feature amount.
Therefore, even an object existing in the screen does not have a portion having a density close to the background, and the feature amount such as an area changes due to this, so that the target object may not be accurately extracted.

Therefore, the applicant has proposed an intruding object monitoring device (special feature) as an improvement plan for extracting an object even if one object is separated into a plurality of labels or a part of a plurality of objects overlap. See Japanese Patent Application No. 5-162593: A conventional method 2) is proposed. This is configured by adding changing area combining means or changing area separating means, matching degree calculating means, changing area integrating means, etc. to the conventional method 1. By combining or dividing a plurality of labels, a target object is obtained for each combination. By calculating the degree of coincidence with and selecting the combination with the highest degree of coincidence, the target object can be accurately extracted.

[0004]

However, in the conventional method 2, the number of combinations of n labels is (2 ⁿ -1). Therefore, when the number of labels is large, the degree of coincidence is increased for a huge number of combinations. There is a problem that calculation becomes necessary and the processing time becomes too long to be practical. Further, the depth of the surveillance plane is not usually taken into consideration, and therefore, there is a problem that the object extraction cannot be performed accurately when the surveillance plane has a depth. Therefore, an object of the present invention is to enable the target object to be extracted at a higher speed in the case where the target object is divided into a plurality of labels and extracted.
Another object is to enable accurate extraction of the target object even when the monitoring plane has a depth.

[0005]

In order to solve such a problem, the present invention adopts any one of the following (1) to (6) as a method of integrating the labels of the change areas. (1) The distance between the labels on the image is obtained, and the set of labels whose distance is a fixed value or less is integrated as one object. (2) A distance threshold between labels is set separately in the vertical direction and the horizontal direction, and a set of labels in which at least one of them has a distance equal to or less than the threshold is integrated as one object. (3) One label set whose distance on the image is less than a certain value
Labels are combined into one group, and the set of labels having the highest degree of coincidence with the shape model of the object is integrated as an object. (4) The distance threshold between labels is set separately in the vertical direction and the horizontal direction, and labels in which at least one of them has a distance equal to or less than the threshold value are grouped into one group, and the labels are combined only in that group. Then, the set of labels having the highest degree of coincidence with the shape model of the object is integrated as the object.

(5) The target object is in contact with the plane of the surveillance area due to the positional relationship (distance, angle) between the plane of the surveillance area and the image pickup means such as a camera, the focal length of the lens, and the change area label extraction position. Under the condition that the target object occupies the size (width, height) on the screen, the horizontal distance is less than the object width and the vertical distance is A set of labels satisfying at least one of two conditions of being less than or equal to the height is integrated as one target object. (6) Depending on the positional relationship (distance, angle) between the plane of the surveillance area and the image pickup means such as a camera, the focal length of the lens, and the change area label extraction position, the target object is in contact with the plane of the surveillance area. First, the size (width, height) that the target object occupies on the screen is roughly estimated, and for each label of the change area, the horizontal distance is less than the object width, and the vertical distance is less than the object height. The group of labels satisfying at least one of the two conditions of is defined as one groove, and the combination of labels is performed only in the group, and the group of labels having the highest degree of coincidence with the shape model of the object is integrated as the object. To do so.

[0007]

FIG. 1 is a block diagram showing a first embodiment of the present invention. In the figure, 1 is an image pickup unit such as a television camera, 2 is an A / D conversion unit, 3 is an input image storage unit at the current time, 4 is a change region extraction unit, 5 is a change region binary image, and 6 is a change. It is an inter-region distance calculation means. Further, 12 is a change area label integration means, which means A uses the change area distance threshold of 10A, and means B uses the change area horizontal distance and vertical distance threshold of 10B to integrate change areas. To do. Reference numeral 11 is an object shape model, 14 is an object coincidence degree calculation means, 16 is an object position / coincidence degree storage section, 17 is a motion extraction means, 18 is an object movement storage section, 19 is result calculation / output means, and 20 is a monitoring result. Show.

The image captured by the image pickup means 1 is
It is digitized by the A / D conversion means 2 and stored in the input image storage section 3. The changed area extracting means 4 generates an area having a density change of a certain level or more as a changed area grayscale image 5. The changing area grayscale image 5 is input to the changing area distance calculating means 6, where the distance between the circumscribed rectangles is calculated for all combinations of labels of the changing areas. Here, the horizontal distance and the vertical distance between the labels, and further the straight line distance are calculated. The changing area label integrating means A refers to the inter-changing area threshold value set to 10A and integrates a label group having a distance equal to or less than this threshold value as one object area.

On the other hand, the label integrating means B has separate distance threshold values in the horizontal direction and the vertical direction, and one or both of them are integrated as a single object area with a label group equal to or less than the threshold value. With respect to the label group of this integration result, the object coincidence degree calculating means 14 calculates the object coincidence degree using the object shape model 11, and stores the result in the object position / coincidence degree storage unit 16. The motion extracting means 17 extracts the motion by temporally corresponding to the object, and the object motion storage unit 1
8 is stored. The result calculation / output means 19 calculates and outputs a necessary result from the object position / coincidence and the movement of the object. Reference numeral 20 indicates a monitoring result such as presence / absence of a monitoring target, the number of monitoring targets, or a monitoring target speed.

FIG. 2 shows a first example of integration of changing area labels. FIG. 10A shows an example of the change area of the monitor image, and here is an example in which seven labels 1 to 7 exist.
As the distance between these labels, the distance between the x and y directions of the circumscribed rectangle of the two labels is the horizontal distance d.
Assuming that x is the vertical distance dy, the inter-label distance d can be expressed as follows: d = √ (dx ² + dy ² ) ... (1) If the circumscribed rectangles partially overlap, the distance is set to "0". That is, it can be said that the label integrating means A is used to perform the integration based on the distance between the labels. Note that FIG. 2B shows the label-to-label distance of the change area image of FIG. 2A, and FIG. 2C shows the label-to-label distance ≦ 20.
Those satisfying the above are shown by enclosing with "○". Further, FIG. 2D shows the result of the combination.

Furthermore, the conditions for integration also differ depending on the shape of the object. That is, in the case of a shape close to a circle or a square, integration may be performed based on the distance between labels, but in the case of an object whose aspect ratio is not “1” such as a rectangle, the distance in the vertical direction and the distance in the horizontal direction. Different object thresholds may give better object extraction results. In addition, depending on the shape and background of the object, it may be possible to sufficiently extract the object only by integrating the image in one horizontal or vertical direction. Therefore, the criterion for integration may be determined from the horizontal distance between labels, the vertical distance, and a combination thereof. FIG. 3 is a second example of integration of changing area labels based on this viewpoint.

In the change area image as shown in FIG. 2A, the distance between the labels in the x direction is shown in FIG. 3A, and the distance between the labels in the x direction is shown in FIG. 3B. Shown respectively. Those enclosed by "○" in Fig. 3 (a) are d
In FIG. 3B, a group of labels satisfying x ≦ 8 and dy ≦ 18 is shown. FIG. 3C shows the process of integrating the label sets. Here, if the labels that satisfy both dx ≦ 8 and dy ≦ 18 are integrated, the labels 1, 2, 3 and the labels 4, 5, 5, 6 and 7 satisfy these conditions. The labels are integrated as shown in FIG. That is, it can be said that the label integrating means B is used to perform the integration based on both the horizontal distance and the vertical distance between the labels.
Note that the integration may be performed based on only one of the horizontal distance and the vertical distance.

FIG. 4 is a block diagram showing a second embodiment of the present invention. This is the changing area integrating means 1 of FIG.
It is characterized in that a changing area label group creating means 12 'is provided instead of 2, and an in-group label combination creating means 13 and an in-group label combination selecting means 15 are added. FIG. 5 is an explanatory diagram for explaining the change area label integration operation in FIG. 4, that is, an example of label integration that is effective even when the label contains noise.

It is now assumed that the label groups 1 and 2 obtained by integrating the changing area labels by the changing area label group creating means 12 'based on the inter-label distance are as shown in FIG. 5A, and the label 8 is noise. Then, the in-group label combination creating unit 13 creates label combinations for the groups 1 and 2 as shown in FIG. 5B, and calculates the degree of coincidence with the object model for each combination, and the degree of coincidence is calculated among them. The in-group label combination selection means 15 extracts the highest combination of the above as one object. In this example, the objects 1 and 2 are extracted as shown in FIG. 5C, and the label 8 is excluded as noise.

In this case as well, it is needless to say that the labels may be combined not only by the distance between the labels as described above but also by the distance between the labels in the horizontal and vertical directions. Also,
In the configuration of FIG. 4, the object coincidence degree calculating means 14 is an essential constituent element as a label integrating means, but the object coincidence degree calculating means 14 of FIG. 1 is only used as an auxiliary, and therefore, depending on the case. May be omitted.

FIG. 6 is a block diagram showing a third embodiment of the present invention, which is different from the one shown in FIG. 1 in that the representative position 7 of the changing area label and the camera parameter 8 showing the relationship between the camera and the monitoring area. , And a calculation means 9 for calculating the object size 10 'on the screen. That is,
Whereas the above example ignores the depth of the monitoring area, it shows an example that can deal with the case where this cannot be ignored. This will be described below with reference to FIGS.

That is, in this example, even when the monitoring area has a depth as shown in FIG. 7A, for example, the camera parameter (camera parameter) is set under the constraint that an object is in contact with the monitoring area. Using the focal length f, the distance H between the camera and the surveillance plane, and the angle θ) 8 of the camera with respect to the surveillance plane, the two-dimensional coordinates observed on the screen are perspective-transformed into the coordinates in the three-dimensional real space, It is based on the principle that the size and movement of an object in space can be measured (see Japanese Patent Application No. 5-181267).

As described above, when the depth of the monitoring area cannot be ignored, if a group of variable area labels is created by the same method as described above, even if the distance between labels on the image is the same, the distance between labels after conversion to the real space is increased. Since the distance of changes depending on the label position on the image, if the integration is performed with the same distance threshold on the image, the far one is easy to integrate and the near one is difficult to integrate. FIG. 7B shows an example of a change area image of a monitoring area having a depth. Here, change region labels from 1 to 10 are extracted. 7 (c), (d)
In the same manner as above, the result when the integration is performed based on a constant label distance is shown. (C) shows an example of the integration result by the distance threshold that is too large, and (d) shows an example of the integration result by the distance threshold that is too small.

That is, in reality, labels 1 to 1 are displayed on the screen.
Even if there are three objects 3 and 4 to 6 and 7 to 10, if the inter-label distance threshold is too large, two objects 1 to 6 and 7 to 10 as shown in FIG. On the other hand, when the inter-label distance threshold value is reduced, the labels 1 to 3 and 4 to 6 are correctly integrated as shown in FIG. 7D, but the labels 7 to 9 and 10 are separated. It becomes an object.

Therefore, according to the present invention, a reference label is selected in order from the label in front of the change area, and when the target object is present at the detection position based on the object model and the camera parameters, the target object on the screen is determined. The size of the label is calculated backwards, and the threshold value of the label distance during integration is changed according to the size. That is, the integration threshold is set low for a distant object and is increased for a near object.

FIG. 7E shows the distance between the object and the camera,
The relationship with the integrated distance threshold is shown. This can be obtained by multiplying the distance threshold in real space by a function of how much the unit size such as height and width of the object model changes on the screen depending on the distance between the camera and the object. Figure 7
(F) shows an example of the integration result according to the present invention. This is the result of integration using the integrated distance threshold shown in FIG.

FIG. 8 is a block diagram showing a fourth embodiment of the present invention, in which the changing area integrating means 12 shown in FIG. 6 is replaced with a changing area label group creating means 12 'and an in-group label combination creating means is provided. 13 and an in-group label combination selecting means 15 are added. The size of the target object on the screen is back-calculated from the object model and the camera parameter, and the variable area label group creating means 12 'determines the inter-label distance threshold (width) at the time of integration according to the size. , Height) to create a label group, the in-group label combination creating means 13 creates a combination only within the group, and the in-group label combination selecting means 15 selects the combination having the highest degree of coincidence with the object model. It is selected by. That is, by further focusing on the label group in comparison with the ones shown in FIGS. 6 and 7, even when there are multiple objects or noises on the screen in the monitoring area with depth, the number of label combinations is reduced, and high speed and high accuracy are achieved. The object is to be extracted.

[0023]

According to the present invention, the following effects can be expected. (1) Even when one object is divided into a plurality of labels and extracted on the change area screen, the label distance is calculated, and the label set whose distance is equal to or less than a certain threshold is set to 1
Since it is extracted as one object, the object can be extracted at high speed. In this case, the inter-label distance is divided into the horizontal (x) and vertical (y) directions, and the labels are integrated using the distance threshold for each direction, so that the aspect ratio is "1".
Even with respect to such target objects, the object can be extracted at high speed and with high accuracy. (2) A plurality of labels on the change area screen are divided into label groups in which the distance between labels is less than or equal to a threshold value, and the labels are combined only within that group, and the label with the highest degree of agreement with the object model is displayed. By selecting a combination, it is possible to reduce the number of label combinations even when there are a plurality of objects or noises on the screen, and to extract the objects at high speed and with high accuracy. In this case as well, it is of course possible to divide the inter-label distance into the x and y directions and integrate the labels using the distance threshold for each direction. (3) The size of the target object on the screen is calculated back from the object model, the camera parameters, and the position of the target object, and the threshold value of the label distance at the time of integration is changed according to the size. Thus, it is possible to extract an object at high speed and with high accuracy even in a monitoring area having a depth. Also in this case, divide multiple labels into label groups with a distance between labels equal to or less than the threshold value, perform label combinations only within that group, and select the label combination with the highest degree of agreement with the object model. Thus, the number of label combinations can be further reduced, and an object can be extracted at high speed and with high accuracy.

[Brief description of drawings]

FIG. 1 is a block diagram showing a first embodiment according to the present invention.

FIG. 2 is an explanatory diagram of a first integrated example of changing area labels in FIG.

FIG. 3 is a diagram illustrating a second integration example of changing area label integration in FIG. 1.

FIG. 4 is a block diagram showing a second embodiment according to the present invention.

5 is an explanatory diagram of an example of integration of changing area labels in FIG.

FIG. 6 is a block diagram showing a third embodiment according to the present invention.

7 is an explanatory diagram of an example of integration of changing area labels in FIG.

FIG. 8 is a block diagram showing a fourth embodiment according to the present invention.

FIG. 9 is a block diagram showing a conventional example.

[Explanation of symbols]

1 ... Imaging means (TV camera), 2 ... A / D conversion means,
3 ... Input image storage unit, 4 ... Change area extracting means, 5 ... Change area binarized image, 6 ... Change area label distance calculating means,
7 ... Change region label representative position, 8 ... Camera parameter,
9 ... Object size calculation means, 10 '... Object size on screen,
10A ... Distance threshold between changing regions, 10B ... Horizontal distance threshold between changing regions, Vertical distance threshold between changing regions, 11
... object shape model, 12 ... changing area label integrating means A,
B, 12 '... Change area label group creating means, 13 ...
In-group label combination creating means, 14 ... Object coincidence degree calculating means, 15 ... In-group label combination selecting means, 16 ... Object position / coincidence degree storage section, 17 ... Motion extracting means, 18 ... Object movement storage section, 19 ... Result Calculation / output means, 20 ... Monitoring result.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Ogose Yamada 1-1 Tanabe Nitta, Kawasaki-ku, Kawasaki-shi, Kanagawa Fuji Electric Co., Ltd. Within the corporation

Claims (6)

[Claims] 1. An image pickup means for inputting a time-series image, a change area extraction means for extracting a change area on an image due to a time change, and a storage means for storing definition data regarding a shape of an object to be monitored on a screen. And a feature extraction means for extracting the position and feature amount of each label from the change area image on the screen,
In an object monitoring device using image processing for recognizing an object to be monitored from a change area, for each label extracted as the change area, one set of labels that is within a certain distance from the label on the screen 2. An object monitoring apparatus, comprising: an integrating unit that integrates the object as an object and an object extracting unit that extracts the integrated region as an object. 2. Instead of the integrating means, vertical and horizontal distances between objects are set separately, and a set of labels in which at least one of the distances is a fixed value or less is integrated as one object. The object monitoring apparatus according to claim 1, further comprising integration means. 3. A group creating means, instead of the integrating means, for each label extracted as the change area, a set of labels within a certain distance from the label on the screen is one group. 2. The object monitoring device according to claim 1, further comprising: an integration unit that combines labels only within the group and integrates a set of labels having the highest degree of coincidence with the shape model of the target object. apparatus. 4. In place of the group creating means, vertical and horizontal distances between the objects are set separately, and a label set in which at least one of the distances is a fixed value or less is defined as one group. 4. The object monitoring apparatus according to claim 3, further comprising a group creating means for controlling the group. 5. The condition that the target object is in contact with the plane of the monitoring area, instead of the integrating means, depending on the positional relationship between the plane of the monitoring area and the imaging means, the focal length of the lens, and the change area label extraction position. Below, the calculation means for roughly estimating the size (width, height) occupied by the target object on the screen and the horizontal distance on the screen from the label for each label extracted as the change area A label that satisfies at least one of the two conditions that the width is equal to or smaller than the width or the vertical distance is equal to or smaller than the height of the target object. The object monitoring device according to claim 1. 6. The condition that the target object is in contact with the plane of the monitoring area, instead of the integrating means, depending on the positional relationship between the plane of the monitoring area and the imaging means, the focal length of the lens, and the change area label extraction position. Below, the calculation means for roughly estimating the size (width, height) occupied by the target object on the screen and the horizontal distance on the screen from the label for each label extracted as the change area A group creating unit that makes one group a label that satisfies at least one of the two conditions that the width is less than or equal to or the vertical distance is less than or equal to the height of the target object, and the label is created only within that group. The object monitoring apparatus according to claim 1, further comprising: an unifying unit that performs a combination to unify a set of labels having a highest degree of coincidence with the shape model of the target object.