JP4572190B2 - Object detection device - Google Patents

Description

  The present invention relates to, for example, an object detection device using an imaging device, and more particularly to an object detection device that detects a target object to be detected from an image signal input from the imaging device in order to monitor an intruding object in an imaging field of view. The present invention relates to an object detection method.

For example, an object is monitored using an imaging device such as a television camera (TV camera). In addition, a technique in which an apparatus or a system automatically monitors instead of manned monitoring is being studied.
As an example, in monitoring using a difference method, a difference in luminance (or pixel value) between an image obtained from a TV camera or the like and a reference background image is detected, and the detected value is a predetermined threshold value (threshold value). Monitoring is performed on the assumption that an object exists or is likely to be present in a large change area compared to (1).
Further, as the object detection condition, for example, a condition related to a mask area, a condition related to size determination, a condition related to a warning area, or the like is used.

JP 2002-135644 A JP 2002-279429 A Supervised by Hideyuki Tamura, “Introduction to Computer Image Processing”, Soken Publishing, 1985, p. 149-153 Tsuneya Shigeyuki, et al., “Object Tracking Using Fixed-Viewpoint Pan / Tilt Stereo Camera”, Information Processing Society of Japan, Computer Vision and Image Media Research Group, 2001, No 127-15

However, the conventional object detection apparatus and object detection method have a problem that, for example, when the optical axis direction and the angle of view of the TV camera change, the object detection condition must be changed. If no change is made, there is a problem that accurate object detection cannot be performed.
The present invention has been made in view of such a conventional situation. For example, even when the optical axis direction or the angle of view of a TV camera changes, an object detection apparatus capable of performing accurate object detection. And an object detection method.

In order to achieve the above object, the object detection apparatus according to the present invention detects an image of an object included in an image to be detected as follows.
That is, the object detection target image visual field change detection means detects a change related to the visual field of the image to be detected. The detection condition setting means sets a detection condition used for detecting the image of the object based on the change detected by the object detection target image visual field change detection means. The object image detection unit detects an image of the object included in the image that is the object detection target using the detection condition set by the detection condition setting unit.
Therefore, since the image of the object included in the image is detected using the detection condition set based on the change regarding the field of view of the image that is the object detection target, for example, the change regarding the field of view of the image that is the object detection target Even in such a case, accurate object detection can be performed.
Here, various images may be used as the image to be detected, the object, the change in the field of view of the image to be detected, and the detection condition.
In addition, when a change in the field of view occurs in the image that is the object detection target, various methods may be used as a method for setting the detection condition by the detection condition setting unit. For example, in response to the change, A method of correcting the detection condition may be used, or a method of correcting the image to be detected by the object corresponding to the change may be used, or both. A method may be used that corrects.

In the object detection device according to the present invention, as one configuration example, the object detection target image visual field change detection means is an optical axis direction of a device that captures an image (image imaging device) as a change related to the visual field of the image to be detected. And / or a change in the angle of view of the device that captures an image.
Therefore, for example, accurate object detection can be performed even when the optical axis direction or the angle of view of the device that captures an image changes.
Here, various apparatuses may be used as an apparatus for capturing an image, and for example, a TV camera or the like can be used.
In addition, as a change in the optical axis direction of the apparatus that captures an image, for example, a change in the horizontal direction or a change in the vertical direction is used. As an example, assuming that the horizontal direction is the X-axis direction and the vertical direction is the Y-axis direction, the amount of change in the optical axis direction can be expressed by a shift in coordinates in the X-axis direction and a shift in coordinates in the Y-axis direction.
In addition, as the change in the angle of view of the device that captures an image, for example, a change in the horizontal direction or a change in the vertical direction is used. As an example, if the horizontal direction is the X-axis direction and the vertical direction is the Y-axis direction, the amount of change in the angle of view can be expressed by the spread or narrowing of the coordinates in the X-axis direction and the spread or narrowing of the coordinates in the Y-axis direction. Alternatively, it can be expressed by an angle in the X-axis direction and an angle in the Y-axis direction.
In addition, a change in the optical axis direction and a change in the angle of view of an apparatus that captures an image may occur due to operation or control by a person or an apparatus, for example, and may occur due to an outdoor environment or an indoor environment such as a wind. There is a case.

In the object detection apparatus according to the present invention, as one configuration example, a detection condition serving as a reference for setting a detection condition used to detect an image of an object with respect to an image wider than an image that is an object detection target. (Reference detection condition) is set. Then, the detection condition setting means sets a detection condition used for detecting the image of the object based on the reference detection condition and the change detected by the object detection target image visual field change detection means.
Therefore, since the detection condition is set based on the reference detection condition set for a wider image than the image that is the object detection target, for example, in the case where a change in the field of view of the image that is the object detection target occurs In addition, an appropriate detection condition can be set, and an image of the object can be detected appropriately.
Here, various images may be used as an image that is wider than the image that is the object detection target. For example, a wider field of view (for example, a frame) than the field of view of the image that is the object detection target (for example, the frame) An image having a frame) can be used. For example, an image that includes all images to be detected by the object can be used.
Various detection conditions may be used as a reference.
In addition, as a reference detection condition, for example, a plurality of reference detection conditions having the same role can be set in a wide range in an image wider than an image to be detected by an object. In this case, a plurality of reference detection conditions having the same role are dispersed over a wide range, so that even when there is a change in the field of view of the image to be detected, the image to be detected by the object is changed. Often, one or more of a plurality of reference detection conditions having similar roles are included.
The reference detection condition is, for example, variably or fixedly set in the apparatus in advance, or variably or fixedly set in advance by a person, and stored in, for example, a memory.

Below, the structural example which concerns on this invention is shown further.
For example, an image that is wider than the image that is the object detection target is set as a monitoring area image, and is generated by combining a plurality of images as an example. Further, the plurality of images are acquired by being imaged by an imaging means such as a TV camera, for example.
As one configuration example, one or more of a template, a reference background image (reference background image), a mask area, a warning area, size information, and speed information are used as detection conditions.
As an example of the configuration, a plurality of templates are set to be separated from each other (that is, distributed), for example, in an image wider than the image to be detected by the object.
As one configuration example, a reference background image for an image to be detected by an object is extracted from an image wider than the image to be detected by an object.

As one configuration example, the amount of change in the optical axis direction is detected by template matching. For example, a part having a luminance or pixel value that matches or is similar to a predetermined template set in the reference background image is detected from the image that is the object detection target, and the amount of deviation between the template and the detection part is determined. To detect. Then, the amount of deviation is corrected to eliminate the deviation due to the change in the optical axis direction between the object detection target image and the reference background image.
As an example of the configuration, the amount of change in the angle of view (for example, a value representing how many times) is detected based on the ratio of the focal length before the change in the angle of view and the focal length after the change in the angle of view. In addition, for example, an angle of view is operated, an angle of view of an imaging unit (for example, a camera) that captures an image that is an object detection target, or an angle of view changing unit that changes the angle of view of the imaging unit. The amount of change in the angle of view is detected in response to the change in the angle of view due to (for example, control of the camera pan head or imaging lens). Then, the amount of change in the angle of view is corrected to eliminate the shift due to the change in the angle of view between the image to be detected by the object and the reference background image.

As one configuration example, a difference between the luminance or pixel value of the image to be detected by the object and the luminance or pixel value of the reference background image is detected, and a portion where the detected difference is equal to or greater than a predetermined threshold or exceeds the predetermined threshold. Detect as a change part. Such a change portion can be regarded as being likely to correspond to an image of a moving or changing object.
As an example of the configuration, even if there is a change in the luminance or pixel value of the image in the mask area, the object image is not detected as being present, that is, the object image is not detected.
As one configuration example, for a warning area, when an image of an object that is partly or wholly included in the warning area is detected and when an image of an object is detected in another area, The notification mode for notifying the object detection is made different.
As an example of the configuration, for information on the size of an object to be detected, an image of an object that is entirely contained within a predetermined number of times the size is detected, or a predetermined number of times the size. It is assumed that an image of an object that includes at least a part within the range is detected.
As one configuration example, for the information on the moving speed of the object to be detected, an image of an object that is entirely included in the range determined from the moving speed is detected, or the range determined from the moving speed. It is assumed that an image of an object including at least a part thereof is detected.

As one configuration example, the object image detection unit determines whether or not a portion included in the image to be detected by the object satisfies the detection condition set by the detection condition setting unit, and determines that the detection condition is satisfied. The detected part is detected as an image of an object included in the image to be detected (object image part).
As one configuration example, the detection condition setting unit is configured to detect a reference detection condition based on a change detected by the object detection target image visual field change detection unit, an image wider than an object detection target image, or an object detection. One or more of the target images are corrected, and a detection condition used to detect the object image is set for the image to be detected. As a specific example, the detection condition setting means compensates for a change detected by the object detection target image visual field change detection means, and has a wider image than the coordinates of the reference detection condition or the image to be the object detection target. Correct one or more of the coordinates and coordinates of the image to be detected by the object, and match the coordinates of the detection condition to be the reference and the coordinates of the image to be detected by the object to the image to be detected by the object To set detection conditions.

  As described above, according to the object detection apparatus and the object detection method according to the present invention, for example, a detection condition serving as a reference is set for a wide image compared to an image to be detected by an object, and for example, a change in optical axis direction or A change in the field of view of the image to be detected, such as a change in the angle of view, is detected, and based on the detected change, a detection condition used to detect an image of the object such as a mask area or a warning area is set. Since the image of the object included in the image that is the object detection target is detected using the set detection condition, for example, the field of view of the image that is the object detection target such as a change in the optical axis direction or a change in the angle of view. Even when a change occurs, accurate object detection can be performed.

Embodiments according to the present invention will be described with reference to the drawings.
In the object detection apparatus and the object detection method according to the present embodiment, when the angle of view or the optical axis direction of the TV camera changes, the change is detected, and based on the change, the mask area, the warning area, and the detection should be detected. The detection conditions such as the information on the size of the object and the information on the speed of the object to be detected are corrected. Thereby, in the object detection apparatus and the object detection method according to the present embodiment, even when the angle of view of the TV camera is changed or the optical axis direction is changed, the change amount is automatically detected, and the detection condition By correcting, accurate object detection with high reliability can be performed.

FIG. 1 shows an example of a hardware configuration of an object detection device (video monitoring device) according to the present embodiment.
The object detection apparatus according to the present embodiment includes an imaging device (TV camera) 1, a processing device 2, an operation unit 3, an external storage device 4, an output monitor (monitoring monitor) 5, and a warning lamp 6.
The imaging device 1 includes an imaging unit 11, an electric swivel base (camera pan head) 12, and a zoom lens (imaging lens) 13.
The processing device 2 includes an image input unit (image input I / F) 21, a pan head control unit (head control I / F) 22, a lens control unit (lens control I / F) 23, and an operation input unit (operation input I). / F) 24, image memory 25, MPU (Micro Processing Unit) 26 which is a small processing device, work memory 27, external input / output unit (external input / output I / F) 28, image output unit (image output I / F) 29, an alarm output unit (alarm output I / F) 30, and a data bus 31.
The operation unit 3 includes a joystick 41 and buttons 42 and 43.

The imaging unit 11 is connected to the image input unit 21, the camera platform 12 is connected to the camera platform control unit 22, the imaging lens 13 is connected to the lens control unit 23, the operation unit 3 is connected to the operation input unit 4, The external storage device 4 is connected to the external input / output unit 28, the output monitor 5 is connected to the image output unit 29, and the warning lamp 6 is connected to the alarm output unit 30. The image input unit 21, the pan head control unit 22, the lens control unit 23, the operation input unit 24, the image memory 25, the MPU 26, the work memory 27, the external input / output unit 28, the image output unit 29, and the alarm output unit 30 Are connected to the data bus 31.
An imaging unit 11 (TV camera 1) that is mounted on the camera platform 12 and includes an imaging lens 13 images a monitoring target (field-of-view range). The captured video signal is stored in the image memory 2 5 from the image input unit 21 via the data bus 31. Data such as a program recorded in the external storage device 4 is read into the work memory 27 via the external input / output unit 28, or conversely, stored in the external storage device 4 from the work memory 27. The MPU 26 analyzes the image stored in the image memory 25 in the work memory 27 in accordance with a program stored in the external storage device 4 and loaded into the work memory 27 when the apparatus is executed. The MPU 26 controls the imaging lens 13 from the data bus 31 via the lens control unit 23 according to the processing result, or controls the camera platform 12 via the camera platform control unit 22 to control the imaging unit 11 ( Changing the imaging field of view of the TV camera 1), displaying an image relating to, for example, an intruding object detection result on the output monitor 5 via the image output unit 29, and lighting the warning lamp 6 via the alarm output unit 30. Do things.
In this example, the imaging device 1 and the processing device 2 are installed in the immediate vicinity via a cable or the like, for example, but various other arrangements may be used. For example, the imaging device 1 May be connected to the processing apparatus 2 at a remote location via a network or the like.
In the following embodiments, description will be made by taking the hardware configuration of the object detection apparatus shown in FIG. 1 as an example.

An object detection apparatus and an object detection method according to the first embodiment of the present invention will be described.
FIG. 2 is a flowchart for explaining the processing process according to this example.
The processing process shown in FIG. 11 performs, for example, step S1 (initialization processing), step S2 (image input processing), and step S5 in which operations similar to those of the monitoring method processing process applying the difference method shown in FIG. 11 are performed. (Difference processing), step S6 (binarization processing), step S8 (mask processing), step S9 (labeling processing), step S10 (processed image display), step S14 (object presence determination processing), step S15 (alarm The processing procedure of predetermined steps S3, S4, S7, S11, S12, and S13 is added to the processing procedure of (monitor display processing). Therefore, in this example, a processing procedure (steps S3, S4, S7, S11, S12, and S13) different from the processing process shown in FIG. 11 will be described in detail, and description of similar processing procedures will be omitted or simplified. To do.

First, initialization processing is performed (step S1), and image input processing is performed (step S2).
Next, in the optical axis direction change detection step, a change in the optical axis direction of the TV camera 1 is detected by a template matching method (step S3).
Such detection processing will be described with reference to FIG.
FIG. 6A shows a position 81 of a template used for template matching in the image 71, and FIG. 6B shows an image 72 at the moment when the optical axis direction is changed due to shaking of the TV camera 1 or the like. .
First, a template position 81 is set in advance in the monitoring area as shown in FIG. As the template position 81, for example, a portion that is not shaken by the wind, such as a roof or eaves of a building, is designated. Next, using the image at the position 81 of the template of the reference background image as a template image, the position having the highest degree of coincidence with the template image in the input image is detected. As the degree of coincidence, for example, a normalized correlation value r (Δx, Δy) can be applied,
This is represented by Equation 1.

Here, f (x, y) represents an input image, and g (x, y) represents a template image (partial image of the reference background image). For example, the upper left corner of the image with respect to the coordinate axis of the image is the origin (0, 0). (X0, y0) represents the upper left coordinates of the template position 81, and D represents the size of the template image (for example, horizontal 50 pixels, vertical 40 pixels). The normalized correlation value r (Δx, Δy) takes a value of “−1 ≦ r (Δx, Δy) ≦ + 1”, and “+1.0” when the input image and the template image completely match. It becomes. The template matching process is performed when Δx and Δy are scanned within the search range (horizontal Mx, vertical My), that is, when “−Mx ≦ Δx ≦ + Mx, −My ≦ Δy ≦ + My” is changed. This is a process for detecting the amount of change (Δx, Δy) at the position where the correlation value r (Δx, Δy) is the largest.
The search ranges Mx and My represent the maximum amount of change in the optical axis direction of the TV camera to be detected, and are experimentally obtained. For example, Mx = 50 and My = 50 (see, for example, Non-Patent Document 1). .)
In FIG. 3B, a position 81 is a set template position, and a position 82 is a position detected by the template matching method. That is, the template image at the position 81 at the time of setting the detection condition is moved to the position 82 with the optical axis direction being changed due to the shaking of the TV camera 1 or the like. Therefore, the change amount (Δx, Δy) of the position where the degree of matching obtained at the time of template matching becomes the maximum is the change amount in the optical axis direction of the TV camera.

Next, in the reference background image position correcting step, “g ′ (x, y) = g (x−Δx, y−Δy)” based on the change amount (Δx, Δy) of the TV camera 1 in the optical axis direction. The position shift of the reference background image with respect to the input image is corrected (step S4). Here, g (x, y) represents a reference background image before correction, and g ′ (x, y) represents a reference background image after correction. In this embodiment, the reason for performing such correction is that the difference method is used to detect the luminance change region of the input image, and the input image and the reference background image when the optical axis direction changes due to shaking of the TV camera 1 or the like. If a difference process is executed as it is, a large difference will occur at the location where the position shift has occurred.
Next, difference processing is performed using the reference background image whose position has been corrected (step S5), and binarization processing is performed (step S6).
Next, in the mask area positional deviation correction processing step, the positional deviation between the input image and the mask area caused by the change in the optical axis direction of the TV camera 1 is corrected in the same manner as in the reference background image position correction step described above ( Step S7).
For example, when the mask region is m (x, y), the image is translated as “m ′ (x, y) = m (x−Δx, y−Δy)” to correct the positional deviation. . Here, m (x, y) represents a mask area before correction, and m ′ (x, y) represents a mask area after correction. By doing in this way, even in a scene where the change in the optical axis direction of the TV camera 1 occurs, the mask process can be executed without setting the mask area wider.
Next, mask processing is performed using the mask region whose position has been corrected (step S8), labeling processing is performed (step S9), and processed image display processing is performed (step S10).

Next, in the warning area correction step, the position of the warning area is translated by an amount of change (Δx, Δy) in the optical axis direction of the TV camera 1 (step S11). In this example, a warning area is set in a rectangular shape, for example, a warning area 201 shown in FIG. 12, and the upper left coordinates and lower right coordinates (that is, the coordinates of two vertices in a diagonal relationship) are used. Specifies the rectangle. That is, when the warning area before correction is “upper left coordinates (wx0, wy0) −lower right coordinates (wx1, wy1)”, the corrected warning area is “upper left coordinates (wx0−Δx, wy0− By setting “Δy) −lower right coordinates (wx1−Δx, wy1−Δy)”, the warning area can be corrected.
Next, in the size condition calculating step, for example, the reference size of the object to be detected as described with reference to FIG. 13 is corrected (step S12). For example, the correction is obtained by performing conversion such that “y1 ′ = y1−Δy, y2 ′ = y2−Δy” with respect to the reference object positions y1 and y2 set in advance in Expressions 2 and 3. The result is substituted into y1 and y2 as in “y1 = y1 ′, y2 = y′2”, and is applied to Expression 2 and Expression 3.
Next, in the speed condition calculation step, based on the reference sizes dx3 and dy3 obtained in the size condition calculation step, for example, information on the speed of the object to be detected with 2/3 values of dx3 and dy3 and To do.
Next, it is determined whether or not an object exists using the object detection conditions corrected as described above (step S14). If it is determined that the object exists, an alarm is output and a monitor is displayed. (Step S15).

Step S4 (correction of the position of the reference background image), step S7 (correction of the position of the mask area), step S11 (correction of the position of the warning area), step S12 (correction of the size condition), step S13 described above With each processing of (speed condition calculation), in the object detection apparatus and the object detection method of this example, for example, when the TV camera 1 is shaken by the wind or when the TV camera 1 is operated, the optical axis direction of the TV camera 1 is changed. Even in such a case, the change amount in the optical axis direction of the TV camera 1 can be automatically detected, and the detection condition can be corrected based on the change amount in the optical axis direction. Therefore, due to a change in the optical axis direction of the TV camera 1, for example, the dead zone must be set wide, so that the range in which an object can be detected becomes narrow, or the size of the change area is determined and within the warning area Therefore, it is possible to eliminate the fact that the determination of whether or not the object exists is not accurately performed, and it is possible to realize highly reliable and accurate object detection.
As described above, in this example, the shaking of the TV camera 1 can be detected and corrected.

Here, in this example, the correction is performed by shifting the positions of the reference background image, the mask area, the warning area, and the setting values y1 and y2 related to the size condition, which are the basis of the object detection condition. As another configuration example, the input image may be shifted and corrected without correcting the information that is the basis of the object detection condition.
Further, in this example, lens control of the imaging lens 13 of the TV camera 1 is not considered, but as a further configuration example, a focal length change amount r is calculated in consideration of lens control, and a mask region or the like is calculated. It is also possible to carry out a combination of processes for increasing the size of r by r times.

An object detection apparatus and an object detection method according to the second embodiment of the present invention will be described.
In the object detection apparatus and the object detection method of the present example, the detection conditions for the mask area, the warning area, and the size information are set on the preliminarily created monitoring area image. When the optical axis direction is manipulated, the monitoring condition is calculated based on the reference detection condition set on the monitoring area image.
FIG. 4 is a flowchart of the reference detection condition setting process.
An example of the flow of the reference detection condition setting process will be described with reference to FIG.
First, in the initialization process step, initialization of external devices, variables, image memory, etc. for executing the setting process is performed (step S21).
Next, in the setting reading processing step, for example, the setting of the reference detection condition recorded in the external storage device 4 is read (step S22). Here, when the reference detection condition is set for the first time, the setting operation is started in the state of the reference detection condition initialized in the initialization process step. From the second time onward, the last set reference detection condition is read. The setting work is started.

Next, from the camera pan head control step (step S23) to the area scan completion determination step (step S25), processing for inputting an image within a predetermined range from the angle of view serving as the reference of the TV camera 1 is performed.
Specifically, in the camera pan head control step, the pan head is controlled in order to scan, for example, a range of 50% of the reference angle of view in nine times (step S23). As an example, when a 1 / 2-type CCD (element size: 6.4 mm in width, 4.8 mm in length) is used as an image sensor and the focal length is set to 10 mm, the reference angle of view is 35.5 ° in the horizontal direction, The vertical direction is 27.0 °. In the camera head control step, each time the process is executed, from the reference angle of view, 17.75 ° to the left, 13.50 ° to the top, 13.50 ° to the top, 17.75 ° to the right, and 13.50 ° up, 17.75 ° left, no movement, 17.75 ° right, 17.75 ° left and 13.50 ° down, 13.50 ° down, 17.75 right Control the camera head 12 at 13.50 ° and down.
Next, in the image input step, an input image having a horizontal size of 320 pixels and a height of 240 pixels is obtained from the TV camera 1 (step S24).
Next, in the area scan completion (end) determination step, when the camera head control is executed a predetermined number of times (9 times in this example), the process branches to the image composition processing step (step S26). If it has not been executed a predetermined number of times, the process branches to the camera head control step (step S23) (step S25).

When the above processing is completed, the images 91 to 99 as shown in FIGS. 5A to 5I are obtained.
Next, in the image composition processing step, the images 91 to 99 shown in FIG. 10 are synthesized to form an image of one frame (step S26).
In this example, by synthesizing the images 91 to 99 so as to overlap each other by 50% of the image size, an image of one frame as shown in FIG. 6 can be generated. The image synthesized in this way is obtained by expanding the reference angle of view by 50% in the top, bottom, left, and right directions. This image will be described as a monitoring area image.
Here, in this example, the monitoring area image is created with the number of scans of 9 and the overlap amount of 50%. However, other modes may be used. For example, the number of scans is 25 (horizontal and vertical division into 5) ) And the overlap amount can be 75%. Note that if the monitoring area image is set too wide with respect to the reference angle of view, it may be difficult to combine images due to the parallax of the camera. For example, the monitoring area image has the reference angle of view. On the other hand, it is preferable to keep about 50% in the vertical and horizontal directions. Further, for example, by using a fixed viewpoint camera, it is possible to solve such a parallax problem. The fixed viewpoint camera is a camera designed so that the viewpoint position remains constant even when pan / tilt control is performed. When this camera is used, the viewpoint position of the camera does not change even if the optical axis direction changes. Does not occur. An application of such a fixed viewpoint camera has also been studied (for example, see Non-Patent Document 2).

Next, in the mask area setting step, a mask area is set for the monitoring area image synthesized in the image synthesis processing step (step S27). The mask area is, for example, like the area 111 in the monitoring area image 101 shown in FIG. 6A when the monitor uses the operation unit (operating device) 3 to specify the vertex of the polygon. Specify to surround the part of the plant that may be shaken by the wind.
Next, in the mask setting completion determination step, for example, “YES” / “NO” (“Yes” or “No”) is selected as to whether or not the setting operation of the mask area 111 is completed. The monitoring person selects “YES” when the setting of the mask area 111 is completed by the operation of the operation unit 3, and “NO” when the setting of the mask area 111 is not completed. "Is selected (step S28). In this way, in the mask setting completion determination step, if “YES” is selected, the process branches to the warning area setting step (step S29), and if “NO” is selected, the mask area setting step (step) Branch to S27).

Next, in the warning area setting step, for example, an area 112, an area 113, an area 114, etc. in the monitoring area image 101 shown in FIG. In this manner, an area with a particularly high alert level is designated (step S29). Information on whether or not there is an intruding object in such a warning area 112 to 114 is reflected in the type of alarm in the alarm / monitor display step (step S58 in FIG. 7). For example, in the warning area 112 to 114 If there is an intruding object, the warning light 6 is turned on in red, and if there is no intruder in the warning areas 112 to 114, the warning light 6 is turned on in yellow.
Next, in the warning area setting completion determination step, similarly to the mask setting completion determination step (step S28) described above, the monitoring staff selects whether or not the setting of the warning areas 112 to 114 is completed, and the warning area 112 to If the setting of 114 is completed, the process branches to the size setting step (step S31). If the setting of the warning areas 112 to 114 is not completed, the process branches to the warning area setting step (step S29) (step S30). ).

Next, in the size setting step, as described with reference to FIG. 13, for example, a size serving as a reference for the intruding object is set (step S31). In the example shown in FIG. 6A, the rectangle 115 and the rectangle 116 are set. That is, values such as y1 and y2 shown in FIG. 13 are determined according to the rectangle 115 and the rectangle 116 set by the monitor, and preparation of parameters for performing the calculations of Expression 2 and Expression 3 is completed.
Next, in the size setting completion determination step, similarly to the mask setting completion determination step (step S28) described above, the monitor is made to select whether or not the setting of the reference size of the intruding object has been completed. If the setting is completed, the process branches to the template setting step (step S32). If the setting of the size is not completed, the process branches to the size setting step (step S31) (step S32).

Next, in the template setting step, a template position for detecting a change in the optical axis direction of the TV camera 1 is set (step S33). As the position of the template, for example, a portion that is not shaken by wind or the like such as a roof or eaves of a building is designated. In this example, as a preferred mode, a plurality of template positions are set, such as a plurality of rectangles 117, 118, 119, and 120 shown in FIG. This is because at least one template is displayed on the screen even when the imaging lens 13 of the TV camera 1 is operated or the optical axis direction is changed. Note that matching accuracy can be improved by performing matching using a plurality of templates (see, for example, Patent Document 1).
Next, in the template setting completion determination step, whether or not the setting of the position of the template for detecting the change in the optical axis direction of the TV camera 1 has been completed, as in the mask setting completion determination step (step S28) described above. When the setting of the template position is completed, the process branches to the setting storage processing step (step S35). When the setting of the template position is not completed, the process branches to the template setting step (step S33). (Step S34).
In the setting storage processing step, the image obtained from the camera pan head control step (step S23) to the template setting completion determination step (step S34) and the information on the detection condition as a reference are stored in the external storage device 4 for example. Save to.
This completes the setting of the reference detection condition.

FIG. 7 is a flowchart illustrating the processing process according to this example.
In the processing process shown in the figure, for example, steps S41 (initialization processing), step S43 (image input processing), and step S48 in which operations similar to those of the monitoring method processing process applying the difference method shown in FIG. 2 are performed. (Difference processing), step S49 (binarization processing), step S51 (mask processing), step S52 (labeling processing), step S53 (processed image display), step S57 (object presence determination processing), step S58 (alarm Monitor display process) and a process equivalent to the processes of the other steps S3, S4, S7, S11, S12, and S13 shown in FIG. (Optical axis direction change detection process), Step S47 (Reference background image position correction process), Step S50 (Mask area position deviation correction) ), Step S54 (warning area correction process), step S55 (size condition calculation process), and step S56 (speed condition calculation process) are added with the process steps of predetermined steps S42, S44, and S45. It has become. For this reason, in this example, a processing procedure (steps S42, S44, and S45) different from the processing process shown in FIG. 2 will be described in detail, and description of similar processing procedures will be omitted or simplified.

First, initialization processing is performed (step S41).
Next, in the camera head control step, the operation of the operation unit 3 by the monitor is input, and the camera head 12 and the imaging lens 13 are controlled (step S42). In the camera pan head control step, for example, when the monitor moves the joystick 41 of the operation unit 3 up and down, left and right, the camera pan head 12 is controlled in conjunction therewith to change the optical axis of the TV camera 1 up and down and left and right. . Further, when the monitor presses the buttons 42 and 43 attached to the operation unit 3, the focal length of the imaging lens 12 is increased or decreased. Thereby, zooming in and zooming out are performed, and the angle of view of the TV camera 1 can be changed.
In this example, the camera pan head control step is configured to perform pan / tilt and zoom control of the camera by the user's operation, but as another configuration example, the position of the detected object on the image, A configuration in which pan / tilt or zoom control is automatically performed in accordance with the size, the movement amount, or the like may be used.
Next, an image input process is performed (step S43).
Next, in the view angle change calculation step, the view angle change of the TV camera 1 is calculated from the ratio between the focal length of the imaging lens 12 when the reference detection condition is set and the focal length of the imaging lens 12 after the view angle operation. (Step S44). For example, when the focal length f0 of the imaging lens 12 when the reference detection condition is set is 10 mm and the focal length f1 of the imaging lens 12 after the change in the angle of view is 12 mm, the change amount r of the angle of view is set. Is f = f1 ÷ f0 = 1.2 times (the angle of the visual field is changed from 35.49 ° to 29.96 ° in the horizontal direction).
Next, in the monitoring area image size correcting step, in order to correct the change in the angle of view of the TV camera 1, the size of the monitoring area image s (x, y) is multiplied by r, and this is s ′ (x, y ), The size of the monitoring area image is corrected (step S45). Here, in this example, for the monitoring area image, the coordinates s when the horizontal direction of the image is the x axis, the vertical direction of the image is the y axis, and the upper left point of the monitoring area image is the origin (0, 0). (X, y). In this example, processing is performed with reference to the coordinate system of the monitoring area image.

Next, in the optical axis direction change detection step, the current change in the optical axis of the TV camera 1 is detected using a template for detecting the change in the optical axis direction set on the monitoring area image (step S46). ).
For example, when the position of the template set on the monitoring area image s (x, y) is (xt, yt) and the vertical and horizontal sizes are (sx, sy), the above-described monitoring area image size correction step (step) On the monitoring area image s ′ (x, y) whose field angle change is corrected in S45), the position of the template is (r × xt, r × yt), and the vertical and horizontal sizes thereof are (r × sx, r × sy). Using the template on the monitoring area image s ′ (x, y) corrected for the change in the angle of view, the input image is scanned by the template matching method to detect the matching position (xm, ym).
Since the matched position (xm, ym) matches the position (r × xt, r × yt) of the template on the monitoring area image s ′ (x, y) corrected for the change in the angle of view, the TV camera 1 shakes. Even in a scene where the optical axis of the camera changes from time to time, the template can be captured at a fixed position in the monitoring area image.
That is, even if the TV camera 1 is shaken and the optical axis of the camera is changed, the position of the input image can be determined on the monitoring area image s ′ (x, y). Note that the upper left coordinates of the input image are represented by (r × xt-xm, r × yt-ym) on the monitoring region image s ′ (x, y). That is, the matching position (xm, ym) is represented by the upper left coordinates of the template position on the input image (where the coordinate axis is the origin (0, 0) at the upper left of the input image). Since the position matches the template position (r × xt, r × yt) on the monitoring area image s ′ (x, y), the upper left coordinate of the input image is ( r * xt-xm, r * yt-ym).
Further, the TV camera 1 is converted by converting the input image into a coordinate system on the monitoring area image s ′ (x, y) and comparing it with a reference detection condition set on the monitoring area image s ′ (x, y). The intruding object detection process can be executed even when the angle of view is operated or when the TV camera 1 is shaken.

Here, in the optical axis direction change detection process (step S46) described above, a plurality of templates 117 to 120 are prepared as shown in FIG. 6A and input using one of the templates. Matching with the image is performed to determine the matching position. (Pattern example 1) (Pattern example 2) and (Pattern example 3) of this process are shown.
In (Pattern example 1), matching with the input image is performed using all templates, and the position (xm, ym) where the normalized correlation value is the largest and the template used at that time are specified. In this example pattern, the processing time may be long.
In (pattern example 2), matching is performed only for templates existing in the input image, and the position (xm, ym) where the normalized correlation value is the largest is specified. For example, as for the size and position of the input image in the coordinate system of the monitoring area image in FIG. 6A, the control information of the camera platform 12 such as how much the panning and tilting from the initial position and the focus of the imaging lens 13 are performed. It can be uniquely determined from the distance information. For this reason, it is possible to select a template in the input image.
In (Pattern example 3), the shaking of the TV camera 1 is further considered in (Pattern example 2) described above. Specifically, a range that is larger (for example, slightly) than the input image is set, matching is performed only for templates existing in the range, and the position where the normalized correlation value is the largest (xm, ym) Is identified.

Next, in the reference background image position correction step, for example, (r × xt-xm, r × yt-ym) to (r × xt-xm + 319, r × yt-ym + 239) are obtained from the monitoring region image s ′ (x, y). Is extracted and used as a reference background image (step S47). In this example, it is assumed that the size of the image to be processed is 320 pixels wide and 240 pixels long.
Here, in the reference background image position correction process (step S4) shown in FIG. 2, the change in the optical axis direction of the TV camera 1 is corrected by translating the reference background image created in advance. However, in the reference background image position correction process (step S47) of this example, the monitoring region image created in a wider range than the image to be processed is reduced. Since the reference background image is extracted from the image, the difference process can be executed without reducing the number of pixels.
Next, difference processing is performed (step S48), and binarization processing is performed (step S49).
Next, in the mask area position deviation correction step, the mask area set on the monitoring area image s ′ (x, y) is set to (r × xt−xm) as in the above-described reference background image position correction step (step S47). , R * yt-ym) to (r * xt-xm + 319, r * yt-ym + 239), and applies as a mask image in the subsequent mask processing step (step S51) (step S50).
Next, mask processing is performed (step S51), and labeling processing is performed (step S52).
Next, in the warning area correction step, the warning area set on the monitoring area image s ′ (x, y) is set to (r × xt−xm, r) as in the above-described reference background image position correction step (step S47). × yt−ym) to (r × xt−xm + 319, r × yt−ym + 239), and in the subsequent object presence determination step (step S57), a determination condition as to whether or not an intruding object exists in the alert area (Step S54).

Here, FIG. 8 shows a specific example of the above-described mask area misalignment correction processing (step S50) and warning area correction processing (step S54).
FIG. 4A shows an example of a mask area (or a warning area) 141 set for the monitoring area image 131. FIG. 5B further shows an input image range 142. In FIG. 5C, an area cut out from the mask area (or the warning area) 141 based on the input image range 142 is indicated by hatching. As an example, a format for grasping the coordinates of each vertex of the mask area on the input image as shown in FIG. 4D can be used. As another example, the format shown in FIG. In this manner, a format in which each pixel of the input image is scanned to grasp the pixels in the mask area can be used, or another format can be used.
Next, in the size condition calculation step, the coordinates of the circumscribed rectangle (for example, the rectangle 184 shown in FIG. 10D) of the change area detected by the difference method are displayed on the monitoring area image s ′ (x, y). Conversion into coordinates is performed to calculate the size of the intruding object as a reference (step S55). Here, for example, if a circumscribed rectangle 223 as shown in FIG. 13B is detected, the distance y3 from the lower end of the image obtained from the detection result to the lower end of the circumscribed rectangle 223 has corrected the change in the angle of view. The distance y3 ′ = 480− (r × yt−ym + 240) + y3 on the monitoring area image s ′ (x, y), and y3 ″ = y3 ′ ÷ in the coordinate system on the monitoring area image s (x, y). r. Note that the size of the input image is 320 pixels wide and 240 pixels long, and the size of the monitoring area image is 640 pixels wide and 480 pixels long (up, down, left, right, and 50% enlargement of the reference angle of view). Therefore, by applying Equations 2 and 3 with y = y3 ″, the reference sizes dx3 and dy3 on the monitoring region image s (x, y) are calculated, and the reference sizes on the input image are calculated. The lengths are calculated as dx3 ′ = dx3 × r and dy3 ′ = dy3 × r.

Next, in the speed condition calculation step, for example, the speed of the object for which the 2/3 times the reference sizes dx3 ′ and dy3 ′ on the input image calculated in the size condition calculation step is to be detected. (Step S56).
Then, an object presence determination process is performed (step S57), and an alarm / monitor display process is performed (step S58). The detection conditions calculated in the warning area correction process (step S54), the size condition calculation process (step S55), and the speed condition calculation process (step S56) are used in the object presence determination process (step S57), and the difference method is used. It is determined whether or not the change area detected by the step is an object to be detected.
As described above, in this example, it is possible to detect and correct the shaking of the TV camera 1 and the change in the focal length. In this example, appropriate correction can be performed by preparing detection conditions in advance corresponding to a wide angle of view.

Here, in this example, the configuration in which the size of the monitoring area image is corrected according to the amount of change r of the focal length of the lens is shown, but as another configuration example, the size of the input image is multiplied by 1 / r. Such a configuration is also possible.
Further, in this example, the processing from the camera head control process (step S23) to the image composition process (step S26) shown in FIG. 4 is combined from the image group composed of the plurality of images 91 to 99 shown in FIG. 6 shows a configuration for generating the monitoring region image 101 as shown in FIG. 6A. As another configuration example, the imaging lens 13 is zoomed out, and the image 101 shown in FIG. It is also possible to adopt a configuration in which an image corresponding to the angle of view is acquired by one shooting. 6B will be described as an example. When a monitoring area image is acquired by photographing at an angle of view of the image 102, an angle of view in a normal process (for example, an image at the time of execution of the flow of FIG. 7). It is also possible to use a shooting mode in which a high-definition (high pixel) image can be obtained as compared with the angle of view of 121 or image 122.

An example of a method for calculating absolute coordinates from control amounts of pan / tilt / zoom with reference to an example of synthesizing the monitoring area image 101 shown in FIG. 11A from the images 91 to 99 shown in FIG. Indicates.
That is, in this example, a 1/2 type CCD (element size is 6.4 mm in width and 4.8 mm in length) is used as the image pickup device, and the focal length is set to 10 mm, as shown in FIG. The image 101 is synthesized. In this case, the angle of view of the input image is 35.5 ° in the horizontal direction and 27.0 ° in the vertical direction, and is synthesized with an overlap amount of 50% in the vertical and horizontal directions, so the monitoring region image 101 is 71.0 ° in the horizontal direction. The angle of view is 54.0 ° in the vertical direction. If the number of pixels of the input image is 320 pixels wide and 240 pixels long, the monitoring area image 101 is composed of 640 pixels wide and 480 pixels high. Accordingly, the monitoring area image 101 has a resolution of 0.055 ° / pix horizontal and 0.056 ° / pix vertical per pixel. As an example, the center of the monitoring area image 101 is set to the origin of the direction of the TV camera 1 (pan 0 °, tilt 0 °), pan is changed to −5 °, and tilt is changed to + 3 ° by the operation of the camera platform 12. Assume that the focal length has changed to 15 mm. Then, the center of the input image moves in the coordinate system of the monitoring area image 101 to (640 ÷ 2-5 ÷ 0.055, 480 ÷ 2 + 3 ÷ 0.056) = (229, 294). In addition, since the angle of view of the input image becomes 24.1 ° in the horizontal direction and 18.2 ° in the vertical direction due to the change in the focal length, the angle of view of the input image is in the coordinate system of the monitoring area image 101 and is This corresponds to 24.1 ÷ 0.055 = 438 pixels, and corresponds to 18.2 ÷ 0.056 = 325 pixels in the vertical direction. That is, the input image is in the range of (229-438 / 2, 294-325 / 2) to (229 + 438/2, 294 + 325/2) = (10, 131) to (448, 456) of the monitoring area image 101. I understand.

In addition, an example of a method capable of specifying absolute coordinates from information other than the template or pan / tilt / zoom control amounts will be described.
In other words, absolute coordinates can be calculated by a matching process using a pan / tilt / zoom control amount and a template. As another configuration example, a specific area existing in a monitoring area such as a door may be used. Focusing on the object, the absolute coordinates can be calculated using the position of the door knob, the apparent size of the door, or the like. For example, instead of using the zoom control amount, the amount of change in the angle of view can be reduced by using the apparent amount of change on the image of the object of interest before and after the change in the angle of view. It is possible to detect.
In this way, in addition to the method of grasping the position and size of the current input image using the template, for example, from the information of camera turning (pan, tilt) and zoom information without using the template, It is also possible to grasp the position and size of the input image. Note that it is not possible to deal with camera shake, for example, from camera turning information or zoom information. In addition, when processing using a template, it is possible to use information such as camera turning (pan, tilt) in combination.
Also, other processing that can grasp the current position of the input image with respect to the reference position may be performed without depending on the template or camera turning (pan, tilt) information or zoom information. It is possible to use detection conditions prepared at the angle of view.
Also, an example of detection condition switching and update methods will be shown.
In other words, in this example, the intruding object is monitored using one detection condition set for the monitoring region image 101, but a plurality of detection conditions may be used as another configuration example. For example, it is possible to set detection conditions for daytime, dusk, and night and switch them every hour, and when there is no wind based on information from an external sensor such as an anemometer It is also possible to temporarily invalidate the mask area. In addition, based on the detection result of the intruding object, for example, a detection area other than the intruding object, that is, an area where the difference between the luminance of the input image and the luminance of the reference background image is equal to or greater than a predetermined threshold is added to the mask area. It is also possible to update the detection condition.

Next, specific examples of effects according to the above-described embodiments (first embodiment to second embodiment) will be described.
With reference to FIGS. 6A and 6B and FIGS. 9A, 9B, and 9C, effects of the present embodiment will be described.
FIG. 6A shows an example of the monitoring area image 101 and various reference detection conditions 111 to 120 set on the monitoring area image 101, as described with reference to FIG. FIG. 6B illustrates an example of the reference angle of view 121, the angle of view 122 of the monitor, and the imaging range 102 of the imaging device 1 by an operation in the optical axis direction.
FIG. 9A shows the display conditions 161 to 163 overlaid on the input image 151 at the angle of view and the angle of view before operation in the optical axis direction. FIG. 9B shows the detection conditions 161a to 161c obtained when the processing according to the first embodiment of the present invention is applied and displayed on the input image 152 in an overlapping manner. FIG. 9C shows the detection conditions 164, 162 a, 165-167 obtained when the processing according to the second embodiment of the present invention is applied, superimposed on the input image 153.

  FIG. 6A shows a mask area 111, a warning area 112 to 114, sizes 115 and 116 serving as a reference for an intruding object to be detected, and templates 117 to 120 for detecting changes in the optical axis direction. . As an example, assuming that the monitoring target in the normal time is an intruding object entering and exiting the entrance of the building and the angle of view 121 shown in FIG. In order to monitor the site gate at the same time, it is assumed that the angle of view 122 is changed to that shown in FIG. Here, an image 151 shown in FIG. 9A has a normal angle of view, and a mask area 161, a warning area 162, and a template 163 for detecting a change in the optical axis are detected with respect to the angle of view. Is set. At this time, if the monitor changes to the angle of view 122 shown in FIG. 6B, for example, when the process according to the first embodiment of the present invention is applied to the conventional detection condition (here, In addition, when the lens control is taken into consideration), the detection condition is as shown in FIG. 9B because the detection condition is translated by the amount of change in the optical axis direction of the TV camera 1.

In FIG. 9B, the mask area 161a cannot cover the entire area where the vegetation exists, and a part of the vegetation exists outside the mask area 161a. There is a possibility of erroneous detection as an object. In addition, the gate of the site, which is actually a warning area, is imaged due to the change in the angle of view, but this is not set as a warning area, and it actually warns the intruding object in the area that is the warning area. It is determined that it does not exist in the area. Furthermore, the template 163a for detecting the change in the optical axis direction of the TV camera 1 protrudes outside the image 152, and the optical axis direction change detection process is not stably performed.
On the other hand, when the process according to the second embodiment of the present invention is applied to the detection conditions, as shown in FIG. 9C, from the reference detection conditions set on the monitoring area image, the mask area 164, the alert Templates 166 and 167 for detecting changes in the optical axis direction of the areas 162a and 165 and the TV camera 1 can be appropriately calculated. Therefore, even in a scene where the angle of view of the TV camera 1 is manipulated or the optical axis direction changes, the dead zone cannot be set sufficiently if it is within the monitoring area image, or the warning area. Can not be set appropriately, and the problem that the change of the optical axis direction of the TV camera 1 cannot be detected stably can be solved. Since the detection condition can be calculated in accordance with the operation of the angle of view of the TV camera 1 and the change in the optical axis direction, it is possible to realize highly reliable and accurate object detection.

As shown in the above-described embodiments (first embodiment to second embodiment), in this embodiment, an object detection device that detects an object in the imaging field based on image signals sequentially input from the imaging device 1 In the object detection method, a change area detection process for detecting a change area of the image signal, an optical axis direction change detection process for detecting an amount of change in the optical axis direction of the imaging apparatus 1, and a change in the optical axis direction A detection condition calculation process for calculating a detection condition for detecting an object in the imaging field based on the image, and an object determination process for determining whether or not the change region satisfies the detection condition. An object in the imaging field is detected.
In the present embodiment, in the object detection device and the object detection method for detecting an object in the imaging field based on the image signal sequentially input from the imaging device 1, a change area detection process for detecting the change area of the image signal; An angle-of-view change detection process for detecting an angle-of-view change amount of the imaging apparatus 1, and a detection condition calculation process for calculating a detection condition for detecting an object in the imaging field based on the angle-of-view change, An object determination process for determining whether or not the change region satisfies the detection condition is performed, and an object in the imaging field of the imaging device 1 is detected.

Further, in the object detection apparatus and the object detection method according to the present embodiment, for example, the imaging apparatus 1 includes a camera head 12 that changes the angle of view of the imaging apparatus 1. In the detection condition calculation process, the detection condition is calculated when the camera pan head 12 is controlled to change the angle of view of the imaging apparatus 1.
Further, in the object detection apparatus and the object detection method according to the present embodiment, for example, the imaging apparatus 1 includes a camera head 12 that changes the angle of view of the imaging apparatus 1. Then, a monitoring region image creation process is performed for creating a monitoring region image based on the input image signal while controlling the camera platform 12 and changing the angle of view of the imaging device 1. The detection condition calculated by the detection condition calculation process is calculated based on the reference detection condition set for the monitoring area image and the angle of view of the imaging device 1.

Further, in the object detection apparatus and the object detection method according to the present embodiment, for example, as the detection condition, a mask area that specifies an area where no object is detected, and a warning area that distinguishes the importance of the detected object, , At least one of size information indicating the size of the object to be detected and speed information indicating the moving speed of the object to be detected.
In the object detection apparatus and the object detection method according to the present embodiment, for example, the mask area is represented by a polygon having at least three or more vertices on the monitoring area image. In the detection condition calculation process, the coordinates of the vertex are converted from the coordinates in the monitoring area image to the coordinates in the input image based on the change in the angle of view and the change in the optical axis direction.
Further, in the object detection apparatus and the object detection method according to the present embodiment, for example, a warning area that distinguishes the importance of a previously detected object is represented by a rectangle on the monitoring area image. In the detection condition calculation process, the coordinates of the rectangle are converted from the coordinates in the monitoring area image to the coordinates in the input image based on the change in the angle of view and the change in the optical axis direction.
Further, in the object detection apparatus and the object detection method according to the present embodiment, for example, the size information indicating the size of the object to be detected is expressed by a rectangle on the monitoring area image. In the detection condition calculation process, the coordinates of the rectangle are converted from the coordinates in the monitoring area image to the coordinates in the input image based on the change in the angle of view and the change in the optical axis direction.
In the object detection apparatus according to the present embodiment, for example, the speed information is calculated based on the size information.

  In the present embodiment, in the object detection device and the object detection method for detecting an object in the imaging field based on the image signal sequentially input from the imaging device 1, the imaging device 1 that sequentially images the monitoring field range, and the imaging device A camera pan head 12 that changes the angle of view of 1, an image input interface 21 that sequentially converts video signals acquired by the imaging device 1 into image signals, and the image signals converted by the image input interface 21. It has a processing function. Then, for example, the processing function detects a change area of the image signal, detects a change in the optical axis direction or a change in the angle of view of the imaging apparatus 1, and based on the change in the optical axis direction or the change in the angle of view. Then, a detection condition for detecting an object in the imaging field is calculated, and an object in the imaging field of the imaging device 1 is detected by determining whether or not the change region satisfies the detection condition.

Therefore, in the object detection apparatus and the object detection method according to the present embodiment, for example, a change area is detected from an image signal captured by the TV camera 1, and a change in the direction of the optical axis or a change in the angle of view of the TV camera is detected. And calculating a detection condition for detecting an object in the imaging field based on the change in the optical axis direction and the change in the angle of view, and determining whether the change region satisfies the detection condition, Even when the optical axis direction and the angle of view of the TV camera 1 are changed, the influence can be suppressed and the object can be detected.
As a specific example, conventionally, there is a possibility that preset detection conditions cannot be applied due to a change in the optical axis direction of the TV camera 1 or a view angle operation. In such a case, there is a problem that the dead zone cannot be set sufficiently, and a problem that it is impossible to accurately determine the size of the change area or whether it exists in the alert area. It has occurred that the detection condition has to be set again.
On the other hand, in this embodiment, a change in the optical axis direction of the TV camera 1 and a view angle operation are detected, and a detection condition is calculated according to the amount of change, thereby changing the optical axis direction of the TV camera 1 and the image angle. It is possible to realize an object detection apparatus and an object detection method that can suppress the influence of the angle change and perform highly reliable and accurate object detection.

As described above, in this embodiment, the positional deviation (movement amount) of the camera is calculated, and for example, by correcting the coordinates of the mask area and the like for the object detection conditions such as the mask area, the warning area, and the background image. The object can be accurately detected by moving the detection condition by the calculated deviation.
As one configuration example, an object detection condition is set in an image capturing a wide field of view, and the input image is selected according to the angle of view and orientation (position on the wide field image) of the camera when the input image is acquired. By calculating the detection conditions for use, the object can be accurately detected. Further, as an image capturing a wide field of view, for example, a composite image obtained by connecting a plurality of images can be used.
Also, as one configuration example, in order to always include one or more templates set in the input image regardless of the angle and direction of the camera at the time of acquiring the input image, the position on the image capturing a wide field of view A plurality of templates for detecting the deviation are set.

In the object detection device and the object detection method according to the present embodiment, an image that is an object detection target is configured by an image (input image) captured by the imaging device 1, and a plurality of images captured by the imaging device 1 are included. The combined result (monitoring area image) forms a wider image than the image that is the object detection target, and the image of the object that is the detection target is configured by the image of the intruding object such as a person. In the object detection apparatus and the object detection method according to the present embodiment, a template, a reference background image, a mask area, a warning area, size information, and speed information are used as detection conditions. Is set in the monitoring region image, and a detection condition used for detecting an image of the object with respect to the input image is set based on the reference detection condition.
Further, in the object detection apparatus and the object detection method according to the present embodiment, the MPU 26 changes the optical axis direction of the input image based on the result of template matching, the operation of the pan head control unit 23, the operation of the lens control unit 23, and the like. The object detection target image visual field change detection means is configured by the function of detecting the change in the angle of view and the angle of view, and the detection condition setting means is configured by the function of the MPU 26 correcting the coordinates, for example, and setting the detection condition for the input image. The object image detecting means is configured by the function of the MPU 26 detecting the image of the moving object from the input image using the detection condition, and the imaging means is configured by the function of the imaging unit 11 of the imaging device 1. The angle-of-view changing means is configured by the function of the camera platform 12 of the imaging apparatus 1 and the function of the imaging lens 13.

The background of the technology related to the present invention will be described below. Note that the matters described here are not necessarily limited to the conventional technology.
An object monitoring apparatus using an imaging apparatus such as a TV camera has been widely used. In a monitoring system using such an object monitoring device, in the case of manned monitoring in which an observer (operator) detects an intruding object such as a human being or a car entering the monitoring visual field while looking at an image displayed on the monitor. Depending on the skill level of the observer, health condition, etc., the monitoring ability may vary, and problems such as oversight of intruding objects may occur. Therefore, instead of the conventional manned monitoring, a system that automatically detects an intruding object from an image input from an image input means such as a camera and obtains a predetermined notification or alarm treatment is required. It is coming.
In order to realize such a system, it is necessary to have a function of detecting an object to be monitored such as an intruder using a predetermined monitoring method. As an example of such a monitoring method, a method called a difference method has been widely used. The difference method is a method in which an input image obtained from a TV camera is compared with a reference background image created in advance, that is, an image in which an object to be detected is not reflected, a difference in luminance value is obtained for each pixel, and the difference value is obtained. Is processed with a predetermined threshold value, and an area having a large difference value is detected as a change area.

With reference to FIG.10 and FIG.11, the process of the above-mentioned difference method is demonstrated. FIG. 10 is a diagram for explaining an example of a procedure for detecting a change area by the difference method, and FIG. 11 is a flowchart showing an example of a typical processing procedure of a monitoring method to which the difference method is applied.
First, an example of a procedure for detecting a luminance change region of an input image that is sequentially input from the imaging device by the difference method will be described with reference to FIG.
In FIG. 9, (A) an image 171 is an input image that is sequentially input from the imaging device, and (B) an image 172 is an image that does not show a target object to be detected prepared in advance.
The difference between the luminance values for each pixel is calculated by a subtractor (subtractor) 175 that receives these two images 171 and 172, and (C) a difference image 173 is obtained. Next, the binarizer 176 having the difference image 173 as an input performs threshold processing on each pixel of the difference image 173 with a threshold Th (experimentally determined, for example, Th = 20). While the pixel value is set to “0”, the pixel value of a pixel equal to or greater than the threshold Th is set to “255”, and (D) a binarized image 174 is obtained. As a result, the humanoid object 181 shown in the input image 171 is calculated as a region 182 (difference change region of the luminance of the input image) where the difference is generated by the differentiator 175, and the pixel value “255” is calculated by the binarizer 176. The image 183 is detected. Various studies have been made on the processing of the difference method (see, for example, Patent Document 2).

Next, an example of a processing procedure of a typical object detection method to which the above-described difference method is applied will be described using the flowchart of FIG.
In the same figure, in the initialization processing step, external devices, variables, image memory, etc. for executing the monitoring method based on the difference method are initialized (step S61). In the image input step, an input image having a horizontal size of 320 pixels and a height of 240 pixels is obtained from the TV camera (step S62). In the difference processing step, a luminance value difference (image 173) for each pixel between the input image (image 171) obtained in the image input step and a reference background image (image 172) created in advance is calculated (step 173). S63). In the binarization processing step, the pixel value of the pixel in which the pixel value (difference value) of the difference image obtained in the difference processing step is less than a predetermined threshold Th (for example, Th = 20) is set to “0”, and is equal to or greater than the threshold Th. A binarized image (image 174) is obtained by setting the pixel value of this pixel to “255” (step S64). In this example, the pixel value of one pixel is calculated by 8 bits, that is, one pixel has a value from “0” to “255”.
Through the processing from the image input step to the binarization processing step (steps S61 to S64), the luminance change region in the input image shown in FIG. 10 is detected.
Next, in the mask processing step, “255” pixels in a preset mask region (a region to be processed as a dead zone region in which no object is detected) with respect to the binarized image obtained in the binarization processing step. The pixel value of the pixel having the value is corrected to “0” (step S65).

Here, the necessity of such mask processing will be described with reference to FIG.
FIG. 3A shows a monitoring area 191, and is assumed to be used for detecting an intruding object that has entered the monitoring area 191. In the monitoring area 191, an entrance of a building, a plant, and the like are shown. FIG. 5B shows a luminance change region of the input image detected when the difference method is applied in the monitoring region 192. In FIG. 5B, in addition to the object 202 to be detected, areas 203 and 204 of vegetation swayed in the wind that cause a difference in brightness between the input image and the reference background image are also detected. It is determined that there are three objects 202, 203, and 204. When such an event occurs, conventionally, a mask area 205 as shown in the monitoring area 193 in FIG. 3C is set, and the inside of the mask area 205 is processed as a dead zone (an area where the presence of an object is not determined). Method is used. Such mask processing is not limited to the difference method, but is used in various object detection methods, and is widely used as a method for improving the performance of the monitoring method.

Next, in the labeling processing step, a block of pixels whose pixel value is “255” in the modified binarized image obtained in the mask processing step can be detected, and each block can be numbered and distinguished. (Step S66). In the processed image display step, for example, a block of pixels in which the pixel value numbered in the labeling processing step is “255” is displayed superimposed on the input image, and the luminance change region of the input image detected by the difference method is displayed. Displayed visually and intelligibly (step S67).
Next, it is determined whether or not an object to be detected is reflected in the luminance change region of the input image detected by the difference method. Specifically, in the object presence determination step, for each of the numbered change areas, whether the change area matches the detection conditions based on the detection conditions such as size, area, speed, and detection position. If the detection condition is met, branch to the alarm / monitor display step because there is an object to be detected. If the detection condition is not met, the image input step is performed assuming that there is no object to be detected. (Step S68).
For example, when the reference size of the change area is used as the detection condition, a circumscribed rectangle 184 is calculated for the numbered change area 183, and the width dx and height dy of the circumscribed rectangle 184 are obtained and detected. Comparison is made with the standard size of the condition (see FIG. 10 above). However, since the apparent size of the object to be detected changes depending on the position observed on the screen, it is necessary to correct the reference size of the detection condition according to the position on the screen.

Such correction processing will be described with reference to FIG.
FIG. 8A shows a reference size of an object to be detected, which is set when the monitoring system is installed, in a region 211. The reference size of the object to be detected is set based on the size of an object actually captured at a position distant from the TV camera and an object reflected at a position close to the TV camera. In FIG. 3A, an object 221 is an image of an object imaged at a position away from the camera, and an object 222 is an image of an object imaged at a position close to the camera. In FIG. 2A, two images are displayed so as to be easily understood. Here, as the information on the size of the object to be detected, the detection positions y1, y2, circumscribed rectangle widths dx1, dx2, and circumscribed rectangle heights dy1, dy2 are recorded for each of the objects 221 and 222. The For example, it is assumed that an object is imaged at the position shown in the area 212 in FIG. In this case, since the object 223 does not exist at the position shown in FIG. 5A, it is necessary to estimate the reference size of the position of the object 223 from the information of the reference size of the object to be detected. is there. The estimation of the reference size is performed by interpolation based on the detection positions y1, y2, and y3. The reference sizes dx3 and dy3 of the object imaged at the position y = y3 of the object 223 are calculated by Expressions 2 and 3. As a result, the reference size of the object detected at an arbitrary position can be calculated.

In the object presence determination step described above, the detection position y is calculated for the change region 183 detected by the difference method shown in FIG. 10, and the reference magnitude is calculated based on the detection position y using the above formulas 2 and 3. Dx3 and dy3 are obtained. As a result, even when the reference size of the object to be detected is not set, the reference size can be calculated by interpolation. Then, an allowable amount (for example, rx in the x direction and ry in the y direction) is set with respect to the reference sizes dx3 and dy3, and the width dx and the height dy of the circumscribed rectangle 184 of the change area 183 are “dx3-rx”. When satisfying ≦ dx ≦ dx3 + rx, dy3−ry ≦ dy ≦ dy3 + ry ”, it is determined that there is an object to be detected in the change region 183.
Here, rx and ry are calculated as, for example, “rx = 0.5 × dx3, ry = 0.3 × dy3”, and fluctuations of 50% and 30% of the reference size, respectively. Can be tolerated. That is, the object to be detected in the change region of the size of these ranges in the range of 50% to 150% of the reference size in the horizontal direction and the range of 70% to 130% of the reference size in the vertical direction. Is determined to exist.

Here, in the above description, the reference size of the object to be detected is used as the detection condition of the object to be detected. However, the position change amount of the change area in the input image sequentially input from the TV camera ( That is, an apparent movement speed) may be calculated, and it may be determined whether or not the object is to be detected according to the position change amount. In this case, for example, if the moving speed of the object to be detected is 5 km / h (1.4 m / s) and the input image is input at a cycle of 100 ms, the object to be detected for each image input moves 0.14 m. Will do. At this time, assuming that the horizontal width of the object to be detected is 0.5 m, the object moves by about 1/3 of the reference size dx3 calculated by the above equation 2. Therefore, for determining whether or not the object is to be detected, for example, the object to be detected when the position change amount of the change region is 2/3 or less of dx3 (an example having a margin of 2 times). It is determined that In this way, for example, it is possible to distinguish between a person to be detected and a vehicle that should not be detected in the monitoring area.
If it is determined in the object presence determination step that there is an object to be detected, then in the alarm / monitor display step, for example, an alarm indicating that an intruding object has been found is sounded to inform the monitoring staff (step S69). In the alarm / monitor display step,
As shown in FIG. 12A, when a warning area 201 is set in the monitoring area and a part of the change area 202 is included in the warning area 201, the detected object is more important. Assuming that it exists in a high position (in the example of FIG. 12A, the entrance of the building is a warning area), a red warning light is turned on to alert the observer and change into the warning area 201 When a part of the area 202 is not included, a yellow warning lamp may be turned on.

  In the conventional method using the difference method as described above, the mask area, the reference size of the object to be detected, the detection condition of the warning area are set in advance, and when the luminance change area of the input image is detected, The detection condition is used when determining whether or not an object to be detected exists in the detected change area. These detection conditions need to be reset to appropriate values when, for example, the angle of view or the optical axis direction of the TV camera is changed. That is, in order to change the monitoring area, the angle of view and the optical axis direction of the TV camera are operated by operating the imaging lens (for example, the imaging lens 13 in FIG. 1) and the camera platform (for example, the camera platform 12 in FIG. 1). It is necessary to reset the above-described detection conditions every time the value is changed. Therefore, there is a problem that the monitoring operation of the intruding object cannot be performed during the resetting operation of the detection condition. Furthermore, when the TV camera is shaken by the wind, for example, outdoors, the optical axis direction of the TV camera changes and the input image is blurred. In such a case, it may be determined that there is a moving object in the input image. In the conventional method, for such a problem, a mask area is set wider, or the reference size is reduced. The allowable amount (in the above example, rx, ry) is set larger, or the warning area is set wider. For this reason, it is necessary to set a wide dead zone, the range in which an object can be detected becomes narrow, the size of the change area cannot be accurately determined, and whether it exists in the alert area. There was a problem that such a determination could not be made accurately. Such problems are not limited to the object detection method based on the difference method, but a method using a mask area, a method for determining the size of a detected area, and setting a warning area to change the type of alarm inside or outside the warning area. It is a problem that exists in common in the method.

As described above, the object detection method that has been widely used conventionally has a problem that, for example, when the angle of view or the optical axis direction of the TV camera is changed, the detection condition must be reset. In a scene where the optical axis direction changes due to camera shake, etc., the dead zone must be set wide, and the problem that the range in which an object can be detected becomes narrow, the size of the change area, or the warning area There is a problem in that it is impossible to accurately determine whether or not it exists within.
On the other hand, in the object detection apparatus and the object detection method according to the embodiment of the present invention, for example, when the angle of view or the optical axis direction of the TV camera is changed, or the optical axis direction changes when the TV camera shakes. Even in such a case, accurate object detection with high reliability can be performed by detecting and correcting the change.

Here, the configurations of the object detection apparatus and the object detection method according to the present invention are not necessarily limited to those described above, and various configurations may be used. The present invention can also be provided as, for example, a method or method for executing the processing according to the present invention, a program for realizing such a method or method, or a recording medium for recording the program. It is also possible to provide various devices and systems.
The application field of the present invention is not necessarily limited to the above-described fields, and the present invention can be applied to various fields.
In addition, as various processes performed in the object detection apparatus and the object detection method according to the present invention, for example, a control program stored in a ROM (Read Only Memory) in a hardware resource including a processor, a memory, etc. A configuration controlled by execution may be used, and for example, each functional unit for executing the processing may be configured as an independent hardware circuit.
Further, the present invention can be grasped as a computer-readable recording medium such as a floppy (registered trademark) disk or a CD (Compact Disc) -ROM storing the above control program, or the program (itself). The processing according to the present invention can be performed by inputting the program from the recording medium to the computer and causing the processor to execute the program.

It is a figure which shows the structural example of the object detection apparatus which concerns on the Example of this invention. It is a figure which shows an example of the flow of a process of the object detection system using the difference method which concerns on 1st Example. (A) is a figure which shows an example of the template position for detecting the change of the optical axis direction of a TV camera, (B) is a figure for demonstrating an example of a template matching process. It is a figure which shows an example of the flow of a process of the setting of the reference | standard detection condition which concerns on 2nd Example. (A)-(I) shows each image and is a figure for demonstrating an example of the process which synthesize | combines a monitoring area | region image. (A) is a figure which shows an example of the synthetic | combination monitoring area | region image and reference | standard detection conditions, (B) is a figure for demonstrating regarding the view angle and optical axis direction of TV camera. It is a figure which shows an example of the flow of a process of the object detection system using the difference method which concerns on 2nd Example. (A)-(E) is a figure for demonstrating the specific example of a mask area | region position shift correction process and a warning area correction process. (A)-(C) are figures which show an example of the image obtained by an imaging device, and detection conditions. It is a figure which shows an example of the process sequence of a difference method. It is a figure which shows an example of the flow of a process of the object detection system using a difference method. (A) is a figure which shows an example of the monitoring area | region in a difference method, (B) is a figure which shows an example of the change area | region detected in the difference method, (C) is a figure which shows an example of the mask area | region in a difference method. It is. (A) is a figure for demonstrating an example of the setting method of the magnitude | size used as the reference | standard which determines with the object which should be detected, (B) is a figure for demonstrating an example of the interpolation method of the magnitude | size used as a reference | standard. It is.

Explanation of symbols

  1 .... Imaging device (TV camera) 2 .... Processing device 3 .... Operation unit 4 .... External storage device 5 .... Output monitor 6 .... Warning lamp 11 .... Imaging unit 12 .... Camera head, 13 ... Imaging lens, 21 ... Image input unit, 22 ... Pan head control unit, 23 ... Lens control unit, 24 ... Operation input unit, 25 ... Image memory, 26 ... MPU, 27 ... Work memory 28 ... External input / output unit 29 ... Image output unit 30 ... Alarm output unit 41 ... Joystick 42,43 ...

Claims (3)

In an object detection method for detecting an image of an object included in a captured image obtained from an imaging device in which the direction of the optical axis and the angle of field of view are variable,
Creating an image wider than the captured image to be detected by zooming out or synthesizing a plurality of captured images;
A detection condition setting step for setting a reference background image, a template, a mask area or a warning area, and size information as detection conditions used for detecting an image of an object for the wide image;
An angle-of-view change calculation step of calculating a ratio of the focal length of the lens of the imaging device when creating the wide image and the focal length of the lens in the captured image that is the object detection target as a change amount of the angle of view. When,
By multiplying or dividing the calculated amount of change in the angle of view by multiplying or dividing the x-coordinate and y-coordinate when the horizontal direction of the image is the x-axis and the vertical direction is the y-axis, An image size correction step for performing coordinate conversion of the captured image,
After the image size correction step, a template matching process between the template set for the wide image and the captured image to be the object detection target is used to change the direction of the optical axis in the captured image to be the object detection target. An optical axis direction change detection step for detecting a change in the corresponding position;
A region detection step of detecting a region of an object image from the captured image to be the object detection target by using a difference method , using the reference background image translated based on the amount of change in the position ;
A mask area or a warning area correction step for extracting the mask area or the warning area from the wide image translated based on the detected change amount of the position;
The coordinates of the area detected in the area detection step are converted into the coordinate system on the wide image after the image size correction step, and the reference at the vertical positions y1 and y2 of the reference object set in advance on the wide image is obtained. the width or height d1, d2, when the vertical position after the transformation of the area and y3, the reference width or height d3 of the vertical position y3, d3 = (d1-d2 ) · (y3- a size condition calculating step of calculating by y2) / (y1-y2) + d2 ;
The area detected in the area detection step using the mask area or the warning area extracted in the mask area or the warning area correction step and the reference width or height d3 calculated in the size condition calculation step An object presence determination step for determining whether or not is an object to be detected;
An object detection method characterized by comprising: In an object detection device that detects an image of an object included in a captured image obtained from an imaging device in which the direction of the optical axis and the angle of field of view are variable,
Means for creating an image wider than the captured image to be detected by zooming out or synthesizing a plurality of captured images;
Detection condition setting means for setting a reference background image, a template, a mask area or a warning area, and size information as detection conditions used for detecting an image of an object with respect to the wide image;
Field angle change calculation means for calculating a ratio between the focal length of the lens of the imaging device when creating the wide image and the focal length of the lens in the captured image as the object detection target as the amount of change in the angle of view. When,
By multiplying or dividing the calculated amount of change in the angle of view by multiplying or dividing the x-coordinate and y-coordinate when the horizontal direction of the image is the x-axis and the vertical direction is the y-axis, Image size correction means for performing coordinate conversion of the captured image,
After the coordinate transformation, a template matching process between the template set for the wide image and the captured image that is the object detection target is used to cope with a change in the direction of the optical axis in the captured image that is the object detection target. An optical axis direction change amount detecting means for detecting a change amount of the position;
Area detecting means for detecting an area of an image of an object from a captured image to be detected by the object using a difference method , using the reference background image translated based on the amount of change in the position ;
A mask area or a warning area correction means for extracting the mask area or the warning area from the wide image translated based on the detected change in position;
The coordinates of the area detected by the area detecting means are converted into the coordinate system on the wide image after the coordinate conversion, and the reference widths at the respective vertical positions y1 and y2 of the reference object set in advance on the wide image Alternatively, the reference width or height d3 at the vertical position y3 when the height is d1, d2 and the converted vertical position of the region is y3 is d3 = (d1-d2) · (y3-y2) Size condition calculating means for calculating by / (y1-y2) + d2 ,
The area detected by the area detection means using the mask area or the warning area extracted by the mask area or the warning area correction means and the reference width or height d3 calculated by the size condition calculation means Object presence determination means for determining whether or not is an object to be detected;
An object detection apparatus comprising: The object detection method according to claim 1,
The detection condition is, before Symbol template, a plurality of sets to include the roof or eaves or doorknob building,
The optical axis direction change amount detecting step uses pan and tilt information of the image pickup apparatus together in the template matching process.
After the optical axis direction change detection step, based on the detected change in position, a predetermined region is extracted from the wide image and used as a reference background image.
A warning step for outputting or displaying a different type of warning depending on whether the determined object is inside or outside the warning area when it is determined in the object presence determination step ;
The step of creating pre SL, when creating the wide image by the zoom out, the imaging device is used in shooting mode such as high-definition image can be obtained as compared with the captured image to be the object detected An object detection method characterized by the above.