JP2013190943A - Object detector and intercom system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To attain detection of an object such as a person at high speed and at low cost by reducing the number of information dimensions in a local area.SOLUTION: An object detector has: an image input part 1 which generates an image frame group; a time variation extraction part 2 which calculates the maximum value of variation of luminance, and a degree of dispersion of the luminance of the same part from a reference frame to a frame before the predetermined number among the generated image frames, and determines whether pixels are at dynamic states or static states for every pixel from a calculated luminance value and the calculated degree of dispersion; an interframe feature extraction part 3 which calculates feature quantity of motions by creating histogram consisting of two values of the dynamic state/static state calculated by forming cells consisting of a plurality of pixels for every cell, and after that normalizing the histogram; and a detection part 4 constituted of a cascade identifier by AdaBoost to be constituted of a strong identifier by which existence of an object such as a person or a vehicle by using learning samples classified by the object and the background and the feature quantity from the feature quantity, and detects the existence of the object from a pick up video of a camera.

Description

  The present invention relates to an object detection device that detects an object from a captured image of a camera, and an interphone system including the object detection device.

  In recent years, with the increase and sophistication of crime, the demand for security cameras with high security functions is increasing even in ordinary households. An example of such a security camera is a sensor camera as described in Patent Document 1. In Patent Document 1, if a sensor camera is connected to an intercom and the captured person's characteristics match the pre-stored characteristics, it is determined that the person is not a suspicious person, and no warning is issued. It is comprised so that it may judge.

On the other hand, the development of technology for detecting an object such as a person from an image is remarkable, and there is a technique for detecting a person from an image using HOG feature amounts as described in Non-Patent Document 1. In addition, as a study for improving performance for human detection using HOG feature amounts, a detection method (Patent Document 2) in which a spatio-temporal feature using a result of pixel state analysis is combined with a HOG feature has been reported.
Further, the methods of Non-Patent Document 1 and Patent Document 2 have feature information of several gradations corresponding to an angle for each local region, and thus become a high-dimensional feature amount. For reducing the amount of high-dimensional information, efforts have been made to reduce the amount of information by numerically convolving the feature amount of the conventional method (Non-patent Document 2).

JP 2009-272819 A JP 2009-301104 A

N.Dalal and B. Triggs, “Histograms of Oriented Gradients for Human Detection”, IEEE Computer Vision and Pattern Recognitio, pp. 886-893, 2005 Matsushima Chika, Yamauchi Satoshi, Yamashita Takayoshi, Fujiyoshi Hironobu: “Object Detection Using Relational Binarized HOG Features and Real AdaBoost Binary Selection”, MIRU2010

The sensor camera described in Patent Document 1 does not start and does not issue an alarm unless the human sensor performs a sensing operation. Therefore, when the suspicious person is in the imaging range of the camera but at a distance where the human sensor does not react, the imaging is not performed and no alarm is issued.
Therefore, when there is a person in the imaging range of the camera, even if the person is away from the camera, if the camera can recognize it, the crime prevention function can be greatly enhanced. The same applies to a suspicious vehicle other than a person, for example, and it is effective for crime prevention if it can be detected from an image.

  However, the configuration in the case of implementing the technique for detecting an object from the images of Patent Document 2 and Non-Patent Document 1 is not realistic because the amount of data to be calculated is large and the calculation unit is expensive. Further, it is not preferable to reduce the dimension of the feature amount by convolution as in Non-Patent Document 2, because it leads to a reduction in the amount of information that the feature amount should originally have.

  Therefore, in view of such problems, the present invention can reduce calculation processing by reducing the number of information dimensions in a local region, and can realize detection of an object such as a person or a vehicle at high speed and at low cost. It is an object of the present invention to provide an interphone system that improves the crime prevention function by providing the object detection device described above and incorporating this object detection function into an interphone device.

In order to solve the above-mentioned problems, the invention of claim 1 is directed to an image frame group generation means for generating an image frame group consisting of continuous images on a time axis based on a captured image of a camera, and a plurality of generated image frames A luminance change calculating means for calculating a maximum value of a change amount of luminance of the same part for each pixel from the reference frame to a predetermined number of previous or predetermined number of image frames from an arbitrary image frame as a reference frame, and a reference frame And a dispersion degree calculating means for calculating the dispersion degree of the brightness from the brightness change information of the same part for each pixel from the reference frame to the image frame a predetermined number before or after the predetermined number, and a maximum value of the brightness change amount is a predetermined value A pixel that is greater than the luminance value and whose luminance variance is greater than the predetermined variance is determined to be a moving state, and the maximum other change in luminance is the predetermined luminance value. A pixel state determining means for determining a pixel having a lower or lower luminance dispersion degree than a predetermined dispersion state as a static state, and forming a cell composed of a plurality of pixels in a reference frame, and moving state and static state 2 for each cell. A feature amount calculating means for creating a histogram composed of values, further normalizing the histogram to calculate a feature amount of motion, and an object detecting means for detecting the presence of an object such as a person or a vehicle from the calculated feature amount of motion. The object detection means includes a cascade type discriminator using Adaboost composed of learning samples classified by the object and the background and strong discriminators learned using the feature quantity. It is characterized by detecting the presence.
According to this configuration, since the two-dimensional motion information based on the interframe difference is used as the feature amount, the number of dimensions in the local region of interest can be suppressed to two dimensions in the interframe difference feature, compared to the nine dimensions of the HOG feature. it can. Therefore, it becomes possible to detect an object such as a suspicious person or a suspicious vehicle at high speed and at low cost.

The invention of claim 2 is characterized in that an entrance cordless handset for a visitor to call a resident, a security camera for picking up a suspicious person, etc., and a function for responding to a call from the entrance cordless handset, in addition to an image picked up by the security camera. An intercom system having a living room parent device equipped with a monitor for displaying images, wherein the security camera or the living room parent device comprises the object detection device according to claim 1. It has a control unit that performs at least one of alarming, zoom display or pan / tilt display processing of a detected object, recording, and lighting of external illumination when the presence of an object is detected from a captured image of a camera. .
According to this configuration, when a suspicious person appears in the image captured by the security camera, it is recognized and an operation such as warning is performed, so that the security function can be improved. In particular, since the object is detected from the captured image compared to the conventional security camera that detects the suspicious person with the human sensor, it is possible to detect the suspicious person even at a distance that the human sensor cannot detect the suspicious person. It is effective in improving the crime prevention function.

According to the invention of claim 3, in addition to a function for a visitor to call a resident, an entrance cordless handset provided with a camera for photographing the visitor and a function for responding to a call from the entrance cordless handset An intercom system having a living room base unit equipped with a monitor for displaying an image, wherein the front door unit or the living room base unit includes the object detection device according to claim 1, and the room detection unit is configured such that the object detection device is a camera. When the presence of an object is detected from the picked-up video, a control unit that performs at least one of alarming, zoom display or pan / tilt display processing of the detected object, recording, and lighting of external illumination is provided.
According to this configuration, when a suspicious person appears in the captured image of the entrance slave camera, it is recognized and an operation such as warning is performed, so that the crime prevention function can be improved. In addition, it is possible to recognize when a bicycle or the like at the front door has moved, and to grasp theft or the like.

According to the object detection device of the present invention, by using the two-dimensional motion information based on the interframe difference as a feature amount, the number of dimensions in the local region of interest is 2 for the interframe difference feature with respect to 9 dimensions of the HOG feature. Therefore, detection of an object such as a person or a vehicle can be realized at high speed and at low cost.
In addition, according to the intercom system of the present invention, when a suspicious person is reflected in a captured image of a security camera or a camera of a front door unit, it is recognized, and a predetermined operation such as zoom display or notification is performed. The crime prevention function can be improved.

It is a functional lineblock diagram showing an example of an object detection device concerning the present invention. It is explanatory drawing of the image frame group which shows the relationship between an image frame and a luminance value. It is a block diagram which shows an example of an intercom system, and has shown each apparatus with the block diagram. It is a display explanatory drawing of a living room main | base station monitor.

  DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments embodying the present invention will be described below in detail with reference to the drawings. FIG. 1 is a functional configuration diagram showing an example of an object detection apparatus according to the present invention, in which a video signal picked up by a camera is input, and an image frame composed of continuous images on a time axis is generated and output at a predetermined interval. An image input unit 1 that performs an arbitrary image frame as a reference frame, a time change extraction unit 2 that extracts a temporal change in luminance for each pixel, and an inter-frame feature extraction unit that calculates a feature vector serving as an inter-frame difference feature for each pixel 3. A detection unit 4 that detects an object in an image frame from the learning sample and the calculated feature amount, and a result output unit 5 that outputs a detection result are provided. Of these components, the image input unit 1, the time change extraction unit 2, the inter-frame feature extraction unit 3, and the detection unit 4 constitute an object detection device, and the CPU or DSP executes a predetermined program. Is done. The whole including the result output unit 5 can be realized by a personal computer, for example.

  The image input unit 1 receives, for example, a captured video signal from a surveillance camera or a camera built in an entrance unit of an interphone system, and generates and outputs an image frame (still image) every 0.03 seconds, for example. In this way, the image input unit 1 outputs an image frame composed of digital data continuous on the time axis.

The time change extraction unit 2 extracts a time change amount of luminance from an arbitrary pixel of the image frame output from the image input unit 1. Specifically, two change amounts are calculated: a change amount T that represents the magnitude of the change in luminance of each pixel and a stability S that represents the absence of change in luminance. This operation is finally performed on all pixels in the frame.
FIG. 2 is an explanatory diagram of an image frame group showing the relationship between the image frame and the luminance value. A reference frame is K, a past frame that is an image frame output immediately before the reference frame K, and an image frame that is output immediately thereafter. The relationship of the future frame which is an image frame is shown. In addition, the luminance value of a pixel at an arbitrary coordinate in the reference frame K is denoted by Ik, and the luminance at the same coordinate in each image frame is denoted by Ik (± n). Hereinafter, a description will be given with reference to FIG.

  First, a pixel whose luminance value changes rapidly is detected. First, the amount of change between the luminance values Ik-n and Ik of each frame from frame K to N frames before is obtained. Then, the maximum value of this change amount is calculated and set as the change amount T of this coordinate. This change amount T is calculated by the following equation (1). In Equation 1, j is an integer from 1 to n.

  On the other hand, the detection of the pixel stability S focuses here on N frames after K. The stability S is calculated by the following equation (2). This stability S is a dispersion of luminance values from the frame K to K + N, and the stability S becomes small in a stable state.

  Next, threshold processing is performed on the obtained change amount T and stability S of each pixel, and when both are greater than or equal to the threshold, the moving state is set, and otherwise, the static state is set. The time change extraction unit 2 outputs luminance information binarized (in two gradations) for each pixel in this way.

  Note that the threshold value may be a fixed value set in advance. However, in the case of a fixed value, the determination varies greatly depending on the installation environment and brightness of the camera. It is preferable to determine a more appropriate value. Moreover, it is possible to adapt the stability S adapted to the future frame to the past frame, and it is also possible to adapt the change amount T adapted to the past frame to the future frame. It is.

The interframe feature extraction unit 3 calculates a feature vector that is an interframe difference feature based on the static state / motion state of each pixel calculated by the time change extraction unit 2.
First, a cell composed of a plurality of pixels is created, and a static state / motion state histogram (motion state histogram) for each pixel calculated by the time change extraction unit 2 is created for each cell region. Based on the motion state histogram, the feature vector is calculated as follows.
The feature value f is obtained by normalizing the histogram created in each cell in units of blocks formed by a plurality of cells. Note that normalization is performed to remove noise such as abnormal values.

For example, when one cell is composed of 5 × 5 pixels and one block is composed of 3 × 3 cells, the feature quantity f of the cell (i, j) in i rows and j columns is two-dimensional (2 in a moving state and a static state). Therefore, if this is Fij = {fi, fj}, the feature quantity Vk of the k-th block is 18 (2 × 3 × 3) dimensions because one block is 3 × 3 cells, It is expressed by a formula.
Vk = (Fi j, Fi + 1 j, Fi + 2 j, Fi j + 1, Fi + 1 j + 1, Fi + 2 j + 1, Fi j + 2, Fi + 1 j + 2, Fi + 2 j + 2}
Normalization is performed from the feature amount Vk by the following equation (3).

v is a feature vector, which is a feature amount after normalization. This normalization is performed by moving the block one cell at a time.
Therefore, the feature value f is normalized many times depending on the area of different blocks. For example, when the input image is 30 × 60 pixels, normalization is performed for 4 blocks in the horizontal direction and 10 blocks in the vertical direction, for a total of 40 blocks. Is implemented.
In this case, the feature vector normalized for each block is 40 blocks × 18 dimensions = 720 dimensions.

  The detection unit 4 is a learning sample classified according to a person and a background to detect a person, or a learning sample classified according to a vehicle such as a bicycle or a car and a background to detect a vehicle, and the feature amount. An object is detected using a well-known cascade booster based on Adaboost. When the object is detected in this way, an object detection signal including the object coordinate information, region information, and the like is output.

  The result output unit 5 includes a sound report unit that sounds an alarm when the detection unit 4 detects an object, a video display unit such as an LCD monitor that displays the detected video, a notification unit that reports to the outside, and the like. In response to the coordinate information output by 4, the video display unit highlights the area of the object in addition to the display of the input video. For example, the highlighting is performed by covering the detected object with a rectangular window frame.

  As described above, since the two-dimensional motion information based on the interframe difference is used as the feature amount, the number of dimensions in the local region of interest can be suppressed to two dimensions in the interframe difference feature, compared to the nine dimensions of the HOG feature. It is possible to detect an object such as a person or a vehicle, particularly a moving object, at high speed and at low cost.

  Next, an intercom system including the object detection device will be described. FIG. 3 shows a configuration diagram of the intercom system, 6 is an entrance cordless device for visitors to call out residents, and 7 is installed in a parking lot etc. to image suspicious persons etc. A sensor camera 8 is installed in the dwelling unit, and a monitor 80 that displays images taken by the entrance cordless handset 6 and the security camera 7 in addition to a function that a resident responds to a call from the entrance cordless handset 6. It is a living room master equipped with.

  The front cordless handset 6 includes a call button 61, a microphone 63a and a speaker 63b for talking, an acoustic unit 63 for controlling the microphone 63a and the speaker 63b, a slave camera 64 for capturing a visitor, and an image of the slave camera 64. A slave unit video processing unit 65 that performs video modulation processing, a slave unit CPU 66 that controls the entire entrance slave unit 6, a slave unit IF 67 for communicating with the living room master unit 8, and the like are provided.

  The sensor camera 7 collects ambient sounds, a camera 71 for imaging a suspicious person, a camera image processing unit 72 that performs modulation processing of an image captured by the camera 71, a human sensor 73 that senses the approach of a person, and the like. A microphone 74a for output, a speaker 74b for outputting sound transmitted from the living room master unit 8, an acoustic unit 74 for controlling the microphone 74a and the speaker 74b, an illumination 75 for night photography, and a camera CPU 76 for controlling the entire sensor camera 7. A camera IF 77 and the like for communicating with the room master unit 8 are provided.

  In addition to the monitor 80, the living room base unit 8 includes a call button 81 for responding to a call from the front door unit 6, a microphone 82a for collecting the voice of the resident, and a voice collected by the front side unit 6 and the sensor camera 7. And so forth, a speaker 82b for reporting the sound, a sound unit 82 for controlling the microphone 82a and the speaker 82b, and a master image processing unit 84 for receiving an output signal from the object detection unit 85 and generating an image to be displayed on the monitor 80. An object detection unit 85 for detecting an object from the captured video, a memory 86 for storing the video and the like, a touch panel 89 as an operation unit installed on the monitor 80 for performing various operations, and a video displayed on the monitor 80. In addition, a master unit CPU 87 for controlling the entire room master unit 8, a master unit IF 88 for communicating with the entrance slave unit 6 and the sensor camera 7 are provided. The object detection unit 85 is the object detection device.

The intercom system configured as described above operates as follows. However, since the calling from the front door 6, the response operation thereof, the activation of the camera 71 by the sensing operation of the sensor camera 7, and the like are the same as those in the prior art, the description thereof will be omitted. The operation when an object is detected from 71 captured images will be mainly described.
First, a case where the sensor camera 7 is activated will be described. When the sensor camera 7 is activated by the sensing operation of the human sensor 73 or the sensor camera activation operation by the operation of the touch panel 89 of the room master unit 8, a captured image is transmitted to the room master unit 8. The transmitted video is displayed on the monitor 80 via the object detection unit 85 and the parent device video processing unit 84.

At this time, the object detection unit 85 has the configuration of the object detection device shown in FIG. 1. If a moving object such as a person or a vehicle exists in the transmitted video, the object detection signal is detected by detecting the presence of the object. Output.
Upon receiving this object detection signal, the master image processing unit 84 creates a rectangular display frame surrounding the detected object from the coordinate information included in the object detection signal, and displays the image of the camera 71 displayed on the monitor 80. Display the frame created by overlapping. FIG. 4 shows the monitor 80 displayed in this manner, and a display in which a suspicious person (object) detected by the object detection unit 85 is covered with a frame F is shown in the captured image of the camera 71.

On the other hand, in response to the detection operation of the object detection unit 85, the master CPU 87 performs a predetermined operation. For example, an alarm sound is generated from the speaker 82 b of the living room base unit 8, and the captured image is recorded in the memory 86. In addition, in response to an object detection signal from the object detection unit 85, zoom processing and pan / tilt processing of the captured image are performed, and the detected suspicious person or the like is enlarged on the monitor 80 or displayed in the center of the screen. Implement the control.
In addition, the microphone 74a of the sensor camera 7 may be activated so that ambient sound is reported by the room master unit 8, or an alarm sound that is a threatening sound may be generated from the speaker 74b.

  As described above, when a suspicious person appears in the captured image of the sensor camera 7 and recognizes the suspicious person and performs a predetermined operation such as alarm sound generation or zoom processing, the crime prevention function can be improved. it can. In particular, in order to detect an object from a captured image with respect to a conventional security camera that detects a suspicious person by the human sensor 73, the human sensor 73 can detect the suspicious person even at a distance where the suspicious person cannot be detected. It is possible to improve the crime prevention function.

Next, a case where the slave camera 64 is activated will be described. When the visitor presses the call button 61 or the room base unit 8 performs a predetermined operation, the handset camera 64 starts imaging, and the captured video is displayed on the monitor 83 of the room base unit 8 via the object detection unit 85. Is done.
However, since the image captured by the handset camera 64 when the call button 61 is pressed becomes a captured image of the visitor, the detection operation of the object detection unit 85 is not required. Therefore, here, the operation when the slave camera 64 starts imaging by operating the living room master unit 8 will be described.

When the handset camera 64 is activated by the operation of the room base unit 8 to start imaging, the object detection unit 85 starts monitoring the suspicious person or the like. When an object such as a suspicious person is detected, an object detection signal is output, and the master image processing unit 84 generates a display frame F surrounding the area of the object and is displayed on the monitor 80 as in the case of the sensor camera 7. A display frame F is displayed along with the captured image of the slave camera 64.
Similarly, the base unit CPU 87 also sounds an alarm sound from the speaker 82b of the living room base unit 8 to record a captured image. As in the case of the sensor camera 7, the detected image may be enlarged and displayed on the monitor 80 by performing zoom processing and pan / tilt processing of the captured image.

  In this way, when an object such as a suspicious person is shown in the captured image of the slave camera 64, it is recognized and a predetermined operation such as an alarm or zoom display is performed, so that the crime prevention function is improved. Can do. Moreover, since the bicycle etc. of a front door moved can be recognized, it is also possible to grasp theft.

In addition, although the sensor camera 7 provided with the human sensor 73 has been described as an example of the security camera, the object detection device can be satisfactorily applied even to a security camera without the human sensor 73. If it is within the imaging range of the camera, it can be recognized even when a parked car or bicycle moves, and it is possible to grasp theft.
Moreover, although the case where the object detection part 85 was provided in the living room main | base station 8 was demonstrated, the object detection part 85 may be provided near the camera, and in the case of the said form, it is provided in the entrance cordless handset 6 or the sensor camera 7. Also good.

  In the above embodiment, the configuration in which the object detection device is incorporated in the intercom system has been described. However, the object detection device may be incorporated in another device. For example, the present invention may be applied to a crime prevention system that detects a suspicious person, or may be applied to a case where a pedestrian or a bicycle is detected in ITS (Intelligent Transport Systems).

  1 .. Image input unit (image frame group generation unit) 2 .. Time change extraction unit (luminance change calculation unit, degree of dispersion calculation unit, pixel state determination unit) 3 .. Interframe feature extraction unit (feature amount calculation) Means), 4 ··· Detection unit (object detection means), 5 · · Result output unit, 6 · · · Entrance cordless handset, 7 · · Sensor camera (security camera), 8 · · Base unit for home, 64 · · cordless handset Camera (camera), 85... Object detection unit (object detection device), 87... Master CPU (control unit).

Claims (3)

Image frame group generation means for generating an image frame group consisting of continuous images on the time axis based on captured images of the camera;
Luminance change that calculates the maximum value of the amount of change in luminance of the same part for each pixel from the reference frame to the image frame that is a predetermined number before or after the predetermined number of frames, using an arbitrary image frame among the plurality of generated image frames A calculation means;
A dispersion degree calculating means for calculating a dispersion degree of brightness from brightness change information of the same part for each pixel from the reference frame to the image frame a predetermined number before or after the reference frame;
A pixel in which the maximum value of the luminance change is greater than a predetermined luminance value and the variance of the luminance is greater than the predetermined variance is determined to be a moving state, and the other maximum value of the luminance change is A pixel state determination unit that determines a pixel having a predetermined luminance value or less, or a pixel having a luminance dispersion degree equal to or less than a predetermined dispersion degree as a static state;
In the reference frame, a cell composed of a plurality of pixels is formed, a histogram composed of the binary values of the moving state and the static state is created for each cell, and the histogram is normalized to calculate a feature amount of motion. Means,
An object detection means for detecting the presence of an object such as a person or a vehicle from the calculated feature amount of the movement,
The object detection means includes a cascade type discriminator using Adaboost composed of learning samples classified by object and background and a strong discriminator learned using the feature quantity, and the object is detected from the captured image of the camera. An object detection device characterized by detecting the presence of an object. A monitor that displays an image captured by the security camera in addition to a function for responding to a call from the entrance slave unit, a security camera for imaging a suspicious person, etc. An intercom system having a base unit with
The security camera or the room master unit comprises the object detection device according to claim 1,
When the object detection device detects the presence of an object from the image captured by the security camera, the living room master unit is at least one of alarming, zoom display or pan / tilt display processing of the detected object, recording, and lighting of external illumination. An intercom system comprising a control unit that implements the above. In addition to a function for a visitor to call a resident, an entrance cordless handset equipped with a camera for imaging the visitor, and a monitor for displaying an image captured by the camera in addition to a function for responding to a call from the entrance cordless handset An intercom system having a living room base unit equipped with
The entrance child machine or the living room parent machine comprises the object detection device according to claim 1,
When the object detection device detects the presence of an object from the image captured by the camera, the living room master device is at least one of alarming, zoom display or pan / tilt display processing of the detected object, recording, and lighting of external illumination. An intercom system comprising a control unit that implements.

Images