JP2007326380A - Security device and monitoring method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To effectively carry out a monitoring function to eliminate OFF setting of a monitoring mode by a user. <P>SOLUTION: A main control device 3 extracts features of recognition images output by a second image recognition device 2 for monitoring movement of a monitoring target from recognition images based on a temperature difference of the monitoring target. When it is determined that the monitoring target is a person, based on a window frame output by a first image recognition device 1 for recognizing the window frame on the basis of an image obtained by photographing the monitoring target, the main control device 3 determines the movement of the monitoring target inside and outside the window frame output by the second image recognition device 2. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a security device and a monitoring method.

  In the field of car security, a Doppler sensor is used to detect a suspicious person approaching a vehicle, wandering in the vicinity of the vehicle, and unauthorized intrusion into the vehicle (see, for example, Patent Document 1). The Doppler sensor detects the movement of an object by radiating a microwave transmitted from an antenna to a monitoring target and monitoring the frequency of the reflected microwave. That is, when the monitoring target is moving, the frequency of the reflected wave changes due to the Doppler effect, and the movement of the object can be detected by detecting this frequency change.

By the way, the simple attachment type car security device has a mechanism in which a body with a built-in Doppler sensor is attached to a sun visor or the like, and a warning is issued or a remote control is notified when a moving object is detected (for example, Non-Patent Document 1). According to the car security device disclosed in Non-Patent Document 1, the monitoring mode is turned on when the user leaves the vehicle, and turned off when the user approaches, so there is no need to switch the setting mode and the monitoring mode is entered. The Doppler sensor detects the suspicious person's wandering, approaching and intruding, and emits a warning sound before the crime is committed. For this reason, a suspicious person can be threatened and repelled.
Japanese Patent Laid-Open No. 2005-259065 “OMRON Carmoni500”, Internet URL <http: // www. carmoni. com / details / carmoni500. html> (viewed April 22, 2006)

  By the way, since the Doppler sensor reacts to all moving objects in principle, it reacts to dead leaves falling on the window glass of the vehicle, rain drops, etc. Therefore, even though there are no intruders. Warnings are often issued and malfunctions often occur. In particular, when the sensitivity is set high on a rainy day or the like, erroneous detection frequently occurs in response to rainwater flowing through the window glass. Conversely, when the sensitivity is set low, it is assumed that the original purpose cannot be achieved. As a result, the user often sets the monitoring mode of the car security device to OFF and maintains a state in which the prepared monitoring function is not used.

  The present invention has been made to solve the above-described problems, and a security device capable of effectively operating a monitoring function so as to reduce false detection and consequently prevent the user from setting the monitoring mode to OFF. The purpose is to provide a monitoring method.

The security device according to the present invention includes a first image recognition device that identifies a window frame by image recognition from a captured image of a monitoring target, and a second that detects a monitoring target by generating a recognition image based on the temperature distribution of the monitoring target. When the monitoring target is determined to be a person from the recognition image output by the image recognition device and the second image recognition device, it is determined whether the monitoring target is inside or outside the window frame. And a main control device for notifying abnormality according to

  In the security device of the present invention, the main control device may extract the feature of the recognition image output by the second image recognition device and determine that the monitoring target is a person.

  Further, in the security device of the present invention, the main control device determines that the vehicle is wandering near the vehicle when the monitoring target is detected only within the window frame, and the intrusion into the vehicle when the monitoring target is detected outside the window frame. It is also possible to make a determination and to notify each of them separately.

  The security device according to the present invention includes a first sensor that outputs a captured image of a monitoring target, a second sensor that measures a temperature distribution of the monitoring target, and image recognition from a captured image output from the first sensor. When the window frame is identified, a recognition image based on the temperature distribution output from the second sensor is generated, and when the monitoring object is determined to be a person by the generated recognition image, the monitoring object is the window frame. And a main control device that determines whether or not there is an abnormality and reports an abnormality according to the determination result.

  The monitoring method according to the present invention includes a step of identifying a window frame by image recognition from a captured image of a monitoring target, a step of generating a recognition image based on the temperature distribution of the monitoring target and detecting the monitoring target, and the recognition image Determining whether the monitoring target is a person based on the recognition image, and determining whether the monitoring target is a person based on the recognition image, and determining whether the monitoring target is inside or outside the window frame. Detecting an abnormality according to the result.

  According to the present invention, the monitoring function can be effectively operated so that erroneous detection is reduced and, as a result, the user does not set the monitoring mode to OFF.

(Embodiment 1)
FIG. 1 is a block diagram showing an internal configuration of a security device according to Embodiment 1 of the present invention. Here, since car security is assumed, it is assumed that the security device shown here is installed at a position where the entire interior of the vehicle can be seen, such as a sun visor or a room mirror in the vehicle.

  The security device includes a CCD (Charge Coupled Device) sensor image recognition device 1 as a first image recognition device, an infrared sensor image recognition device 2 as a second image recognition device, and a main control device 3. .

  The CCD sensor image recognition device 1 includes a CCD camera that is used for a monitoring camera or the like and picks up a visible image and its image recognition device. The infrared sensor image recognition device 2 is an infrared camera that picks up an infrared image and its image recognition device. Have

  The CCD sensor image recognition device 1 recognizes a captured image to be monitored and generates a recognition image, identifies a window frame in the vehicle from the recognition image, and outputs the recognition image and the window frame shape to the main control device 3. To do. The infrared sensor image recognition device 2 monitors the movement of the monitoring target from the recognition image based on the temperature difference from the monitoring target and outputs the recognition image to the main control device 3. The main control device 3 performs feature extraction of the recognition image output by the infrared sensor image recognition device 2, and when it is determined that the monitoring target is a person, the window frame shape output by the CCD sensor image recognition device 1 Based on the above, the movement of the monitoring target inside and outside the window frame output by the infrared sensor image recognition device 2 is determined, and the occurrence of abnormality is notified to the outside (for example, a remote controller). In addition, the infrared sensor image recognition apparatus 2 used here is used as an apparatus that captures a monitoring target using an infrared camera and recognizes a moving target from a generated temperature image.

  FIG. 2 is a block diagram showing an internal configuration of main controller 3 shown in FIG. The main control device 3 includes a recognition image alignment control unit 31, a window frame shape holding unit 32, a monitoring target determination unit 33, a dynamic detection unit 34, and a notification output unit 35. When the security device is set to a setup mode (a mode that is executed prior to a monitoring mode for discriminating movements to be monitored), the recognition image alignment control unit 31 performs the CCD sensor image recognition device 1 and the infrared sensor image recognition device. The recognition image output by 2 is acquired, alignment is performed so that both recognition images overlap, and the mode is changed to the monitoring mode at the timing when the alignment is completed, and the dynamic monitoring function by the dynamic detection unit 34 is activated.

  The window frame shape holding unit 32 holds the window frame shape data identified by the CCD sensor image recognition device 1 and supplies the data to the movement detection unit 34. In addition, an output from the monitoring target determination unit 33 is supplied to the movement detection unit 34. The monitoring target determination unit 33 determines whether the monitoring target is an object or a person based on the recognition image output from the infrared sensor recognition device 2, and outputs the determination result to the dynamic determination unit 34. Here, since the monitoring target is a person and the shape can be specified in advance, a method for extracting the shape feature of the person is employed. If the shape cannot be specified in advance, a background comparison method that sequentially compares the reference image and the input image or a frame comparison method that compares the input image that changes from moment to moment with the previous frame image is used. When these methods are used, the difference between the average luminance values for each block obtained by dividing the image is calculated, and there is a motion or shape change when the number of blocks exceeds the difference determined by threshold processing for each block. When the number of blocks is counted and the number corresponds to the size of the person to be monitored, a signal that is effective for determining the person is output.

  The behavior detection unit 34 is activated only when the security device is set to the monitoring mode, and as a result of feature extraction by the monitoring target determination unit 33, a determination result that the monitoring target is a person is obtained. The movement detection unit 34 reads shape data relating to the window frame inside the vehicle from the window frame shape storage unit 32, and based on this window frame shape, the movement of the monitoring target inside and outside the window frame output by the infrared sensor image recognition device 2 is detected. It determines and notifies the outside (for example, remote controller) that abnormality has occurred via the notification output unit 35. Specifically, the movement detection unit 34 determines that the suspicious person is approaching the vehicle or is wandering outside the vehicle when the movement of the monitoring target is detected only within the window frame, and the monitoring target is detected even outside the window frame. When the movement is detected, it is determined that the vehicle has entered the vehicle and is output to the notification output unit 35. The notification output unit 35 distinguishes between approaching or wobbling outside the vehicle and intrusion into the vehicle, and issues a warning and an alarm, respectively, and notifies the outside such as a remote controller.

  3 to 5 are flowcharts for explaining the operation of the security device according to the first embodiment. The operations of the CCD sensor image recognition device 1, the infrared sensor image recognition device 2, and the main control device 3 are divided. Explains. FIG. 5 also explains each step of the monitoring method in the security device according to the embodiment of the present invention. Hereinafter, the operation of the first embodiment of the present invention shown in FIGS. 1 and 2 will be described in detail with reference to the flowcharts of FIGS.

  First, in the flowchart of FIG. 3, the CCD sensor image recognition apparatus 1 recognizes an image acquired by a CCD camera that captures the entire interior of the vehicle and identifies the window frame in the vehicle (steps S <b> 31 and S <b> 32). Subsequently, it is determined whether or not the window frame has been recognized (step S33). When the window frame is recognized, the window frame shape and position are incorporated so as not to be affected by the monitoring target (so as not to be overwritten by the movement of the monitoring target). Is stored in a memory (not shown) (step S34). Then, the data relating to the identified window frame shape is transferred to the main controller 3. The main control device 3 determines that the monitoring target displayed in the window frame is outside the vehicle, and the monitoring target displayed outside the window frame is in the vehicle. That is, the main controller 3 determines that an event that has occurred outside the vehicle (an event that has occurred outside the window glass) is detected outside the window frame when motion is detected only within the window frame. However, if a motion is detected, it is determined that the event has occurred in the vehicle.

  In the flowchart of FIG. 4, the infrared sensor image recognition apparatus 2 generates an infrared image (temperature difference image) from temperature difference information detected by an infrared camera that captures the entire interior of the vehicle, and identifies a moving monitoring target by image recognition. (Steps S41 and S42). Subsequently, the recognized image obtained by the image recognition is transferred to the main control device 3 (step S43).

  In the flowchart of FIG. 5, the main control device 3 first determines the mode of the security device with reference to an internal flag (not shown) assigned to an arbitrary address position in the internal memory (step S51). As a result of the mode determination, when it is determined that the security device is in the setup mode, the main control device 3 acquires a recognition image from each of the CCD sensor image recognition device 1 and the infrared sensor image recognition device 2 (step S52). Then, the recognition image alignment control unit 31 performs alignment so that both of the recognition images overlap in order to detect the movement of the monitoring target inside and outside the window frame, which will be described later (step S53). Here, the alignment is realized, for example, by adjusting the direction of the optical axis of the CCD camera or infrared camera manually or automatically. In the setup mode, for example, a person stands in the scene and adjustment is performed based on the recognized image of the person.

  After the alignment operation by the recognized image alignment control unit 31 is completed, the main control device 3 updates the above-described internal flag and sets the monitoring mode (step S54). Subsequently, as a result of further mode determination, the main control device 3 acquires the window frame shape identified by the CCD sensor image recognition unit 1 and determines the window frame shape holding unit 32 when it is determined that the security device is in the monitoring mode. (Step S55). On the other hand, the monitoring target detection unit 33 performs feature extraction of the recognition image output by the infrared sensor image recognition device 2, and as a result, performs dynamic detection of whether the monitoring target is a person or an object (step S56). . Here, when it is determined that the person is a person, a signal necessary for starting the movement detection by the movement detection unit 34 is output. If it is determined that the person is not a person, the movement detection unit 34 is not activated, and the process of this time is finished.

The movement detection unit 34 activated as described above is based on the window frame shape held in the window frame shape holding unit 32, and the movement of the monitoring target based on the recognition image output by the infrared sensor image recognition device 2 is detected inside and outside the window frame. Is discriminated (step S57). Since the vehicle body exists in the area outside the window frame and infrared rays from the outside are blocked, it is possible to determine whether the window frame is inside or outside. Specifically, when the movement detection unit 34 detects the movement of a person only within the window frame, the movement detection unit 34 determines that the person approaches the vehicle or the suspicious person wanders outside the vehicle, and detects the movement of the person outside the window frame. Sometimes, it is determined that a suspicious person has entered the vehicle, and regarding the occurrence of these events, a warning and a warning are distinguished and output to the notification output unit 35 (steps S58 and S59).
Receiving this, the notification unit output unit 35 notifies the user that the abnormality has occurred by issuing a warning sound or a warning sound within the security device, and further notifies the user of the occurrence of the event by using a remote controller or the like.

  According to the first embodiment of the present invention described above, the main control device 3 performs combined monitoring of the recognition of the temperature difference of the monitoring target by the infrared image recognition device 2 and the recognition of the shape by the CCD camera image recognition device 1. Therefore, it recognizes whether the monitoring target is a person or an object and performs dynamic detection only when the monitoring target is a person, so it reacts even if dead leaves fall on the window glass of the vehicle or rain drops flow on the window glass. It is possible to achieve a highly accurate monitoring function. As a result, the user does not set the monitoring mode to OFF, and the monitoring function of the security device can always be operated effectively. In addition, since the CCD camera image recognition device 1 identifies the window frame shape and transfers it to the main control device 3, the main control device 3 can identify whether the event occurred inside the vehicle or the event outside the vehicle. It is possible to make a distinction according to the type of event such as an alarm, and to notify the user via a remote control or the like.

(Embodiment 2)
FIG. 6 is a block diagram showing an internal configuration of the security device according to Embodiment 2 of the present invention. The difference from the first embodiment shown in FIG. 1 is that the CCD sensor 4 and the infrared sensor 5 are connected to the main controller 6, and the main controller 6 has an image recognition function. Therefore, although the load on the main control device 6 increases, the load shared by the CCD sensor 4 and the infrared sensor 5 is reduced correspondingly, and the security device can be made compact.

  FIG. 7 is a block diagram showing an internal configuration of main controller 6 shown in FIG. Here, the main control device 6 is configured by the main control device (FIG. 2) according to the first embodiment (recognition image alignment control unit 31, window frame shape holding unit 32, monitoring target determination unit 33, dynamic detection unit 34). The notification output unit 35) further includes a captured image acquisition unit 61, an image recognition unit 62, a temperature difference detection unit 63, and a recognition image generation unit 64. In FIG. 7, the CCD sensor 4 connected as the first sensor supplies the captured image to be monitored to the captured image acquisition unit 61 of the main control device 6. The infrared sensor 5 connected as the second sensor measures the temperature difference from the monitoring target and supplies it to the temperature difference detection unit 63.

  The image recognition unit 62 recognizes the captured image output via the captured image acquisition unit 61 and supplies the recognized image to the recognized image alignment control unit 66. In addition, the image recognition unit 62 identifies the window frame in the vehicle and detects the window frame. The shape is held in the shape holding unit 65. On the other hand, the recognition image generation unit 64 generates a recognition image based on the temperature difference output from the infrared sensor 5 and supplies the recognition image to the recognition image alignment control unit 66 and the monitoring target determination unit 67. The recognition image alignment control unit 66 performs alignment so that the respective recognition images overlap each other, and the dynamic detection unit 68 receives the recognition image output from the recognition image generation unit 64 in the monitoring target determination unit 67. As in the first embodiment, it is activated when it is determined that the person is a person, and the movement of the person inside and outside the window frame is determined from the recognized image, and the occurrence of abnormality is notified to the outside by the notification output unit 69. .

  FIG. 8 is a flowchart for explaining the operation of the security device according to Embodiment 2 of the present invention. FIG. 8 also shows each step of the dynamic monitoring method in the security device according to the embodiment of the present invention. The operation of the security device according to Embodiment 2 of the present invention shown in FIGS. 6 and 7 will be described in detail below with reference to the flowchart of FIG.

  In the flowchart of FIG. 8, the main control device 6 acquires a captured image captured by the CCD sensor 4 with the captured image acquisition unit 61, and also transmits temperature difference information measured by the infrared sensor 5 to the temperature difference detection unit 63. (Step S81). The captured image acquired here is output to the image recognition unit 62, a recognition image is generated by the image recognition unit 62, and the detected temperature difference information is output to the recognition image generation unit 64, and the recognition image generation unit 64. A recognition image is generated based on the temperature difference information detected in (Step S82).

  Subsequently, main controller 3 refers to the internal flag to determine the mode of the security device (step S83). As a result of the mode determination, when it is determined that the security device is in the setup mode, the main control device 3 acquires a recognition image from each of the image recognition unit 62 and the recognition image generation unit 64 (step S84). Then, the recognition image alignment control unit 66 performs alignment so that both recognition images overlap in order to detect the movement of the monitoring target inside and outside the window frame (step S85).

  After the alignment operation by the recognized image alignment control unit 31 is completed, the main control device 3 updates the above-described internal flag and sets the monitoring mode (step S86). Subsequently, as a result of further mode determination, the main control device 3 acquires the window frame shape identified by the image recognition unit 62 and holds it in the window frame shape holding unit 65 when it is determined that the security device is in the monitoring mode. (Step S87). On the other hand, the monitoring target detection unit 67 performs feature extraction of the recognition image output by the recognition image generation unit 64, and as a result, performs dynamic detection only when the monitoring target is a person (step S88). Here, when it is determined that the person is a person, a signal necessary for starting the movement detection by the movement detection unit 68 is output.

  The movement detection unit 68 activated as described above determines the movement of the monitoring target inside and outside the window frame output by the recognition image generation unit 64 based on the window frame shape held in the window frame shape holding unit 65 (step). S89). Specifically, when the movement detection unit 68 detects a person's movement only within the window frame, the movement detection unit 68 determines that the person approaches the vehicle or hangs around the suspicious person outside the vehicle, and detects the movement of the person outside the window frame. Sometimes, it is determined that a suspicious person has entered the vehicle, and the occurrence of these events is distinguished from a warning and a warning and output to the notification output unit 69 (steps S90 and S91). In response to this, the notification unit output unit 69 notifies the outside that an abnormality has occurred by issuing a warning sound or a warning sound within the security device, and further notifies the user of the occurrence of the event by using a remote controller or the like.

  According to the second embodiment of the present invention described above, the main control device 3 generates an in-vehicle image acquired through the CCD sensor 4 and a recognition image based on the temperature difference information detected through the infrared sensor 5. By monitoring in combination, the movement is detected only when the monitoring target is a person, so there is no reaction even if dead leaves fall on the window glass of the vehicle or rain drops fall on the window glass. High monitoring function can be realized. As a result, the user does not set the monitoring mode to OFF, and the monitoring function of the security device can always be operated effectively. Note that the load on the CCD sensor 4 and the infrared sensor 5 is reduced by the increase in the load on the main control device 6 as compared with the first embodiment, and the security device can be configured compactly.

  In the first and second embodiments of the present invention described above, the description has been made on the premise of car security, but both can be applied to home security. The first image recognition device (or first sensor) is the image recognition device 1 (CCD sensor 4), and the second image recognition device (or second sensor) is the image recognition device 2 (infrared sensor 5). However, any sensor can be used as long as it can recognize the shape of the monitoring object and the temperature difference, and the combination thereof is also arbitrary.

  Further, in the first and second embodiments of the present invention described above, the example in which the security device is installed from the front to the rear in the vehicle has been described. However, a security device incorporating a wide-angle sensor is installed near the center of the vehicle interior, or It is also possible to install a security device incorporating a plurality of sensors in each direction, and in this case, higher-precision car security can be realized.

  Further, in cooperation with a car navigation device mounted on a vehicle, when an unexpected situation occurs by recording an image captured and acquired by the CCD sensor image recognition device 1 together with time information and position information possessed by the car navigation device. It can also be applied as a verification means. Furthermore, in cooperation with a car stereo or a car navigation device, it is possible to output warnings and alarms at a high volume from a speaker. In this case, a threatening effect against a suspicious person can be obtained.

  The main control device 3 is constituted by an IC chip including a microprocessor, a ROM, a RAM, and peripheral LSIs in terms of hardware, and the microprocessor sequentially reads and executes a program recorded in the ROM. Thus, the peripheral LSI including the RAM is controlled to execute the functions of the main control devices 1 and 6 described above. At this time, the window frame shape holding units 32 and 65 are allocated and stored in the RAM, and the recognized image alignment control units 31 and 66, the monitoring target determination units 33 and 67, the dynamic detection units 34 and 68, and the notification unit 36 are stored. 69, the captured image acquiring unit 61, the image recognizing unit 62, the temperature difference detecting unit 63, and the recognized image generating unit 64 are realized by the microprocessor sequentially reading out and executing the program recorded in the ROM. Is. Note that at least one of the above-described configurations may be realized by dedicated hardware. In this case, higher speed processing can be expected.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design and the like within the scope not departing from the gist of the present invention.

It is a block diagram which shows the internal structure of the security apparatus which concerns on Embodiment 1 of this invention. It is a block diagram which shows the internal structure of the main controller shown in FIG. It is a flowchart explaining operation | movement of the security apparatus which concerns on Embodiment 1 of this invention. It is a flowchart explaining operation | movement of the security apparatus which concerns on Embodiment 1 of this invention. It is a flowchart explaining each process of the dynamics monitoring method in the security device concerning Embodiment 1 of this invention. It is a block diagram which shows the internal structure of the security apparatus which concerns on Embodiment 2 of this invention. It is a block diagram which shows the internal structure of the main controller shown in FIG. It is a flowchart explaining each process of the dynamics monitoring method in the security device concerning Embodiment 2 of this invention.

Explanation of symbols

1 CCD sensor image recognition device (first image recognition device)
2 Infrared sensor image recognition device (second image recognition device)
3, 6 Main controller 4 CCD sensor (first sensor)
5 Infrared sensor (second sensor)
31, 66 Recognition image alignment control unit 32, 65 Window frame shape holding unit 33, 67 Monitoring target determination unit 34, 68 Dynamic detection unit 35, 69 Notification output unit

Claims (5)

A first image recognition device for identifying a window frame by image recognition from a captured image to be monitored;
A second image recognition device for generating a recognition image based on the temperature distribution of the monitoring target and detecting the monitoring target;
When it is determined from the recognition image output by the second image recognition device that the monitoring target is a person, it is determined whether the monitoring target is inside or outside the window frame, and an abnormality is detected according to the determination result. A main control device for informing
A security device comprising:   The security device according to claim 1, wherein the main control device extracts a feature of a recognition image output by the second image recognition device and determines that a monitoring target is a person.   The main control device determines that it is wandering in the vicinity of the vehicle when the monitoring target is detected only within the window frame, and determines that the vehicle enters the vehicle when the monitoring target is detected even outside the window frame, The security device according to claim 1, wherein the notification is performed with distinction. A first sensor that outputs a monitored image to be monitored;
A second sensor for measuring the temperature distribution of the monitoring target;
The window frame is identified by image recognition from the captured image output from the first sensor, and a recognition image based on the temperature distribution output from the second sensor is generated, and the monitoring target is generated from the generated recognition image. When it is determined that the person is a person, it is determined whether the monitoring target is inside or outside the window frame, and a main control device that notifies abnormality according to the determination result;
A security device comprising: Identifying a window frame by image recognition from a captured image to be monitored;
Generating a recognition image based on the temperature distribution of the monitoring target to detect the monitoring target;
Determining whether the monitoring target is a person based on the recognition image;
When determining that the monitoring target is a person based on the recognition image, determining whether the monitoring target is inside or outside the window frame, and notifying abnormality according to the determination result;
A monitoring method comprising:

Images