JP2005341450A - Sensing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sensing apparatus in which certainty in invader detection is improved. <P>SOLUTION: A sensing apparatus comprises an imaging section 10 for acquiring a picked-up image resulting from imaging a monitoring target space; a transmitting/receiving section 14 for sending a sonic wave to the monitoring target space and receiving a sonic wave from the monitoring target space as an acoustic signal; an image processing section 12 for extracting a variable area from a differential image that is a differential between the picked-up image and a reference background image; and an information processing section 20 for dividing the variable area into image areas of size may be obtained if an invader is present at a position of a real space corresponding to the variable area, when a peak corresponding to an object existing at the position of the real space corresponding to the variable area is present in an acoustic signal received by the transmitting/receiving section 14, and for detecting the invader based on a predetermined image feature amount for each of the divided areas. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a sensing device that detects an intruder based on image information and acoustic information.

  In recent years, security systems that detect intruders or the like that have entered a monitoring space have been widely used. As a sensing device used in such a security system, there is one that detects an intruder or the like using an image acquired by an imaging device such as an image sensor. In a sensing device using such an image, a reference background image is recorded, and a method for detecting an intruder based on a difference image between a captured image and a background image captured every moment, There is a method for detecting an intruder based on a difference image between images.

  For example, in Japanese Patent Laid-Open No. 3-97080, a difference image of luminance between a captured image and a background image is generated, pixels whose difference values are equal to or greater than a predetermined threshold are extracted as variation pixels, and continuous variation pixels are extracted. A technique is disclosed in which, when the number is equal to or greater than a predetermined determination value, it is determined that a changing area constituted by changing pixels is an intruder and an alarm driving signal is output. Further, a method of calculating a moving distance and a moving speed of an object by associating a fluctuating region between a plurality of captured images and tracking (tracking) the moving state, and detecting an intruder based on these values Are also used.

Japanese Patent Laid-Open No. 3-97080

  With intruder detection using conventional image sensors, light and shadows other than the intruder are also captured in the captured image, and even if the image area moves, it may be detected as an intruder, which may be falsely reported. It was. In order to avoid this, light and shadow are distinguished from intruders using the degree of similarity and change of texture such as the correlation between the reference background image and the captured image and the degree of preservation and change of the edge of the fluctuation region. The method is used.

  However, as shown in FIG. 14, when the intruder 50 is captured while being included in the light region 52, or as shown in FIG. 15, the intruder 50 and the intruder's shadow 54 are in contact with each other and imaged. In such a case, there is a problem that it is difficult to distinguish light or shadow from an intruder even by the above method.

  The present invention has been made in view of the above-described problems of the prior art, and an object of the present invention is to provide a sensing device with improved intruder detection accuracy.

  The present invention includes: an imaging unit that acquires a captured image obtained by imaging a monitoring target space; and a transmission / reception unit that transmits a sound wave to the monitoring target space and receives a sound wave from the monitoring target space as an acoustic signal, A sensing device that detects an intruder in the monitoring target space based on a captured image acquired by an imaging unit and an acoustic signal received by the transmission / reception unit, the difference between the captured image and a reference background image A signal corresponding to an image processing unit that extracts a fluctuation region included in a captured image based on the difference image, and an object that exists at a position in a real space corresponding to the fluctuation region is received by the transmission / reception unit. It is determined whether or not an acoustic signal exists, and a signal corresponding to an object existing at a position in real space corresponding to the fluctuation region exists in the acoustic signal received by the transmission / reception unit. If determined, the variable region is partitioned into a plurality of image regions of a size that would be obtained when an intruder is present at a position in real space corresponding to the acoustic signal. And an information processing unit for detecting an intruder based on a predetermined image feature amount for each region.

  Here, it is preferable that the information processing unit determines whether or not the variation area is greater than or equal to a predetermined image area, and performs partition processing when the variation area is greater than or equal to the predetermined image area. .

  In addition, it is preferable that the information processing unit detects an intruder using a degree of change in texture of the partitioned area between the captured image and the background image. That is, it is preferable that the information processing unit detect an intruder based on an area in which the degree of texture change is maximum among the partitioned area group.

  According to the present invention, it is possible to more reliably detect an intruder from an intruder image whose characteristics are ambiguous due to light or shadow.

  As shown in FIG. 1, the sensing device 100 according to the embodiment of the present invention includes an imaging unit 10, an image processing unit 12, a transmission / reception unit 14, an acoustic processing unit 16, a storage unit 18, and an information processing unit 20. The The sensing device 100 can be configured by a computer including an imaging device such as an image sensor and a camera, a sound wave transmission / reception device, and the like.

The imaging unit 10 includes an imaging device such as an image sensor or a camera. The imaging unit 10 converts an optical image of the monitoring space into an electrical image signal and outputs the electrical image signal to the image processing unit 12. It is also preferable that the imaging unit 10 has a function of delivering the image signal to the image processing unit 12 after performing pre-processing such as amplification, filtering, and digitization on the image signal. In the present embodiment, the captured image is converted into a digitized image including discrete pixel groups, and each pixel is input to the image processing unit 12 and stored in the storage unit 18 as a captured image I expressed by a luminance value. And retained. Hereinafter, a captured image I ⁽ⁿ⁾ obtained by the n-th measurement among the captured images I that are repeatedly captured is shown.

The transmission / reception unit 14 includes an ultrasonic pulse transmitter and an ultrasonic sensor. The transmission / reception unit 14 isotropically transmits ultrasonic pulses to the monitoring space at predetermined time intervals using an ultrasonic pulse transmitter, and receives ultrasonic waves as acoustic signals from the monitoring space. At this time, an ultrasonic pulse reflected by an object or wall existing in the monitoring space is received as an acoustic signal S (t) that varies with time. When the monitoring distance is sufficiently short, the time from transmission to reception of ultrasonic waves can be ignored, and by matching the transmission timing of ultrasonic waves with the imaging timing of the imaging unit 10, the same monitoring space at approximately the same time The situation can be sensed. Hereinafter, the acoustic signal S (t) repeatedly measured and obtained in synchronization with the n-th imaging is indicated as a signal S ⁽ⁿ⁾ (t).

  The received acoustic signal S (t) is transmitted to the acoustic processing unit 16 and is stored and held in the storage unit 18. Moreover, it is also preferable that the transmission / reception unit 14 has a function of passing the acoustic signal S (t) to the acoustic processing unit 16 after performing pre-stage processing such as amplification, filter processing, and digitization processing.

  In this embodiment, a case where ultrasonic waves are used as measurement signals will be described as an example. However, the present invention is not limited to this, and sound waves in an audible range may be used. However, ultrasonic waves have the advantage that they cannot be heard by humans. The observation time of the acoustic signal S (t) is preferably changed according to the distance to be monitored. For example, if the distance to be monitored is L (m) and the sound speed is C (m / s), the observation time t (second) of the acoustic signal S (t) necessary for monitoring can be calculated by t = 2L / C. it can.

  Hereinafter, a process of detecting an intruder from the monitoring space using the captured image I acquired by the imaging unit 10 and the ultrasonic acoustic signal S (t) acquired by the transmission / reception unit 14 will be described. The function of each part of the sensing device 100 can be executed by a processing device of a computer by making each step of the flowchart shown in FIG. 2 a control program that can be executed by the computer.

In step S10, an image of the monitoring space is acquired. The imaging unit 10 acquires a captured image I of one frame at a predetermined time interval. Here, it is assumed that the captured image I ⁽ⁿ⁾ is acquired at the n-th imaging timing. The captured image I ⁽ⁿ⁾ is transferred to the image processing unit 12.

In step S12, the variable region extraction unit 22 in the image processing unit 12 extracts the variable region from the captured image I ^(n), and determines whether or not the variable region is included. Here, as shown in FIG. 1, the image processing unit 12 includes a variable region extraction unit 22, a tracking unit 24, and an image feature amount calculation unit 26. The image processing unit 12 receives the digitized captured image and performs background difference processing, binarization processing, tracking processing, and image feature amount calculation processing.

In the fluctuation region extraction unit 22, a difference image D ⁽ⁿ⁾ is generated by taking a difference value between pixels corresponding to each other in the captured image I ⁽ⁿ⁾ and the reference background image B. The background image B is an image captured in a state where no intruder or the like exists in the monitoring space, and is acquired in advance before starting the intruder detection process, and is stored and held in the storage unit 18. . Next, the difference image is binarized based on the magnitude relationship between the luminance (characteristic value) of each pixel included in the obtained difference image D ⁽ⁿ ) and a predetermined threshold value. That represents each image included in the captured image I ⁽ⁿ⁾ by ^{I (n) (i, j} ), represents each pixel contained in the background image B by B (i, j), the threshold is a T _hv Then, the value of each pixel D ⁽ⁿ⁾ (i, j) included in the difference image D ⁽ⁿ⁾ can be determined by Equation (1).

As a result, as shown in FIG. 3, _binarization is performed into a fluctuation pixel (1) whose characteristic value fluctuates more than the threshold value _Thv and a non-fluctuation pixel (0) whose fluctuation is smaller than the threshold value _Thv. Is represented as a difference image D ⁽ⁿ⁾ . The calculated difference image D ⁽ⁿ⁾ is stored and held in the storage unit 18 to be a reference for tracking processing for the next captured image.

In the present embodiment, the background image B is an image obtained by capturing the surveillance space in advance in a situation where there is no intruder, but it is also preferable to update the background image B with an image captured every predetermined time. Further, by using the captured image acquired last time as the background image B, a difference image D ⁽ⁿ⁾ between consecutive frames may be obtained and used for processing.

Here, if the captured image I ⁽ⁿ⁾ includes a variable region, the process proceeds to step S14. On the other hand, ^if the fluctuation area is not included in the captured image I ⁽ⁿ⁾ , the process proceeds to step S18. Below, the case where the fluctuation area is included in the captured image I ⁽ⁿ⁾ will be described.

In step S14, the tracking process of the fluctuation region is performed by comparing the difference image D ⁽ⁿ⁾ acquired by the current imaging with the difference image acquired in the past. In the tracking unit 24, substantially continuous variable pixel groups included in the binarized difference image D ⁽ⁿ⁾ are grouped as one variable region, and each variable region is labeled with a unique label (number). Subsequently, the past difference image held in the storage unit 18 is read, and the past difference image corresponding to the tracking search range set around each variable region included in the difference image D ⁽ⁿ⁾ If a variable region exists in the region, the variable regions estimated to be regions in which the same subject is imaged based on the similarity of the feature amount such as the size and shape of the region are associated with each other. The difference image D ⁽ⁿ⁾ and the labeling information are sent to the image feature quantity calculation unit 26.

In step S16, the image feature amount calculation unit 26 calculates various feature amounts related to the image based on the difference image D ⁽ⁿ⁾ and the labeling information. The image feature amount calculation unit 26 includes, for example, an image distance calculation unit, a region size calculation unit, and a movement vector calculation unit.

  The image distance calculation unit calculates the distance from the imaging device included in the imaging unit 10 to the object imaged in each variable region. By assuming that the floor surface of the monitoring space is flat and the object imaged in the fluctuation region is in contact with the floor surface, the depression angle, the installation height, and the fluctuation region of the image pickup unit 10 in the image of the fluctuation region Based on the position, it is possible to estimate a linear distance from the imaging device to the object imaged in the fluctuation region.

For simplicity, a case will be described in which a variable region is extracted in front of the imaging unit 10 as shown in FIG. The vertical size of the difference image D is Y, the length to the upper end of the fluctuation area in the image is y _h , the length to the lower end of the fluctuation area in the image is y _f , and the distance between the pixels is p. As shown in FIG. 5, when the depression angle of the imaging device is θ, the installation height is H, and the focal length is F, an image in real space from the imaging device to the upper end of the object imaged in the fluctuation region using Equation (2) The distance R _h can be calculated, and the image distance R _f in the real space from the imaging device to the lower end of the object imaged in the fluctuation region can be calculated using Equation (3). When higher accuracy is required, correction may be performed in consideration of the characteristics (lens, CCD) of the imaging unit 10.

In the following, the image distance R _f obtained for the i-th fluctuation region included in the difference image D ⁽ⁿ⁾ of the n-th measurement is the image distance R _Vif ⁽ⁿ⁾ , and the image distance R _h is the image distance R. _Vih ⁽ⁿ⁾ . i is the labeling number of the variable region. Further, a position vector P _Vi ⁽ⁿ⁾ from the imaging device in the real space to the position corresponding to each fluctuation region is also calculated.

The area size calculation unit calculates the size of the image of each variable area included in the difference image D ⁽ⁿ⁾ . For example, a rectangular area circumscribing each variable area is obtained, and the size of the rectangular area is set as the size of each variable area.

In the movement vector calculation unit, the movement vector of the fluctuation region labeled as the same subject image by the tracking unit 24 is calculated. The movement vector is the speed and direction of movement of the object in real space, and the position vector P _Vi ⁽ estimated by the image distance calculation unit between a plurality of difference images obtained from captured images taken at different times. It can be calculated from the fluctuation of ⁿ⁾ .

  Note that the processing in the image distance calculation unit, region size calculation unit, and movement vector calculation unit is an example, and the method is not limited to the processing method of the present embodiment as long as the same feature amount can be calculated.

In step S18, the received wave fluctuation extraction unit 28 obtains a differential waveform from the acoustic signal S ⁽ⁿ⁾ (t). On the acoustic signal S ⁽ⁿ⁾ (t), as shown in FIG. 6A, a pulse signal reflected by an object existing in the monitoring space is superimposed. Therefore, the received wave fluctuation extraction unit 28 reads out the acoustic signal S ^(n-1) (t) (FIG. 6B) acquired at the previous measurement from the storage unit 18 and acquires the acoustic signal acquired at the previous measurement. Envelope signals of the signal S ^(n-1) (t) and the acoustic signal S ⁽ⁿ⁾ (t) acquired this time are respectively obtained. The envelope signal can be obtained by existing processing. The difference waveform d ⁽ⁿ⁾ (t) (FIG. 6C ^{) is} obtained by taking these differences using Equation (4). A peak appears in the differential waveform d ⁽ⁿ⁾ (t) when there is an object that has newly appeared in the monitoring space or when the object has moved.

In step S20, the peak included in the differential waveform d ⁽ⁿ⁾ (t) is extracted, and when the peak is included, the distance to the position corresponding to the peak is calculated. This step S20 is made into a subroutine, and is executed by the received wave fluctuation extracting unit 28 and the acoustic feature quantity calculating unit 30 according to the flowchart shown in FIG.

In step S20-1, it extracts a region exceeding the threshold T _h of the difference waveform d ^{(n) (t)} as a peak. Here, it is also preferable to detect a peak using a threshold value T _h (t) expressed as a function of time t after reflection of the ultrasonic wave from the transmission / reception unit 14. The threshold value Th (t) is preferably set to a smaller value as the reflected wave of an object at a farther distance based on the attenuation of the sound wave distance, that is, as the time t increases. Further, the threshold value _Th (t) can be set based on the image feature amount obtained in step S16.

For example, as shown in FIG. 8, when a differential waveform d ⁽ⁿ⁾ (t) and a threshold value T _h (t) are represented, a region t _S0 ⁽ⁿ⁾ <t having an amplitude equal to or greater than the threshold value _Th (t). <T _E0 ⁽ⁿ⁾ , t _S1 ⁽ⁿ⁾ <t <t _E1 ⁽ⁿ⁾ ,... _Are extracted as peak regions.

In step S20-2, the peak position having the maximum value in each peak region is extracted. For example, at t _S0 ⁽ⁿ⁾ <t <t _E0 ⁽ⁿ⁾ , the time t ₀ ⁽ⁿ⁾ having the maximum value is the peak position. Similarly, for t _S1 ⁽ⁿ⁾ <t <t _E1 ⁽ⁿ⁾ , the peak position is determined from t ₁ ⁽ⁿ⁾ ,.

In step S20-3, the distance to the object corresponding to each peak position is calculated. Hereinafter, the image distance has been calculated on the basis of the image signal R _Vif ^(n), shown in order to distinguish the R _Vih ^(n), a distance corresponding to the peak position t _i ⁽ⁿ⁾ and the acoustic distance R _Ai ⁽ⁿ⁾ . The acoustic distance R _Ai ⁽ⁿ⁾ can be calculated using Equation (5). Here, C (m / s) is the speed of sound.

In step S22, the counter m is initialized to 1. The counter m is used as a label for sequentially specifying and processing each variable region included in the difference image D ⁽ⁿ⁾ in the following processing. The processing from step S22 to step S32 is performed in the representative region extraction unit 32 of the information processing unit 20.

In step S24, matching between the image distance of the mth fluctuation region and the acoustic distance is performed. Specifically, it is investigated whether or not a peak exists between the minimum distance R _Vmf ⁽ⁿ⁾ and the maximum distance R _Vmh ⁽ⁿ⁾ of the mth fluctuation region on the difference image D ^(n). The acoustic distance R _Aj ⁽ⁿ⁾ to the peak is obtained. As shown in FIG. 9, this processing is performed by _setting the minimum distance R _Vmf ⁽ⁿ⁾ and the maximum distance R _Vmh ^{(n) of} the mth fluctuation region from the peak group of the acoustic signals extracted in step S20. It is confirmed whether or not a peak corresponding to the corresponding region (the hatched region in FIG. 9) exists in the differential waveform S ⁽ⁿ⁾ (t). If there is a peak corresponding to any fluctuation region, the acoustic distance R _Aj ⁽ⁿ⁾ corresponding to the fluctuation region is extracted and the process proceeds to step S26, and no peak corresponding to the fluctuation region exists. If YES, the process proceeds to step S38. When there are a plurality of peaks corresponding to the fluctuation region, a representative value of the acoustic distance is obtained. The representative value may have a maximum peak, for example, or may be an average value of a plurality of acoustic distances.

  In step S26, it is determined whether or not the image size of the mth fluctuation region obtained by the area region size calculation unit is larger than a predetermined threshold value. When the m-th fluctuation region is larger than the predetermined threshold, it can be determined that there is a high possibility that an image of an intruder or the like is captured in a state where the image overlaps with or is in contact with light. On the other hand, when the mth fluctuation region is equal to or less than the predetermined threshold, it can be determined that the image of the intruder or the like is not superimposed or touched with light or shadow. If the m-th variable region has an area equal to or smaller than the predetermined threshold, the process proceeds to step S32. If the area is larger than the predetermined threshold, the variable region is extracted and processed in step S28. To migrate.

In step S28, if there is an intruder of general human size at each acoustic distance R _Aj ⁽ⁿ⁾ extracted in step S24, the image size that would be obtained as an intruder is calculated. As shown in FIG. 10, the size of the imaging region changes according to the distance from the imaging unit 10 even when people of the same size are imaged. Therefore, it is assumed that a human having a general physique (for example, a physique having a height of 160 cm and a width of 50 cm) exists at the acoustic distance R _Aj ⁽ⁿ⁾ , and the size of the image region when the human is imaged is obtained.

At this time, the acoustic distance R _Aj ⁽ⁿ⁾ is more likely to correctly indicate the distance to the object than the image distances R _Vmf ⁽ⁿ⁾ and R _Vmh ^{(n), and} thus corresponds to a general person. When estimating the size of the image area, it is preferable to perform calculation using the acoustic distance R _Aj ⁽ⁿ⁾ .

For example, as shown in FIG. 10, when calculating the width of the fluctuation region in which the person in front of the acoustic sensor is projected, the actual size considered as the width of a general person is W _min , and the acoustic distance is R _Aj ^{(n ) When} the horizontal width of the captured image I ⁽ⁿ⁾ is H _x and the horizontal angle of view of the image sensor is φ, the width w of the human image region at the acoustic distance R _Aj ⁽ⁿ⁾ is simply expressed by Equation (6). Can be calculated automatically. In addition, the fluctuation region can be accurately obtained by performing correction in consideration of the characteristics of the imaging unit 10 and the like.

Similarly, it is possible to obtain the height h when the smallest person is projected on the image, where H _min is the actual size considered as the height of the person. Further, by correcting in the angular direction, the minimum human size in all areas on the image can be obtained.

In step S30, an image area that is most likely to be an image area in which an intruder is imaged is extracted from the m-th fluctuation area, and it is determined whether an intruder image is captured. As shown in FIG. 11, the size corresponding to the fluctuation region 42 in the captured image I ⁽ⁿ⁾ and the background image B stored in the storage unit 18 is the size that can be obtained as the intruder calculated in step S28. It divides into the following division area 40 Then, the degree of change in the texture of the image of each partition area 40 in the captured image I ⁽ⁿ⁾ and the background image B is obtained. Then, the partition area having the largest degree of texture change is determined as the representative area in which the intruder has been imaged. Here, as the degree of change in texture, the normalized correlation value between the partitioned area 40 of the captured image I ⁽ⁿ⁾ and the corresponding partitioned area 40 of the background image B, or the degree of increase or decrease of the edge can be used.

Further, when the variable region 42 is partitioned, it is preferable that the image region is partitioned by overlapping a predetermined amount of the partitioned region 40 with the acoustic distance R _Aj ⁽ⁿ⁾ as the center, as shown in FIG. In order to more reliably detect an image area including an image obtained by capturing an intruder, it is desirable to increase the amount of overlapping of the partitioned areas 40 as much as possible. However, considering the processing speed, it is desirable to minimize the amount of overlapping. From these balances, it is preferable that the overlapping width is ¼ or more and ／ or less of the width of the image size that will be obtained as an intruder. In particular, it is more preferable that the overlapping width is about ½ of the width of the image size that would be obtained as an intruder.

  In this way, after matching between the image distance and the acoustic distance, if the fluctuation area is large, the area where the intruder may have been imaged is extracted by dividing it into a plurality of divided areas. A person can be detected reliably and at high speed.

If the degree of texture change in the representative area between the captured image I ⁽ⁿ⁾ and the background image B is equal to or greater than a predetermined threshold, the process proceeds to step S34. If it is smaller than the predetermined threshold, the process proceeds to step S36.

In step S <b> 32, the size is compared with the size in the case where a person who is considered to have the smallest mth fluctuation region is captured as a human. The size of the image area in which a person considered to be the smallest person (for example, a physique having a height of 100 cm and a width of 30 cm) is obtained in the same manner by changing the value of the person's width W _min in Equation (6). it can. If the m-th fluctuation region is larger than the size when a person considered to be the smallest as a human is captured, the process proceeds to step S34. Otherwise, the process proceeds to step S36.

  In step S34, the instantaneous attribute level is increased in the instantaneous attribute level setting unit 34, assuming that the variation image corresponds to a human. The instantaneous human attribute is a parameter indicating the probability that a person is captured in a captured image for each measurement, and the higher the parameter value, the higher the probability that a person is captured in the captured image. It is shown. On the other hand, in step S36, it is determined that the imaged object is a small animal (dog, cat, mouse, etc.), and the instantaneous small animal attribute level is increased. The instantaneous small animal attribute is a parameter indicating the probability that a small animal is captured in a captured image for each measurement. The higher the parameter value, the higher the probability that a small animal is captured in the captured image. It is shown.

On the other hand, if the matching between the image distance and the acoustic distance fails, an acoustic distance R _Aj ⁽ⁿ⁾ corresponding to a distance smaller than the minimum distance R _Vmf ⁽ⁿ⁾ to the maximum distance R _Vmh ⁽ⁿ⁾ _exists in step S38. It is determined whether or not to do so. As shown in FIG. 12, when the acoustic distance R _Aj ⁽ⁿ⁾ exists at a distance closer to the distance range from the minimum distance R _Vmf ⁽ⁿ⁾ to the maximum distance R _Vmh ⁽ⁿ⁾ , as shown in FIG. There is a high possibility that small animals such as small birds and insects in the vicinity of the sensor are imaged, and seemingly appear as if an object is imaged at the image distance R _Vj ⁽ⁿ⁾ of the difference image D. If the acoustic distance R _Aj ⁽ⁿ⁾ is present at a distance shorter than the distance range from the minimum distance R _Vmf ⁽ⁿ⁾ to the maximum distance R _Vmh ⁽ⁿ⁾ , the _{process proceeds} to step S40. The process proceeds to S42.

In step S40, it is investigated whether a small animal such as a bird or an insect is imaged based on the size of the mth fluctuation region. The size of the variation image that should appear in the difference image D ⁽ⁿ⁾ when a small animal of the maximum size that can be considered as a small animal exists at the acoustic distance R _Aj ⁽ⁿ⁾ is obtained, and the variation image actually included in the difference image D is obtained. It is determined whether the subject is a small animal or a human by comparing with the size. The size on the image of the largest small animal is compared with the size of the fluctuation region. If the size of the fluctuation region is equal to or larger than the size of the largest small animal, the process proceeds to step S44, and the size of the small animal having the largest fluctuation region size. If smaller, the process proceeds to step S46.

  On the other hand, in step S42, the luminance of the mth fluctuation region is examined. If the luminance (average luminance) of the variable region is equal to or greater than a predetermined threshold, the process proceeds to step S48 as a variable region caused by intangible light, and the luminance (average luminance) of the variable region is predetermined. If it is smaller than the threshold value, the process proceeds to step S50 assuming that the region is a fluctuation region caused by an intangible shadow.

  In step S46, the instantaneous attribute level setting unit 34 increases the instantaneous small animal attribute level. On the other hand, in step S44, the instantaneous attribute level setting unit 34 increases the instantaneous attribute level, assuming that the imaged object is likely to be a human. In step S48, the instantaneous light attribute setting unit 34 increases the instantaneous light attribute level. The instantaneous light attribute is a parameter indicating the probability that intangible light is captured in the captured image for each measurement, and the higher the parameter value, the more invisible light is captured in the captured image. This indicates that the probability is high. On the other hand, in step S50, the instantaneous shadow attribute setting unit 34 increases the instantaneous shadow attribute setting. The instantaneous shadow attribute is a parameter indicating the probability that an intangible shadow is captured in a captured image for each measurement. The higher the parameter value, the more invisible shadow is captured in the captured image. This indicates that the probability is high.

  In step S52, it is confirmed whether or not processing has been performed for all the variable regions included in the difference image D (n). If the process from step S24 to S50 is performed for all the variable areas, the process proceeds to step S56, and if any variable area remains, the process proceeds to step S54. In step S54, the value of the counter m is incremented by 1, and the processing from step S24 is repeated for the next fluctuation region.

  In step S <b> 56, the accumulated attribute level calculation unit 36 calculates the accumulated attribute level. The accumulated attribute level is a parameter calculated based on the instantaneous attribute level and the acoustic feature amount obtained by a plurality of measurements, and indicates whether the fluctuation region is a person, a small animal, light, or a shadow. is there. For example, it is possible to calculate the accumulated person attribute level by adding the instantaneous person attribute level a predetermined number of times. In this way, it is possible to more accurately determine whether or not the fluctuation region is a person by using the accumulated person attribute degree obtained by integrating the instantaneous person attribute degree a plurality of times. Similarly, the accumulated attribute level corresponding to each instantaneous attribute level can be calculated by integrating the other instantaneous attribute levels for a predetermined number of times of measurement.

  In step S58, the overall determination unit 38 determines whether there is an intruder in the monitoring space using at least one of the instantaneous attribute level and the accumulated attribute level. For example, it is determined that there is an intruder in the monitoring space when the accumulated person attribute level has a value equal to or greater than a predetermined threshold. In addition, it is also preferable to perform a combination determination with other accumulation attribute levels, image feature amounts, and acoustic feature amounts. If it is determined that there is an intruder, the process proceeds to step S60 to perform processing such as issuing an alarm, and if it is determined that there is no intruder, the process ends. Further, the processing may be repeated from step S10.

  As described above, according to the present embodiment, it is possible to reliably detect an object existing in the monitoring space and to specify whether or not the object is an intruder.

It is a block diagram which shows the sensing apparatus in embodiment of this invention. It is a figure which shows the flowchart of intruder detection in embodiment of this invention. It is a figure explaining the production | generation method of the difference image in embodiment of this invention. It is a figure explaining the method to calculate the image distance in embodiment of this invention. It is a figure explaining the method to calculate the image distance in embodiment of this invention. It is a figure explaining the production | generation method of the differential waveform of the acoustic signal in embodiment of this invention. It is a figure which shows the flowchart of the subroutine of step S20 in embodiment of this invention. It is a figure explaining the method of peak detection in an embodiment of the invention. It is a figure which shows the relationship between the maximum distance in a difference image, and the minimum distance. It is a figure which shows the example of the change of the size of the person with respect to the distance from an imaging part. It is a figure which shows the example of the division area group which divided the fluctuation area in embodiment of this invention. It is a figure which shows the case where matching with image distance and acoustic distance was not able to be taken in embodiment of this invention. It is a figure which shows the condition where the image distance and the acoustic distance were not able to be taken in embodiment of this invention. It is a figure which shows the example of the captured image imaged by the intruder's image and light overlapping. It is a figure which shows the example of the captured image imaged in contact with the image of the intruder, and the shadow of the intruder.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Imaging part, 12 Image processing part, 14 Transmission / reception part, 16 Sound processing part, 18 Storage part, 20 Information processing part, 22 Fluctuation area extraction part, 24 Tracking part, 26 Image feature-value calculation part, 28 Received wave fluctuation extraction part , 30 acoustic feature amount calculation unit, 32 representative region extraction unit, 34 instantaneous attribute level setting unit, 36 accumulated attribute level calculation unit, 38 comprehensive determination unit, 100 sensing device.

Claims (4)

An imaging unit that acquires a captured image obtained by imaging the monitoring target space;
A transmission / reception unit for transmitting sound waves to the monitoring target space and receiving sound waves as acoustic signals from the monitoring target space;
A sensing device that detects an intruder in the monitoring target space based on a captured image acquired by the imaging unit and an acoustic signal received by the transmission / reception unit,
An image processing unit that extracts a variation area included in the captured image based on a difference image that is a difference between the captured image and a reference background image;
Determining whether a signal corresponding to an object existing at a position in real space corresponding to the fluctuation region is present in the acoustic signal received by the transceiver;
When it is determined that a signal corresponding to an object existing at a position in real space corresponding to the fluctuation region is present in the acoustic signal received by the transmission / reception unit,
The variable region is divided into a plurality of image regions of a size that would be obtained when an intruder exists at a position in the real space corresponding to the acoustic signal, and a predetermined value for each of the divided regions is obtained. An information processing unit for detecting an intruder based on the image feature amount;
A sensing device comprising: The sensing device according to claim 1,
The information processing unit determines whether or not the fluctuation region is a predetermined image area or more,
A sensing apparatus that performs partition processing when the variation area is greater than or equal to the predetermined image area. The sensing device according to claim 1 or 2,
The information processing unit detects an intruder using a degree of change in texture of the partitioned area between a captured image and a background image. The sensing device according to claim 3,
The said information processing part detects an intruder based on the area | region where the degree of change of a texture is the maximum among the divided area groups.

Images