TW201635247A - Abnormality determination system based on sound and image changes - Google Patents

Abstract

The present invention relates to an abnormality determination system based on sound and image changes for determining whether an abnormality occurs in a region. The abnormality determination system mainly includes a camera device, a sound receiving device, and an abnormality determination device. The camera device shoots the region and generates a regional image. The sound receiving device receives sound from the region and generates a regional sound signal of the region. As such, the abnormality determination device can determine whether an abnormality occurs in respect of the received regional image in order to determine whether the image is abnormal or normal, and determine whether an abnormality occurs in respect of the received regional sound signal of the region in order to determine whether the sound is abnormal or normal. When determining the image is abnormal and/or the sound is abnormal, the abnormality determination device issues an abnormality determination signal.

Description

Abnormal judgment system based on sound and image changes

The present invention relates to an abnormality judging system, and more particularly to an abnormality judging system based on sound and image changes.

With the increase in household income and the gradual affluence of the people, the quantity and value of financial and valuable items stored in the home are constantly increasing, making home security more and more important in modern society. Although valuables can be stored in the bank safe, it is not convenient to use, especially for women's jewelry and household passbooks, bank ATM cards, cash and other storage, and depositing safes will also make valuables Lose the decorative effect in the family.

Now that people's income has increased, the number of people who travel abroad has increased. As a result, many homes are unoccupied and unattended for a certain period of time. This situation has given the thieves an opportunity to take advantage of. Therefore, the safe living and living environment has attracted more and more people's attention, and security facilities such as security doors and security windows have emerged.

In recent years, a new type of security facilities-image surveillance system has been favored by users because of its wisdom and friendliness, and has been widely used in various industries. However, the traditional image monitoring system can only provide images of the monitoring area, so that the monitoring of the monitoring area, the identification, analysis, identification, and tracking of the target object are completely dependent on the person.

In order to improve the above problems, intelligent image monitoring has an automatic and intelligent image analysis capability, which can automatically capture images taken by the camera without human intervention. Dynamic analysis can identify and distinguish dynamic targets in the scene, obtain information such as size, quantity, direction, speed, etc., and can generate alarms, videos, tracking, etc. in the fastest and best way when abnormal conditions occur. Therefore, the number of personnel is greatly reduced, the labor intensity is reduced, the false positives and false negatives are minimized, and the investment in the storage device is reduced. However, traditional intelligent image monitoring is very sensitive to light, and it is easy to cause false positives and false negatives due to lack of light or light.

In order to improve this problem, manufacturers have used infrared shielding or induced pressure to enhance false positives and false negatives. However, infrared sensing or pressure sensitive wafers have certain deficiencies. The infrared sensing method has only the way of erecting the radiation, so there is a possibility of being crossed, and it is impossible to sense the entire space. Since infrared rays are water-soluble and absorbed, they are easily affected in places where moisture is too high, and there is a case of misjudgment. In addition, the pressure-sensitive wafer method also has the drawback of infrared rays on the one hand, that is, it is impossible to sense the space, and at the same time, a large number of patches are required, which inevitably requires more cost investment and implementation difficulty when the approximate effect is achieved.

The main object of the present invention is to provide an abnormality judging system based on sound and image changes, which emits an abnormality judgment signal when sound and image appear more than a certain degree at the same time, so as to avoid excessive judgment of individual images or sounds. The false positive rate also avoids increasing the cost of soft and hardware to reduce the false positive rate.

Based on the above object, the present invention is based on an abnormality determination system for sound and image changes for determining whether an abnormality occurs in a region. The abnormality determination system mainly includes a photographing device, a sound receiving device, and an abnormality determining device. The photographing device photographs the area and produces an area image, and the sound receiving device picks up the area and produces an area area sound signal. In this way, the abnormality determining device can determine whether there is an abnormality for the received region image, and determine the image. It is judged that the abnormality or the image judgment is normal, and it is judged whether there is an abnormality with respect to the received regional area sound signal, and it is judged that the sound judgment is abnormal or the sound judgment is normal. When the abnormality determining device determines that the image determination is abnormal or/and the sound determination is abnormal, an abnormality determination signal is issued.

The advantages and spirit of the present invention will be further understood from the following detailed description of the invention.

10‧‧‧Photographing device

20‧‧‧Sound receiver

30‧‧‧Abnormal judgment device

401, 402‧‧‧ regional imagery

501, 502‧‧‧ regional audio signals

601‧‧ ‧ decibel table

602‧‧‧ distance decibel table

The first A~B diagram is a schematic diagram of an abnormality judgment system based on sound and image changes of the present invention.

The second A to B diagrams are schematic diagrams showing the operation mode of the abnormality determining apparatus of the present invention.

The third A~3B diagram is a schematic diagram of the abnormal judgment of the present invention.

The fourth A to the fourth B diagrams are schematic diagrams of the abnormal sound judgment of the present invention.

Please refer to the first A~B diagram. The first A~B diagram is a schematic diagram of the abnormality judgment system based on sound and image change in the present invention. As shown in FIG. A, the abnormality determination system of the present invention mainly includes a photographing device 10, a sound receiving device 20, and an abnormality determining device 30. The photographing device 10 photographs the area and produces an area image, and the sound receiving device 20 picks up the area and produces an area area sound signal.

For example, if the area monitored by the abnormality judging system based on the sound and image changes of the present invention is the wall of the community, and the monitoring of whether there is a small over-the-wall and invading the community, the photographing apparatus 10 and the sound receiving apparatus 20 can adopt "video synchronization". For the model, or add a live radio "microphone" next to the original monitor. In order to monitor the behavior of overturning the wall and falling over the ground, the photographic device 10 is aligned with the wall and continuously monitors the image changes above the wall, while the sound receiving device 20 can use the microphone array or the pointing microphone to perform sound monitoring on the wall and the wall adjacent to the wall. control.

Briefly, the abnormality determining device 30 can determine the received region image If there is an abnormality, it is judged that the image judgment is abnormal or the image judgment is normal, and it is judged whether there is an abnormality with respect to the received area area sound signal, and it is judged that the sound judgment is abnormal or the sound judgment is normal. When the abnormality determining device determines that the image determination is abnormal or/and the sound determination is abnormal, an abnormality determination signal is sent to the monitoring center or related personnel, and the amplifier of the place where the event occurs may be used to issue an alarm or light a warning light to scare off the intruder.

As shown in Figure B, in order to allow users to use the system more flexibly In addition to the abnormality of monitoring images and sounds, it also sends an abnormality judgment signal to the monitoring center or related personnel for the case where only the image judgment is abnormal, the sound judgment is abnormal, and the image judgment is abnormal or the sound judgment is abnormal. You can also use the amplifier at the location where the event occurred to sound an alarm or illuminate a warning light to scare off the intruder.

However, there is a need for further explanation regarding the above abnormality judgment mechanism.

Please refer to the second A~B diagram, and the second A~B diagram is a schematic diagram of the operation mode of the abnormality determining apparatus of the present invention. In the abnormality determination system of the present invention, in order to cooperate with the above-described various abnormality determination devices, the abnormality determination device 30 of the present invention needs to first determine an abnormality in image determination or abnormality in sound determination. As shown in FIG. 2A, the abnormality determining device 30 of the present invention determines that the image is abnormal and the second degree of variation of the regional region sound signal is that the first degree of variation of the received region image exceeds the first variation comparison value. When the first change comparison value is exceeded and the sound is judged to be abnormal, an abnormality determination signal is issued. However, the abnormality determining device 30 does not need to first determine the image determination abnormality or the sound determination abnormality, and the first variation degree of the region image received by the abnormality determining device 30 may exceed the first variation comparison value and the regional region sound. The second change in the signal is greater than the first When the comparison value is changed, an abnormality judgment signal is issued, as shown in the second B diagram. Underneath, the image judgment abnormality and the sound judgment abnormality will be described in more detail.

Please refer to the third A~3B diagram, and the third A~3B diagram is the indication of the abnormal judgment of the present invention. intention. As shown in FIG. 3A, the abnormality determining device 30 first performs shooting at the first time, and then performs shooting at the second time, and then receives the area image 401 received at the first time and received at the second time. The resulting region image 402 is compared to the difference image. However, the regional image 401 may also be the average of the regional images received during the reservation time. In other words, the area image 401 can be an image prepared in advance, or an image can be set for each hour, so that the reference image can be automatically changed with the change of the external environment such as day or night, and no artificial designation is required. Which time point is the reference image. Among them, if the photographing device 10 is of a rotary type, the captured area image may lack relativity, so the image matching and alignment technique can be used to find the area image 401 received at the first time, and in the second The area image 402 received at the time is aligned to compensate for variations caused by the photographing apparatus 10 in the moving photographing mode.

For example, the area image 401 originally only photographed the wall and the nearby vegetation. However, the area image 402 has an abnormal object of about 50 x 200 cm on the original wall. In this case, the difference image belonging to the first degree of change compared between the first time and the second time by the abnormality determining device 30 determines that the image determination is abnormal when the first variation comparison value is exceeded.

For example, if the first change comparison value is a proportional value, then the exception can be based on The proportion of the object occupying the entire area image is used as a criterion for image anomaly (for example, an anomalous object occupies more than 10% of the entire area image). In order to achieve this effect, the abnormality determining device 30 first uses the depth map technique to determine the probability of the abnormality of the abnormal object in the difference image. Cloth, and find a large enough difference to avoid interference with noise such as leaves, leading to misjudgment. Then, based on the foreign matter probability distribution, the abnormality determining means 30 analyzes the abnormal object from the area image 402 received at the second time by the background subtraction technique.

When the anomalous object in the area image occupies more than the first variation ratio When the ratio value (for example, 10%) is compared, and the second degree of change of the regional area sound signal exceeds the second variation comparison value, an abnormality determination signal is issued. Of course, if only the range of the anomalous objects in the area image is more than 10%, a judgment signal can be issued.

The part of the sound judgment abnormality is roughly the same in principle. As shown in Figure 3B, the sound The tone receiving device 20 first receives the sound at the first time, and then performs the sound collection at the second time, and then according to the regional sound signal 501 received at the first time and the regional sound signal 502 received at the second time. Compare the difference decibels.

Next, the abnormality determining means 30 is based on the area decibel received at the first time. The value 501, and the region decibel value 502 received at the second time, compares the difference decibels, wherein the first time is earlier than the second time. Similarly, the region decibel value 501 received at the first time can be replaced with the region decibel value received during the reservation time.

Thus, when the first variation of the regional image exceeds the first variation comparison value When the difference decibel belonging to the second degree of variation exceeds the second variation comparison value, the abnormality determining means 30 issues an abnormality determination signal. In the case of abnormal sound judgment, the decibel value of the region in the sound frequency band that can be emitted by the physical object (for example, small) can be selected to compare the differential decibel to exclude the unmarked object (for example, extremely low frequency or super high audio). The sound interferes, and the sound frequency band is 20 Hz to 4 kHz.

It should be noted that the sound receiving device 20 can be directly measured against the decibel meter. The measured sound produces a decibel value of the digital signal. If a general microphone is used, a conversion mechanism is required to produce a decibel value of the digital signal.

Please refer to the fourth A~4B picture, and the fourth A~4B picture is the abnormal sound judgment of the present invention. Schematic diagram. As shown in FIG. 4A, the abnormality determining device 30 processes the received regional region audio signal into a digital signal, and generates a regional decibel value based on the decibel comparison table 601. Among them, the decibel comparison table 601 includes a decibel value of a plurality of digital signals and matching digital signals. Simply put, the decibel table can convert a simple regional sound signal into a shell value.

However, in the actual test, it is necessary to determine that the abnormality determining device 30 is not located. The decibel value of the ground is estimated as the decibel value of a specific location of the adjacent sound receiving device 20. Therefore, it is necessary to compensate the energy attenuation loss caused by the distance according to the actual or estimated distance value to determine the estimated decibel value.

As shown in the fourth B diagram, in the process of converting the decibel value of the digital signal, After the regional area sound signal is processed into a digital signal, the sound source energy attenuation calculation program is further included, and the area decibel value is generated based on the distance based on the distance decibel comparison table 602. The distance decibel table 602 includes a plurality of digital signals and a decibel value matching the digital signal and the distance. The decibel comparison table 602 can calculate the equation by the sound source energy attenuation, or by repeatedly experimenting in advance, and storing the obtained estimated decibel value in the distance value and the estimated decibel value comparison table, so as to be used for actual use, according to the user. Enter the distance to convert the estimated decibel value for a particular location. Since the equation for calculating the energy attenuation of the sound source is an application of the general natural principle, it is not exclusive to the present invention, and has been widely used in various fields, and has long been well known, and therefore will not be described herein.

In the above description, the required distance value can be estimated by using image technology. It is not necessary for the user to manually input between the sound receiving device 20 and the monitored place (such as a wall) the distance. That is to say, in the case where the photographing device 10 and the sound receiving device 20 are integrated or physically in contact with each other, the above distance may be a photographing distance. In order to derive the photographing distance, the distance between the abnormal object and the photographing device 10 can be judged by the depth map technique.

In order to distinguish the decibel number, it is corrected by the distance if it is at the sound receiving device 20 The measured position has a decibel value of 60 dB, which can be displayed as 60 dB/0 m on the display; if the specific position (eg, wall) is measured, the sound receiving device 20 is 20 M and the estimated decibel value is 20 dB. At 70dB, it displays the estimated decibel value of the specific position, and can process the measured value of the measured value and the input distance value to display the estimated decibel value of 70dB/20m; or at the same time each specific position ( The estimated decibel values of the sound receiving device 20 to the noise source range are displayed on the display, such as 60dB/0m, 61dB/2m, 65dB/10m...70dB/20m (however, the estimated decibel value is limited to sound). The linear path of the receiving device 20 and the sound source and its adjacent area, if the deviation is too far, depends on the specific location of the sound source and the sound receiving device 20.

With the above detailed description of the preferred embodiments, it is desirable to describe this more clearly. The invention is not limited by the specific embodiments disclosed herein. On the contrary, the intention is to cover various modifications and equivalents within the scope of the invention as claimed.

10‧‧‧Photographing device

20‧‧‧Sound receiver

30‧‧‧Abnormal judgment device

Claims (19)

An abnormality judging system based on sound and image changes for judging whether an area has an abnormality, the abnormality judging system comprising: a photographing device for photographing the area and generating an area image; a sound receiving device, The method is used for collecting the sound of the area, and generating an area area sound signal; and an abnormality determining device, determining whether an abnormality is detected for the received image of the area, and determining that an image is abnormal or an image is judged to be normal. And determining whether there is an abnormality in the received regional area audio signal, and determining that a sound determination is abnormal or a sound determination is normal; wherein the abnormality determining device determines that the image determination is abnormal or/and the sound is abnormally determined At the time, an abnormal judgment signal is issued. The abnormality judging system based on sound and image change according to claim 1, wherein the abnormality judging device is the image in a first variation of the received image of the region exceeding a first variation comparison value. The abnormality determination signal is issued when it is determined that the abnormality and a second degree of fluctuation of the regional area sound signal exceeds a first variation comparison value and the sound is abnormally determined. An abnormality judging system based on sound and image change according to claim 1, wherein a first variation degree of the image of the region received by the abnormality determining device exceeds a first variation comparison value, and the region region When the second variation degree of the sound signal exceeds a second variation comparison value, the abnormality determination signal is issued. An abnormality judging system based on sound and image change as described in claim 3, wherein the abnormality judging means receives the image of the area received at a first time and the second time received The area image compares a difference image, wherein the first time is earlier than the second time. An abnormality judging system based on sound and image changes as described in claim 3, wherein the image of the area received at the first time and the image of the area received at the second time are aligned so that The variation caused by the photographing device in the mobile photography mode is compensated. An abnormality judging system based on sound and image changes as described in claim 4, wherein the difference image that the abnormality judging device compares in the first time and the second time belongs to the first degree of variation exceeds The abnormality determination signal is issued when the first variation comparison value and the second variation degree of the regional region audio signal exceed the second variation comparison value. Anomalies based on sound and image changes as described in item 3 of the patent application scope a judging system, wherein the abnormality determining device compares a difference image according to the image of the area received at a first time and the image of the area received during a predetermined time. The abnormality judging system based on sound and image change according to Item 7 of the patent application, wherein the difference judging device compares the difference image belonging to the first degree of change at the first time and the reservation time. The abnormality determination signal is issued when the first variation comparison value and the second variation degree of the regional region audio signal exceed the second variation comparison value. An abnormality judging system based on sound and image change as described in claim 4 or 7, wherein the abnormality judging device uses a depth map technique to determine an abnormality probability distribution of an abnormal object in the difference image. And according to the foreign matter probability distribution, the abnormality determining device uses the background subtraction technique to analyze an abnormal object from the region image received at the second time. An abnormality judging system based on sound and image change according to claim 9, wherein when the range occupied by the abnormal object in the region image exceeds a proportional value belonging to the first variation comparison value, and the region region When the second degree of change of the sound signal exceeds the second variation comparison value, the abnormality determination signal is issued. The abnormality judging system based on the sound and image change described in claim 3, wherein the abnormality judging device processes the received regional area sound signal into a digital signal, and outputs the data based on a decibel comparison table. An area decibel value, the decibel comparison table includes a plurality of digital signals and a decibel value matching the digital signal. An abnormality judging system based on sound and image change as described in claim 3, wherein the abnormality judging device processes the received regional area sound signal into a digital signal, and based on a photographing distance, based on a distance The decibel table produces a region decibel value, and the distance decibel table includes a plurality of digit signals and a decibel value matching the digit signal and the photographing distance. The abnormality judging system based on the sound and image change described in claim 7, wherein the abnormality judging device processes the received regional area sound signal into a digital signal, and based on a photographing distance, based on a distance Decibel comparison table generates a regional decibel value, the distance decibel comparison table includes a plurality of digital signals and a decibel value matching the digital signal and the photographic distance, wherein the photographic distance is determined by a depth map technique The distance between the anomalous object and the photographic device. An abnormality judging system based on sound and image changes as described in claim 11 or 12 or 13 wherein the abnormality judging means is based on the decibel value of the region received at a first time, and in a second Time received The received decibel value of the region is compared to a difference decibel, wherein the first time is earlier than the second time. An abnormality judging system based on sound and image changes as described in claim 11 or 12 or 13 wherein the abnormality judging means is based on the decibel value of the area received at a first time, and at a reservation time The decibel value of the region received in the region is compared to a difference decibel. An abnormality determining system based on sound and image changes as described in claim 14 or 15, wherein when the first degree of variation of the image of the area exceeds a first variation comparison value, and the second degree of variation When the difference decibel exceeds the second variation comparison value, the abnormality determination signal is issued. The abnormality judging system based on sound and image change as described in claim 16 wherein only the decibel value of the region in a sound frequency band that can be emitted by the physical object is selected to compare the difference decibel. An abnormality judging system based on sound and image changes as described in claim 17 of the patent application, wherein the sound frequency band is 20 Hz to 4 kHz. An abnormality judging system based on sound and image changes for monitoring a place, the abnormality judging system comprising: a photographing device for photographing the place and generating a scene image; a sound receiving device for aligning the location for sound collection and generating a local area sound signal; and an abnormality determining device, wherein the first variation of the received image of the location exceeds a first variation comparison value And an abnormality determination signal is sent when a second degree of change of the sound signal of the location area exceeds a second variation comparison value.