JP2012048689A - Abnormality detection apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an abnormality detection apparatus according to acoustics in which an erroneous detection does not frequently occur even under the environment where various acoustics similar to an abnormal sound are present.SOLUTION: The present invention relates to an abnormality detection apparatus according to acoustics, including: a sound collection section for acquiring acoustic signals being generated in a monitoring area where an important object is preserved; an acoustic analysis section for analyzing the acoustic signals collected by the sound collection section to detect a predetermined abnormal sound; a person detection section for detecting a person located in an important object area which is included in the monitoring area and preset near a place where the important object is preserved; an abnormality determination section for determining an abnormal state within the monitoring area on the basis of an abnormal sound detection result of the acoustic analysis section and a person detection result of the person detection section; and an output section for outputting a determination result of the abnormality determination section. The abnormality determination section determines an abnormal state when a person is detected by the person detection section within a predetermined time before and after the predetermined abnormal sound is determined by the acoustic analysis section.

Description

  The present invention relates to an abnormality detection apparatus using sound for detecting an abnormal state in a monitoring area based on acoustic information.

  Conventionally, an acoustic abnormality detection device has been proposed that detects abnormal sounds with abnormal waveforms such as burglary compulsive voices, gunshots, and screams of customers from acoustic signals acquired by the sound collection unit and reports them to security companies. (Patent Document 1). In addition, in order to reduce false detection due to ambient noise, an acoustic abnormality detection device that detects the presence of screams from ambient noise by analyzing the spectrum of the acoustic signal has also been proposed (Patent Document 2).

JP 5-6718 JP 9-251583 A

  However, since the above-described conventional abnormality detection device using sound detects an abnormal state in the monitoring area only from the sound signal, sound similar to abnormal sound such as surrounding announcement sounds, TV sounds, employee cheers, etc. There was a problem of erroneously detecting the signal. In order to reduce such false detection, if the detection sensitivity is lowered, it may be a cause of detection failure. Therefore, it is difficult to adjust the detection sensitivity.

  By the way, as a result of analyzing the cases of many burglary cases, the inventor found that if a burglary case occurred, at least in the vicinity of important items including storage such as a safe in the time zone before and after the occurrence of abnormal sounds such as screams and gunshots I found out that there are many people who are close to or important. For example, if a robber himself (or an employee forced by a burglar) approaches an important object after detecting an employee scream when the burglar threatens the employee, or after taking an important object For example, when detecting a scream from an employee.

  Therefore, the present invention focuses on the characteristics relating to the detection timing of the abnormal sound and the person position as described above, and does not frequently cause false detections even in an environment where various sounds similar to the abnormal sound exist. The purpose is to be able to detect an abnormal state.

  In order to solve such a problem, a sound collection unit that acquires an acoustic signal generated in a monitoring area in which an important object is stored, and the acoustic signal collected by the sound collection unit are analyzed to obtain a predetermined abnormality. An acoustic analysis unit for detecting sound, a person detection unit for detecting a person included in the monitoring object and located in the important material area set in the vicinity of the storage location of the important material, and the acoustic analysis unit Based on the sound composed of an abnormality determination unit that determines an abnormal state in the monitoring area based on an abnormal sound detection result and a person detection result of the person detection unit, and an output unit that outputs the determination result of the abnormality determination unit In the abnormality detection device, the abnormality determination unit determines an abnormal state when a person is detected by the person detection unit within a predetermined time before and after the acoustic analysis unit determines that the predetermined abnormal sound is detected. An anomaly detection device characterized by To.

With this configuration, the acoustic analysis unit of the present invention determines whether or not an abnormal sound such as a scream has occurred in the monitoring region by analyzing the acoustic signal collected by the sound collection unit. In addition, the person detection unit of the present invention detects whether or not a person is present in an important object area that is an area included in at least a monitoring area and includes a storage area for important objects. . Then, the abnormality determination unit of the present invention is based on the time determined by the acoustic analysis unit that an abnormal sound such as scream has occurred in the monitoring region, and within a predetermined time before that time or a predetermined time after that time. When the person is detected in the important object area by the person detection unit, the abnormal state is determined. And the output part of this invention outputs the said determination result.
Thereby, for example, when a robber who has taken important objects is detected by the person detection unit, and then a scream uttered by an employee is detected by the acoustic analysis unit, it can be determined that the inside of the monitoring area is in an abnormal state. Also, for example, when the employee is threatened, the scream that the employee uttered is detected by the acoustic analysis unit, and then a robber who approaches the important object area to detect the important object is detected by the person detection unit. Sometimes it can be determined that the monitoring area is in an abnormal state. As described above, the present invention takes into consideration the indoor situation such as whether or not a person exists in the important object area even if the monitoring area is an environment where various sounds similar to abnormal sounds exist. As a result, frequent occurrence of false detections can be suppressed.

Further, as a preferred aspect of the present invention, the person detection unit includes an imaging unit that sequentially acquires an image of an area including the monitoring area, a person image extraction unit that extracts a person image from the image, and a person image. A tracking unit that tracks a position on the image; a storage unit that stores a movement trajectory of the person image from a tracking result of the tracking unit; and that stores a position of the important object region on the image in advance; and the predetermined time And a person detecting means for calculating a time during which the person image stays in the important object region from the movement locus of the person image in the image, and the abnormality determining unit is calculated by the person detecting means. Assume that an abnormal state is determined when the retention time of the person image exceeds a predetermined retention time.
With this configuration, the imaging unit of the person detection unit of the present invention sequentially acquires images of the area including the monitoring area. Then, the human image extraction means of the human detection unit of the present invention extracts a human image from the image. Then, the tracking means of the person detection unit of the present invention tracks the position on the image for each extracted person image, and uses the movement locus for each person image as a storage unit. And the person detection means of the person detection part of this invention calculates the time when a person image stays in the important object area | region memorize | stored in the memory | storage part. And the abnormality determination part of this invention determines with an abnormal state, if the residence time of the person image calculated in the person detection means exceeds predetermined residence time. In this way, by tracking the position of the person with the image, the movement status and positional relationship of the person before and after detecting the scream can be taken into account in the determination of the abnormal state, and the abnormal state can be determined more accurately. it can. For example, a person image that has just temporarily crossed the important object area can be excluded from the determination that it is located in the important object area, so that it is possible to eliminate false detection factors by employees in normal conditions, and abnormalities with high accuracy. The state can be detected.

  As described above, the anomaly detection device of the present invention is prone to erroneous detection even in an environment where various sounds similar to the anomalous sound exist by actively using the location information of the person in the monitoring area. An abnormal state in the monitoring area can be detected without doing so.

The figure which showed typically the structure and arrangement | positioning image of the abnormality detection apparatus which concern on an Example. The block diagram which shows the structure of the management apparatus which concerns on a 1st Example. Diagram showing movement trajectory information Flow chart showing processing in the control unit Flowchart showing person detection processing according to the first embodiment. Flow chart showing scream detection processing Flow chart showing abnormality determination processing Typical cases that are judged as abnormal The block diagram which shows the structure of the management apparatus which concerns on a 2nd Example. Flowchart showing person detection processing according to the second embodiment.

(First embodiment)
Hereinafter, as an embodiment of the present invention, a room in a building where a safe in a building is set as a monitoring area, and acoustic information in the monitoring area and image information obtained by imaging a predetermined area in the monitoring area are used. An embodiment for detecting an abnormal state will be described with reference to the drawings.

FIG. 1 is a diagram schematically showing the overall configuration and arrangement of the abnormality detection device 1. The abnormality detection device 1 includes a management device 2, an imaging device 3, and a sound collection device 4.
In this embodiment, the abnormality detection device 1 uses the room where the safe 7 is installed as a monitoring area, detects a scream as an abnormal sound generated in the monitoring area, and within a predetermined time before and after the scream is detected. When a person image is continuously detected around the safe 7 for a predetermined time, it is determined that the state is abnormal, and a process of notifying the security center device 6 located in a remote place via the security device 5 is performed.

  The management device 2 is installed in an office or a security room (not shown) in the building, and is based on the monitoring image transmitted from the imaging device 3 connected to the management device 2 and the acoustic signal transmitted from the sound collection device 4. An abnormal state detection process is performed. Moreover, the management apparatus 2 alert | reports an abnormal condition by outputting a warning by an audio | voice or an image display to the administrator, guard, etc. who exist in an office etc. at the time of the detection of an abnormal condition. Further, the management device 2 is connected to the security device 5 and transmits an abnormal signal to the security device 5 when an abnormal state is detected.

The image pickup apparatus 3 is a so-called surveillance camera that includes an image pickup element such as a CCD element or a C-MOS element, optical system components, and the like. The imaging device 3 is installed on the upper part or ceiling of the indoor wall 9 and is installed so as to capture an image of the monitoring area from above. The imaging device 3 captures the monitoring area at predetermined time intervals and sequentially transmits monitoring images to the management device 2. The time interval at which the input image is captured is 1/5 second, for example.
The range captured as the monitoring image includes an important object region 8 illustrated by a one-dot chain line. Here, the important object area 8 means an area in a predetermined range around the safe 7 in which important objects are stored. Here, the predetermined range is a range that is likely to be located when a person is operating the safe 7. The important object area 8 is preset as a partial area on the monitoring image by the management apparatus 2 and is referred to in the abnormal state detection processing by the management apparatus 2. In this embodiment, specifically, the important object area 8 is set to an area corresponding to a radius of 3 m with the safe 7 as the center.

  The sound collecting device 4 is configured to include a microphone, an amplifier, an A / D converter, and the like, and is an electric circuit that converts sound in a monitoring space into a digital signal (acoustic signal). The sound collection device 4 is installed at a position where sound generated at least in a region including the monitoring region can be collected, converts the sound into an acoustic signal, and outputs the sound signal to the management device 2. Note that the amplifier is set in advance so that the volume of the sound signal output when a scream is generated in the room where the safe 7 is installed falls within the range of 70 to 100 dB.

  The security device 5 connected to the management device 2 is installed in a not-shown office room, security room or the like in the same building as the monitoring area. When the security device 5 receives the abnormality signal from the management device 2, the security device 5 transmits the abnormality to the security center device 6 via a wide area communication network such as a public telephone line, and the security device resident in a security center (not shown) at a remote location. Inform the employee of the occurrence of an abnormality.

  FIG. 2 shows the configuration of the management device 2. The management device 2 has a computer function and includes a storage unit 21, a control unit 22, an input unit 23, an output unit 24, and a communication unit 25.

  The communication unit 25 is a communication interface such as a LAN or USB, and communicates with the imaging device 3, the sound collection device 4, and the security device 5.

  The input unit 23 is an information input device such as a keyboard, a mouse, a touch panel, or a portable storage medium reading device. An administrator or the like can input various setting information, operation information, and the like such as the important object area 8 to the management apparatus 2 using the input unit 23.

The output unit 24 is an interface for outputting processing results from the control unit 22 to various external devices. The output unit 24 is connected to a sound output device such as a speaker or a buzzer, for example, and outputs an abnormal signal that causes the sound output device to sound a warning sound according to an instruction from the control unit 22. The output unit 24 is connected to a display output device such as a display, for example, and causes the display output device to display and output a warning message according to an instruction from the control unit 22. An administrator or the like can detect an abnormal state in the monitoring area by confirming the display output or the audio output from the output unit 24 as a warning output. Note that the administrator or the like can also confirm the setting information of the abnormality detection device 1, the monitoring image in the monitoring area, and the like by confirming the display output.
The output unit 24 may include a communication interface that transmits an abnormal signal to the external security device 5. Thereby, when an abnormal state is detected by the process of the control unit 22, the management apparatus 2 can notify a signal indicating the abnormal state to an external monitoring center or the like. In this case, the output unit 24 may be an interface device common to the communication unit 25.

  The storage unit 21 is an information storage device such as a ROM, RAM, or HDD. The storage unit 21 stores various programs and various data, and inputs / outputs such information to / from the control unit 22. The various data includes region information 211, vowel feature value 212, movement trajectory information 213, person detection time 214, and scream detection time 215.

  The area information 211 is information on the important object area 8 expressed in the coordinate system of the monitoring image. For example, when the important object area 8 is defined as a circle centered on the safe 7, the area information 211 stores the center coordinates and radius data of the circle.

The vowel feature value 212 is vowel feature value data created in advance, and is referred to in order to extract a vowel from an acoustic signal in the vowel extraction unit 224 described later. The vowel feature quantity 212 is stored in association with an identifier representing each vowel type for each vowel type.
The original data of the vowel feature 212 is a sample acoustic signal of the scream ending collected from many speakers. In advance, a spectral envelope parameter representing the frequency feature amount of the scream ending is extracted from each of these sample acoustic signals, and the distribution of the parameter is learned for each type of vowel. In this example, an 8th-order LPC cepstrum (LPC: Linear Predictive Coding) is used as a parameter representing the spectral envelope, and a GMM (Gaussian Mixture Model) is used as the distribution. In addition, three types of “a”, “e”, and “o” are used as vowels at the end of the scream. That is, as the vowel feature value 212, the GMM of the “PC” LPC cepstrum, the GMM of the “PC” LPC cepstrum, and the GMM of the “PC” LPC cepstrum are stored in the storage unit 21, respectively.

  The movement trajectory information 213 tracks the movement trajectory of the person image by analyzing the monitoring image acquired by the imaging device 3 and storing the coordinates of the person image included in the monitoring image on the monitoring image in time series. Information generated so that it can be used. As a specific example of the movement trajectory information 213, as shown in FIG. 3, the time at which the monitoring image is acquired and the unique image given to the person image when the person image is included in the monitoring image. The image ID that is the identifier, the coordinates of the center of gravity of the person image on the monitoring image, and the time that the person image is continuously located in the important object area 8 when the person image is located in the important object area 8 It is stored in the storage unit 21 as a table for associating a certain continuous location time. As will be described later, the image ID is analyzed by the control unit 22 between the person image in the monitoring image at the past time and the person image at the current time, and the person image at the past time and the person image at the current time are analyzed. Are determined to be the same, a common image ID is assigned, and if it is determined that the person image at the past time and the person image at the current time are not the same, a new image ID is assigned.

  The person detection time 214 is a time list in which the time when the person image is detected in the important object area 8 is added as needed, and is recorded by the person detection means 223 described later. In the present embodiment, when a person image has been detected in the important object region 8 for a predetermined time, the first time at which the person image was detected is added to the person detection time 214.

  The scream detection time 215 is a time list in which the time when the scream was detected in the monitoring area is added as needed, and is recorded by the scream determination means 225 described later.

  The control unit 22 is an arithmetic device such as a CPU or a DSP, for example, and executes various types of information processing according to programs stored in the storage unit 21. In the present embodiment, the control unit 22 analyzes the acoustic signal acquired by the sound collecting device 4 and the monitoring image acquired by the imaging device 3, and outputs a signal corresponding to the content of the abnormality when an abnormal state is detected. An abnormality detection process to be output to the output unit 24 is performed. In addition, the control unit 22 performs processing for storing input information such as setting information and operation information from the input unit 23 in the storage unit 21. The control unit 22 is functionally configured to include a person image extraction unit 221, a tracking unit 222, a person detection unit 223, a vowel extraction unit 224, a scream determination unit 225, and an abnormality determination unit 226. .

  The person image extraction unit 221 performs person image extraction processing for extracting one or a plurality of person images from the monitoring images for each of the monitoring images captured every predetermined time by the imaging device 3. Specifically, for example, the person image extraction unit 221 stores a monitoring image obtained by imaging a monitoring space in a state where no one is present in advance in the storage unit 21 as a background image, and the background image and the monitoring image are recorded. A difference pixel is extracted by comparing with an image. Then, among the difference pixel group configured to include the difference pixels adjacent to each other, a difference pixel group having a shape satisfying a predetermined condition and a size greater than or equal to a predetermined value is extracted as a human image representing a person. In addition to the above method, the person image extraction unit 221 may extract a person image by combining various image processes such as edge detection. Information relating to the person image extracted by the person image extraction means 221 (for example, the amount of the image feature such as the position of the person image and the color histogram) is stored in the storage unit 21 and is used for tracking the person image by the tracking means 222 described later. Used.

  The tracking unit 222 performs tracking processing of a person image each time the person image extraction unit 221 extracts a person image for each monitoring image acquired by the imaging device 3 at predetermined time intervals. In the person image tracking process, the tracking unit 222 extracts each person image extracted by the person image extracting unit 221 from the newly captured monitoring image at the past time and stored in the storage unit 21. And the movement path of the person corresponding to the person image in the monitoring space is stored in the movement trajectory information 213 of the storage unit 21. Specifically, the tracking unit 222 captures information related to each person image extracted from the new monitoring image (for example, the amount related to the position of the person image and the image feature) and is stored in the storage unit 21 last time. The person images representing the same person are associated with each other by comparing with the information about the person image extracted from. Then, the same image ID is assigned to the person images associated with each other as the person image representing the same person, and the coordinates on each monitoring image captured in time series are associated with the image ID and the movement locus of the storage unit 21. Save the information 213.

The person detection means 223 determines whether or not each person image tracked by the tracking means 222 is located in the important object region 8 and is continuously located with respect to the person image being located. The process of counting up the continuous location time, which is the current time, is performed. That is, referring to the movement trajectory information 213, the continuous location time corresponding to the image ID of the person image located in the important object region 8 at the time of the previous frame is read. And the process which adds the time of the frame period of an image acquisition with respect to the said continuation location time is performed, and the process which writes in the movement trace information 213 is performed.
In addition, the person detecting means 223 determines the person in the important object area 8 when the continuation location time is equal to or longer than the residence time T0 set in advance and stored in the storage unit 21. When the image is detected, the person flag in the storage unit 21 is turned ON, and a process of adding the time when the person image is first detected in the important object region 8 to the person detection time 214 in the storage unit 21 is performed. Here, the person flag is a flag indicating that a person image has been detected in the important object area 8.

  The vowel extraction means 224 extracts the vowel part of the sound from the acoustic signal collected by the sound collection device 4 and outputs the extraction result to the scream determination means 225. That is, the vowel extraction unit 224 calculates the frequency feature amount of the input sound signal for each frame, compares each frequency feature amount with the vowel feature amount 212, and determines whether each frame has the frequency feature amount of the vowel. The determination result is output to the scream determination means 225.

  Preferably, the vowel extraction unit 224 also sends information on the vowel type (“A”, “E”, or “O”) of the frame to the scream determination unit 225 when a frame having the frequency characteristic amount of the vowel is determined. Output. The vowel “A”, “E” or “O” extracted by the vowel extraction means 224 is a vowel typically uttered as a scream ending. Examples of typical screams that end with "A" include "Kya", "Gya", "Wow", "Wow", and "Awow". Typical examples of screams that end with "E" For example, you can list "help me", "stop me", "help me", "stop me" as an example of a typical scream that ends with "o" Can be mentioned.

  The frequency feature amount calculated by the vowel extraction means 224 is the same type as the vowel feature amount 212, and in this example, is an eighth-order LPC cepstrum. Values suitable for speech analysis are set in advance for the frame length and frame period. In this example, the frame length is 20 ms and the frame period is 10 ms. Specifically, the vowel extraction unit 224 calculates the distance between the frequency feature amount of each frame of the input acoustic signal and the vowel feature amount 212 of each vowel, and compares it with a preset vowel determination threshold value. If the distance between the frequency feature quantity and one of the vowel feature quantities 212 is equal to or less than the vowel determination threshold value, it is determined that the frame is a vowel and a vowel type corresponding to the vowel feature quantity 212. On the other hand, the vowel extraction unit 224 determines that a frame having no vowel feature value 212 whose distance is equal to or smaller than the vowel determination threshold is not a vowel.

In this example in which the vowel feature value 212 is stored in the GMM, the distance D between each vowel feature value 212 and the frequency feature value of the frame is calculated by the following equation.
D = Σwi × sqrt ((x−mi) × (x−mi) / σ ² i) (1)
However, x is a vector representing the frequency feature quantity of the frame, i is a normal distribution index (i = 1,..., K) constituting the distance calculation target vowel feature quantity 212, and wi is the i-th normal distribution. A weighting factor, mi represents an average vector of the i-th normal distribution, σ ² i represents a variance vector of the i-th normal distribution, Σ represents a summation about i, and sqrt (·) represents a square root.

  The scream determining unit 225 determines whether or not the processing target frame has a volume (power) equal to or higher than a preset scream volume threshold, thereby determining whether or not the volume is a scream volume. Perform the process. Moreover, the process which detects a scream is performed by determining whether the vowel part extracted by the vowel extraction means 224 continues more than preset scream determination time. Note that values suitable for detection of the scream ending are preset in the scream volume threshold and the scream determination time. In this example, the scream volume threshold is 70 dB, and the scream determination time is 200 ms. Incidentally, in this example in which the frame period is set to 10 ms, the scream determination time converted into the number of frames is 20 frames.

  As described above, the utterance contents of the entire screaming voice are various, but the utterance contents to be extracted can be reduced to at most three types by paying attention to the ending of the screaming. As a result, the trouble that the utterance content fails to detect an unexpected scream is significantly reduced. Also, by having a vowel part, coughing, sneezing, horning, door closing sound, squeaking sound, falling sound of cans, etc., reduce the trouble of misdetecting sounds other than screams with high volume and long duration as screams be able to. In addition, it is possible to reduce the problem of erroneously detecting a normal volume conversation voice as a scream according to the volume condition of the vowel part. Further, it is possible to reduce the problem of erroneously detecting a loud vowel such as a laughing voice as a scream according to the condition of the duration time of the vowel part.

  Here, if a section in which a vowel is continuous is included in a word of loud conversational speech, there is a possibility of erroneous detection as a scream. Therefore, the scream determination means 225 refers to the vowel type extracted by the vowel extraction means 224 to determine whether each section in which the same vowel continues is the scream part (determination section) or not. That is, it is determined whether sections having different vowel types are screamed as different vowel parts. Thus, for example, even if a section of “Ao” in which “A” and “O” are consecutive is included in a word of a loud conversation voice, and the length of the section of “AO” reaches the scream determination time, Since the “a” section and the “o” section are separate determination sections, it is possible to reduce the problem of erroneously detecting this loud conversation voice as a scream.

  Based on the person detection time 214 of the storage unit 21 that is the detection result of the person detection unit 223 and the scream detection time 215 of the storage unit 21 that is the result of the scream detection process of the scream determination unit 225, the abnormality determination unit 226 It is determined whether or not a person image is continuously detected around the safe 7 within a predetermined time before and after the detection, and an abnormal state determination process in the monitoring area is performed. Specifically, the abnormality determination unit 226 determines whether or not a person image is detected in the important object region 8 in a time period within a predetermined first abnormality determination time T1 in the previous time period from the time when the scream was detected. Is determined by referring to the scream detection time 215 and the person detection time 214. In addition, the abnormality determination unit 226 determines whether or not a person image is detected in the important object region 8 in a time zone within a predetermined second abnormality determination time T2 in a time zone after the time when the scream is detected. The determination is made by referring to the scream detection time 215 and the person detection time 214. If it is determined that there is an abnormal condition within the monitoring area, the abnormality determination unit 226 causes the output unit 24 to output a warning.

Hereinafter, an example of the flow of processing executed by the control unit 22 of the management device 2 according to the present embodiment will be described based on the flowcharts of FIGS.
FIG. 4 is a flowchart showing an abnormality detection process in the control unit 22.

  Prior to the operation, various initial settings such as setting of area information 211 are performed by the administrator or the like using the input unit 23 of the management apparatus 2 (S1). In the initial setting, first, when an administrator who has confirmed that a room is unattended starts up the management apparatus 2, the person image extraction unit 221 of the control unit 22 uses a monitoring image captured at a predetermined time after the startup. A background image is generated and stored in the storage unit 21. Further, in the initial setting, in addition to the above, the residence time T0, the first abnormality determination time T1, the second abnormality determination time T2, the area information 211, and the scream volume threshold are input and registered as necessary. When the initial setting is completed, the abnormality detection process in steps S3 to S8 is repeated each time a new monitoring image is input from the imaging device 3.

Next, returning to FIG. 4, the person detection process in step S3 will be described. The person detection process is a process performed by the person image extraction unit 221, the tracking unit 222, and the person detection unit 223 every time a monitoring image is acquired by the imaging device 3 (that is, every frame period of the monitoring image).
FIG. 5 is a flowchart illustrating person detection processing according to the present embodiment. Below, a person detection process is demonstrated based on the flowchart of FIG.

  As shown in FIG. 5, first, when a monitoring image at the current time is acquired from the imaging device 3 (S30) and the monitoring image is transmitted via the communication unit 25, the person image extraction unit 221 displays the monitoring image. And the background image are compared, and a human image extraction process is performed (S32).

  Next, the control unit 22 determines whether or not one or more person images have been extracted in the person image extraction process of step S32 (S34). If no person image is extracted (S34-No), the monitoring image acquired in step S30 is stored as a new background image in the storage unit 21 (S44), the person detection process is terminated, and the process proceeds to step S5. Here, the reason why the background image is updated when the person image is not extracted is to cope with the case where the background image changes with time.

  When a person image is extracted in the person image extraction process in step S32 (S34-Yes), the tracking unit 222 performs the person image tracking process described above (S36).

  Next, the person detection unit 223 determines whether or not each person image subjected to the tracking process by the tracking unit 222 is located in the important object region 8 (S38). Then, the person detection means 223 performs a process of counting up the continuous location time, which is the time in which the person image located in the important object region 8 is continuously located in the important object region 8 (S38). ).

  Next, the person detecting means 223 refers to the continuation location time of the movement trajectory information 213 and determines whether or not there is a person image whose continuation location time is longer than the preset residence time T0 (S40). When there is no person image whose continuation location time is longer than the residence time T0 (S40-No), the person detection process is terminated, and the process proceeds to step S5.

  On the other hand, when there is a person image whose continuation location time is equal to or longer than the residence time T0 (S40-Yes), the person detection unit 223 turns the person flag ON (S42). In addition, the person detection unit 223 performs a process of adding the time when the person image is first detected in the important object area 8 to the person detection time 214 of the storage unit 21 (S42). Thereafter, the person detection process is terminated, and the process proceeds to step S5.

Subsequently, returning to FIG. 4, the scream detection process in step S <b> 5 will be described. The scream detection processing is performed by the vowel extraction unit 224 and the scream determination unit 225, and is performed for each predetermined frame period for the acoustic signal transmitted by the sound collection device 4 and stored in the storage unit 21. Process. That is, the control unit 22 confirms whether or not an acoustic signal having a frame period length is newly stored in the storage unit 21. When the frame period arrives, the control unit 22 reads out frame data of a preset frame length from the storage unit 21, performs windowing processing using a Hamming window function, and screams the windowed frame data. Perform detection processing.
FIG. 6 is a flowchart illustrating the scream detection process according to the present embodiment. Below, a scream detection process is demonstrated based on the flowchart of FIG.

  When the frame data is read from the storage unit 21 by the control unit 22, the scream determination unit 225 calculates the volume (power) of the frame data (S52) and compares the calculated volume with a scream volume threshold ( S54). If the volume does not exceed the scream volume threshold (S54-No), the scream determination means 225 determines that there is no scream, the control unit 22 resets the value of the scream counter (S70), and ends the scream detection process. Then, the process proceeds to step S5. On the other hand, if the volume exceeds the scream volume threshold (S54-Yes), the scream determination unit 225 proceeds with the process to step S56 because there is a possibility that a scream has occurred.

  Next, the vowel extraction unit 224 calculates the frequency feature amount of the frame data (S56). Then, the vowel extraction means 224 sequentially reads out the vowel feature quantity “a” 212, the vowel feature quantity “e” 212, and the vowel feature quantity 212 “o” from the storage unit 21, It is determined whether the frame data is a vowel by comparing the frequency feature amount of the frame data (S56). In addition, the vowel extraction unit 224 specifies the vowel type when the frame data is determined to be a vowel.

  When it is determined that the frame data is not a vowel (S58-No), the vowel extraction unit 224 outputs to the scream determination unit 225 that a vowel has not been extracted. Upon receiving this output, the scream determining means 225 determines that there is no scream. And the control part 22 resets the value of a scream counter (S70), complete | finishes a scream detection process, and advances a process to step S8.

  On the other hand, if it is determined that the frame data is a vowel (S58-Yes), the vowel extraction unit 224 checks whether or not the specified vowel type is the same as the previously specified vowel type and is specified this time. The vowel type is stored in the storage unit 21 (S60). In the next confirmation, the current vowel type stored in the storage unit 21 is referred to as the previous vowel type.

If the vowel type specified this time is the same as the vowel type specified last time (S60-Yes), the vowel extraction unit 224 outputs to the scream determination unit 225 that the vowel is continuously extracted. Upon receiving this output, the scream determining means 225 increments the scream counter by 1 (S62).
On the other hand, if the vowel type specified this time is not the same as the vowel type specified last time (S60-No), the vowel extraction unit 224 outputs to the scream determination unit 225 that a new vowel has been extracted. Upon receiving this output, the scream determining means 225 sets 1 to the scream counter (S64).
When the value of the scream counter is updated in step S62 or step S64 in this way, the scream determination means 225 compares the scream counter with the scream determination time (S66). If the scream counter does not exceed the scream determination time (S66-No), it is determined that the scream determination unit 225 is continuing the determination, and the control unit 22 ends the scream detection process until the next frame period arrives, step S8. Proceed to the process.

  When the scream counter has exceeded the scream determination time (S66-Yes), the scream determination means 225 turns on the scream flag (S68). Further, the scream determining means 225 performs a process of adding the time when the scream was detected to the scream detection time 215 of the storage unit 21 (S68). Then, the control part 22 resets the value of a scream counter (S70), complete | finishes a scream detection process, and progresses to step S8.

Next, returning to FIG. 4, the abnormality determination process in step S8 will be described. The abnormality determination process in step S8 is a process performed by the abnormality determination means 226 based on the person detection time 214 in the storage unit 21 and the scream detection time 215 in the storage unit 21.
FIG. 7 is a flowchart illustrating the abnormality determination process according to the present embodiment. Hereinafter, the abnormality determination process will be described based on the flowchart of FIG.

  As shown in FIG. 7, first, the abnormality determination unit 226 determines whether or not the scream flag in the storage unit 21 is ON (S80).

  When the scream flag is ON in step S80 (S80-Yes), the abnormality determination unit 226 refers to the person detection time 214 in the storage unit 21 and refers to the current abnormality time before the first abnormality determination time T1 (with the current time). It is determined whether or not a person has been detected at the same time (S82).

If a person is detected before T1 from the current time in step S82 (S82-Yes), the abnormality determination unit 226 outputs a warning signal to the output unit 24. When the output unit 24 receives a warning signal from the abnormality determination unit 226 of the control unit 22, the output unit 24 performs a warning output process (S84).
In the warning output process, the output unit 24 performs a predetermined warning output. For example, a warning sound corresponding to the type of abnormal state is sounded from a speaker connected to the output unit 24, and the type of abnormal state is output to the display connected to the output unit 24 as a warning message. An administrator or the like can detect an abnormal state in the monitoring area by checking these warning sounds and warning messages. Further, the output unit 24 notifies the security device 5 of the abnormal state via the communication unit 25. Thereby, the management apparatus 2 can notify the abnormal state to the security center apparatus 6 via the security apparatus 5, and can notify the abnormality to the security staff who is resident and monitored at the external monitoring center. .

  When the scream flag is not ON in step S80 (S80-No), the abnormality determination unit 226 determines whether the person flag in the storage unit 21 is ON (S86).

  When the person flag is ON in step S86 (S86-Yes), the abnormality determination unit 226 refers to the scream detection time 215 in the storage unit 21 and refers to the current abnormality time before the second abnormality determination time T2 (with the current time). It is determined whether or not a scream is detected at the same time (S88). If a scream is detected before T2 from the current time in step S88 (S88-Yes), the abnormality determination unit 226 outputs a warning signal to the output unit 24 as described above.

  If no person is detected before T1 from the current time in step S82 (S82-No), if no scream is detected before T2 from the current time in step S88 (S88-No), or an alarm is output in step S84 When the process is performed, the abnormality determination unit 226 performs a process of resetting the scream flag and the person flag (that is, “0”) (S90).

  If the flag is reset in step S90, or if the person flag is not ON in step S86 (S86-No), the process returns to step S3, and thereafter, the processing from step S3 to step S8 is repeated in the same manner as described above.

  Next, a specific operation of the abnormality detection device 1 according to the present embodiment will be described using an example in which it is determined as an abnormal state. FIG. 8 is a diagram showing a typical case in which an abnormal state is determined.

  FIG. 8A is a diagram in which the monitoring images 100 from time t to time t + 3 are arranged in time series. In this case, a person image A in FIG. 8A is a person image of an employee who is bound by an intrusion robbery. The person image B is a person image of an intruder robbery who has moved to the safe 7 after threatening and binding the person image A as an employee with a weapon such as a knife.

  FIG. 8B is a diagram in which the trajectory images 110 from time t to time t + 3 are arranged in time series. The locus image 110 is drawn by superimposing the important object region 8 of the region information 211 on the image drawn so as to match the position on the image of the monitoring image 100, and is further stored in the movement locus information 213. It is an image in which the movement trajectory up to the current time is plotted for each image ID.

  In this case, it is assumed that the residence time T0 is set in advance as the “2” time interval, and the first abnormality determination time T1 and the second abnormality determination time T2 are set as the “150” time interval in advance by the administrator or the like.

  First, the state at the time t will be described. At time t, the person images A and B are extracted from the monitoring image 100 newly acquired at step S30 by the person image extracting means 221 at step S32. The person images A and B are each tracked in step S36 and associated with the person images A and B at the past time. However, since the person images A and B are not located in the important object area 8 at this time (and before this time), the duration time is not counted and the person flag is also OFF. . On the other hand, at time t, it is assumed that the person image A as an employee is tightly bound to a person image B as a robber, and at this time, a scream was made by the employee. Therefore, the scream flag is set to ON in the scream detection process in step S <b> 5, and the time t is stored as the scream detection time 215 in the storage unit 21. Therefore, in the abnormality determination process in step S8, it is determined in step S80 that the scream flag is ON (S80-Yes), and in step S82, whether or not a person has been detected before the time T1 at the time of scream detection. Determined. Here, as described above, since no person image is detected in the important object area 8 at time t, No is determined in step S82, and the scream flag is reset to OFF in step S90. Thus, at time t, abnormality determination means 226 does not determine abnormality.

  Next, the state at the time t + 1 will be described. At time t + 1, as in the case of time t, person images A and B are extracted in step S32 from the monitoring image 100 newly acquired in step S30, and are tracked and processed in step S36. It is associated with the human images A and B at time t. At this time, as in the time t, since the person images A and B are not located in the important object area 8, the continuation time is not counted and the person flag is also OFF. Therefore, since the person flag and the scream flag are both OFF, in the abnormality determination process in step S8, No is determined in step S80 and No in step S86, and no abnormality is determined.

  Next, the state at the time t + 2 will be described. Since the person image B is located in the important object area 8 at the time t + 2, the process of counting up the continuous location time in the storage unit 21 is performed in step S38. However, since the continuation location time is less than the residence time T0 (= “2” time interval), the person flag remains in the OFF state. Therefore, since the person flag and the scream flag are both OFF, in the abnormality determination process in step S8, No is determined in step S80 and No in step S86, and no abnormality is determined.

  Next, the state at the time t + 3 will be described. Since the person image B is located in the important object area 8 at the time t + 3 as in the time t + 2, a process of counting up the continuous location time in the storage unit 21 is performed in step S38. Then, since the continuation location time becomes “2” time interval and becomes the residence time T0 or more, the result is “Yes” in step S40, the person flag is turned ON, and the first detected time at the person detection time 214 in the storage unit 21. The time t + 2 is recorded. Therefore, since the person flag is ON and the scream flag is OFF, in the abnormality determination process in step S8, No is determined in step S80, Yes is determined in step S86, and scream is detected in step S88 before T2 time when the person is detected. It is determined whether or not. As described above, the second abnormality determination time T2 is set to “150” time interval, and since the time t when the scream is detected is within this time zone, Step S88 becomes Yes, and in Step S84. Alarm output processing is performed. In this way, the management device 2 can detect an abnormal state in the monitoring area at the time t + 3 and can notify the security center device 6 of the abnormal state via the security device 5 and is resident in the external monitoring center. Abnormality can be notified to the monitoring security guard.

(Second embodiment)
In the first embodiment described above, whether or not a person is present in the important object region 8 is determined based on the monitoring image from the imaging device 3. However, in the present embodiment, infrared rays emitted from the human body of a person located in the important object region 8 are received, and the human detection signal from the infrared sensor that detects the presence or absence of the human body or the like based on the change in the amount of received light is used. Based on this, it is different from the first embodiment in that it is determined whether or not a person is present in the important object region 8. It is assumed that the infrared sensor is installed in advance so that the detection area of the person using infrared rays substantially coincides with the important object area 8.

  FIG. 9 is a block diagram illustrating a configuration of the management apparatus 2 according to the second embodiment. As shown in FIG. 9, the infrared sensor is connected to the communication unit 25, the human image extraction unit 221 and the tracking unit 222 are not present in the control unit 22, the region information 211 and the movement trajectory information 213 in the storage unit 21. This is different from the first embodiment in that there is no. Further, the content of the person detection process in the person detection means 223 of the control unit 22 is different from that of the first embodiment. FIG. 10 is a flowchart showing person detection processing according to the second embodiment.

  As shown in FIG. 10, the person detection means 223 of the control unit 22 determines whether or not a person detection signal is received from the infrared sensor via the communication unit 25 in step S46. When the person detection signal is not received from the infrared sensor (S46-No), the person detection unit 223 ends the person detection process and proceeds to step S5.

  When the person detection signal is received from the infrared sensor (S46-Yes), the person detection means 223 turns the person flag ON (S48). In addition, the person detection unit 223 performs a process of adding the current time to the person detection time 214 in the storage unit 21 (S48). Thereafter, the person detection process is terminated, and the process proceeds to step S5.

  In the present embodiment, the person located in the important object region 8 is detected by the infrared sensor. However, the present invention is not limited to this, and the person may be detected by a microwave sensor or an ultrasonic sensor.

  The present invention is not limited to the above embodiments, and may be implemented in various different embodiments within the scope of the technical idea described in the claims. Moreover, the effect described in the Example is not limited to this.

In each of the above-described embodiments, an abnormal state in the monitoring area is detected by detecting a scream as an abnormal sound. However, the present invention is not limited to this, and an abnormal state in the monitoring area may be detected by detecting an abnormal sound other than a scream, for example, an abnormal sound such as a gunshot or a roar. For example, when a sound volume equal to or higher than a preset abnormal sound volume threshold is detected, it can be simply regarded as an abnormal sound such as a gunshot or anger. That is, after the volume is calculated in step S52, it is compared with the abnormal volume threshold, and if the threshold is higher than the abnormal volume threshold, the scream flag is turned ON, and the abnormality is detected at the scream detection time 215 of the storage unit 21. This can be realized by recording the sound detection time.
Further, the destructive sound generally has a remarkable feature in the rise and fall of the detected sound, such that the loudness is generally detected rapidly and then attenuated and reduced in a short time. Therefore, when a destructive sound is detected as an abnormal sound, the sound rise rate, peak volume, fall decay rate, and duration of the sound signal acquired by the sound collector 4 are calculated, and these values are set in advance. It is possible to determine whether or not the sound is a destructive sound by collating with a threshold having a destructive sound characteristic. In addition, by analyzing the frequency component of the sound that is emitted when a safe or the like is struck in advance, whether or not there is a specific frequency component in the acoustic signal at the time of detection, It is possible to greatly improve the certainty of determination. In addition, since it is difficult to destroy a strong box such as a safe, it is detected as an abnormal sound when a similar destructive sound is observed multiple times in a short time. It is also effective to impose conditions such as determining

In each of the above embodiments, when a scream is detected and when a person is detected in the important object area 8, the respective times are stored in the scream detection time 215 and the person detection time 214 of the storage unit 21, and based on these time information. Thus, an abnormal state in the monitoring area is detected by determining whether or not a person has been detected within a predetermined time before and after detecting the scream. However, the present invention is not limited to this. When a scream is detected, a timer is started to determine whether or not a person has been detected in the important object region 8 before the second abnormality determination time T2 has elapsed. An abnormal state in the monitoring area may be detected. At this time, when the scream is detected again at the time after the time when the scream is detected (assuming that the second abnormality determination time T2 has elapsed), the time when the first scream was detected and the subsequent scream were detected. A process of extending the second abnormality determination time T2 by the difference from the time is performed. Thereby, even if it is a case where a new scream is detected, when a person is detected within the important object area | region 8 within the 2nd abnormality determination time T2 from the time, it can determine with it being in an abnormal state.
Similarly, when a person is detected in the important object area 8, the timer starts counting, and before the timer passes the first abnormality determination time T1, it is determined whether or not a scream is detected. An abnormal state in the monitoring area may be detected. At this time, if a person is detected again at a time after the time when the person is detected in the important object region 8 (assuming that the first abnormality determination time T1 has elapsed), the person at the time of the first person detection and the person after the first person is detected. A process of extending the first abnormality determination time T1 by the difference from the detection time is performed. Thereby, even if it is a case where a person is newly detected, when a scream is detected within the 1st abnormality determination time T1 from the time, it can determine with it being in an abnormal state.

  In each of the above-described embodiments, the second abnormality determination time T2 is a fixed value set in advance by an administrator or the like. However, the second abnormality determination time T2 is not limited to this, and may be changed according to the situation in the monitoring area. For example, in the first embodiment, when a plurality of person images tracked by the tracking unit 222 are close to each other and the shape change rate of these person images is large, the situation may be a dispute. It's likely that the situation is tight. In such a case, a process of extending the second abnormality determination time T2 may be performed. As a result, even if a predetermined second abnormality determination time T2 has elapsed since the detection of scream, if a person image is detected in the important object region 8 within the extended time, it is detected as an abnormal state. Can do.

  In the first embodiment, when a person is detected within the important object region 8 within a predetermined time before and after the scream is detected, the abnormal state is determined. However, the present invention is not limited to this, and after detecting a scream, an abnormal state may be determined by detecting a specific movement of the person image caused by the scream based on the movement trajectory of the tracking unit 222. Here, the specific movement of the person image caused by scream is, for example, the movement of the person image quickly toward the entrance of the room immediately after detecting the scream, or the other person image immediately after detecting the scream. For example, a quick approach. These movements correspond to movements of bandits trying to escape from the room due to screams and movements of bandits trying to harm the discoverers. That is, by detecting a specific movement of the person image caused by such a scream from the movement trajectory information 213, it can be detected that the monitoring area seems to be more abnormal.

The embodiment of the present invention has been described above. In the first embodiment, the abnormality detection device 1 functions as the abnormality detection device of the present invention. The sound collecting device 4 functions as the sound collecting unit of the present invention. Further, the vowel extraction means 224 and the scream determination means 225 of the control unit 22 function as an acoustic analysis unit of the present invention. In addition, the image pickup device 3 and the person image extraction unit 221, the tracking unit 222, and the person detection unit 223 of the control unit 22 function as a person detection unit of the present invention. Moreover, the abnormality determination means 226 of the control unit 22 functions as the abnormality determination unit of the present invention. Further, the imaging device 3 functions as an imaging unit of the present invention. The storage unit 21 functions as the storage unit of the present invention. Further, the person image extraction means 221 of the control unit 22 functions as the person image extraction means of the present invention. The tracking unit 222 of the control unit 22 functions as the tracking unit of the present invention. Further, the person detection means 223 of the control unit 22 functions as the person detection means of the present invention.
On the other hand, in the second embodiment, the infrared sensor and the person detection means 223 of the control unit 22 function as the person detection unit of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Abnormality detection apparatus 2 ... Management apparatus 3 ... Imaging apparatus 4 ... Sound collector 5 ... Security apparatus 6 ... Security center apparatus 7 ... Safe 8 ... Important thing Area 9 ... Wall 21 ... Storage part 22 ... Control part 23 ... Input part 24 ... Output part 25 ... Communication part 211 ... Area information 212 ... Vowel feature quantity 213 ... Movement trajectory information 214... Person detection time 215... Scream detection time 221... Human image extraction means 222... Tracking means 223 ... Person detection means 224. ..Scream determining means 226. Abnormality determining means 100... Monitoring image 110... Trajectory image A, B.

Claims (2)

A sound collection unit for acquiring an acoustic signal generated in a monitoring area where important items are stored;
Analyzing the acoustic signal collected by the sound collecting unit, and detecting a predetermined abnormal sound; and
A person detection unit that detects a person located in an important object area that is included in the monitoring area and is set in the vicinity of the important object storage location;
An abnormality determination unit that determines an abnormal state in the monitoring region based on an abnormal sound detection result of the acoustic analysis unit and a person detection result of the person detection unit;
An abnormality detection device by sound composed of an output unit that outputs a determination result of the abnormality determination unit,
The abnormality detection unit, wherein the person detection unit determines an abnormal state within a predetermined time before and after the acoustic analysis unit determines that the predetermined abnormal sound is detected, and determines that the abnormal state is detected apparatus. The person detection unit
Imaging means for sequentially acquiring an image of an area including the monitoring area;
Human image extraction means for extracting a human image from the image;
Tracking means for tracking the position on the image for each person image;
A storage unit for storing the movement trajectory of the person image from the tracking result of the tracking unit, and storing in advance the position of the important object region on the image;
Human detection means for calculating the time during which the person image stays in the important object region from the movement trajectory of the person image at the predetermined time;
The abnormality detection device according to claim 1, wherein the abnormality determination unit determines that the person image calculated by the person detection unit is in an abnormal state when the retention time of the person image exceeds a predetermined retention time.

Images