KR101520446B1 - Monitoring system for prevention beating and cruel act - Google Patents

Abstract

An embodiment of the present invention relates to a monitoring system for preventing a beating and a cruel act, and aims to prevent the beating and the cruel act, taking place in a monitoring blind spot such as a school, an institute, an army, etc., by using an intelligent voice and image system. The embodiment of the present invention comprises: a sound source processing apparatus for detecting a sound source signal in a monitoring area, and judging whether an abnormal event occurs in the monitoring area based on the detected sound source signal; an image processing apparatus for filming and storing an image of the monitoring area if determined that the abnormal event occurs in the sound source signal, and outputting a first warning signal set in advance and a second warning signal received from the outside; a central monitoring apparatus for receiving the sound source signal detected in the sound source processing apparatus and an image signal filmed in the image processing apparatus if determined the abnormal event occurs in the sound source signal, recognizing an action of an object included in the image signal, and transferring the second warning signal set in advance to the image processing apparatus; and a manager terminal having a real-time detection application, connected to a central monitoring apparatus through a wire and wireless communication net by the real-time detection application, receiving information of the sound source signal, an execution signal, and the second warning signal from the central monitoring apparatus, and transferring a control command signal to the central monitoring apparatus.

Description

[0001] MONITORING SYSTEM FOR PREVENTION BEATING AND CRUEL ACT [0002]

An embodiment of the present invention relates to a surveillance system for beating and harshness prevention for monitoring whether a beating or harsh behavior occurs in a monitored blind spot.

As the number of people continues to increase in modern society, various incidents related to them are increasing. As a result, video surveillance systems have been introduced in indoor and outdoor space security surveillance and traffic security surveillance. Video-based surveillance systems aim to detect / recognize / track suspicious objects and identify their behavior. However, in the conventional image-based surveillance system, the surveillance operator must monitor several monitors at the same time, and there is an inconvenience to observe the monitor for a long time.

In order to solve such difficulties, there is a need to develop an intelligent audio-video surveillance system. The intelligent voice and image based surveillance security system automatically detects when an abnormal event occurs in the surveillance area and informs surveillance operator that the system operation is more effective.

Patent Registration No. 10-1377029 'Step-by-step self-alarming type smart CCTV monitoring device and monitoring method using intelligent hybrid monitoring control module' Open Patent Publication No. 10-2014-0033262 'Indoor Monitoring System and Indoor Monitoring Method'

An embodiment of the present invention provides a surveillance system for preventing beatings and harsh behavior that can prevent beatings and harsh behaviors occurring in a blind spot in schools, academies, military, etc. by using an intelligent voice and image system.

The surveillance system for preventing beating and harshness according to an embodiment of the present invention detects a sound source signal in a surveillance area and determines whether an abnormal event in the surveillance area is generated based on the sensed sound source signal Sound source processing device; An image processing device for photographing and storing an image of the surveillance area and outputting a predetermined first alarm signal and a second alarm signal received from the outside when it is determined that an abnormal event has occurred in the sound source signal; Receiving a sound source signal sensed by the sound source processing apparatus and a video signal photographed by the image processing apparatus when an abnormal event is determined to have occurred in the sound source signal and recognizing an action of the object included in the video signal, A central monitoring device for transmitting a predetermined second alarm signal to the image processing apparatus; And a real-time detection application, and is connected to the central monitoring apparatus and the wired / wireless communication network by execution of the real-time detection application, receives the sound source signal, the video signal, and the second alarm signal information from the central monitoring apparatus, And an administrative terminal for transmitting a control command signal to the device.

Wherein the sound source processing device comprises: a first communication unit for transmitting and receiving data with the central monitoring device; A mode selector for inputting selection information for a first mode or a second mode for sensing the sound source signal; A sound source sensing unit for sensing the sound source signal; A level measurement unit for measuring a sound source level of the sensed sound source signal when the first mode is selected; A class level classifier for classifying a class level of the sound source frame to generate a sound source frame from the sensed sound source signal when the second mode is selected; An event determination unit for determining whether an abnormal event occurs in the monitoring area by applying the sound source level or the rating level to a predetermined event generation algorithm; A first memory unit for storing the sound source signal, the sound source level, the level level, and an event generation algorithm; And a first control unit for controlling operations of the components constituting the sound source processing apparatus.

The class level classifier may include a feature extraction unit for generating a sound source frame from the sound source signal and extracting the sound source frame from the sound source frame; A model selection unit for calculating a likelihood of each of the sound source models and the sound source models and selecting a sound source model having a maximum likelihood value; And a level classifier for classifying the class level of the sound source frame by a hierarchical approach.

Wherein the event determination unit determines that an abnormal event has occurred in the monitoring area when the measured sound source level exceeds a preset reference level; And a second event determiner for determining that an abnormality has occurred in the surveillance region if the ratio of the number of classification events classified as abnormal events to the number of classification levels for a predetermined time period is equal to or greater than a threshold value.

The image processing apparatus includes a second communication unit for transmitting and receiving data with the central monitoring apparatus; A photographing unit photographing an image of the surveillance area; A second memory unit for storing the photographed image and the first alarm signal; And a second controller for controlling an operation of each component constituting the image processing apparatus.

The central monitoring apparatus includes a communication module for transmitting and receiving data to and from the sound source processing apparatus, the image processing apparatus, or the administrator terminal; A behavior recognition module that recognizes a behavior of an object included in the image signal; A data output module for outputting data transmitted and received through the communication module and recognition result data by the behavior recognition module; A storage module that stores data transmitted and received through the communication module and stores a plurality of behavior templates corresponding to an action of an object included in behavior information of the recognized object; And a control module for controlling operations of the components constituting the central monitoring apparatus and transmitting a second alarm signal to the image processing apparatus or the administrator terminal according to the behavior of the recognized object, A data acquisition unit for acquiring joint information and depth information of an object included in the video signal; A behavior template generating unit for generating an input action template from the joint information and the depth information of the acquired object; And an estimator for matching the input action template with the plurality of behavior templates and outputting a behavior most similar to the input action template to estimate a behavior of the object, The positions of the respective joints can be connected by a straight line to form an image.

Wherein the estimating unit calculates the similar scores by matching the input behavior template with the plurality of behavior templates, extracts a behavior template having a maximum score among the plurality of behavior templates, and compares the similarities of the plurality of behavior templates Calculating a minimum acceptable score to compare whether the score of the behavior template with the maximum score is greater than the minimum acceptable score and if the score of the behavior template with the maximum score is greater than the minimum acceptable score, The behavior of the object can be estimated by outputting the meaningful behavior.

The surveillance system for preventing beating and harshness according to an embodiment of the present invention determines whether or not an abnormal event is generated by using a sound source signal detected in a surveillance region, and simultaneously captures and stores an image in the corresponding region, It is possible to effectively prevent the occurrence of beating and harsh actions in the blind spot by transmitting the voice signal and the photographed video signal to the central monitoring device and / or the administrator terminal and realizing an accident response measure in real time.

1 is a schematic view of a surveillance system for preventing beatings and harshness according to an embodiment of the present invention.
2 is a block diagram schematically showing the sound source processing apparatus of FIG.
3 is a block diagram schematically showing the video processing apparatus of FIG.
4 is a block diagram schematically illustrating the central monitoring apparatus of FIG.
FIG. 5 is a diagram illustrating an operation when a surveillance system for preventing beatings and harshness is applied to a defense system according to an embodiment of the present invention.
6 is a diagram showing a behavior template generated by the behavior template generating unit of FIG.

The terms used in this specification will be briefly described and the present invention will be described in detail.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. Also, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term, not on the name of a simple term, but on the entire contents of the present invention.

When an element is referred to as "including" an element throughout the specification, it is to be understood that the element may include other elements as well, without departing from the spirit or scope of the present invention. Also, the terms "part," " module, "and the like described in the specification mean units for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software .

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

1 is a block diagram schematically showing a monitoring system for preventing beating and harsh behavior according to an embodiment of the present invention, FIG. 2 is a block diagram schematically showing the sound source processing apparatus of FIG. 1, FIG. 4 is a block diagram schematically showing the central monitoring apparatus of FIG. 2, and FIG. 5 is a block diagram schematically showing a monitoring apparatus for preventing beatings and harshness according to an embodiment of the present invention. FIG. 6 is a diagram showing a behavior template generated by the behavior template generation unit of FIG. 4. FIG.

Referring to FIG. 1, a surveillance system for preventing beatings and harshness according to an embodiment of the present invention includes a monitoring blind spot of a school, a school, an army or the like, an elevator inside a building or a building, A sound processing unit 11, an image processing unit 15, a central monitoring unit 20, and a central monitoring unit 20. The system includes a sound source processing unit 11, an image processing unit 15, a central monitoring unit 20, And an administrator terminal 30.

In addition, the surveillance system for preventing beatings and harshness can effectively prevent beatings and harsh behaviors by complementing the surveillance and post-war coping functions by using both sound sources and images.

In the meantime, the present invention (i.e., FIGS. 1 and 5) shows military and military quarters as a blind spot, but it is natural that such a system can be applied to other blind spots. 1, the sound source processing device 11 and the image processing device 15 are integrated into a single device, which is denoted by reference numeral 10 (i.e., a data processing device).

The sound source processing device 11 detects a sound source signal in the surveillance area and determines whether or not an abnormal event in the surveillance area is generated based on the sensed sound source signal.

2, the sound source processing apparatus 11 includes a first communication unit 111, a mode selection unit 112, a sound source detection unit 113, a level measurement unit 114, A classification unit 115, an event determination unit 116, a first memory unit 117, and a first control unit 118.

The first communication unit 111 is a device for transmitting / receiving data to / from the central monitoring device 20, and is formed of a wired / wireless communication network. The wired / wireless communication network connects a predetermined communication channel with the central monitoring device 20 based on a protocol stack (for example, a TCP / IP protocol or a CDMA protocol) defined in the communication network, Information for executing the present monitoring system is transmitted and received by using a communication protocol defined in the communication program (for example, Hyper-Text Transfer Protocol (HTTP), Wireless Application Protocol (WAP) / Mobile Explorer) However, the present invention does not limit the type of network, but may be applied to various wired / wireless networks such as Wi-Fi, zigbee, bluetooth, 3G, 4G, LTE and LTE- A communication method may be applied.

The mode selector 112 is a device for inputting selection information for a first mode or a second mode for detecting a sound source signal from a user. The first mode and the second mode are classified according to a method of detecting a sound source signal, and a detailed description of each mode will be described later. The mode selector 112 provides a plurality of keys for operating the tone generator 11, and generates a selection signal corresponding to a user's selection of keys and transmits the generated selection signal to the first controller 118. At this time, the mode selection unit 112 may be an input device such as a keypad, a pointing device such as a touch pad, or a touch screen.

The sound source detecting unit 113 may be a sound source detecting sensor for detecting a sound source signal. Here, the sound source sensor may be a dynamic microphone, a condenser microphone, a ribbon microphone or the like depending on the microphone structure, and a directional microphone, a non-directional microphone, a supergain microphone, or the like may be used depending on the directivity. These sound source sensors may be distributed in one or more numbers according to the surveillance area. The sound source sensing unit 113 receives a sound source signal from the sound source sensor, samples the sound source signal at a predetermined interval, and stores the sampled sound signal in the first memory unit 117.

The level measuring unit 114 converts the sound source signal detected by the sound source detecting unit 113 into a predetermined sound source level when the first mode is selected. The level measuring unit 114 provides the first event determiner 116a with the level of the converted sound source level to determine whether an abnormal event has occurred by the control of the first controller 118. [

When the second mode is selected, the rating level classifier 115 generates a sound source frame from the sound source signal sensed by the sound source sensing unit 113, and classifies the level level of the sound source frame. The class level classifying unit 115 performs a feature extraction of the sound source signal and a sound source class recognition, and a second event determiner (not shown) to determine whether an abnormal event has occurred by the control of the first controller 118 116b.

More specifically, the rating level classifier 115 includes a feature extractor 115a, a model selector 115b, and a level classifier 115c.

The feature extraction unit 115a generates a sound source frame from the sound source signal, and extracts the sound source frame from the sound source frame. That is, the feature extraction unit 115a generates a sound source frame in units of a predetermined time from a sound source signal, and extracts a sound source feature from the generated sound source frame. In this case, the sound source feature can be used as a feature vector of several tens of MFCCs and delta extracted by the Mel-frequency cepstral coefficient (MFCC) feature extraction technique. Generally, the human auditory sense is sensitive to frequency changes in the low frequency range, but insensitive to frequency changes in the high frequency range. The present invention utilizes the MFCC feature, which is a feature vector that can be further extracted in a low frequency region and can be customized through various parameters as compared with a high frequency region. For this purpose, the feature extraction unit 115a pre-processes the time-domain sound source signal to boost high-frequency energy, performs a Fourier transform (FFT) to obtain a spectrum in the frequency domain, The sum of the magnitudes in each band is obtained by mapping the triangular filter bank corresponding to the scale (Mel scale), and the MFCC feature vector is obtained by performing discrete cosine transform on the filter bank output value. In this case, in order to extract the MFCC feature vector, a sound source frame is generated by applying a window to the sound source signal. In the present invention, in order to increase the accuracy of event detection, . Meanwhile, since the MFCC feature extraction method is commonly used in the field of sound source recognition technology, a detailed description thereof will be omitted. MFCC features can be used in combination with MFCC features and Delta features that are time-dependent changes in MFCC. In one embodiment of the invention, a feature vector combining MFCC 20th order and Delta feature 20th order can be used.

The model selection unit 115b calculates a likelihood between a sound source feature extracted by the feature extraction unit 115a and each sound source model and selects a sound source model having a maximum likelihood value. The model selection unit 115b extracts the MFCC feature vector from the sound source frame, and calculates the likelihood of the extracted MFCC feature vector and each sound source model. Then, the sound source model having the highest likelihood value is selected by ML (Maximum Likelihood) criterion. To this end, the first memory unit 117 stores a model of various sound source events generated by training a sound source feature using a Gaussian Mixture Model (GMM).

The level classifying unit 115c classifies the class levels of the sound source frames by a hierarchical approach. The level classification unit 115c classifies the gradation of the sound source frame by a hierarchical approach. Here, the sound source class can be roughly divided into two categories. That is, the sound source class such as screaming, crying, normal conversation, and announcement can be classified into the vocal category because it sounds from a person's neck. In the case of collision, door, step, empty elevator, (Non-Vocal) category. However, in the present invention, it is needless to say that the voice and non-voice events are not limited to the above-described types and can be variously set according to the surveillance region and the system design. According to the present invention, by classifying the classes by a hierarchical approach method, it is possible to improve the misrecognition between the sound source classes in each category of speech / non-speech. In addition, the level classification unit 115c uses optimized sound source characteristics, can be reconfigured according to the system installation environment, can be applied to various places according to the configuration of the level classification unit 115c, . Further, the level classifying unit 115c can be configured by subdividing each of them by event learning according to the installation environment.

The event determination unit 116 determines whether an abnormal event occurs in the surveillance region by applying a sound source level or a rating level to a predetermined event generation algorithm.

For this, the event determination unit 116 includes a first event determination unit 116a and a second event determination unit 116b.

The first event determiner 116a determines that the abnormal event occurs in the surveillance area when the sound source level measured by the level measurer 114 exceeds a preset reference level. That is, when the first mode is selected by the mode selection unit 112, the first event determination unit 116a determines that an abnormal event has occurred in the surveillance region when the size of the sound source level exceeds the reference level.

The second event determination unit 116b determines that the abnormality event has occurred in the surveillance region if the ratio of the number of classification times of the class level for a predetermined time by the class level classifying unit 115 to the number of classified abnormal events is not less than a threshold value do. That is, when the second mode is selected by the mode selection unit 112, the second event determination unit 116b classifies the classified sound source frame into a normal or abnormal event, and then generates an abnormal event Based on the number of times, it is determined whether an abnormal event has occurred or not. At this time, the second event determiner 116b accumulates class level classification results for a certain number of sound source frames, and determines whether an abnormal event has occurred in the surveillance area based on the accumulated classification results. Meanwhile, the abnormal event may be defined as a collision, a scream, a crying, and a normal event may be defined as a dialogue, a footstep, a door open / close, a silence, etc. However, the present invention does not limit the types of abnormal events and normal events .

The first memory unit 117 stores the sound source signal detected by the sound source detection unit 113, the sound source level measured by the level measurement unit 114, the level level classified by the level level classification unit 115, (EEPROM), an SDRAM (Synchronous Dynamic Random Access Memory), or an HDD, and stores the result information determined by the first determination unit or the second determination unit. However, the present invention is not limited to the kind of storage device. Here, the information stored in the first memory unit 117 can be updated. In addition, the first memory unit 117 may be equipped with a control program for controlling power supply to the tone generator 11, as well as an event generation algorithm for determining whether a normal event has occurred.

The first control unit 118 is an apparatus for controlling the operation of each component constituting the tone generator 11. That is, the first control unit 118 detects the sound source signal in the surveillance region, and generates an abnormal event in the surveillance region by the first event determination unit or the second event determination unit based on the sensed sound source signal A mode selection unit 112, a sound source detection unit 113, a level measurement unit 114, a level level classification unit 114, The operation unit 115, the event determination unit 116, and the first memory unit 117. [

When it is determined that an abnormal event has occurred in the sound source signal, the image processing apparatus 15 captures and stores an image of the surveillance region, and outputs a first alarm signal set in advance and a second alarm signal received from the outside.

3, the image processing apparatus 15 includes a second communication unit 151, a photographing unit 152, a second memory unit 153, and a second control unit 154, .

The second communication unit 151 is a device for transmitting and receiving data to and from the central monitoring device 20, and is formed of a wired / wireless communication network. The wired / wireless communication network connects a predetermined communication channel with the central monitoring apparatus 20 based on a protocol stack defined in the communication network, and connects the monitoring monitoring system 20 to the monitoring system 20 using a communication protocol defined in the communication program And transmits and receives information for executing the service. However, the present invention does not limit the type of network, but can be applied to various wired / wireless communication methods such as Wi-Fi, ZigBee, Bluetooth, 3G, 4G, LTE, LTE-A and its equivalent.

The photographing unit 152 is a digital and analog camera for photographing an image of a surveillance region according to a control command of the central monitoring apparatus 20 when it is determined that an abnormal event has occurred in a sound source signal. The surveillance area is photographed. Such a camera is set to photograph the surveillance area only when it is determined that the driving is normally stopped and an abnormal event is generated in the sound source signal. The photographing unit 152 may be integrally installed with the sound source detecting sensors individually or with the sound source detecting sensors incorporated therein. The photographing unit 152 may be distributed in at least one number according to the surveillance area. The photographing unit 152 may include a CCD camera for photographing an image of a surveillance region, or a depth camera for photographing joint information and depth information of the object.

Although not shown, the present image processing apparatus 15 may further include an alarm output unit. The alarm output unit may include a display and a speaker, and may generate an alarm sound or an alarm screen (i.e., a first alarm signal) preset when an abnormal event occurs. The alarm output unit may generate a second alarm signal according to a control command of the central monitoring device 20 when it is determined that an abnormal event is finally generated from the central monitoring device 20. [ For example, the second alarm signal may be a larger and more specific voice signal than the first alarm signal. In other words, the alarm output unit can output a video signal or a voice signal input from a plurality of channels. At this time, the first alarm signal can be displayed as a box in the abnormal event occurrence area detected in the image of the display screen, and the user can be notified of an alarm by highlighting the border of the image of the channel determined to be abnormal or the whole screen of the image . In addition, the second alarm signal is output as an alarm sound, which is a sound source signal input from a plurality of channels through a speaker.

The second memory unit 153 may be a RAM, a ROM, an EEPROM, an SDRAM, or an HDD. The second memory unit 153 may store a first alarm signal and an image of the monitored surveillance area. But is not limited to.

The second control unit 154 is an apparatus for controlling the operation of each component constituting the image processing apparatus 15. That is, when it is determined that an abnormal event is generated in the sound source signal, the second control unit 154 photographs and stores the image of the surveillance region, and outputs a first alarm signal, which is set in advance, and a second alarm signal, The photographing unit 152, the alarm output unit, and the second memory unit 153 to control the operations of the second communication unit 151, the photographing unit 152, the alarm output unit,

The central monitoring apparatus 20 controls the operation of the sound source processing apparatus 11 and the image processing apparatus 15 as well as controls the operation of the sound source processing apparatus 11 and the image processing apparatus 15, In the case of the military, it may be a device for monitoring the operational status of the critical situation room. That is, when the central monitoring apparatus 20 determines that an abnormal event has occurred in the sound source signal, the central monitoring apparatus 20 receives the sound source signal detected by the sound source processing apparatus 11 and the image signal captured by the image processing apparatus 15 , And recognizes the behavior of the object included in the video signal, and then transmits a preset second alarm signal to the image processing apparatus 15.

4, the central monitoring apparatus 20 includes a communication module 210, a behavior recognition module 220, a data output module 230, a storage module 240, Module 250. < / RTI >

The communication module 210 is a device for transmitting and receiving data to and from the sound source processing device 11, the video processing device 15 or the administrator terminal 30, and is formed of a wired / wireless communication network. The wired / wireless communication network connects a predetermined communication channel with the sound source processing apparatus 11, the image processing apparatus 15 or the administrator terminal 30 on the basis of a protocol stack defined in the communication network. The sound source processing apparatus 11, Information for executing the monitoring system using the communication protocol defined in the communication program included in the device 15 or the administrator terminal 30 (that is, information on the sound source signal, the video signal, A control command to the apparatus 11 or the image processing apparatus 15). However, the present invention does not limit the type of network, but can be applied to various wired / wireless communication methods such as Wi-Fi, ZigBee, Bluetooth, 3G, 4G, LTE, LTE-A and its equivalent.

The behavior recognition module 220 is a device for recognizing the behavior of an object included in the image signal received from the image processing apparatus 15 and includes a data acquisition unit 221, a behavior template generation unit 222, and an estimation unit 223 ).

The data acquiring unit 221 acquires joint information and depth information of an object. The data acquisition unit 221 may acquire the joint information and the depth information of the object using the joint tracking information provided by the depth camera. This depth camera can acquire RCB color image, depth image and joint tracking information in real time. In this way, the behavior recognition module 220 can increase the recognition rate of the motion by weighting the joint information of the arms and legs, which are relatively moved in the human body.

The behavior template generating unit 222 receives the joint information and the depth information of the object, and generates the behavior template. In addition, the behavior template generating unit 222 may generate an action template to be stored in the storage module 240, or may input the information of the currently input object to generate an input action template. Here, the behavior template is an image having a certain size of joint information of an object at a predetermined time (t).

Specifically, the behavior template connects the position of each joint with a straight line having a specific scalar value based on the moment point of the object, and forms an image thereof. However, according to the position and the viewpoint of the depth camera of the human joint information, the coordinate values projected to the 2D image are different. Accordingly, the behavior template generating unit 222 generates a behavior template robust against translation and rotation using depth information. That is, the behavior template generation unit 222 performs a transformation operation to generate a behavior template that is robust to transformation and a rotation operation to generate a rotation-resistant behavior template. First, the behavior template generation unit 222 performs a transformation operation to obtain a scale of a human body ratio according to a distance with joint information and depth information of the input object. In order to perform the rotation operation, the behavior template generating unit 222 calculates the normal vector using the center point and the depth information of both shoulder points in the joint information of the object. Here, when the normal vector is calculated, the scale of the depth information is much larger than the position value of the joint. Therefore, the normalization operation is performed through the conversion operation described above and then externally calculated. Then, the behavior template generating unit 222 calculates and corrects the rotation difference between the normal vector of the template registered as an action and the normal vector calculated according to the input information. Here, the rotation calculation of the joint position is performed after the conversion operation. Since the position of the joint is expressed based on the center point, the position in the XY plane is fixed. Therefore, the behavior template generation unit 222 calculates and corrects only the rotation in the ZY plane. The formula of such a rotation operation is expressed by the following equation (1).

 [Equation 1]

Here,? Can be obtained by a normal vector (Nref) of a template registered as an action and a normal vector (Nin) calculated according to the input information. Accordingly, the behavior template generating unit 222 may correct the position (x ') of the joint by rotating only the x-axis coordinate (x) projected on the 2D image. That is, the action template generated by the above operation is expressed as shown in FIG.

The estimating unit 223 calculates the similar scores by matching the input behavior template with a plurality of behavior templates, extracts a behavior template having a maximum score from the plurality of behavior templates, compares the similarities of the plurality of behavior templates Calculating a minimum acceptable score to compare whether the score of the behavior template with the highest score is greater than the minimum acceptable score and if the score of the behavior template with the highest score is greater than the minimum acceptable score, To estimate the behavior of the object.

The estimator 223 compares the input action template generated by the behavior template generator 222 with the behavior template stored in the storage module 240 to estimate the behavior of the object. That is, the estimator 223 determines whether the currently input action template is similar to the action template stored in the storage module 240, and estimates whether the action of the object corresponds to beating or harsh action. For example, if the image of the currently input input action template corresponds to an image corresponding to a beating action and a harsh action, which is a behavior template stored in the storage module 240, the estimating unit 223 determines whether the object of the image signal is beaten It can be assumed that the person is doing something that corresponds to the action.

Specifically, the estimator 223 calculates a similarity score by matching the input behavior template with the behavior template stored in the storage module 240 using a template matching algorithm. In addition, since the estimation unit 223 increases the likelihood of false estimation if the similarity between behaviors is high, the probability of false estimation can be further reduced by using a negative template rejection algorithm. Here, the negative template rejection algorithm compares the similarities of the defined behaviors existing in the behavior template set, calculates a minimum acceptance score for each action, and uses the minimum acceptance score as a reference value when calculating similar scores. That is, when the input behavior template is compared with the stored behavior template and the behavior with the greatest similarity is estimated, the estimation unit 223 does not return the estimated value if the minimum acceptable score is not exceeded.

The data output module 230 is a device for outputting data transmitted and received through the communication module 210 and recognition result data by the behavior recognition module 220. As an apparatus, A liquid crystal display (LCD), an LED (Liquid Emitting Diode), an OLED (Organic LED), and an AMOLED (Active Matrix OLED). For example, the data output module 230 can display the detected surveillance area as a box, and can highlight the border of the image determined as abnormal or the entire screen, thereby externally displaying the surveillance area.

The storage module 240 stores data transmitted and received through the communication module 210 and stores a plurality of behavior templates corresponding to the behavior of the objects included in the behavior information of the object recognized by the behavior recognition module 220 As an apparatus, it may be composed of RAM, ROM, EEPROM, SDRAM or HDD, but the present invention does not limit the kind of storage device. In addition, the storage module 240 stores the information acquired by the data acquisition unit 221. Specifically, the storage module 240 stores the joint information and the depth information of the user acquired by the data acquisition unit 221. In addition, the storage module 240 stores a gesture template. The action template is basic information for recognizing the behavior of the object. Various action templates are stored in the storage module 240 to recognize various actions of the object. In addition, the storage module 240 may group a plurality of behavior templates generated according to a predetermined period into one set, define the behavior templates as a set of behavior templates, and store the set of behavior templates.

The control module 250 controls the operation of the components constituting the central monitoring apparatus 20 and controls the operation of the image processing apparatus 15 according to the behavior of the object recognized by the behavior recognition module 220. [ Or to the administrator terminal 30. [ That is, when it is determined that an abnormal event has occurred in the sound source signal, the control module 250 receives the sound source signal sensed by the sound source processing device 11 and the image signal captured by the image processing device 15, A communication module 210, a behavior recognition module 210, and a communication module 210. The communication module 210 is configured to recognize a behavior of an object included in a video signal and transmit a predetermined second alarm signal to the image processing device 15 when the behavior of the object corresponds to beating and harsh behavior. The module 220, the data output module 230, and the storage module 240. [ In addition, when it is determined that an abnormal event has occurred and / or when it is determined that the behavior of the object corresponds to beating and harsh behavior, the control module 250 transmits the determination result to the administrator terminal 30 .

The administrator terminal 30 is connected to the central monitoring device 20 through a wired / wireless communication network, and is a terminal device of a manager that monitors the overall beating and harshness prevention system as a whole. The manager terminal 30 is provided with a real-time detection application. The manager terminal 30 is connected to the central monitoring apparatus 20 through a wired / wireless communication network by execution of a real-time detection application. The manager terminal 30 receives a sound source signal, And transmits a control command signal to the central monitoring apparatus 20. [ In addition, when an abnormal event is generated in the surveillance area, the administrator terminal 30 can transmit the corresponding information to a police station, a 119 fire station, or a military unit, and instruct the surveillance area to go to the surveillance area. Through this, the beating and harshness in the surveillance area can be prevented from continuing.

The PDA (Personal Digital Cellular) phone, the PCS (Personal Communication Service) phone, the PHS (Personal Handyphone System) phone, and the PDA (Wideband CDMA) phone, a dual band / dual mode phone, a GSM (Global Standard for Mobile) phone, a MBS (Mobile Broadband System) phone, a CDMA- A PDA (Personal Digital Assistant), a hand-held PC (a hand-held PC), a notebook computer, a laptop computer, a WiBro, etc., which may include a communication function such as a DMB (Digital Multimedia Broadcasting) phone, a smart phone, (WiBro) terminal, an MP3 player, an MD player, and an IMT-2000 (International Mobile Telecommunication-2000) terminal that provides an international roaming service and an extended mobile communication service. Held Based wireless communication device, and a terminal equipped with a microprocessor capable of performing a multimedia playback function.

The operation of the surveillance system for preventing beatings and harshness according to an embodiment of the present invention will be described in detail with reference to a military environment.

5, the surveillance system for preventing beatings and harshness according to an embodiment of the present invention includes a CCTV image processing device 15 and a sound source detecting device, which are normally in a human condition, . In this state, when a beating and harsh action sound is generated through the sound source sensing device, it is sensed and the detected sound source is analyzed through comparison with the existing sound source DB to determine whether an abnormal event has occurred. At this time, if it is determined that an abnormal event has occurred, the image of the surveillance area recorded and stored through the image processing device 15 is transmitted to the central monitoring device 20 and is displayed on the accident situation control center screen. The central monitoring device 20 wirelessly encrypts the incident situation information and transmits the information to the administrator terminal 30 of the commander in real time and allows the administrator terminal 30 to determine the incident situation, It is possible to send a countermeasure to the control unit 20, and separately to dispatch it to the field or to instruct various accident prevention and countermeasures.

Therefore, according to the surveillance system for preventing beatings and harshness according to an embodiment of the present invention, it is determined whether an abnormal event is generated using the detected sound source signal in the surveillance region, Capturing and storing the video image, and transmitting the sensed voice signal and the photographed video signal to the central monitoring apparatus 20 and / or the administrator terminal 30 to perform an accident response measure in real time, It is possible to effectively prevent occurrence of harsh behavior.

It is to be understood that the present invention is not limited to the above-described embodiment, and that various modifications and changes may be made by those skilled in the art without departing from the spirit and scope of the present invention as defined by the appended claims. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention.

10: data processing device 11: sound source processing device
15: image processing apparatus 20: central monitoring apparatus
30: Manager terminal 111:
112: Mode selection unit 113: Sound source sensing unit
114: Level measurement unit 115: Level level classification unit
116: Event determination unit 117: First memory unit
118: first control unit 151: second communication unit
152: photographing section 153: second memory section
154: second control section 210: communication module
220: behavior recognition module 230: data output module
240: storage module 250: control module

Claims (7)

A sound source processing unit for detecting a sound source signal in the surveillance area and determining whether an abnormal event in the surveillance area is generated based on the sensed sound source signal;
An image processing device for photographing and storing an image of the surveillance area and outputting a predetermined first alarm signal and a second alarm signal received from the outside when it is determined that an abnormal event has occurred in the sound source signal;
Receiving a sound source signal sensed by the sound source processing apparatus and a video signal photographed by the image processing apparatus when an abnormal event is determined to have occurred in the sound source signal and recognizing an action of the object included in the video signal, A central monitoring device for transmitting a predetermined second alarm signal to the image processing apparatus; And
A central monitoring device connected to the central monitoring device through a wired / wireless communication network and receiving a sound source signal, a video signal, and second alarm signal information from the central monitoring device by execution of the real time sensing application, And transmitting the control command signal to the manager terminal,
The central monitoring device
A communication module for transmitting and receiving data to and from the sound source processing device, the image processing device, or the administrator terminal;
A behavior recognition module that recognizes a behavior of an object included in the image signal;
A data output module for outputting data transmitted and received through the communication module and recognition result data by the behavior recognition module;
A storage module that stores data transmitted and received through the communication module and stores a plurality of behavior templates corresponding to an action of an object included in behavior information of the recognized object; And
And a control module for controlling operations of the components constituting the central monitoring apparatus and transmitting a second alarm signal to the image processing apparatus or the administrator terminal according to the behavior of the recognized object,
The behavior recognition module
A data acquisition unit for acquiring joint information and depth information of the object using a depth image of the object and joint tracking information provided by the depth camera;
A behavior template generating unit for generating an input action template from the joint information and the depth information of the acquired object; And
And an estimator for matching the input behavior template with the plurality of behavior templates and outputting a behavior most similar to the input behavior template to estimate the behavior of the object,
Wherein the action template forms an image by connecting the positions of the respective joints with straight lines around the reference point of the body of the object.
The method according to claim 1,
The sound source processing apparatus
A first communication unit for transmitting and receiving data with the central monitoring device;
A mode selector for inputting selection information for a first mode or a second mode for sensing the sound source signal;
A sound source sensing unit for sensing the sound source signal;
A level measurement unit for measuring a sound source level of the sensed sound source signal when the first mode is selected;
A class level classifier for classifying a class level of the sound source frame to generate a sound source frame from the sensed sound source signal when the second mode is selected;
An event determination unit for determining whether an abnormal event occurs in the monitoring area by applying the sound source level or the rating level to a predetermined event generation algorithm;
A first memory unit for storing the sound source signal, the sound source level, the level level, and an event generation algorithm; And
And a first control unit for controlling the operation of each component constituting the sound source processing apparatus.
delete The method of claim 2,
The event determination unit
A first event determination unit determining that an abnormal event has occurred in the monitoring area when the measured sound source level exceeds a predetermined reference level; And
And a second event determiner for determining that an abnormality has occurred in the surveillance region when the ratio of the number of classification events classified as abnormal events to the number of classification levels for the predetermined time period is greater than or equal to a threshold value. Monitoring system for.
The method according to claim 1,
The image processing apparatus
A second communication unit for transmitting / receiving data to / from the central monitoring device;
A photographing unit photographing an image of the surveillance area;
A second memory unit for storing the photographed image and the first alarm signal; And
And a second control unit for controlling the operation of each component constituting the image processing apparatus.
delete The method according to claim 1,
The estimation unit
Calculating a similarity score by matching the input action template with the plurality of behavior templates, extracting a behavior template having a maximum score from the plurality of behavior templates,
Comparing the similarities of the plurality of behavioral templates to calculate a minimum acceptable score to compare whether the score of the behavioral template with the highest score is greater than a minimum acceptable score,
Wherein the behavior of the object is estimated by outputting a behavior of the behavior template having the maximum score if the score of the behavior template having the maximum score is larger than the minimum allowable score.

Images