JP2007228459A - Monitoring system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a monitoring system capable of discriminating audible sound in a monitoring zone and its environment, in addition to a video image in the monitoring zone, and issuing warning of abnormality. <P>SOLUTION: In a monitoring system for taking a picture in a monitoring zone by a camera C, a microphone M for gathering sound in the monitoring zone, in which the camera C takes a picture, is provided. A video signal from the camera C is subjected to image processing by an image processing means 15. Phoneme is discriminated by a voice/non-voice recognizing means 11 from the sound gathered by the microphone M, and from the time seres data of the phoneme, whether it is abnormal sound to be warned is determined by the abnormal voice/abnormal sound recognizing means 12a. If the sound is determined as being abnormal, the abnormal voice from the abnormal voice/abnormal sound recognizing means 12a is converted into warning character data, by a character data generating means 14 and the warning character data is superimposed on the image data. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a monitoring system in which a surveillance area is photographed by a surveillance camera and voices in and around the surveillance area are recorded so that the surveillance area can be monitored through a communication cable, the Internet, a mobile phone network, or the like.

  Conventionally, there are monitoring systems that monitor abnormalities in the monitoring area with video and audio. When the sound volume level is higher than a predetermined threshold, such as when there is a noise or a loud sound, the monitoring camera is activated. It is what you do. Referring to FIG. 10, the optical unit 2 is mounted on the camera unit 1, and the camera unit 1 has a camera control circuit unit 4 for processing an image signal from the CCD 3 and an overall control circuit unit 5 for controlling the entire camera. The information from the sound sensor 8 and the optical sensor 9 is input to the overall control circuit unit 5, and the overall control circuit unit 5 is in an abnormal state only when the sound sensor 8 detects a sound volume above a certain level. And the camera control circuit unit 4 is controlled to take a picture. The video imaged in the event of an abnormality is displayed as a video image on a monitor 7 connected to the camera unit 1 via the cable 6, and the status of the monitoring area can be monitored. (For example, see Patent Document 1)

Japanese Patent Laying-Open No. 2003-333583 (paragraphs [0014] to [0017] of the specification, drawing 1)

  However, the conventional monitoring system starts shooting the monitoring area when the volume level of the sound that can be heard in the monitoring area exceeds a predetermined threshold, and the monitor constantly monitors the monitor in order to detect an abnormality. However, the monitoring side only has to determine what kind of content or cause the audio information is from on the monitor image. In other words, it may be an impact sound when a simple object is dropped, or it may be activated by a sound caused by a factor that is not recognized as abnormal, and the image is projected. The cause of the sound generated in or around the area cannot be grasped, and although it is not abnormal, the image is frequently projected with noise, which causes annoyance to the surveillance staff. In addition to video, it was not a monitoring system that could detect abnormal situations by voice.

  The present invention has been made in view of the above-described problems, and provides a monitoring system capable of identifying an audible sound in and around a monitoring area in addition to an image of the monitoring area and warning an abnormality. The purpose is to provide.

The present invention has been made to achieve the above object, and the invention of claim 1 is a monitoring system for recording a captured image of a camera that captures a surveillance area in a monitoring device.
A microphone for picking up sound of the monitoring area photographed by the camera is attached to the camera, and the monitoring device performs image processing on a video signal output from the camera by image processing means, and from the sound picked up by the microphone. The phoneme is recognized by the voice / non-voice recognition means, and after the predetermined voice / non-voice is recognized by the abnormal voice / abnormal sound identification means from the time-series data of the phoneme, it is an abnormal voice or abnormal sound to be warned. If the abnormal voice / abnormal sound from the abnormal voice / abnormal sound identification means is converted to warning character data by the character data output means, the warning character data is converted into the image processing means. It superimposes on the image data output from and outputs, It is the monitoring system characterized by the above-mentioned.

  The invention according to claim 2 is the monitoring system according to claim 1, wherein the monitoring device has a sound output means for notifying the abnormal sound / abnormal sound.

  According to a third aspect of the present invention, in the monitoring system according to the first or second aspect, the monitoring device includes first transmission means for transmitting image data on which the warning character data is superimposed to a network. is there.

  According to a fourth aspect of the present invention, the monitoring device includes a second sending unit that converts the image data superimposed with the warning character data into data that can be received by a mobile phone and sends the data to a network. The monitoring system according to claim 1, 2 or 3.

Further, the invention of claim 5 is a monitoring system for recording a captured image of a camera that captures a monitoring area in a monitoring device.
The camera and the microphone are installed in a predetermined monitoring area, and the monitoring device converts the video signal of the camera and the sound picked up by the microphone into digital data and sends it to the network; A monitoring device that receives the information on the network via the network, the monitoring device acquires the image data from the camera and stores it in the storage means, and the sound data from the microphone is voice / non-voice recognition means Is recognized by the abnormal voice / abnormal sound identification means from the time-series data of the phoneme, and then it is determined whether or not it is an abnormal voice or abnormal sound to be warned. If it is determined that there is an abnormality, the abnormal voice / abnormal sound from the abnormal voice / abnormal sound identification means is converted into warning character data by the character data output means, and the warning character data is output from the storage means. A monitoring system, characterized in that superimposed on the image data.

  According to a sixth aspect of the present invention, in the monitoring system according to the fifth aspect, the monitoring device includes a first transmission unit that transmits the warning character data to the network by superimposing the warning character data on the image data. It is.

  The invention according to claim 7 is the monitoring system according to claim 5 or 6, characterized in that the monitoring device comprises a sound output means for notifying the abnormal sound / abnormal sound.

  In the invention according to claim 8, the monitoring device superimposes the warning character data on the image data, converts the image data into data that can be received by a mobile phone, and transmits the data to a network. The monitoring system according to claim 5, 6 or 7.

  According to a ninth aspect of the present invention, the monitoring device includes time data sending means, superimposes the warning character data on the image data, and based on time information from the time data sending means, The monitoring system according to any one of claims 1 to 8, wherein an abnormality occurrence date and time is superimposed and output corresponding to the position information.

  The speech / non-speech recognition means is not limited to a neural network, but the neural network includes a feature database unit for storing phoneme characteristic parameters, and a teacher signal is transmitted from the feature database unit to the neural network. In addition to improving the discrimination efficiency, it is possible to increase the optimal voice / non-speech discrimination efficiency to be warned according to the location where the surveillance camera is installed. preferable.

  According to a first aspect of the present invention, in a surveillance system for photographing and recording a surveillance area with a camera, a microphone for picking up sound of the surveillance area photographed by the camera is attached to the camera, and a surveillance device is output from the camera. The video signal is image-processed by the image processing means, the phoneme is recognized by the voice / non-speech recognition means from the sound picked up by the microphone, and the predetermined voice is detected by the abnormal voice / abnormal sound identification means from the time series data of the phoneme. / After recognizing non-speech, it is determined whether it is an abnormal sound or abnormal sound to be warned. If it is determined as abnormal, abnormal sound / abnormal sound from the abnormal sound / abnormal sound identifying means is converted into character data. Since the monitoring system is characterized in that it is converted into warning character data by the output means and the warning character data is superimposed on the image data output from the image processing means, The voice recognition means detects non-voice and voice phonemes such as security buzzer sounds and screams, and the voices or non-voices that are recognized as abnormal as the monitoring target by the abnormal voice / abnormal sound identification means for the phonemes output in time series It is possible to determine whether or not the alarm is present, and a warning message informing the abnormality on the monitor screen can be displayed superimposed on the video at the time of occurrence of the abnormality, which makes it easier to recognize the abnormality than a simple video. There is an advantage that the efficiency of the monitoring work can be improved.

  Further, in the invention according to claim 2, since the monitoring device has a sound output means for notifying the abnormal sound / abnormal sound, the warning system displays the warning text on the video. In addition, the warning information notifying the abnormality can be notified by voice, and there is an advantage that the abnormality can be recognized more easily than a simple video.

  According to a third aspect of the present invention, in the monitoring system according to the first or second aspect, the monitoring device includes a first transmission unit that transmits image data on which the warning character data is superimposed to a network. Therefore, when an abnormality due to voice or non-voice is recognized, the abnormality can be distributed to the Internet, and a specific person such as a monitor can monitor the abnormal state of a remote monitoring area. There are advantages.

  According to a fourth aspect of the present invention, the monitoring device includes a second transmission unit that converts the image data superimposed with the warning character data into data that can be received by a mobile phone and transmits the data to a network. Since the monitoring system according to claim 1, 2 or 3, the abnormality in the monitoring area can be distributed to the mobile phone network via the Internet, and a specific person such as a monitor is away There is an advantage that an abnormal state can be confirmed by a mobile phone even at a place.

In the invention of claim 5, in the monitoring system for recording the captured image of the camera that captures the monitoring area in the monitoring device,
The camera and the microphone are installed in a predetermined monitoring area, and the monitoring device converts the video signal of the camera and the sound picked up by the microphone into digital data and sends it to the network; A monitoring device that receives the information on the network via the network, the monitoring device acquires the image data from the camera and stores it in the storage means, and the sound data from the microphone is voice / non-voice recognition means Is recognized by the abnormal voice / abnormal sound identification means from the time-series data of the phoneme, and then it is determined whether or not it is an abnormal voice or abnormal sound to be warned. If it is determined that there is an abnormality, the abnormal voice / abnormal sound from the abnormal voice / abnormal sound identification means is converted into warning character data by the character data output means, and the warning character data is output from the storage means. Since it is a monitoring system characterized by superimposing it on image data, the voice / non-voice recognition means detects non-voice and voice phonemes such as security buzzer sounds and screams, and the phonemes output in this time series are The abnormal voice / abnormal sound identification means determines whether or not it is abnormal voice or abnormal sound. If it is determined to be abnormal, a warning message notifying the abnormality is superimposed on the video and displayed on the monitor screen. There is an advantage that it is easier to recognize the abnormality than the image. In addition, a camera and a microphone are set in each monitoring area, a monitoring device is installed in a distant place, and a plurality of monitoring areas can be monitored at one place, and there is an advantage that the monitoring efficiency is good.

  According to a sixth aspect of the present invention, in the monitoring system according to the fifth aspect of the present invention, the monitoring device includes first transmission means for superimposing the warning character data on the image data and transmitting the image data to a network. Therefore, since the abnormal information of the monitoring area is distributed via the Internet, there is an advantage that the monitoring area can be monitored from a remote place.

  Further, in the invention of claim 7, since the monitoring device is provided with a sound output means for notifying the abnormal sound / abnormal sound, the warning text is displayed in the monitoring system according to claim 5 or 6. In addition to being displayed on the video, it is possible to notify a specific keyword that warns of an abnormal situation by voice, so there is an advantage that it is easier to recognize the abnormality than a simple video.

  In the invention according to claim 8, the monitoring device superimposes the warning character data on the image data, converts the image data into data that can be received by a mobile phone, and transmits the data to a network. Since the monitoring system according to claim 5, 6 or 7, when an abnormal situation due to voice or non-speech is recognized, a warning text is distributed to the Internet together with a video to There is an advantage that a specific person can confirm the abnormality of the monitoring area even if it is a place away.

  According to a ninth aspect of the present invention, the monitoring device includes a time data sending unit, superimposes the warning character data on the image data, and based on time information from the time data sending unit, The monitoring system according to any one of claims 1 to 8, wherein an abnormality occurrence date and time is superimposed and output in correspondence with the position information, so that the date and time when the abnormality occurred together with the image and the monitoring area (location) There is an advantage that can be specified.

  If the speech / non-speech recognition means is a neural network, learn keywords such as “fire, robo, help, car” and phonemes such as security buzzer, which represent warning items to be monitored in the surveillance area. Thus, there is an advantage that it is possible to deal with monitoring items in various monitoring areas and to provide a versatile monitoring system.

  Hereinafter, a monitoring system according to the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of the present invention, FIG. 2 is a functional block diagram of a monitoring device, and FIG. 3 is a block diagram of a monitoring system. FIG. 4 is a diagram of a neural network showing an example of a voice / non-voice recognition processing unit. FIG. 5 is an explanatory diagram for explaining a method of selecting an image when an abnormality occurs, and FIG. 6 is a diagram showing a display screen of a mobile phone. 7 is a block diagram showing another embodiment of the present invention, FIG. 8 is a functional block diagram of the monitoring apparatus, and FIG. 9 is a block diagram of the monitoring system.

(Embodiment 1)
With reference to FIGS. 1-6, the monitoring system of one Embodiment of this invention is demonstrated. As shown in FIG. 1, in the monitoring system of this embodiment, a microphone M and a camera C are connected to a monitoring device 10, and the monitoring device 10 is connected to a client computer PC via the Internet N. The mobile phone terminal T is connected to the mobile phone line network D via the. Mike M picked up sounds around ₁ or in the monitoring zone E, the camera C is photographed in the surveillance area E _1, and the audio and video information is fed to the monitoring device 10, the monitoring person, the client computer monitoring the surveillance zone E ₁ a PC or a mobile phone T can be monitored by the video and audio information. The monitoring device 10 is installed in each of the plurality of monitoring areas E _{1 to} En, and the microphone M and the camera C are connected to the monitoring device 10 via a communication cable. Of course, there may be one or a plurality of monitoring areas monitored by the monitoring device 10, and the microphone M may be integrated with the camera C, or may be arranged adjacent to or separated from the camera C. Good.

  The monitoring device 10 determines whether or not the sound captured by the microphone M is a voice or non-voice related to the monitoring item, and if the sound is a voice or non-voice related to the monitoring item, a situation to be warned (occurrence of an alarm signal) ), The warning character data is output, the image at the time of alarm occurrence is selected, the warning character data is superimposed on this image, and the warning character data is converted into voice and warned. The monitoring device 10 also has an upload function (sending means) for distributing a warning display video in which warning character data is superimposed on the screen to the Internet.

The monitor can monitor the uploaded warning display video on the monitor by the client computer PC via the Internet N, and is uploaded by connecting the Internet N and the mobile phone network D. The warning display video can be displayed on the display screen (display) of the mobile phone terminal T to monitor each of the monitoring areas E _{1 to} En from a remote location. Needless to say, the client computer PC and the mobile phone terminal T cannot confirm the warning display video on the network unless the personal identification can be performed by the password and password or biometric authentication. In addition, it is desirable to install a router or form a firewall so that the network cannot be invaded from the outside.

  Further, the monitoring device 10 will be described with reference to the functional block diagram of FIG. First, the audio signal processing system will be described. The monitoring device 10 is provided with a CPU (Central Processing Control Device), a voice / non-voice recognition means 11 for digitally processing a sound picked up in a monitoring area by a microphone M and recognizing a phoneme, and a voice / non-voice. And a control processing unit 12 to which the phoneme time-series data output by the recognition unit 11 is input. The control processing unit 12 determines how the phoneme time-series data obtained from the speech / non-speech recognition unit 11 is used. Audio for detecting the abnormal content / abnormal sound identifying means 12a and the abnormal voice / abnormal sound identified by the abnormal voice / abnormal sound identifying means 12a as audio data to drive the speaker S It has functions of an abnormal voice data sending means 12b for outputting information, a time data sending means 12d for specifying the date and time when the abnormality occurred, and the abnormal voice data sending means. Abnormal voice output means 13 for outputting abnormal voice data from 2b as a voice signal, and abnormal voice character data sending means for outputting the abnormal voice / abnormal sound identified by the abnormal voice / abnormal sound identification means 12a as a warning character data signal 14. The abnormal sound / abnormal sound identification means 12a may be processed before the control processing unit 12.

  The above phonemes will be described later. In the above voice / non-voice, the voice is a voice of a person who can be understood as a meaning, for example, a drool, a burglar, a fire, a fire, a help, indicating an abnormal state. Non-speech is a voice or sound of an insignificant person, screaming, screaming (baby crying, etc.), security buzzer sound, gachan (glass breaking sound, vehicle crashing sound) , Sound that breaks things). When an abnormality due to voice / non-voice is recognized, the abnormal voice / abnormal sound recognition processing information is notified to the monitor by driving the speaker S, but the character displayed by the voice or warning character data signal driving the speaker S is In the case of voice, I recognized the voice of "Drobbing, robber, fire, fire, help, stop, etc." Please deal with it as soon as possible. In the case of non-speech, etc., "" Security buzzer sound, scream, etc. "was recognized. Please deal with it as soon as possible. Etc.

  Next, the video signal processing system will be described. The monitoring apparatus 10 includes an image processing unit 15 that converts a video signal from the camera C into a digital signal and performs compression processing, and an image that repeatedly stores image data compressed by the image processing unit 15 for each frame or field image. Storage means 16, video output means 17 for expanding the frame or field image stored in image storage means 16 and converting it into an NTSC video signal, and sending the abnormal voice character data to the video signal from video output means 17 The video composition means 18 for superimposing the warning text based on the warning character data signal from the means 14 and outputting it to the monitor W, and the frame or field image A stored in the image storage means 16 from the abnormal voice character data sending means 14 Within a predetermined image size that can be displayed on the mobile phone terminal display shown in FIG. Image size conversion means 19 for converting the image to fit within, and distribution of warning display video in which warning information (alarm occurrence date / time, location of alarm occurrence, warning text) is superimposed on the video to the Internet N and the mobile phone network via the Internet N And sending means 20 for uploading to a web server (not shown).

  As shown in FIG. 6, the display of the mobile phone terminal is composed of display screens A and B. The display screen A is a display area for displaying a monitoring image. This is the area to be displayed. On the display screen B, alarm occurrence date / time, alarm occurrence location, and abnormal voice / abnormal sound recognition processing information are displayed as abnormality information. The screen size of the mobile phone is a maximum of 240 pixels (horizontal) × 320 pixels (vertical) QVGA (Quarter Video Graphics Array) size, and the effective size is 240 pixels (horizontal) × 224 pixels (vertical). is there. The frame or field image (display screen) A is converted by the image size conversion means 19 so as to be within this image size.

  Next, the present embodiment will be described in detail with reference to FIGS. A microphone M and a camera C are connected to the monitoring device 10 via a communication cable, and a sound captured by the microphone M and a video of a monitoring area captured by the camera C are recorded on a VTR, HDD recorder, or DVD recorder RE. Yes. The monitoring device 10 is connected to a monitor W, and an input device such as a keyboard is connected via the serial signal processing unit 21, and the monitoring device 10 can be controlled by this input device.

  The voice / non-speech recognition means 11 includes a voice / non-speech capturing unit 11a that amplifies voice / non-speech sound picked up from or around the monitoring area by the microphone M and converts it into a digital signal by the A / D converter. A short period of preprocessing for speech recognition, such as using a band filter group of about 10 to 30 channels for digitized speech / non-speech, or directly calculating speech spectrum using fast Fourier transform (FFT). A speech analysis unit 11b that performs temporal spectrum analysis to decompose speech / non-speech into phonemes, and a speech recognition processing unit 11c that receives time-series data of phonemes obtained from the speech analysis unit 11b and recognizes speech / non-speech. And a feature database 11d that stores phoneme characteristic parameters and supplies them to the speech recognition processor 11c as teacher signals. In the voice analysis unit 11b, the non-voice is analyzed into a phoneme by performing a short-time spectrum analysis similar to that of the voice. Similar to consonants and vowels, non-speech also has a characteristic pattern.

  The speech recognition processing unit 11c is configured by, for example, a known time delay neural network (TDNN), FIG. 4 shows an outline of the structure diagram, and TDNN recognizes a phoneme which is a unit of sound recognition. Then, speech / non-speech can be recognized from the time-series data of the phoneme. The speech recognition processing unit 11c is classified into each phoneme group, and a TDNN is provided for each group. Time series data of phonemes obtained by performing spectrum analysis on speech / non-speech from the speech analysis unit 11b is input to the input layer I of each TDNN. In the first and second hidden layers H1 and H2, the phonemes are identified and the phonemes identified from the output layer O are output. There are a total of 24 Japanese phonemes (b, d, g, p, t, k, m, n, N, s, sh, h, z, ch, ts, r, w, y, a, i, u, e, o, Q (silence)), phonemes are classified into vowels and consonants, and further divided into voiced and unvoiced consonants. Voiced consonants include burst sounds, friction sounds, flow sounds, and nasal sounds. There are sounds, etc., and unvoiced consonants include plosives, rubbing sounds, frictional sounds, etc., and a TDNN is provided for each group. In the speech recognition processing unit 11c, not only speech but also non-speech is recognized. In the voice analysis unit 11b, non-voices such as a security buzzer and a scream are decomposed and output as phonemes in addition to the voices. Therefore, the voice recognition processing unit 11c recognizes non-voices not only from voices but also from non-voice characteristic patterns. be able to.

  The TDNN in FIG. 4 exemplifies a group for recognizing consonant / b / d / g, and its structure is an input layer (Input Layer) I, first and second hidden layers (Hidden Layer). A multilayer perceptron type composed of H1, H2 and an output layer (Output Layer) O is formed. This type of TDNN is characterized in that it can perform back-propagation (Back-Propagation) learning that reversely propagates an error from the teacher signal based on the teacher signal from the feature database 11d. In the TDNN of FIG. 4, in the input layer I, the horizontal axis represents the time axis and the vertical axis represents the frequency axis. The time axis on the horizontal axis is a frequency pattern of 15 frames obtained by performing spectrum analysis every 10 milliseconds. The frequency axis on the vertical axis divides 0 to 6000 Hz into 16 bands (16 channel spectrum), Time series data (15 frames × 16 dimensions = 240 points), which is a characteristic component of phonemes for 15 frames, is input to the input layer I. In the first hidden layer H1, feature detectors (elements) for detecting local feature components of three frames (30 milliseconds) of the input layer I are arranged. In the second hidden layer H2, feature detectors for detecting more global feature components of 5 frames (50 milliseconds) of the first hidden layer H1 are arranged. The output layer O can recognize the phoneme “b” by using the sum of the output values of the second hidden layer H2 in the time direction as an output. Also, other phoneme groups including vowels can be recognized by the same TDNN. Incidentally, in the speech recognition processing unit 11c, it is not necessary to prepare TDNNs for all phoneme groups, and TDNNs for recognition target phoneme groups may be prepared.

  As described above, in the speech recognition processing unit 11c, the time series data of phonemes is given to the input layer I of the TDNN, and the comparison evaluation is repeatedly performed based on the self-evaluation or the phoneme data input from the feature database unit 11d. A weighting amount between each layer is set, a recognition process of each phoneme (vowel / consonant) is performed, a recognition result of the phoneme is output from the output layer O, and a time-series phoneme (consonant, vowel) from the output layer O ) Is output, and voice / non-voice can be recognized later. The voice recognition processing unit 11c uses known software such as TDNN, a device, or the like.

  The control processing unit 12 has a function of performing arithmetic processing by the CPU. The control processing unit 12 will be described with reference to the functional block diagram of FIG. The voice / non-voice information output from the voice recognition processing unit 11c is input to the abnormal voice / abnormal sound identification unit 12a. The abnormal voice / abnormal sound identification means 12a recognizes the voice / non-voice input information, and can understand the voice / non-voice to be warned in each monitoring area, and can understand the abnormal voice / abnormal sound information (abnormal voice: Drobo, Recognize as abnormal sound information (security buzzer sound, scream, etc.) such as fire, help, stop, etc., and send out voice identification information and alarm signal (abnormal condition detection signal) for voice synthesis and text conversion.

  On the other hand, the time data sending means 12d receives the date / time information of the year, month, day, and time from the time generating section 12c, and from the time data sending means 12d by an alarm signal based on the abnormal voice information from the abnormal voice identifying means 12a. Date and time information is output. As will be described later, this date and time information is output as a warning display video by being superimposed or added to a video signal in a predetermined output format together with an alarm occurrence location and audio identification information (warning information). This warning display image is displayed on the monitor W and on the screen of the client computer PC or the mobile phone terminal T via the Internet N.

  The voice identification information from the abnormal voice data sending means 12b is input to the voice synthesis processing section 13a, an analog voice is created by the existing voice synthesis means, and the synthesized analog signal is amplified by the voice output section 13b. Output to the speaker S. Further, the voice identification information from the abnormal voice data sending means 12b is input to the character data output unit 14, and the character data output unit 14 converts the voice identification information into character data based on the voice identification information. The data is output to the server unit 19 and the network processing unit 20, respectively.

  Next, the video processing system will be described. The image processing means 15 includes a video capturing unit 15a and a video processing unit 15b. In the video capturing unit 15a, an analog such as NTSC which is a video of the monitoring area captured by the camera C is used. The video processing unit 15b decodes the NTSC analog video signal, A / D-converts and compresses it into digital compressed data, and stores it in the image memory (image recording means) 16. Send it out.

  The image memory 16 includes a plurality of frame or field memories, and digital compressed data that is a monitoring video is repeatedly overwritten and recorded on the plurality of frame or field memories. The video output processing unit 17 reads the compressed video data recorded in the image memory 16, decompresses it, encodes it into NTSC, sends it to the character signal synthesis unit (video synthesis unit) 18, and sends it to the character signal synthesis unit (video synthesis unit). In 18, a warning text or the like is superimposed on the video signal and output to the monitor W. Further, in the character signal synthesis unit 18, warning information indicating an abnormal state of the monitoring area, information on occurrence date and time, information necessary for monitoring such as the monitoring area (location) is superimposed on the video signal as character information and output to the monitor W. .

In the image memory 16, as shown in FIG. 5, at the time t ₁ when the abnormal sound / abnormal sound identification unit 12 a starts the abnormality recognition process (when the alarm signal is generated) or when the abnormality recognition process is completed, the control processing unit 12, the overwriting process to the image memory 16 is stopped, and the recorded image F ₀ at the time when an abnormality occurs retroactive to the continuous recorded image recognition processing time t ₀ is processed by the video output processing unit 17, After passing through the signal synthesizer 18, it is output to the monitor W as a video or an image when an abnormality occurs. Further, the control processing unit 12 can output a plurality of images before and after the recorded image F ₀ when the occurrence of the abnormality is detected, by the mobile phone terminal T or the client computer PC of the monitoring device 10.

  The image conversion server unit (image size conversion means) 19 converts the video signal into an image format (JPEG or the like) that can be displayed on the screen of the mobile phone terminal T to a size that can be displayed in the image size (QVGA size or the like). Information necessary for monitoring, such as status information, time of occurrence, and monitoring area (location), is added as character information to a predetermined format and sent to the network processing unit (network sending means) 20.

  The network processing unit (transmission means) 20 uses the alarm information, the date and time of occurrence, the information necessary for monitoring, such as the monitoring area (location), as character information, and the image at the time of occurrence of the alarm. It has a function of delivering (uploading) to T as an output format that can be displayed on a WWW browser via the Internet N.

  The monitoring device 10 is connected to the Internet N via the router R, the client computer PC is connected to the Internet N via the router R, and the mobile phone network D to which the mobile phone terminal T is connected is the router. The client computer PC and the mobile phone terminal T are connected to the Internet N via R, and can monitor even if they are away from the monitoring area. The router has a function to analyze packet data that flows on the network from the protocol, determine which route should be used and transfer it, and also has a function to set the data transmission route. It has the function of not passing any data and is effective in enhancing the security of the system.

(Embodiment 2)
Next, another embodiment of the present invention will be described with reference to FIGS. FIG. 7 is a block diagram showing another embodiment of the present invention, FIG. 8 is a functional block diagram of the other embodiment, and FIG. 9 is a block diagram of a monitoring system of the other embodiment.

  As shown in FIG. 7, this embodiment differs from the above embodiment in that a camera C and a microphone M are installed in each monitoring area, and the camera C and the microphone M receive sound and video captured in the monitoring area on the Internet. Sending means 22 for sending to N is provided. The sending means 22 has a function of converting the video and audio of each monitoring area into a form that can be searched and viewed with a WWW browser and automatically uploading it to a Web server. The camera C is a so-called network camera. It is. In this embodiment, the monitoring device 10 does not need to be installed in the monitoring area, but is installed in a place away via the Internet N, which is different from the above embodiment. The mobile phone terminal T is the same as that in the above embodiment. The description of the same parts as those in the above embodiment is omitted as much as possible.

  The voice / non-voice recognition unit 11 acquires the voice / non-speech picked up by the microphone M as a packet signal received from the network sending unit 20, and the contents of the voice / non-speech by the packet signal are as follows. When this voice or non-speech is recognized as an abnormal voice, the abnormal voice / abnormal sound is input to the abnormal voice / abnormal sound identifying means 12a of the control processing unit 12 and described in the above embodiment. As described above, the abnormal voice / abnormal sound is converted into voice or warning character information and notified by the speaker S, or a warning display video as character information superimposed on the monitoring video is displayed on the monitor W. Similarly, a warning display image can be displayed on the mobile phone terminal T. Further, the image acquisition means 15a 'acquires a digitized image signal, records the image signal in the image storage means 16, and is superposed with warning character information through the same image processing as in the above embodiment.

  In this embodiment, a pair of cameras and microphones are installed in each surveillance area, and the client computer PC or mobile phone terminal via the Internet N only when the sound and image from these cameras detect abnormal sound or sound. It is possible to monitor with warning display character information and images at T, and it is not necessary to install a monitoring device in each monitoring area as compared with the above embodiment, and many monitoring areas can be monitored with a small number of people.

  As described above, the present invention recognizes the abnormalities occurring in the monitoring area by using the neural network to recognize the extracted features of voice and non-voice generated in the field, and the recognition result indicates abnormalities. This is used as a trigger to warn the monitor of abnormality. Moreover, the recognition result of the neural network based on the extracted features is converted into character information, added to the monitoring image at the time of occurrence of the abnormality, and distributed. The monitoring image recorded at the time of occurrence of the abnormal state recorded by the monitoring image is sent to the monitoring monitor of a dedicated supervisor, and at the same time, the client computer PC or mobile phone terminal located away from the monitoring location is connected to the Internet or a mobile phone network. The monitoring image can be distributed via the network.

  In the present invention, when monitoring a monitoring area, the content of abnormal sound in an abnormal state differs depending on whether it is a fire monitoring, an intruder monitoring, or a monitoring system for nursing purposes. Therefore, the voice / non-voice recognition means must recognize different abnormal voices or abnormal sounds, and by providing the voice / non-voice recognition means with a neural network, a versatile monitoring system is provided by learning. can do.

  As an example of use of the present invention, even if the monitor is away from the monitoring area, an abnormal sound is automatically displayed as a warning character in addition to the monitoring video or image to automatically identify the abnormal situation. It can be used as a monitoring system that can be recognized and monitored, and can be used as a monitoring system corresponding to various abnormal situations.

It is a block diagram which shows one Embodiment of the monitoring system of this invention. It is a functional block diagram of the monitoring apparatus of this embodiment. It is a block diagram of the monitoring system of this embodiment. It is a figure of the neural network which shows an example of the phoneme recognition process part in this embodiment. It is explanatory drawing for demonstrating the method to select the image at the time of abnormality generation in this embodiment. It is a figure which shows the display screen of the mobile telephone in this embodiment. It is a block diagram which shows other embodiment of this invention. It is a functional block diagram of the monitoring apparatus of embodiment of FIG. It is a block diagram of the monitoring system of the embodiment of FIG. It is a block diagram which shows the conventional monitoring system.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Monitoring apparatus 11 Voice / non-voice recognition means 11a Voice / non-voice capture part 11b Voice analysis part 11c Voice recognition processing part 11d Feature database part 12 Control processing part 12a Abnormal voice / abnormal sound identification means 12b Abnormal voice data transmission means 12c Time Generation unit 12d Time data transmission means 13 Abnormal voice output means 13a Speech synthesis processing section 13b Audio output section 14 Abnormal voice character data transmission means 15 Image processing means 15a 'Image acquisition means 15a Video capture section 15b Video processing section 16 Image memory (image Recording means)
17 Video output processing unit (video output means)
18 Character signal synthesis unit (video synthesis means)
19 Image conversion server unit (image size conversion means)
20, 22 Network processing unit (transmission means)
21 serial signal processing unit C camera D mobile phone network E _{1 to} En monitoring area M microphone N Internet PC client computer R router RE VTR, DVD recorder, HDD recorder S speaker T mobile phone terminal W monitor

Claims (9)

In a surveillance system that records a photographed image of a camera that photographs a surveillance area in a surveillance device,
A microphone for picking up sound of the monitoring area photographed by the camera is attached to the camera, and the monitoring device performs image processing on a video signal output from the camera by image processing means, and from the sound picked up by the microphone. The phoneme is recognized by the voice / non-voice recognition means, and after the predetermined voice / non-voice is recognized by the abnormal voice / abnormal sound identification means from the time-series data of the phoneme, it is an abnormal voice or abnormal sound to be warned. If the abnormal voice / abnormal sound from the abnormal voice / abnormal sound identification means is converted to warning character data by the character data output means, the warning character data is converted into the image processing means. A monitoring system for superimposing and outputting the image data output from the image data.   The monitoring system according to claim 1, wherein the monitoring device includes a sound output unit that notifies the abnormal sound / abnormal sound.   3. The monitoring system according to claim 1, wherein the monitoring device includes first sending means for sending image data on which the warning character data is superimposed to a network.   The said monitoring apparatus is provided with the 2nd sending means which converts the image data which superimposed the said alert character data into the data which a mobile telephone can receive, and sends to a network. The monitoring system described. In a surveillance system that records a photographed image of a camera that photographs a surveillance area in a surveillance device,
The camera and the microphone are installed in a predetermined monitoring area, and the monitoring device converts the video signal of the camera and the sound picked up by the microphone into digital data and sends it to the network; A monitoring device that receives the information on the network via the network, the monitoring device acquires the image data from the camera and stores it in the storage means, and the sound data from the microphone is voice / non-voice recognition means Is recognized by the abnormal voice / abnormal sound identification means from the time-series data of the phoneme, and then it is determined whether or not it is an abnormal voice or abnormal sound to be warned. If it is determined that there is an abnormality, the abnormal voice / abnormal sound from the abnormal voice / abnormal sound identification means is converted into warning character data by the character data output means, and the warning character data is output from the storage means. Monitoring system, characterized in that superimposed on the image data.   The monitoring system according to claim 5, wherein the monitoring device includes a first sending unit that sends the warning character data to the network by superimposing the warning character data on the image data.   The monitoring system according to claim 5, wherein the monitoring device includes a sound output unit that notifies the abnormal sound / abnormal sound.   The monitoring device includes second sending means for superimposing the warning character data on the image data, converting the image data into data receivable by a mobile phone, and sending the data to a network. The monitoring system according to 5, 6 or 7.   The monitoring device includes time data sending means, superimposes the warning character data on the image data, and based on the time information from the time data sending means, an abnormality occurrence date and time corresponding to the position information of the monitoring area The monitoring system according to any one of claims 1 to 8, wherein the monitoring system is superimposed and output.

Images