JP2014171187A - Monitor camera system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable the preservation of an image by an image format suitable to the kind of the image by determining the kind of the image such as a snap image and a monitor image through the use of the sound of a subject.SOLUTION: A monitor camera system for recording captured image data in a server includes: imaging means for imaging a subject; one or more elements of voice input means capable of receiving the inputs of sound to be uttered by the subject as sound signals; subject state detection means for detecting the state of the subject on the basis of the sound signals input by the sound input means; format change means for changing the format of image data to be recorded in the server; and control means for controlling the operation of the whole device. The control means controls the format change means to change the format of the image data to be recorded in accordance with the detection result of the subject state detection means.

Description

  The present invention relates to a surveillance camera system, and more particularly to a technique suitable for use in a surveillance camera system that transmits video data shot at a remote place via a network such as a LAN.

  In recent years, many network cameras have been proposed in which video captured by a video camera or the like is transmitted via a network such as the Internet or a LAN and can be monitored by a remote client (computer terminal).

  In addition, a surveillance camera system that installs such a network camera in a house for security purposes and transmits a video taken while the user is away to a client such as a security company for recording is also a well-known technique. Furthermore, surveillance cameras are increasingly installed indoors for the purpose of preventing accidents in the home. If a surveillance camera installed in such a house is used, it is possible to take a snap video of a family member or the like, and it is possible to save the data of the snap video in a home personal computer.

  In general, a video that is desired to be saved as a snap video is often a video in which a person who is a subject utters a conversation or the like. Many methods have been proposed, such as a method for detecting the state of a speaker or a subject using the sound of the subject and a method for performing image processing using the detection results (Patent Document 1, Patent Document). 2).

JP 2007-010995 A JP 2007-193824 A

  Video data captured by the monitoring camera is encoded and transmitted in a video format suitable for monitoring, and is recorded by the client. Specifically, encoding is performed in a video format suitable for image quality that can identify a person's face, or encoding in a video format suitable for network capacity for transmission to a client.

  On the other hand, when recording video data taken by the surveillance camera as a snap video, it is necessary to encode and transmit the video data in a video format suitable for viewing and editing, and to record it on the client. Specifically, it is necessary to save the data after encoding with a video format suitable for high image quality or with a still image format that can be printed as a snapshot.

  Therefore, in order to record a snap video using a surveillance camera installed in a house, data of two or more video formats can be recorded simultaneously as video data for different purposes of the video for monitoring and the video for snapping. We have to go. For such a plurality of video recordings, there are problems such as an increase in the capacity of the storage medium of the client, an increase in the load of the recording process of the client, and a limitation of the network line capacity accompanying an increase in transmission data.

  The snap video must be extracted from the recorded video data. Therefore, it is necessary to manually select and edit the video data after all the captured videos are recorded, and very complicated work is required.

As described above, when recording a snap video using a surveillance camera, it is difficult to increase the capacity of a storage medium due to simultaneous recording of the monitoring video and the snap video, or to extract the snap video from all the recorded videos. There was a point.
In view of the above-described problems, an object of the present invention is to determine a video type such as a snap video or a monitoring video using the sound of a subject and to save the video in a video format suitable for the video type. .

  The surveillance camera system of the present invention is a surveillance camera system that records captured video data on a server, and is a photographing means for photographing a subject, and one or more voice input means capable of inputting a sound emitted from the subject as a voice signal. And a subject state detecting means for detecting the state of the subject from the sound signal input by the sound input means, a format changing means for changing the format of the video data recorded in the server, and controlling the operation of the entire apparatus. Control means, and the control means controls the format changing means so as to change the format of the video data to be recorded in accordance with the detection result of the subject state detecting means.

  According to the present invention, the type of video such as snap video or surveillance video is determined using the sound of the subject, and the video is stored in a recording format suitable for the video type. And a snap video can be easily extracted.

It is a block diagram which shows embodiment of this invention and shows the structural example of a surveillance camera system. It is a flowchart which shows an example of the control procedure of the surveillance camera system of 1st Embodiment. It is a flowchart which shows an example of the audio | voice detection procedure of the surveillance camera system of 1st Embodiment. It is a figure explaining the audio | voice spectrum of the surveillance camera system of 1st Embodiment. It is a figure explaining the audio | voice spectrum of the surveillance camera system of 1st Embodiment. It is a flowchart which shows an example of the control procedure of the surveillance camera system of 2nd Embodiment. It is a flowchart which shows an example of the control procedure of the surveillance camera system of 3rd Embodiment.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.
<First Embodiment>
FIG. 1 is a block diagram illustrating a configuration example of a surveillance camera system according to an embodiment of the present invention. This surveillance camera system includes a camera server (device) 100 and a client 120 connected via a network 130.
The imaging unit 101 includes a control circuit (not shown) relating to zoom, focus, exposure, and the like, and controls the optical system according to control data from the lens control unit 108 to capture an optimal video.

  Video data captured by the imaging unit 101 is transmitted to the video processing unit 103, and the encoding unit 104 encodes the video data. Video data encoding methods include methods such as JPEG, H.264, and MPEG4. However, the present embodiment does not depend on the encoding method, and thus detailed description of the encoding method is omitted. .

  In order to enable the imaging unit 101 to rotate to a desired position, the camera platform 102 on which the imaging unit 101 is mounted can be driven by a motor 111. Further, the motor 111 can rotate to a desired position under the control of the pan head control unit 109. Moreover, the microphone 112 for inputting the sound emitted from the subject as a voice signal is provided. The audio data of the subject input from the microphone 112 is transmitted to the audio processing unit 113, and after determining whether or not the subject is suitable for the snap video, the encoding unit 104 encodes the audio data. There is a method such as G.711 for encoding audio data, but since the present embodiment does not depend on the encoding method, detailed description thereof is omitted.

  The video processing unit 103, audio processing unit 113, encoding unit 104, CPU 105, ROM 106, RAM 107, lens control unit 108, and pan head control unit 109 are connected through an internal bus. In addition, a function of transmitting video data and audio data to the network 130 through the communication unit 110 and receiving a request signal for a shooting angle is provided. A computer system including the CPU 105, ROM 106, and RAM 107 described above controls the operation of the entire apparatus. The client 120 can reproduce and record video data and audio data captured by the camera server 100, and functions as a recording server. In addition, it is possible to remotely control the shooting angle of the camera server 100.

  In the present embodiment, subject state detection for detecting the state of a subject such as a conversation or absence from the speech signal of the subject is performed, and the video data encoding method is changed according to the state information indicating the detection result. It is said. A processing procedure (method) for changing the video data encoding method will be described below with reference to the configuration diagram of FIG. 1 and the flowchart of FIG. The processing of the flowchart in FIG. 2 is realized by developing a program stored in the ROM 106 in the RAM 107 and executing it by the CPU 105.

  In S201, the audio signal of the subject input from the microphone 112 is analyzed, and it is determined whether or not the predetermined time is silent. If it is determined in S201 that there is no sound, the encoding unit 104 encodes the video data with MPEG4 format 640 × 480 pixels (VGA size) in S206. The encoded video data is transmitted to the client 120 via the communication unit 110 and the network 130, and the client 120 starts recording the video data.

  In S207, it is determined whether the resident of the house where the camera is installed has returned home. As for the detection method of returning home, there is a method in which door unlocking sound and air conditioning equipment operation sound are registered in advance, and the registered sound and sound input from the microphone 112 are compared. It does not depend on the method.

  Here, regarding the method for comparing the registered sound and the drive sound, a processing procedure (method) for identifying the sound source by comparing the sound signal as a spectrum will be described with reference to the flowchart of FIG. 3 and FIGS. 4 and 5. To do.

In S301, a window function is applied in order to convert the drive sound signal (speech input waveform) of FIG. 4A input from the microphone 112 into data of a predetermined time t.
In S302, the data subjected to the window function is subjected to Fourier transform to obtain an audio spectrum that is power for each frequency.

  In S303, the registered spectrum diagram 4 (b) registered in advance is selected. Registered spectrum FIG. 4B is composed of a spectrum of door unlocking sound and an operating sound spectrum of air conditioning equipment. The spectrum of door unlocking sound is a frequency range Fk and level Lk, and an operating sound of air conditioning equipment. The spectrum of is characteristic of the frequency range Fa and the level La.

In S304, the speech spectrum figure 4 (c) obtained in S302 is compared with the registered spectrum figure 4 (b) to determine whether or not the spectrum of the door unlocking sound has been inputted.
When the spectrum of the door unlocking sound is detected, the door unlocking flag is set to “1” and the process proceeds to the next step.

  In S305, the speech spectrum figure 4 (d) obtained in S302 is compared with the registered spectrum figure 4 (b) to determine whether or not the spectrum of the operating sound of the air conditioning equipment has been input. If the spectrum of the operation sound of the air conditioning equipment is detected, the air conditioning operation flag is set to “1” and the process proceeds to the next step.

In S306, the door unlocking flag and the air conditioning operation flag are confirmed, and if both are “1”, it is determined that the user has returned home.
As described above, the sound signal input from the door unlocking sound or the microphone of the air conditioner or the like is clarified by comparing the frequency signal with the frequency spectrum, and erroneous determination can be prevented.

  Furthermore, by using a band-pass filter that extracts only the frequency band to be compared with the voice spectrum diagram 5 (a), the environmental sound or telephone in which the surveillance camera is installed as shown in the voice spectrum diagram 5 (b). It is possible to eliminate the spectrum such as the ringing tone. As a result, it is possible to improve the accuracy of voice comparison.

When the return home detection method or the like as described above is used, when the resident's return is detected in S207, the transmission of the video data is stopped in S205, and the recording of the video data is stopped in the client 120.
On the other hand, if the sound signal of the subject input from the microphone 112 is analyzed in S201 and it is recognized that the sound has been input, it is detected whether or not the sound input in S202 is a conversation.
Here, as a method example of detecting a conversation, a method using voice recognition will be described. Speech recognition can be performed using general utterance statistics. The features of the learning data in which many utterances are recorded and the voice signal from the microphone 112 are extracted and compared, and the spoken language is estimated.

If the surveillance camera is equipped with such a voice recognition function, it can be determined that the sound is a conversational sound when a child's name or a specific language such as "I'm right" is generated. Further, by using directivity information using a plurality of microphones, the above-described spectrum information, and the like, it is possible to make a highly accurate determination.
In the present embodiment, it does not depend on such a speech recognition algorithm, and it is determined whether or not a conversation is performed using the result of the speech recognition.

If it is determined in S202 that the subject's voice is conversation, the encoding unit 104 encodes the video data with 1920 × 1080 pixels (HD size) in H.264 format in S203. The encoded video data is transmitted to the client 120 via the communication unit 110 and the network 130, and the client 120 starts recording the video data.
If it is confirmed in S204 that the conversation is completed using voice recognition, the transmission of the video data is stopped in S205, and the recording in the client 120 is stopped.

As described above, it is possible to determine the state of the subject such as absence or talking from the audio signal of the subject, and change the video encoding method accordingly to record. As an encoding method, H.264 can be used. The H.264 system, MPEG4 system, JPEG system, etc. are considered.
In the present embodiment, the encoding method and the resolution have been described as examples of changing the recording method. However, the same processing can be performed for changing the compression rate, which is a high-quality factor desired for a snap video. Further, in the present embodiment, the change of the moving image format has been described as an example of changing the encoding method. However, the same processing can be performed for changing the format to a still image format required as a snapshot.

<Second Embodiment>
In the second embodiment, a processing procedure (method) for detecting the number of people who are talking from the audio signal of the subject and changing the format of the video data to be recorded by the client 120 according to the number of people who are talking. This will be described with reference to the configuration diagram of FIG. 1 and the flowchart of FIG. The processing in the flowchart of FIG. 6 is realized by developing a program stored in the ROM 106 in the RAM 107 and executing it by the CPU 105.

  In S601, the encoding unit 104 of the camera server 100 encodes video data in two types of video formats, MPEG4 suitable for monitoring video and H.264 suitable for snap video. The encoded two types of video data are transmitted to the client 120 via the communication unit 110 and the network 130.

In step S602, recording of video data in the MPEG4 format received by the client 120 is started.
In step S603, the process waits until the sound of the subject is input from the microphone 112. When the sound is input, the sound signal is analyzed to determine whether the sound is an uttered sound. If there is a utterance sound in S603, the number of people having a conversation is detected in S604, and the voice processing unit 113 adds the information of the number of people to the voice data.

  Here, a method of using speaker recognition will be described as an example of a method of detecting the number of people having a conversation. Speaker recognition can be performed by using voiceprint information of an individual. A registrant reads out a predetermined text in order to register voiceprint information of an individual in advance. Furthermore, speaker features are extracted by utterance according to a predetermined text, and a typical speech model is learned and constructed. It is possible to identify the speaker by comparing such a speech model with the input speech signal.

If the surveillance camera is equipped with such a speaker recognition function, it is possible to identify the number of speakers and the number of speakers, such as who is speaking in the family and who is visiting a visitor other than the family. . There is also a method that enables highly accurate recognition by combining such speaker recognition and face recognition from video.
In the present embodiment, it does not depend on such an algorithm for speaker recognition, but uses the result of speaker recognition to determine the number of persons and the number of people who are having a conversation.

In step S605, the client 120 confirms information on the number of people added to the audio data. If three or more people are confirmed, the transmission of MPEG4 video data is stopped in S606, and the client 120 stops recording the MPEG4 video data.
In S607, recording of video data in the H.264 format received by the client 120 is started.

  As described above, in the present embodiment, according to the frequency at which a specific speaker is detected by speaker recognition, these pieces of information are added to audio data as a priority for saving as a snap video. The client 120 can change the video format to be recorded according to the priority information.

  In the present embodiment, the number of people who have a conversation is detected as an example of changing the recording method, and the method of recording by changing the video encoding method according to this is described, but in terms of the importance of the conversation, Similar processing can be performed using factors such as conversation time and volume.

<Third Embodiment>
In the third embodiment, a processing procedure (method) for detecting a speaker having a conversation using the above-described speaker recognition and changing a video format and a recording server according to the speaker is shown in FIG. It demonstrates using the flowchart of these. The processing of the flowchart in FIG. 7 is realized by developing a program stored in the ROM 106 in the RAM 107 and executing it by the CPU 105.

In S701, the voice signal of the subject input from the microphone 112 is analyzed, and the speaker is identified by comparing with the voice of the family registered in advance using speaker recognition.
If it is recognized in S701 that the speaker is a family, in S702, the encoding unit 104 of the camera server 100 encodes the video data with 1920 × 1080 pixels in H.264 format, and the home server ( Send to (not shown) and record.

  In S703, the audio signal of the subject input from the microphone 112 is analyzed, and a speaker other than the family is detected using speaker recognition. When a speaker other than the family is detected in S703, in S704, the encoding unit 104 of the camera server 100 encodes the video data with 320 × 240 pixels in the MPEG4 format and is installed in a security company outside the house. Send to server and record.

  In step S <b> 705, the audio signal of the subject input from the microphone 112 is analyzed, and it is detected that the voice is only for the family by using speaker recognition. If the voice of only the family is confirmed in S705, the transmission of the video data to the monitoring server is stopped in S706.

  As described above, it is determined whether or not the video should be monitored by speaker recognition, the video to be monitored is transmitted to the monitoring server, the snap video is transmitted to the home server, and each is recorded in an optimal video format. It becomes possible.

  In the present embodiment, the change of the recording server has been described. However, for the purpose of changing the recording location, it is also possible to change the recording folder and recording medium in accordance with the change of the encoding of the video format. Is possible.

If speaker recognition is used, it is possible to change the encoding method triggered by recognition results such as conversations with family members or conversations with visitors, but not only speaker recognition but also screams, laughter, etc. It is also possible to make use of the voice quality and characteristic sounds such as doors and air conditioning. Such registration of characteristic sounds is registered in advance as a spectrum, and a high-accuracy recognition is possible by comparing the spectrum and characteristics of the input speech.
As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (computer program) that implements the functions of the above-described embodiments is supplied to a system or apparatus via a network or various computer-readable storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program.

DESCRIPTION OF SYMBOLS 100 Camera server 101 Image pick-up part 102 Pan head part 103 Image processing part 104 Encoding part 105 CPU
106 ROM
107 RAM
108 Lens Control Unit 109 Head Control Unit 110 Communication Unit 111 Motor 112 Microphone 113 Audio Processing Unit 120 Client 130 Network

Claims (9)

A surveillance camera system for recording captured video data on a server,
Photographing means for photographing the subject;
One or more voice input means capable of inputting a sound emitted from the subject as a voice signal;
Subject state detection means for detecting the state of the subject from the sound signal input by the sound input means;
Format changing means for changing the format of video data recorded in the server;
Control means for controlling the operation of the entire apparatus,
The surveillance camera system characterized in that the control means controls the format changing means to change the format of video data to be recorded in accordance with the detection result of the subject state detecting means.   The subject state detection means inputs the subject's voice, compares the inputted voice data with a voice model for recognizing a speaker registered in advance, and detects a speaker that emits voice. The surveillance camera system according to claim 1, wherein:   2. The subject state detection unit inputs speech of a subject, compares the input speech data with a speech model for recognizing a language, and detects a spoken language. Surveillance camera system. The subject state detection means detects a video to be monitored and a video to be saved,
The control means controls the format changing means so as to change either a coding method or resolution or compression rate of a moving image and a still image to be recorded according to a detection result of the subject state detecting means. The surveillance camera system according to any one of claims 1 to 3.   The encoding method is H.264. The surveillance camera system according to claim 4, wherein the surveillance camera system is a H.264 system, an MPEG4 system, or a JPEG system, and the resolution includes 1920 × 1080 pixels or 640 × 480 pixels.   The status information according to the detection result of the subject status detection means is transmitted to a server, and the server changes the format of the video data to be recorded according to the status information. The surveillance camera system according to item.   The subject state detection means inputs the sound of the subject and changes the format of the video data to be recorded in the server according to the frequency at which a specific language or a specific speaker is detected from the input sound data. The surveillance camera system according to any one of claims 1 to 5, wherein:   The surveillance camera system according to claim 1, wherein the control unit changes a server that records video data according to a detection result of the subject state detection unit.   The surveillance camera system according to claim 1, wherein the control unit changes a folder in which video data is recorded according to a detection result of the subject state detection unit.

Images