JP2011234033A - Monitoring camera and monitor system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the compressibility of the entire image while sufficiently securing the recognition performance of an object to be monitored.SOLUTION: A monitor system comprises a monitoring camera (1) and a monitoring terminal (2). The monitoring camera (1) comprises: an imaging unit (11) for imaging the object to be monitored; an image coding unit (12) which codes the image outputted from the imaging unit (11) by irreversible compression; an image decoding unit (13) which decodes the image coded by the image coding unit (12); a feature amount extraction unit (14) which receives a first image outputted from the imaging unit (11) and a second image outputted from the image decoding unit (13) and extracts the feature amount of each image; a feature variation detection unit (15) which detects a varied portion between a feature amount extracted from the first image and that extracted from the second image; and a coding control unit (16) which changes the control parameter of the image coding unit (12) on the basis of the varied portion of the detected feature amount.

Description

  The present invention relates to a surveillance camera and a surveillance system in which a surveillance camera and a terminal are connected via a network or the like, and more particularly to a surveillance system that performs surveillance automation using a person recognition technology or the like.

  2. Description of the Related Art In recent years, surveillance systems have been developed that efficiently transmit video from surveillance cameras over an IP network or the like and receive them with a surveillance terminal to efficiently monitor a wide area. FIG. 6 shows the configuration of a conventional monitoring system. The monitoring system includes a monitoring camera 100 and a monitoring terminal 200. The surveillance camera 100 includes an imaging unit 104 including a lens 101, an image sensor 102, and camera signal processing 103, a video encoding unit 105, and a video transmission unit 106. The monitoring terminal 200 includes a video receiving unit 201, a video recording unit 202, a video decoding unit 203, and a video display unit 612.

  The lens 101 forms an optical image to be monitored on the image sensor 102. The image sensor 102 converts the optical image into an electrical signal. The camera signal processing unit 103 converts the signal of the image sensor 102 into luminance and color digital data. The video encoding unit 105 compresses and encodes the video. Network bandwidth is limited, and lossy compression is employed to increase the compression rate. The video transmission unit 106 transmits the encoded video to the monitoring terminal 200. For example, when transmitting on an IP network, the video transmission unit 106 performs transmission control such as IP header processing and retransmission / congestion control.

  In the monitoring terminal 200, the video reception unit 201 receives the encoded video from the monitoring camera 100. The video recording unit 202 records the received video. The video decoding unit 203 reproduces video data by decoding the encoded video. However, since the video transmitted from the monitoring camera 100 is irreversibly compressed, on the monitoring terminal 200 side, the video acquired by the monitoring camera 100 is not completely the same, and deterioration always occurs. The video display unit 204 displays the video of the monitoring camera 100.

  An important requirement in video surveillance is that important information is not missing, not important beauty in consumer cameras. For example, a person's face is very important for identifying a suspicious person. In addition, the network bandwidth is usually limited, and since recording is always performed, video compression with the highest possible compression rate is required. Therefore, the surveillance camera 100 is provided with a region of interest detection unit 107 that detects a human image region as a region of interest from the video, and the video encoding unit 105 lowers the compression rate of the region of interest relative to other regions. Therefore, loss of important information is prevented while achieving high image quality (see, for example, Patent Document 1).

JP 2001-145101 A

  According to the prior art, the image quality of the region of interest can be improved in a relative relationship with other regions, but it is difficult to control with an absolute measure of how much to improve. Therefore, when the network bandwidth becomes more severe and even the image quality of the region of interest has to be reduced to some extent, it may not be possible to determine how much it should be reduced. Therefore, it is difficult to logically and quantitatively improve cost performance by reducing the network bandwidth and the capacity of the video recording unit while ensuring sufficient performance.

  In view of such a problem, an object of the present invention is to increase the compression ratio of the entire video while sufficiently ensuring the recognition performance of the monitoring target in the monitoring system.

  In order to solve the above problems, the present invention has taken the following measures. That is, the monitoring camera decodes the video encoded by the video encoding unit, the video encoding unit that encodes the video output from the imaging unit by irreversible compression, and the video encoding unit. A video decoding unit that receives the first video output from the imaging unit and the second video output from the video decoding unit, and extracts a feature amount in each video; A feature change detecting unit for detecting a change between the feature quantity extracted from the first video and the feature quantity extracted from the second video; and the video based on the detected change in the feature quantity. An encoding control unit that changes the control parameter of the encoding unit is provided. The surveillance system includes the surveillance camera and a surveillance terminal connected to the surveillance camera through a communication line.

  According to this, a control parameter related to video coding is changed based on a change in feature amount necessary for video identification / recognition, which is the final purpose of video monitoring, that is, a degree of deterioration. Therefore, the entire video can be appropriately compressed and encoded with an absolute scale.

  Preferably, the monitoring terminal includes a monitoring target recognition unit that recognizes the monitoring target from a video received from the monitoring camera, and a recognition difficulty level evaluation unit that evaluates a recognition difficulty level of the video received from the monitoring camera, The recognition difficulty level is transmitted to the monitoring camera, and the encoding control unit in the monitoring camera changes the control parameter of the video encoding unit in consideration of the recognition difficulty level transmitted from the monitoring camera. According to this, the control parameter relating to video encoding is adaptively changed based on the recognition difficulty level of the monitoring target fed back from the monitoring terminal.

  Preferably, the feature quantity extraction unit outputs the extracted feature quantity as a feature vector, and the feature change detection unit relates to the feature vector related to the first video and the second video. A difference vector from the feature vector is calculated, and each element of the difference vector is weighted and output as a feature difference vector. Thus, by weighting each element of the difference vector, it is possible to detect a change in the feature amount that accurately reflects the recognition performance.

  More preferably, the monitoring terminal includes a monitoring target recognition unit that recognizes the monitoring target from video received from the monitoring camera, and transmits an algorithm related to the recognition to the monitoring camera. The feature change detecting unit weights each element of the feature difference vector in consideration of the recognition algorithm transmitted from the surveillance camera. According to this, the feature weighting in the monitoring camera is adaptively controlled based on the recognition algorithm fed back from the monitoring terminal.

  According to the present invention, it is possible to increase the compression rate of the entire video while sufficiently ensuring the recognition performance of the monitoring target in the monitoring system. This makes it possible to improve the cost performance of the entire monitoring system by efficiently using the network bandwidth and the video recording unit.

FIG. 1 is a configuration diagram of a monitoring system according to an embodiment of the present invention. FIG. 2 is a configuration diagram of the feature extraction unit. FIG. 3 is a configuration diagram of the error detection unit. FIG. 4 is a configuration diagram of the encoding control unit. FIG. 5 is a configuration diagram of a main part of the monitoring terminal. FIG. 6 is a configuration diagram of a conventional monitoring system.

  FIG. 1 shows a configuration of a monitoring system according to an embodiment of the present invention. The monitoring system according to this embodiment includes a monitoring camera 1 and a monitoring terminal 2. The surveillance camera 1 includes an imaging unit 11 including a lens 111, an image sensor 112, and a camera signal processing unit 113, a video encoding unit 12, a video decoding unit 13, a feature amount extraction unit 14, a feature change detection unit 15, and an encoding. A control unit 16, a video transmission unit 17, a recognition difficulty level reception unit 18, and a recognition algorithm reception unit 19 are provided. The monitoring terminal 2 includes a video receiving unit 21, a video recording unit 22, a video decoding unit 23, a video display unit 24, a monitoring target recognition unit 25, a recognition difficulty level evaluation unit 26, a recognition difficulty level transmission unit 27, and a recognition algorithm transmission unit 28. It has.

  The lens 111 forms an optical image to be monitored on the image sensor 112. The image sensor 112 converts the optical image into an electrical signal. The camera signal processing unit 113 converts the signal of the image sensor 112 into luminance and color digital data. The video encoding unit 12 compresses and encodes the video. The video decoding unit 13 decodes the encoded video. The feature amount extraction unit 14 extracts features to be monitored necessary for recognition processing performed by the monitoring terminal 2 from the video. The feature change detection unit 15 detects a change between the feature amount of the video output from the imaging unit 11 and the feature amount of the video output from the video decoding unit 13. That is, the feature change detection unit 15 detects a change in feature amount before and after encoding. The encoding control unit 16 changes the control parameter of the video encoding unit 12 based on the detection result of the feature change detection unit 15. The video transmission unit 17 transmits the video encoded by the video encoding unit 12 to the monitoring terminal 2.

  H. In the video compression system such as H.264 and MPEG, since the inter-frame prediction encoding is performed with reference to the already decoded image, the video encoding unit 12 and the video decoding unit 13 are actually integrated. For convenience, the video encoding unit 12 and the video decoding unit 13 are displayed separately. Therefore, even if the video encoding unit 12 and the video decoding unit 13 exist separately, there is no particular additional cost.

  The video receiver 21 receives the encoded video from the surveillance camera 1. The video recording unit 22 records the received video. The video decoding unit 23 decodes the received video or the video recorded in the video recording unit 22. The video display unit 24 displays the decoded video. The monitoring object recognition unit 25 recognizes the monitoring object from the video, for example, personal recognition of a person. The recognition difficulty level evaluation unit 26 evaluates the difficulty level of the recognition process in the monitoring target recognition unit 25. The recognition difficulty level transmission unit 27 transmits the recognition difficulty level output from the recognition difficulty level evaluation unit 26 to the monitoring camera 1. The recognition algorithm transmission unit 28 transmits an algorithm for recognition processing performed by the monitoring terminal 2 to the monitoring camera 1.

  The recognition difficulty level receiving unit 18 receives the recognition difficulty level. The encoding control unit 16 receives the recognition difficulty level. When the recognition difficulty level is high, the encoding control unit 16 controls the video encoding unit 12 so that the feature amount change detected by the feature change detection unit 15 becomes smaller. The recognition algorithm receiving unit 19 receives a recognition algorithm. The feature change detection unit 15 receives the recognition algorithm, and changes a parameter related to detection of a feature amount change according to the recognition algorithm.

  Next, each part of the surveillance camera 1 will be described in detail. FIG. 2 shows the configuration of the feature quantity extraction unit 14. The feature quantity extraction unit 14 includes a selector 141, a face detection unit 142, and a face feature extraction unit 143. The face detection unit 143 includes a plurality of specific face direction detection units 144 and a coupler 145. The specific face direction detection unit 144 includes a plurality of weak classifiers 146. The face feature extraction unit 143 includes a face part position detection unit 147 and a face part shape detection unit 148. The face part shape detection unit 148 includes a plurality of Gabor filters 149.

  The selector 141 selects one of the videos from the imaging unit 11 and the video decoding unit 13. The face detection unit 142 detects a human face from the selected video. In the face detection unit 142, a so-called boosting learning device is configured in which weak discriminators 146 for identifying the facial appearance based on the shading pattern of the video are combined in cascade. Further, the face detection unit 142 includes a specific face direction detection unit 144 that performs individual learning for each specific face direction in order to distinguish the specific face direction and to optimize the direction.

  The face-likeness determination by each specific face direction detection unit 144 is combined by a combiner 145, and finally it is determined whether or not it is a human face. In the area determined to be a human face by the face detection unit 142, the human face feature is extracted by the face feature extraction unit 143. First, the face part position detection unit 147 detects the positions of eyes, nose, mouth, and the like, which are important face parts characterizing the human face. Here, each facial part position is searched using a weak discriminator 146 that discriminates from the shading pattern similar to that of the face detection unit 142 and that is specially learned for the facial part. The face component detected by the face component position detection unit 147 is extracted by the face component shape detection unit 148. The face part shape detection unit 148 includes a plurality of Gabor filters 149 for detecting edge information including inclination that is shape information of the face part. The face part position information and the face part shape information are sent to the feature change detection unit 15 at the subsequent stage.

  FIG. 3 shows the configuration of the feature change detector 15. Since there are a plurality of feature amounts, these are collected as a feature vector 151. The feature change detection unit 15 includes a feature vector memory 152, a feature difference vector calculation unit 153, a feature difference vector space conversion unit 154, and a feature difference vector space determination unit 155. The feature vector memory 152 includes a memory 152 a that stores a feature vector related to a video output from the imaging unit 11 and a memory 152 b that stores a feature vector related to a video output from the video decoding unit 13. The feature difference vector calculation unit 153 calculates a feature difference vector that is an error between the feature vector related to the video output from the imaging unit 11 and the feature vector related to the video output from the video decoding unit 13. The feature difference vector space conversion unit 154 converts a feature difference vector space, that is, a difference value between individual feature amounts, according to a predetermined rule. This means that different weighting is performed for each feature amount. This is because the weighting of the feature amount varies depending on the recognition algorithm implemented by the monitoring terminal 2. In order to perform this process accurately, the feature difference vector space determination unit 155 determines a conversion rule based on the recognition algorithm received from the recognition algorithm reception unit 19.

  FIG. 4 shows the configuration of the encoding control unit 16. The encoding control unit 16 includes a feature difference vector magnitude evaluation value calculation unit 161, a feature difference vector threshold comparison unit 162, a feature difference vector threshold determination unit 163, a quantization parameter control unit 164, and a facial part position difference evaluation value calculation unit 165. The prediction mode control unit 166 is configured.

  The feature difference vector magnitude evaluation value calculation unit 161 calculates a comprehensive index indicating the size of the feature difference vector output from the feature change detection unit 15. For example, the absolute value of the vector. As a result, the difference in feature amount is collected into one evaluation value. The feature difference vector threshold value comparison unit 162 compares the threshold value of the feature difference vector that should be in order to ensure desired recognition performance in the monitoring terminal 2 and the evaluation index. The feature difference vector threshold value determination unit 163 determines the threshold value of the feature difference vector.

  In order to ensure the desired recognition performance more accurately, the threshold value of the feature difference vector is updated based on the recognition difficulty level sent from the recognition difficulty level receiving unit 18. For example, when the recognition difficulty level is high, processing for lowering the threshold value is performed. The quantization parameter control unit 164 controls the quantization parameter of the video encoding unit 12 based on the output of the feature difference vector threshold comparison unit 162. When the quantization is roughened, noise due to the quantization increases and the feature difference vector increases. If the threshold value is exceeded, the desired recognition performance cannot be ensured, so control is performed in the direction of finer quantization.

  Even if the quantization noise is reduced, the feature difference vector may not be within the threshold value, which is mainly caused by a shift of the face part position due to an error in inter-frame prediction. In order to cope with this, the facial part position difference evaluation value calculation unit 165 evaluates the error of the facial part position and sends the information to the prediction mode control unit 166. The prediction mode control unit 166 performs measures such as prohibiting inter-frame prediction when the error is large based on the face component position error information from the face component position difference evaluation value calculation unit 165. Note that finer quantization and prohibition of inter-frame prediction cause an increase in the code amount of the face region, but this is compensated by reducing the code amount of the region other than the face.

  Note that the video encoding unit 16 is H.264. The MPEG video coding system such as H.264 is adopted, and since it is an existing technology, only a simple explanation will be given. The MPEG video coding method is a combination of inter-frame predictive coding and intra-frame coding. In intra-frame coding, after converting into frequency components by orthogonal transformation, a portion inconspicuous in visual characteristics is quantized. For this reason, the original information cannot be reproduced, resulting in lossy compression. Thereafter, further compression is performed by entropy coding. This part is lossless compression. Since inter-frame predictive coding requires a reference image as a prediction source, it is obtained by local decoding. Since entropy coding is lossless compression, the previous quantized data is decoded in the reverse order of inverse quantization, inverse orthogonal transform, and inter-frame decoding to obtain a reference image. Inverse quantization, inverse orthogonal transform, and inter-frame decoding are included in the video encoding unit 12 in the MPEG moving image encoding system, but the video decoding unit 13 is intentionally displayed individually for convenience of explanation.

  Next, detailed description of the main part of the monitoring terminal 2 will be given. FIG. 5 shows the configuration of the main part of the monitoring terminal 2. The monitoring target recognition unit 25 includes a face detection unit 251, a face feature extraction unit 252, a feature space conversion unit 253, an individual difference information extraction unit 254, a personal feature information database 255, and a personal identification unit 256. The recognition difficulty level evaluation unit 26 includes a disturbance noise extraction unit 261 and individual difference information versus disturbance noise comparison evaluation unit 262. The face detection unit 251 and the face feature extraction unit 252 are basically the same as the face detection unit 142 and the face feature extraction unit 143 in the feature extraction unit 14 of the monitoring camera 1 (see FIG. 2). In order to facilitate the extraction of individual difference information in the individual difference information extraction unit 254 in the next stage, the feature space conversion unit 253 lowers the weight of the feature amount that is easily affected by disturbance noise, and makes the individual difference more accurate. The feature space is converted so as to increase the weight of the feature amount to be reflected. This information is transmitted from the recognition algorithm transmission unit 28 to the monitoring camera 1 and used for conversion of the feature difference vector space, that is, weighting of the feature amount difference.

  The individual difference information extraction unit 254 generates an evaluation amount that accurately reflects the individual difference from the feature amount. The personal feature information database 255 is a database that stores feature values of individual faces. The personal identification unit 256 compares the feature amount of a specific individual in the personal feature information database 255 with the facial features in the video extracted by the personal difference information extraction unit 254, and determines whether the face in the video is a specific personal face. Identify whether.

  A disturbance noise extraction unit 261 in the recognition difficulty level evaluation unit 26 extracts a feature amount that is easily affected by a disturbance such as illumination from the feature amount. The individual difference information versus disturbance noise comparison / evaluation unit 262 compares the evaluation amount that accurately reflects the individual difference with the feature amount that is easily affected by the disturbance, and generates an evaluation amount that reflects the recognition difficulty level. The evaluation amount reflecting the recognition difficulty level is transmitted to the monitoring camera 1 through the recognition difficulty level transmission unit 27 and is used for performing the encoding control in the encoding control unit 16 more appropriately.

  In addition, this invention is not limited to this embodiment, Various embodiment can be taken in the range which does not deviate from the summary of this invention. For example, although the present embodiment aims to recognize a human face, it is possible to realize both recognition performance and network bandwidth efficiency by applying the same concept to recognition techniques other than faces.

  The monitoring system according to the present invention is useful as a monitoring system that requires efficient use of the network bandwidth and the video recording unit because it can increase the compression rate of the entire video while sufficiently ensuring the recognition performance of the monitoring target. is there.

DESCRIPTION OF SYMBOLS 1 Surveillance camera 11 Image pick-up part 12 Image | video encoding part 13 Image | video decoding part 14 Feature-value extraction part 142 Face detection part 143 Face feature extraction part 15 Feature change detection part 16 Encoding control part 2 Monitoring terminal 25 Monitoring object recognition part 26 Recognition Difficulty evaluation department

Claims (9)

An imaging unit for imaging a monitoring target;
A video encoding unit that encodes the video output from the imaging unit by lossy compression;
A video decoding unit for decoding the video encoded by the video encoding unit;
A feature amount extraction unit that receives the first video output from the imaging unit and the second video output from the video decoding unit, and extracts a feature amount in each video;
A feature change detection unit that detects a change amount between the feature amount extracted from the first video and the feature amount extracted from the second video;
A surveillance camera comprising: an encoding control unit that changes a control parameter of the video encoding unit based on the detected change in the feature amount. The surveillance camera of claim 1,
The feature quantity extraction unit outputs the extracted feature quantity as a feature vector;
The feature change detecting unit calculates a difference vector between a feature vector related to the first video and a feature vector related to the second video, and weights each element of the difference vector as a feature difference vector A surveillance camera characterized by output. The surveillance camera of claim 2,
The encoding control unit converts the feature difference vector output from the feature change detection unit into an evaluation index, and changes the control parameter of the video encoding unit so that the evaluation index falls within a threshold value. Surveillance camera. The surveillance camera of claim 1,
The surveillance camera, wherein the coding control unit controls presence / absence of inter-frame prediction and a quantization parameter of the video coding unit. The surveillance camera of claim 1,
The feature amount extraction unit includes:
A face detection unit for detecting a human face in the input video;
A surveillance camera, comprising: a facial feature extraction unit that extracts density gradients of components such as eyes, nose, and mouth in the detected human face. The surveillance camera of claim 1,
A surveillance camera, wherein the video encoding unit, video decoding unit, feature quantity extraction unit, feature change detection unit, and encoding control unit are configured by one or a plurality of semiconductor chips. A surveillance camera according to claim 1;
A monitoring system comprising: a monitoring terminal connected to the monitoring camera through a communication line. The monitoring system of claim 7,
The monitoring terminal is
A monitoring object recognition unit for recognizing the monitoring object from the video received from the monitoring camera;
A recognition difficulty level evaluation unit that evaluates the recognition difficulty level of the video received from the surveillance camera;
Transmitting the recognition difficulty level to the surveillance camera;
The monitoring system, wherein the encoding control unit in the monitoring camera changes a control parameter of the video encoding unit in consideration of a recognition difficulty level transmitted from the monitoring camera. A surveillance camera according to claim 2;
A monitoring terminal connected to the monitoring camera through a communication line;
The monitoring terminal includes a monitoring target recognition unit that recognizes the monitoring target from the video received from the monitoring camera, and transmits an algorithm related to the recognition to the monitoring camera.
The monitoring system, wherein the feature change detection unit in the monitoring camera weights each element of the feature difference vector in consideration of a recognition algorithm transmitted from the monitoring camera.

Images