JP5971712B2 - Monitoring device and method - Google Patents

Description

  Embodiments described herein relate generally to a monitoring apparatus and method.

  Conventionally, in order to find a specific person such as a criminal, there is a monitoring device that visually confirms image data captured by a camera on a display device. In this monitoring device, there is a demand to acquire image data that makes it easy to visually confirm a person's face, and a face image is detected from image data captured by a camera, and the detected face image is cut out and output. There is.

JP 2010-22601 A

  However, in the above-described prior art, when the shooting conditions are wide-angle, the face size in the image shot by the camera may be reduced, and it is not always possible to acquire an image in which a human face can be easily visually confirmed. . In addition, for a plurality of persons passing in front of the camera, photographing of the camera in the horizontal direction, vertical direction, rotation direction, zoom (angle of view), etc. so as to obtain image data that makes it easy to visually confirm each person's face. It was not easy to adjust the conditions.

ｆ（θ，φ，ψ，ｒ）は、前記評価値である。θ１〜θＮは、Ｎ人分の前記人物の動線を直線で近似した場合の当該人物の動線と前記カメラの撮影方向とがなす角度である。ψ１〜ψＮは、Ｎ人分の前記人物の動線と前記画像データの水平方向の直線とがなす角度である。Ａ、Ｂ、Ｃ、Ｄ（＞＝０）は、θ、φ、ψ、ｒのいずれを重視するかを定めるパラメータである。 In order to solve the above-described problem, the monitoring apparatus according to the embodiment includes an image input unit that inputs image data captured by a camera, and a face that detects a face area representing a human face from the input image data. a detection means, a detection result storing means for storing the detection result of the face detection unit, on the basis of the stored detection result, according to equation (4), vertical angle phi, the horizontal angle of the camera Using at least θ , the angle ψ of the photographing direction in the rotation direction, and the angle of view r as parameters, at least the evaluation value by the evaluation function that evaluates the ease of confirming the face of the person based on the stored face area is maximized. A calculation unit that calculates one shooting condition; and an output unit that outputs the calculated shooting condition.

f (θ, φ, ψ, r) is the evaluation value. θ1 to θN are angles formed by the flow lines of the person and the shooting direction of the camera when the flow lines of the N persons are approximated by straight lines. ψ1 to ψN are angles formed by the flow lines of the N persons and the straight line in the horizontal direction of the image data. A, B, C, and D (> = 0) are parameters that determine which of θ, φ, ψ, and r is important.

The monitoring apparatus according to the embodiment repeats preset detection of an image input unit that inputs image data captured by a camera and a face area in which a human face is represented from the input image data . Based on detection results for the number of iterations of the face detection means, face detection means for continuing until execution, shooting condition storage means for storing shooting conditions of the camera for facilitating confirmation of the face of the person, Determination means for determining whether at least one shooting condition of a vertical direction, a horizontal direction, a shooting direction in a rotation direction, and an angle of view of a person's face matches the stored shooting condition; Output means for outputting a determination result of the determination means.

ｆ（θ，φ，ψ，ｒ）は、前記評価値である。θ１〜θＮは、Ｎ人分の前記人物の動線を直線で近似した場合の当該人物の動線と前記カメラの撮影方向とがなす角度である。ψ１〜ψＮは、Ｎ人分の前記人物の動線と前記画像データの水平方向の直線とがなす角度である。Ａ、Ｂ、Ｃ、Ｄ（＞＝０）は、θ、φ、ψ、ｒのいずれを重視するかを定めるパラメータである。 The method of the embodiment is a method executed by the monitoring apparatus, in which the image input unit inputs the image data captured by the camera, and the face detection unit detects the person's image from the input image data. detecting a face is represented face area detection result storage means, and storing the detection result of the face area, calculation means, based on the detection result of said stored formula (4) accordingly vertical angle φ of the camera, the angle of the horizontal direction theta, the angle of the photographing direction in the direction of rotation [psi, and the angle r as a parameter, confirmation of the face of the person to which the basis of the stored face area Calculating at least one imaging condition that maximizes an evaluation value based on an evaluation function for evaluating ease; and an output unit outputting the calculated imaging condition.

f (θ, φ, ψ, r) is the evaluation value. θ1 to θN are angles formed by the flow lines of the person and the shooting direction of the camera when the flow lines of the N persons are approximated by straight lines. ψ1 to ψN are angles formed by the flow lines of the N persons and the straight line in the horizontal direction of the image data. A, B, C, and D (> = 0) are parameters that determine which of θ, φ, ψ, and r is important.

The method according to the embodiment is a method executed by a monitoring device, and the monitoring device includes a shooting condition storage unit that stores a shooting condition of a camera that makes it easy to check a person's face, and the image input unit includes: The step of inputting image data captured by the camera and the face detection means continue until detection of a face area in which a human face is represented from the input image data is performed for a preset number of iterations. And at least one of the shooting direction in the vertical direction, the horizontal direction, the rotation direction, and the angle of view of the person's face based on the detection result of the face area for the number of repetitions. A step of determining whether or not a condition matches the stored photographing condition; and a step of outputting the determined determination result by the output means.

FIG. 1 is a block diagram illustrating a configuration of a monitoring device according to the first embodiment. FIG. 2 is a conceptual diagram illustrating the shooting direction of the camera. FIG. 3 is a flowchart illustrating an example of the operation of the monitoring apparatus according to the first embodiment. FIG. 4 is a conceptual diagram illustrating the relationship between the flow line and the shooting direction. FIG. 5 is a conceptual diagram illustrating the relationship between the flow line and the shooting direction. FIG. 6 is a conceptual diagram illustrating rotation of the flow line and the captured image of the camera. FIG. 7 is a conceptual diagram illustrating zooming of a captured image. FIG. 8 is a block diagram illustrating a configuration of a monitoring device according to a modification. FIG. 9 is a block diagram illustrating a configuration of a monitoring device according to the second embodiment. FIG. 10 is a flowchart illustrating an example of the operation of the monitoring apparatus according to the second embodiment. FIG. 11 is a block diagram illustrating a hardware configuration of the monitoring device according to the first and second embodiments.

  Hereinafter, a monitoring apparatus and method according to embodiments will be described in detail with reference to the accompanying drawings. The monitoring apparatus and method according to the embodiment are assumed to be used for visually confirming a human face from an image of a security camera (hereinafter referred to as a camera) installed in a street, a building, or a public area. This makes it possible to easily adjust the shooting conditions (horizontal direction, vertical direction, rotation, zoom) of the camera that are easy to check. In this embodiment, a procedure for realizing a problem by detecting a face area as a person area and using facial feature information will be described. However, a technique for detecting a person area whole body other than a face (Watanabe et al., "Co-occurrence Histograms of Oriented Gradients for Pedestrian Detection, In Proceedings of the 3rd Pacific-Rim Symposium on Image and Video Technology" (PSIVT2009), pp. 37-47.) Obviously, the technology is not limited to the technology described in the present embodiment as long as it is a technology for detecting a person region and a technology for measuring characteristic information in the person region.

(First embodiment)
FIG. 1 is a block diagram illustrating the configuration of a monitoring apparatus 100 according to the first embodiment. As illustrated in FIG. 1, the monitoring apparatus 100 includes an input unit 101, a detection unit 102, a detection result management unit 103, an imaging condition calculation unit 104, and an output unit 105. The camera 150 takes an image of a predetermined area. For example, the camera 150 is a monitoring camera or the like that captures an entrance / exit target area on a traffic path, and generates moving image data that is a capturing result. Then, the input unit 101 performs input processing on moving image data from the camera 150. The input unit 101 outputs the input moving image data to the detection unit 102.

  The camera 150 can be installed in at least one place or a plurality of points. The camera 150 is for inputting a face image of a person existing in a predetermined area, and is an ITV (Industrial Television) camera, for example. The camera 150 digitizes optical information obtained through the lens of the camera by an A / D converter, generates frame image data of a predetermined frame rate, and outputs the frame image data to the monitoring apparatus 100. The camera 150 is a PTZ (pan / tilt / zoom) camera and can adjust photographing conditions such as a horizontal direction, a vertical direction, rotation, and zoom.

  FIG. 2 is a conceptual diagram illustrating the shooting direction 24 of the camera 150. As shown in FIG. 2, the shooting direction 24 of the camera 150 is adjusted by adjusting a camera pedestal (not shown) on which the camera 150 is installed, so that the horizontal direction 21 (pan and yaw direction) and the vertical direction 22 (tilt and pitch direction). ), And can be adjusted in the shooting direction 24 (roll direction). The camera 150 can adjust the focus by adjusting the lens position. Adjustment of shooting conditions such as horizontal direction, vertical direction, rotation, and zoom may be performed manually by a user or automatically by an actuator or the like.

  Returning to FIG. 1, the detection unit 102 detects a face area in which a human face is represented from input image data (input image). Specifically, the detection unit 102 obtains coordinates indicating the face region using luminance information on the image in the input image. Here, the literature (Yoshi Mita et al .: “Joint Haar-like feature based on co-occurrence suitable for face detection”, IEICE Transactions (D), vol. J89-D, No8, pp 1791-1801 (2006)). Since this is realized by using the method, it is assumed that this method is used. The information indicating the detection result depending on the orientation and size of the face may be an arbitrary shape, but in this embodiment, for the sake of simplicity, the face area is indicated by rectangular information, and the coordinates of the corner are detected. It will be used as. In addition, by obtaining a correlation value while moving a template prepared in advance in the image, a method that uses the position that gives the highest correlation value as a face region, a face extraction method that uses an eigenspace method or a subspace method, etc. Realization is possible.

  In addition, in the video shot by the camera 150, it is assumed that the detected face of the same person is continuously shown over a plurality of frames due to the movement of the person in front of the camera 150 (movement locus). . Therefore, the detection unit 102 needs to perform tracking processing of a person's face so that the detected same face can be associated as the same person. This means can be realized by using a method of associating the position detected by the optical flow with the position of the face in the next frame, or by using a patent publication (Japanese Patent Laid-Open No. 2011-170711). When detecting a part as a feature point (to be described later), a method of selecting at least one image suitable for searching from images of face regions of a plurality of frames associated with the same person, Any number of images up to the number of detected frames can be used.

  Further, the position of a face part such as an eye or nose is detected as a facial feature point from the detected face area. Specifically, literature (Kazuhiro Fukui, Osamu Yamaguchi: “Face feature point extraction by combination of shape extraction and pattern matching”, IEICE Transactions (D), vol. J80-D-II, No. 8, pp2170 -2177 (1997)). In addition to the above-mentioned eye / nose detection, the literature (Mayumi Yuasa, Akiko Nakajima: “Digital Make System Based on High-Precision Facial Feature Point Detection” 10th Image Sensing Symposium Proceedings, pp 219-224 This can be easily realized by using the technique of (2004)). In any case, it is possible to acquire information that can be handled as a two-dimensional array of images and to detect a facial feature region from the information. Also, in these processes, in order to extract only one facial feature from one image, a correlation value with the template is obtained for all images, and the maximum position and size may be output. To extract facial features, obtain the local maximum correlation value for the entire image, narrow down the candidate face positions in consideration of the overlap in one image, and finally the past images that were input continuously It is also possible to finally find a plurality of facial features at the same time in consideration of the relationship (time transition).

  Regarding the estimation of the face orientation, as disclosed in Japanese Patent Application Laid-Open No. 2003-141551 (“Face Orientation Calculation Method and Device” Toshiba, K. Yamada; Kazuhiro Fukui; Hayato Maki; Ayako Nakajima) It is possible to estimate the face orientation using a plurality of classes (subspaces) learned according to the rotation matrix and face orientation.

  The detection result management unit 103 stores information (detection result) related to the face area detected by the detection unit 102. Specifically, the detection result management unit 103 includes, for each human face detected by the detection unit 102, the coordinates of the face region in the input image, the trajectory (flow line) of the face region detected over a plurality of frame images, It is a database that stores position information of face parts (eyes, nose, etc.) detected from a face area and estimated face orientation.

  The imaging condition calculation unit 104, based on the detection result stored in the detection result management unit 103, makes it easy to confirm the human face, the imaging direction in the vertical direction, horizontal direction, and rotational direction of the camera 150, and the angle of view ( At least one shooting condition of (zoom) is calculated. For example, assuming that detection results for 1 to N persons are obtained for the persons detected by the detection unit 102 in the detection result management unit 103, it is easy to confirm the faces of the persons for the 1 to N persons. The shooting conditions are calculated. Specifically, the imaging condition calculation unit 104 calculates an evaluation value based on an evaluation function for evaluating ease of confirming a person's face based on the detection results stored in the detection result management unit 103 for 1 to N persons. The maximum shooting condition is calculated.

  FIG. 3 is a flowchart illustrating an example of the operation of the monitoring apparatus 100 according to the first embodiment. More specifically, FIG. 3 is a flowchart illustrating calculation of shooting conditions by the shooting condition calculation unit 104. As shown in FIG. 3, when the processing is started, the imaging condition calculation unit 104 acquires the detection result (face detection / tracking result) stored in the detection result management unit 103 (S1). Next, the imaging condition calculation unit 104 determines whether or not the number of detection results acquired (number of acquired persons) is equal to or greater than a predetermined number (N) set in advance (S2). If it is not greater than the predetermined number (S2: NO), the process returns to S1 and waits.

  When there are more than a predetermined number of detection results (S2: YES), the shooting condition calculation unit 104 determines the range that the horizontal angle (pan) of the camera 150 can take, that is, the range of horizontal parameters in the shooting conditions. It sets from the flow line for N persons memorize | stored in the detection result management part 103 (S3).

  FIG. 4 is a conceptual diagram illustrating the relationship between the flow lines d1 to dN for N persons and the shooting direction 24. More specifically, FIG. 4 is a conceptual view of the shooting direction 24 seen from above. As shown in FIG. 4, it is assumed that the trajectory of movement for N persons is given as flow lines d1 to dN. In S3, the shooting condition calculation unit 104 sets the shooting direction (horizontal direction) when each flow line is approximated by a straight line so that the shooting direction 24 of the camera 150 and the flow lines d1 to dN are as parallel as possible. When the formed angles are θ1, θ2,... ΘN in ascending order, the horizontal angle θ is set as in the following equation (1).

  Next, the imaging condition calculation unit 104 calculates the range that can be taken by the vertical angle (tilt) angle φ of the camera 150, that is, the range of the vertical parameter in the imaging condition, for N people stored in the detection result management unit 103. Set from the flow line (S4).

  FIG. 5 is a conceptual diagram illustrating the relationship between the flow lines d1 to dN for N persons and the shooting direction 24. More specifically, FIG. 5 is a conceptual view of the shooting direction 24 seen from the side. As shown in FIG. 5, it is assumed that the trajectory of movement for N people is given as flow lines d1 to dN. In S4, the imaging condition calculation unit 104 sets the average angle of the face direction for each flow line with respect to the frame image as φ1, φ2,... ΦN in ascending order, as shown in the following equation (2). Set the angle φ.

  Next, the imaging condition calculation unit 104 calculates the range that the angle ψ of the rotation direction (roll) of the camera 150 can take, that is, the range of the rotation direction parameter in the imaging condition, for N people stored in the detection result management unit 103. The flow line is set (S5).

  FIG. 6 is a conceptual diagram illustrating the flow lines d1 to dN for N persons and the rotation of the captured image G of the camera 150. As shown in FIG. 6, it is assumed that the trajectory of movement for N persons is given as flow lines d1 to dN. In S <b> 5, the imaging condition calculation unit 104 assumes that the angle formed by each flow line in the captured image G and the horizontal straight line of the captured image G is ψ1, ψ2,. As in 3), the angle ψ in the rotational direction is set.

  Next, the shooting condition calculation unit 104 sets a range that can be taken by the zoom (field angle) of the camera 150, that is, a range of parameter of the field angle in the shooting conditions (S6). Specifically, k values such as r1, r2,... Rk are set with r being a zoom parameter within a range that can be set in advance by adjusting the lens position of the camera 150, which is preset in a memory or the like. . As for the above-described θ, φ, and ψ, the upper limit value and the lower limit value are set within the operating range of the camera pedestal preset in the memory or the like.

  Next, the pan, tilt, roll, and zoom values that maximize the evaluation value based on the above-described evaluation function are calculated for the shooting conditions in which the ranges of the pan, tilt, roll, and zoom parameters are set. Calculation of the evaluation value using this evaluation function is ideal for monitoring the face visually, such as when the face size is large, the face direction is front of the camera 150, and the shooting range is wide, but it is difficult to achieve both. Considering the standard, the following equation (4) is used.

  In Expression (4), A, B, C, and D (> = 0) are parameters that determine which of θ, φ, ψ, and r, which are parameters to be adjusted, are important. For A, B, C, and D, one parameter is enabled (> = 0) so that one or more photographing conditions can be evaluated. g (r) is included in each frame image of each flow line, for example, from the number X of faces included in each frame image of each flow line that are not included in the angle of view (imaging area) by zooming. It is assumed that the number Y of faces whose face size is a predetermined range of S pixels or more and S pixels or less is subtracted. Other functions may be used as long as they can express the trade-off between the face size and the angle of view. Since the evaluation value can be maximized by, for example, linear programming when the value of r is fixed, it can be realized by solving a linear programming problem for each value of r. Also, the face size and angle of view can be calculated if three-dimensional information about the camera and subject is obtained. The evaluation function (objective function) may be any function as long as the face orientation and the face size are adjusted so as to be photographed under a desired condition. This function is maximized or minimized (in this embodiment, the evaluation value is increased). This means that it is easy to confirm the person's face.

  FIG. 7 is a conceptual diagram illustrating zooming of the captured image G. As shown in FIG. 7, the regions R1 and R2 where the face appears frequently obtained by examining the frequency of the face based on the detection result may be limited to the imaging region R and may be enlarged and photographed. Further, by performing the above-described processing, the rotation angle can be calculated so as to correct the inclination of the flow line as shown in FIG. Further, when the detection result is not output over a certain period, a warning may be output from the output unit 105 on the assumption that an appropriate photographing condition cannot be calculated.

  The output unit 105 is a display device or the like, and displays and outputs the shooting conditions calculated by the shooting condition calculation unit 104. The output unit 105 also displays the input image of the input unit 101 and displays the input image superimposed with the face area detected by the detection unit 102. The user can check the shooting conditions output from the output unit 105 to make it easier to check the face of the person shot by the camera 150, such as the horizontal direction, vertical direction, rotation direction, zoom (view angle). ) Etc. can be adjusted easily.

(Modification)
Next, a modified example of the monitoring device 100 will be described. FIG. 8 is a block diagram illustrating a configuration of a monitoring device 100a according to a modification. In the monitoring device 100a, the same components as those of the monitoring device 100 are denoted by the same reference numerals, and the description thereof is omitted.

  As shown in FIG. 8, the monitoring apparatus 100a includes an input unit 101, a detection unit 102, a detection result management unit 103, an imaging condition calculation unit 104, an output unit 105, an imaging condition control unit 106, and feature extraction. Unit 107, person information management unit 108, and recognition unit 109.

  The shooting condition control unit 106 outputs a drive signal to an actuator (not shown) of the camera 150 according to the parameters calculated by the shooting condition calculation unit 104, whereby the angle of the camera 150 in the horizontal direction 21 and the angle of the vertical direction 22 Control the angle of roll direction 23, screen resolution, and screen zoom. The shooting condition control unit 106 allows the camera 150 to automatically change to shooting conditions that make it easy to check the face of a person shot by the camera 150 without the user manually changing the shooting conditions.

  The feature extraction unit 107 identifies feature information from the face area information detected by the detection unit 102 (hereinafter, “face feature” indicates feature information for identifying the individual). Is output as a numerical value. As an image used for feature extraction, an input image obtained by the input unit 101 is used. The feature extraction unit 107 associates the input image from the input unit 101 with the person region detected by the detection unit 102 and detects an image region necessary for feature extraction, which is a face region in this embodiment. Are extracted, and the shading information is used as a feature amount. In case of using a plurality of frames of images, the image correction means may output a plurality of frames.

  Here, the gray value of an area of m pixels × n pixels is used as information as it is, and m × n-dimensional information is used as a feature vector. These are normalized so that the vector and the length of each vector are set to 1 by a method called a simple similarity method, and a similarity indicating the similarity between feature vectors is obtained by calculating an inner product. For details, this can be realized by using the subspace method as described in the literature (written by Elkki Oya, Hidemitsu Ogawa, Makoto Sato, “Pattern Recognition and Subspace Method”, Sangyo Tosho, 1986). Document (Toshiba (Tatsuo Kosakaya): “Image recognition apparatus, method and program”, Japanese Patent Laid-Open No. 2007-4767) discloses the orientation and state of a face using a model for one piece of human image information. You may apply the method of improving a precision by producing the image changed intentionally. When the facial features are obtained from one image, the facial feature extraction is completed by the processing so far. On the other hand, recognition processing with higher accuracy can be performed by calculating with a moving image using a plurality of continuous images for the same person. Specifically, the literature (Kazuhiro Fukui, Osamu Yamaguchi, Kenichi Maeda: “Face Recognition System Using Moving Images” IEICE Research Report PRMU, vol 97, No. 113, pp 17-24 (1997), Kenichi Maeda, Watanabe Sadaichi: Use the mutual subspace method described in "Pattern matching method with local structure", IEICE Transactions (D), vol. J68-D, No. 3, pp 345-352 (1985)). How to explain. Similar to the feature extraction unit, an image of m × n pixels is cut out from images continuously obtained from the input unit, a correlation matrix of the feature vectors is obtained from these data, and an orthonormal vector by KL expansion is obtained. A subspace indicating facial features obtained from successive images is calculated. The subspace calculation method calculates a subspace by obtaining a correlation matrix (or covariance matrix) of feature vectors and obtaining an orthonormal vector (eigenvector) by KL expansion. In the subspace, k eigenvectors corresponding to eigenvalues are selected in descending order of eigenvalues, and expressed using the eigenvector set. In the present embodiment, the correlation matrix Cd is obtained from the feature vector, and diagonalized with the correlation matrix Cd = ΦdΛdΦdT to obtain the eigenvector matrix Φ. This information becomes a partial space indicating the characteristics of the face of the person currently recognized. The feature information such as the partial space output by such a method is used as the individual feature information for the face detected in the input image.

  The person information management unit 108 is a database to be used when searching by the recognition unit 109 to be described later. The facial feature information extracted by the feature extraction unit 107 for each person to be searched, and the gender, age, Information associated with a person such as height and other attribute information that can be discriminated by attribute discriminating means is managed in association with each person. The content actually managed as face feature information and attribute information may be the data itself output by the feature extraction unit 107, or may be an m × n feature vector, a partial space, or a correlation matrix immediately before performing KL expansion. Furthermore, by managing the feature information output by the feature extraction unit 107 together with the person image input at the time of registration from the input unit, it can be used for personal search and search display.

  The recognizing unit 109 performs a calculation indicating the similarity between the facial feature information of the input image obtained by the feature extracting unit 107 and the facial feature information stored in the corresponding person information management unit 108, so that the similarity is higher. Process to return the result in order from the one. Further, as described above, it is possible to narrow down to predetermined attribute information and search only part of the person information management unit 108.

  At this time, as a result of the search process, a person ID managed in order to identify an individual in the person information management unit 108 and an index indicating the similarity as a calculation result are returned in descending order of the similarity. In addition, information managed for each individual may be returned together, but it is basically possible to associate by identification ID, so the search process itself does not need to exchange attached information. Realization is possible. The index indicating similarity is the similarity between partial spaces managed as face feature information. As a calculation method, a method such as a subspace method or a composite similarity method may be used. In this method, both the recognition data in the registration information stored in advance and the input data are expressed as subspaces calculated from a plurality of images, and the “angle” formed by the two subspaces is defined as similarity. . The partial space input here is referred to as an input means space. Similarly, a correlation matrix Cin is obtained for the input data string, and diagonalized with Cin = ΦinΛinΦinT to obtain an eigenvector Φin. The similarity between subspaces (0.0 to 1.0) of the subspaces represented by two Φin and Φd is obtained and used as the similarity for recognizing this. A specific calculation method can be realized by the literature (Erki Oya) introduced in the description of the feature extraction unit 107. It is also possible to improve accuracy by collecting multiple human images that are known to be the same person in advance and identifying whether the person is the person by projecting to the partial space. The same applies to the literature (Fukui and Kosakaya). Processing can also be performed. For high-speed search, a search method using a TREE structure can be used.

  The output unit 105 displays the result obtained by the recognition unit 109 and the input image on the screen. Real-time face search result display that displays in real time the results that are searched by the recognition unit 109 that meet the specified conditions, and the search results that have been searched by the recognition unit 109 are saved as a search history, and the conditions are specified later. By doing so, it is possible to incorporate either or both of offline face search result displays that display only the corresponding search history.

(Second Embodiment)
FIG. 9 is a block diagram illustrating a configuration of a monitoring device 100b according to the second embodiment. In the monitoring device 100b, the same components as those of the monitoring device 100 are denoted by the same reference numerals, and the description thereof is omitted.

  As illustrated in FIG. 9, the monitoring device 100 b includes an input unit 101, a detection unit 102, an output unit 105, an imaging condition storage unit 110, and an imaging condition diagnosis unit 111.

  The photographing condition storage unit 110 is a camera photographing condition that makes it easy to confirm the face of a person photographed by the camera 150. Recommended values such as a predetermined face size, face orientation, viewing angle, background image, or Information such as the pan, yaw direction (horizontal direction), tilt, pitch direction (vertical direction), roll direction (rotation direction), and zoom magnification of the camera 150 is stored. As in the first embodiment, the value determined from the tracking history of the person moving in front of the camera 150 may be recorded, or the set value is recorded while the user visually confirms the screen in advance. May be.

  Based on the detection result of the detection unit 102, the imaging condition diagnosis unit 111 stores at least one imaging condition of an up / down direction, a horizontal direction, a rotation direction and an angle of view of the camera 150 with respect to a human face. It is determined whether or not the shooting conditions stored in the unit 110 match.

  FIG. 10 is a flowchart illustrating an example of the operation of the monitoring apparatus 100b according to the second embodiment. More specifically, FIG. 10 is a flowchart illustrating determination of imaging conditions by the imaging condition diagnosis unit 111. As illustrated in FIG. 10, the imaging condition diagnosis unit 111 determines whether or not the processing of S11 to S17 has been performed an upper limit number of times set in advance (S10). Here, when the upper limit number of repetitions has not been performed (S10: NO), the imaging condition diagnosis unit 111 continues the processes of S11 to S17. When the upper limit of the number of repetitions is performed (S10: YES), the imaging condition diagnosis unit 111 advances the process to S18.

  In S <b> 11, the imaging condition diagnosis unit 111 acquires the detection result (face detection / tracking result) of the detection unit 102. Next, the imaging condition diagnosis unit 111 determines whether or not the number of detection results is zero (S12). If the number is zero (S12: YES), the imaging condition diagnosis unit 111 stores the input image of the input unit 101 as a background image (S13), and increments the number of processing iterations (S17). When the number is not zero (S12: NO), the imaging condition diagnosis unit 111 stores the face size, the face direction, and the flow line, which are detection results of the detection unit 102 (S14 to 16), and increments the number of processing iterations. (S17).

  In S18, the imaging condition diagnosis unit 111 calculates an evaluation value of the consistency of the imaging conditions based on the data stored in the processing of S12 to S17 and the imaging conditions stored in the imaging condition storage unit 110. (S18) The calculation result (determination result as to whether or not to match) is output to the output unit 105 (S19). The output unit 105 displays and outputs whether or not the shooting conditions of the camera 150 are consistent with the shooting conditions stored in the shooting condition storage unit 110 in advance.

  Specifically, the imaging condition diagnosis unit 111 stores a background image when no person is detected from the input image, calculates a difference from the stored background image in S18, and the pixel or an adjacent pixel The posterior probability that is the background can be obtained for each block that summarizes (Hiroaki Nakai, JP 09-81714 A). In addition, the face size of the face detection result is recorded, and the consistency is determined based on the number X of faces not included in the range of the face size greater than or equal to the pixel specified in the imaging condition storage unit 110 and less than or equal to S pixel. You can also In addition, the horizontal angle, the vertical angle, and the in-screen rotation angle of the face direction are recorded, and matching is performed based on the number Y of faces not included in the face direction range specified by the photographing condition storage unit 110. Sex can also be judged. The face orientation can be calculated by the method described above. The angle formed by the flow line and the horizontal straight line of the image from the flow line information obtained by the detection unit 102 is matched based on the number Z of persons not included in the angle range specified by the imaging condition storage unit 110. Sex can also be judged. Further, if the number of persons counted by tracking the person obtained by the detection unit 102 is N, it can be determined that the angle of view is inappropriate when N is smaller than the number of persons specified by the imaging condition storage unit 110.

  Further, it is possible to determine that the registered condition is different from the difference between the background image and the comparison of the color histogram. It is also possible to use feature quantities based on local feature quantities such as edges and corners and the luminance distribution of local areas. For this determination, the feature values may be obtained for the entire screen and the respective coordinate values may be memorized individually, or the entire screen may be divided into small areas and feature information may be calculated and compared for each block.

  FIG. 11 is a block diagram illustrating a hardware configuration of the monitoring devices 100, 100a, and 100b according to the first and second embodiments. As illustrated in FIG. 11, the monitoring devices 100, 100 a, and 100 b include a CPU 1101, a ROM (Read Only Memory) 1102, a RAM (Random Access Memory) 1103, a communication I / F 1104, an HDD 1105, and a display device 1106. , An input device 1107 such as a keyboard and a mouse, and a bus 1108 for connecting them, and has a hardware configuration using a normal computer.

  The programs executed by the monitoring apparatuses 100, 100a, and 100b of the present embodiment are files in an installable format or an executable format, and are CD-ROM, flexible disk (FD), CD-R, DVD (Digital Versatile Disk). Or the like recorded on a computer-readable recording medium.

  The program executed by the monitoring devices 100, 100a, and 100b of the present embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. . Further, the program executed by the monitoring devices 100, 100a, and 100b of the present embodiment may be configured to be provided or distributed via a network such as the Internet. Further, the program of this embodiment may be configured to be provided by being incorporated in advance in a ROM or the like.

  The program executed by the monitoring devices 100, 100a, and 100b according to the present embodiment has a module configuration including each of the above-described configurations. As actual hardware, the CPU 1101 reads the program from the storage medium and executes it. As a result, the respective components are loaded onto the RAM 1103 and the respective components are generated on the RAM 1103.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 100, 100a, 100b ... Monitoring apparatus, 21 ... Horizontal direction, 22 ... Vertical direction, 23 ... Roll direction, 24 ... Shooting direction, 101 ... Input part, 102 ... Detection part, 103 ... Detection result management part, 104 ... Shooting condition Calculation unit 105 ... Output unit 106 ... Shooting condition control unit 107 ... Feature extraction unit 109 ... Recognition unit 108 ... Personal information management unit 110 ... Shooting condition storage unit 111 ... Shooting condition diagnosis unit 150 ... Camera 1101 ... CPU 1102 ... ROM 1103 ... RAM 1104 ... Communication I / F 1105 ... HDD 1106 ... Display device 1107 ... Input device 1108 ... Bus, d1, d2, dN ... Flow line, G ... Photographing Image, R ... shooting area, R1, R2 ... area

Claims (13)

Image input means for inputting image data captured by the camera;
Face detection means for detecting a face area representing a human face from the input image data;
Detection result storage means for storing the detection result of the face detection means;
Based on the stored detection result, according to the equation (1), the vertical angle φ 1 of the camera, the horizontal angle θ 2 , the angle ψ of the shooting direction in the rotation direction, and the angle of view r are used as parameters. Calculating means for calculating at least one photographing condition that maximizes an evaluation value by an evaluation function for evaluating ease of confirmation of the face of the person based on a stored face area ;
Output means for outputting the calculated photographing condition;
A monitoring device comprising:

f (θ, φ, ψ, r) is the evaluation value. θ1 to θN are angles formed by the flow lines of the person and the shooting direction of the camera when the flow lines of the N persons are approximated by straight lines. ψ1 to ψN are angles formed by the flow lines of the N persons and the straight line in the horizontal direction of the image data. A, B, C, and D (> = 0) are parameters that determine which of θ, φ, ψ, and r is important. The face detecting means detects a size of the face region;
The calculating means calculates a shooting condition based on the size of the stored face area, with the face area having a predetermined size;
The monitoring apparatus according to claim 1 . The face detecting means detects a face direction based on a face part included in the face region;
The calculation means calculates shooting conditions based on the stored face orientation, with the face orientation being a predetermined direction.
The monitoring apparatus according to claim 1 . The detection result storage means stores the face area detected by the face detection means over a plurality of frame images,
The calculation means calculates shooting conditions for shooting from a predetermined direction with respect to a trajectory of movement of the person based on the face area stored over the plurality of frame images.
The monitoring apparatus according to claim 1 . Control means for controlling the shooting direction and angle of view of the camera based on the calculated shooting conditions,
The monitoring apparatus as described in any one of Claims 1 thru | or 4 . Image input means for inputting image data captured by the camera;
Detects a face of a person from the input image data is represented the face area, a preset number of iterations, a face detection unit which continues until the implementation,
Shooting condition storage means for storing shooting conditions of the camera for facilitating confirmation of the person's face;
Based on the detection results for the number of repetitions of the face detection means, at least one shooting condition of the camera's face in the vertical direction, horizontal direction, shooting direction in the rotation direction, and angle of view is stored. Determining means for determining whether or not it matches the shooting conditions;
Output means for outputting a determination result of the determination means;
A monitoring device comprising: The photographing condition storage means stores the face size of the person when the person is photographed with the camera as the photographing condition,
The determination unit determines whether or not a face size based on the detection result matches the stored face size;
The monitoring device according to claim 6 . The photographing condition storage means stores the face direction of the person when the person is photographed by the camera as the photographing condition,
The determination means determines whether or not a face orientation based on the detection result matches the stored face orientation;
The monitoring device according to claim 6 . The shooting condition storage means stores, as the shooting condition, the shooting direction of the camera with respect to a trajectory of movement of the person when the person moves in front of the camera,
The determination unit determines whether or not the movement trajectory of the person matches the stored movement trajectory based on the face area detected by the face detection unit over a plurality of frame images;
The monitoring device according to claim 6 . The photographing condition storage means stores, as the photographing condition, a pixel value in the background of the person when the person is photographed by the camera.
The determination means determines whether or not a pixel value of an area other than the face area based on the detection result matches the stored pixel value.
The monitoring device according to claim 6 . The determination unit determines whether the shooting condition based on the detection result of the face detection unit matches the stored shooting condition for all or a part of the image in the image data captured by the camera. To determine,
The monitoring device according to any one of claims 6 to 10 . A method performed by a monitoring device, comprising:
An image input means for inputting image data captured by the camera;
A step of detecting a face area in which a human face is represented from the input image data;
A detection result storing means for storing the detection result of the face area;
Based on the stored detection result, the calculating means calculates the vertical angle φ 1 of the camera, the horizontal angle θ 2 , the shooting direction angle φ 2 in the rotation direction, and the angle of view r according to the equation (1). Calculating at least one imaging condition that maximizes an evaluation value by an evaluation function for evaluating ease of confirming the face of the person based on the stored face area as a parameter ;
An output means for outputting the calculated photographing condition;
Including methods.

f (θ, φ, ψ, r) is the evaluation value. θ1 to θN are angles formed by the flow lines of the person and the shooting direction of the camera when the flow lines of the N persons are approximated by straight lines. ψ1 to ψN are angles formed by the flow lines of the N persons and the straight line in the horizontal direction of the image data. A, B, C, and D (> = 0) are parameters that determine which of θ, φ, ψ, and r is important. A method performed by a monitoring device, comprising:
The monitoring device includes a shooting condition storage unit that stores shooting conditions of a camera that makes it easy to check a person's face,
An image input means for inputting image data captured by the camera;
A step of the face detecting means continues to detect the face of a person from the input image data is represented the face area, a preset number of iterations, until the implementation,
Based on the detection result of the face area for the number of repetitions , the determination unit has at least one shooting condition of the shooting direction in the vertical direction, horizontal direction, rotation direction, and angle of view of the person's face, Determining whether or not it matches the stored shooting conditions;
An output means for outputting the determined determination result;
Including methods.

Images