JP5180630B2 - Monitoring device - Google Patents

Description

  The present invention relates to a monitoring apparatus that performs monitoring by capturing an image of a place where a person to be authenticated is authenticated.

  For example, in order to restrict a person who passes through the entrance / exit, control may be performed such as authenticating the person by an authentication device and opening / closing the entrance / exit according to the authentication result. Furthermore, in this case, in order to monitor a person who attempts to pass through the entrance / exit without being authenticated (an unauthorized passer), an image near the entrance / exit may be captured by a monitoring camera or the like.

In the above case, in order to effectively monitor an unauthorized passerby using a monitoring image captured by a monitoring camera or the like, an authenticated person to be authenticated is selected from persons shown in the monitoring image. It is necessary to identify unauthorized passers. As one of such identification methods, a technique has been proposed in which a person existing in a predetermined range in front of the authentication device is specified as the authentication target when the authentication device authenticates the authentication target ( Patent Document 1).
JP 2004-185484 A

  By the way, a plurality of people come and go near the entrance. In the technology of the above conventional example, in the case where a plurality of persons stand within a predetermined range in front of the authentication device by chance or by unjust intention, it is correctly determined which of the plurality of persons has been authenticated. It may not be possible to specify. In addition, if the shadow of a person, baggage, or a trolley that the person has is reflected in the monitoring image, the person image reflected in the monitoring image cannot be accurately extracted due to the influence of these disturbances, and compared with the person to be authenticated. As a result, a person standing at a position away from the authentication device may be erroneously recognized as a person to be authenticated. In particular, when the authentication device captures the face of the person to be authenticated and analyzes the captured face image to authenticate the person to be authenticated, the person to be authenticated does not need to contact the authentication device. It is possible to receive authentication at a position away from the station. When such an authentication method is used, there is a high possibility that the above problem will occur. Further, in the technology of the conventional example, the information obtained by the authentication device has not been sufficiently utilized to identify the image of the person to be authenticated.

  The present invention has been made in view of the above circumstances, and one of its purposes is that when an image of a place where the person to be authenticated is authenticated is imaged and monitored, it is selected from the captured monitoring images. An object of the present invention is to provide a monitoring device that can specify a person to be authenticated with high accuracy.

  The monitoring device according to the present invention for solving the above-described problem includes an authentication image capturing unit that captures an authentication image including the face of the person to be authenticated, and the face of the person to be authenticated from the authentication image. An authentication unit that detects a face area to be shown and authenticates the person to be authenticated based on an image in the face area; a monitoring image pickup unit that picks up a monitoring image including the person to be authenticated from above; and The face area of the person to be authenticated detected by the authentication unit based on the relationship between the human image extraction means for extracting one or a plurality of human images and the camera parameters of the authentication image capturing unit and the monitoring image capturing unit And a person-to-be-authenticated person specifying means that uses a person image corresponding to the position of the person to be authenticated as a person image.

  In the monitoring apparatus, the person image extraction unit may detect a human head region as a person image.

  Further, in the monitoring device, the authentication-subjected person specifying means includes the face of the authentication-subject person in the monitoring image based on a position where the face area of the authentication-subject person is detected in the authentication image. Then, an estimated range may be calculated, and the person image of the person to be authenticated may be specified using the calculated range.

  Further, in the monitoring device, the authentication-subjected person specifying means further includes the extracted person image based on the size of the face area of the person to be authenticated detected in the authentication image. The person image of the person to be authenticated may be specified.

  Further, in the monitoring apparatus, the monitoring image capturing unit is installed at a position where the position estimated to be the person to be authenticated when the authenticating unit authenticates the person to be authenticated is imaged from above substantially vertically. It is also good to do.

  According to the present invention, using the information on the position of the face of the person to be authenticated in the authentication image captured when authenticating the person to be authenticated, the target is accurately extracted from the person image extracted from the monitoring image. The person image of the certifier can be specified.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  As shown in FIG. 1, the monitoring device 1 according to an embodiment of the present invention includes an authentication device 10, a control unit 11, a storage unit 12, a monitoring image imaging unit 13, an authentication signal receiving unit 14, and an operation. The unit 15 and the output unit 16 are included. In the present embodiment, the monitoring device 1 authenticates a person who wants to pass through the door D, which is an entrance / exit to be monitored, and attempts to pass through the door D by capturing an image around the door D. Monitor people.

  The authentication device 10 is a device for authenticating a person who is going to pass through the door D. As shown in FIG. 1, the authentication image capturing unit 21, the authentication information storage unit 22, the authentication processing unit 23, And an authentication signal transmission unit 24. In addition, each part which comprises the authentication apparatus 10 may be accommodated in one housing, and the image pick-up part 21 for authentication and other parts may each be accommodated in another housing, and may be connected by the communication line. Good. In the following, a person who is authenticated by the authentication device 10 is referred to as a person to be authenticated Pa. In the present embodiment, the authentication device 10 authenticates the person to be authenticated Pa using an image obtained by imaging the face of the person to be authenticated Pa.

  The authentication image capturing unit 21 is an imaging device such as a video camera that captures an authentication image including the face of the person Pa to be authenticated. In order to image the face of the person Pa to be authenticated, for example, the authentication image capturing unit 21 is installed at a height (for example, about 1.5 m) on the wall surface near the door D according to the height of the person. The image capturing direction of the authentication image capturing unit 21 is substantially horizontal with respect to the floor surface. Note that the authentication image capturing unit 21 may continuously capture the authentication image, or may capture the authentication image at the timing when the person to be authenticated Pa presses a predetermined button, for example. The authentication image captured by the authentication image capturing unit 21 is output to the authentication processing unit 23.

  The authentication information storage unit 22 includes, for example, a memory element such as a ROM, and stores authentication information of a person to be authenticated in advance. Specifically, the authentication information storage unit 22 stores authentication information of each of one or more persons (hereinafter referred to as permitted persons) who are permitted to pass the door D. Here, the authentication information is information for distinguishing and identifying a permitted person from other persons, such as a feature amount representing a feature of the permitted person's face.

  The authentication processing unit 23 is configured by an arithmetic device such as a digital signal processor (DSP), for example, and executes authentication processing for authenticating the person Pa to be authenticated. Specifically, the authentication processing unit 23 first detects a face region indicating the face of the person to be authenticated Pa from the authentication image captured by the authentication image capturing unit 21. Here, as a method of detecting the face area of the person Pa to be authenticated from the authentication image, for example, a method described in JP-A-2005-92262 can be used. When a plurality of face areas are detected from the authentication image, the authentication processing unit 23 selects a face area that satisfies a predetermined criterion, such as the largest face area, from the face areas. A face area indicating the face of Pa is selected.

  Further, the authentication processing unit 23 authenticates the person to be authenticated Pa based on the detected image in the face area of the person to be authenticated Pa. That is, by comparing the analysis result obtained by analyzing the image of the detected face area with the authentication information of the permitted person stored in the authentication information storage unit 22, the authenticated person Pa is the permitted person. Judge whether there is. As a method of analyzing the face image and collating with the authentication information, for example, a method described in Japanese Patent Application Laid-Open No. 2007-299186 can be used.

  As a result of the authentication processing, the authentication processing unit 23 outputs a predetermined authentication signal to the authentication signal transmission unit 24 when it is authenticated that the person to be authenticated Pa is a permitted person. This authentication signal includes information indicating that authentication of the person to be authenticated Pa was successful, as well as information on the face area of the person to be authenticated Pa detected from the authentication image (hereinafter referred to as face area information). It is. Specifically, the face area information includes at least position information (position coordinate information or the like) indicating a position where the face area of the person to be authenticated Pa is detected in the authentication image. This position information may be information indicating the center of gravity of the face area, for example, or may be information indicating the center position of a circumscribed rectangle that circumscribes the face area. Further, the face area information may include size information indicating the size of the face area detected in the authentication image (for example, numerical values indicating the width and height of a circumscribed rectangle circumscribing the face area). .

  The authentication signal transmission unit 24 is a communication device such as a LAN interface, for example, and transmits an authentication signal output from the authentication processing unit 23 to the authentication signal reception unit 14 at a timing when the authentication of the user Pa is successful. . The authentication signal transmission unit 24 receives the authentication signal output from the authentication processing unit 23 and outputs a control signal for unlocking the door D to an electric lock (not shown). Thereby, the door D is unlocked only for a predetermined time, and the person to be authenticated Pa can pass through the door D in the meantime.

  The control unit 11 is an arithmetic device such as a CPU or a DSP, for example, and executes various types of information processing according to programs stored in the storage unit 12. In the present embodiment, the control unit 11 analyzes the monitoring image captured by the monitoring image capturing unit 13 and performs processing to output a signal corresponding to the detection of the abnormality to the output unit 16 when the abnormality is detected. . In addition, the specific example of the process which the control part 11 performs in this embodiment is mentioned later.

  The storage unit 12 includes a memory element such as a ROM and a RAM, for example, and stores various programs executed by the control unit 11 and information such as various parameters used by processing executed by the control unit 11. Is done.

  The monitoring image capturing unit 13 is an imaging device such as a video camera that captures a monitoring image including the person to be authenticated Pa from above. In this embodiment, the monitoring image capturing unit 13 continuously captures monitoring images at predetermined time intervals, and sequentially outputs the monitoring images (frame images) captured at each timing to the control unit 11. .

  2A and 2B are diagrams illustrating an example of a state of a space to be monitored by the monitoring device 1 (hereinafter referred to as a monitoring space). FIG. 2A shows a view of the authenticated person Pa as viewed from the side when the authentication apparatus 10 is authenticating the authenticated person Pa, and FIG. 2B shows the same authenticated person. The figure which looked at Pa from the back is shown. As shown in these drawings, the monitoring image capturing unit 13 is installed at a position where it is possible to capture the position where the authenticated person Pa is estimated to exist when the authenticated apparatus 10 authenticates the authenticated person Pa. The imaging direction is downward from the horizontal plane. Thereby, the monitoring image capturing unit 13 captures an image including the same person to be authenticated Pa in an image capturing direction different from the image capturing direction of the authentication image capturing unit 21. FIG. 3 is a diagram illustrating an example of a monitoring image captured by the monitoring image capturing unit 13. In the example of FIG. 3, the door D, the authentication image capturing unit 21, and the person to be authenticated Pa are shown in the monitoring image. Here, by installing the monitoring image capturing unit 13 substantially vertically above the position where it is estimated that the authenticated user Pa exists, in the monitoring image captured by the monitoring image capturing unit 13, It is possible to make it difficult for the person image to overlap with the person images of other persons existing around the person image.

  The authentication signal receiving unit 14 is a communication device such as a LAN interface, for example, and receives the authentication signal transmitted from the authentication signal transmitting unit 24 of the authentication device 10 and stores it in a predetermined temporary storage area. Information included in the authentication signal stored in the temporary storage area is read by the control unit 11.

  The operation unit 15 is a user interface such as a touch panel display, for example, and is used by the administrator of the monitoring device 1 to input information such as various parameters. Information input by the administrator from the operation unit 15 is stored in the storage unit 12 and used for processing executed by the control unit 11. As a specific example, the administrator inputs the camera parameters of the authentication image capturing unit 21 and the monitoring image capturing unit 13 from the operation unit 15. The camera parameters include an installation position of the authentication image capturing unit 21 (hereinafter referred to as an authentication image capturing position C1) and an imaging direction and an installation position of the monitoring image capturing unit 13 (hereinafter referred to as a monitoring image capturing position C2) in real space. ) And an imaging parameter including an imaging direction, an angle of view of the authentication image capturing unit 21, the number of pixels and lens distortion, an angle of view of the monitoring image capturing unit 13, an imaging parameter including the number of pixels and lens distortion, and the like. Further, the administrator inputs information indicating the position of the door D in the monitoring image and the authentication image capturing position C1 in the monitoring image from the operation unit 15.

  The output unit 16 is an interface for outputting a processing result by the control unit 11 to an external device or the like. The output unit 16 is connected to, for example, a sound output unit such as a speaker or a buzzer, and outputs an abnormal signal that causes the sound output unit to execute a warning sound in response to an instruction from the control unit 11. Thereby, the monitoring apparatus 1 can sound a warning sound when abnormality is detected by the process of the control unit 11. The output unit 16 may include a communication interface that transmits an abnormal signal to an external center device or the like. Thereby, when abnormality is detected by the process of the control part 11, the monitoring apparatus 1 can notify the signal which notifies the detection of abnormality to an external monitoring center. In this case, the output unit 16 may be an interface device common to the authentication signal receiving unit 14.

  Hereinafter, specific examples of functions realized by the monitoring device 1 when monitoring is performed using the monitoring image captured by the monitoring image capturing unit 13 will be described. As shown in FIG. 4, the monitoring device 1 functionally includes a person image extraction means 31, a head detection means 32, an authenticated person specifying means 33, a tracking means 34, and an abnormality determination means 35. Consists of including. Further, the authenticated person specifying means 33 includes an authenticated person existence range calculating means 33a and an authenticated person image selecting means 33b. These functions are realized when the control unit 11 executes a program stored in the storage unit 12.

  The person image extraction unit 31 extracts one or a plurality of person images from the monitoring images for each of the monitoring images captured every predetermined time by the monitoring image capturing unit 13. Specifically, for example, the person image extraction unit 31 stores a monitoring image obtained by imaging a monitoring space in a state where no person is present in the storage unit 12 as a background image, and the background image, the monitoring image, and the like. To extract a difference pixel. Then, among the difference pixel group configured to include the difference pixels adjacent to each other, a difference pixel group having a shape satisfying a predetermined condition and a size greater than or equal to a predetermined value is extracted as a human image representing a person. In addition to the above method, the person image extraction means 31 may extract a person image by combining various image processing such as edge detection.

  The person images are extracted from the monitoring images by the number corresponding to the number of persons in the monitoring space at the timing when the monitoring images are captured. When there is a person to be authenticated Pa who has been authenticated by the authentication device 10, a person image of the person to be authenticated Pa (hereinafter referred to as an image of the person to be authenticated) is extracted from the monitoring image by the person image extraction means 31. It will be. The person image extraction unit 31 extracts a person image from the captured monitoring image even when the person to be authenticated Pa does not exist in the monitoring space and the authentication signal is not transmitted from the authentication device 10. Execute. Information about the person image extracted by the person image extraction unit 31 is stored in the storage unit 12 and used for the tracking process of the person image by the tracking unit 34 described later.

  The head detection unit 32 detects a head region indicating a human head from each of one or more person images extracted by the person image extraction unit 31. Specifically, for example, the head detection unit 32 performs image processing such as edge detection, Hough transform, and pattern matching, so that an area having a human head characteristic (for example, an image area of each person image) (for example, A substantially circular region having a size within a predetermined range) is detected as a head region, and information indicating the position and size of the detected head region is output. The person image extracted by the person image extraction means 31 may not necessarily correspond to the actual position where the person stands due to the influence of the shadow of the person, the baggage held by the person, the carriage, or the like. Even in such a case, the position in the monitoring image corresponding to the person's face existing in the monitoring space can be accurately detected by detecting the head region further from the human image by the head detecting means 32. Can be identified well. In the present embodiment, the head detection means 32 is used to increase the accuracy, but the head detection means 32 may be omitted if the human image extraction means 31 can obtain sufficient accuracy.

  The person-to-be-authenticated identification unit 33 extracts a plurality of person images by the person image extraction unit 31 from the monitoring image captured by the monitoring image capturing unit 13 at the timing when the authentication apparatus 10 authenticates the person to be authenticated Pa. If it is determined, the image of the person to be authenticated is specified from the extracted person image. At this time, the person-to-be-authenticated specifying means 33 specifies a person image extracted at a position in the authentication image that has a corresponding relationship with the position where the face area is detected in the authentication image as the person-to-be-authenticated image. The correspondence between the position coordinates in the authentication image and the position coordinates in the monitoring image is derived from the camera parameters described above. The derivation rule is stored in advance in the storage unit 12 as a program of the authenticated person specifying means 33, and the authenticated person specifying means 33 applies the camera parameters stored in the storing unit 12 to the rule prior to the operation. Deriving correspondence. As a result, the person-to-be-authenticated identification means 33 simply identifies the person image located in front of the authentication image capturing unit 21 as the person-to-be-authenticated image when a plurality of person images are shown in the monitoring image. In comparison, the person-to-be-authenticated can be identified with higher accuracy.

  Here, a specific example of the method by which the person to be authenticated specifying unit 33 specifies the image of the person to be authenticated will be described. First, the person-to-be-authenticated person existence range calculation means 33a is based on the face area information included in the authentication signal received from the authentication device 10 and is the area in which the face of the person to be authenticated Pa is estimated to exist in the monitoring image (hereinafter, the object Certifier existence range). Since the face area information used here is confirmed by the authentication apparatus 10 to be the person to be authenticated Pa, the image of the person to be authenticated can be specified with certainty. In addition, since the person-to-be-authenticated person image is specified from the information of the face area narrowed down to one, it is efficient.

  For example, the person-to-be-authenticated person existence range calculation unit 33a projects the position in the authentication image where the face area is detected in the face area information included in the authentication signal onto the imaging surface of the monitoring image, thereby Calculate the range. Specifically, the person-to-be-authenticated person presence range calculating unit 33a includes the position coordinates of the face area in the authentication image, the authentication image capturing position C1 stored in the storage unit 12, and the imaging direction of the authentication image capturing unit 21. And a straight line in the real space connecting the authentication image capturing position C1 and the position of the face of the person Pa to be authenticated. It is estimated that the face position of the person to be authenticated Pa exists in a range along this straight line. Therefore, the authentication subject existence range calculation unit 33a projects the calculated straight line onto the imaging surface of the monitoring image imaging unit 13 using information regarding the monitoring image imaging position C2 and the imaging direction of the monitoring image imaging unit 13. Thus, the epipolar line EP is calculated, and the range along the epipolar line EP is set as the authentication subject existence range. The direction in which the face of the person to be authenticated Pa exists in the monitoring image can be limited by the presence range of the person to be authenticated. A specific example of the process in which the person-to-be-authenticated person presence range calculating unit 33a calculates the epipolar line EP will be described later.

  Next, the person-to-be-authenticated person image selection means 33b uses the information on the person-to-be-authenticated person presence range calculated by the person-to-be-authenticated person calculation range 33a to authenticate the person to be authenticated from the person images extracted by the person image extraction means 31. Select a person. For example, the person-to-be-authenticated image selection means 33b selects, as a person-to-be-authenticated person image, a person image that is present at a position closest to the authentication image capturing position C1 among the person images that are present in the person-to-be-authenticated person existence range.

  At this time, the person-to-be-authenticated image selecting means 33b further selects the person-to-be-authenticated image based on the position of the head region detected by the head detecting means 32 for each of the person images extracted by the person image extracting means 31. You may specify. It is considered that the head of the person to be authenticated Pa appears at a position in the monitoring image corresponding to the position of the face of the person to be authenticated Pa captured by the authentication image capturing unit 21. Therefore, the authentication target person's existence range obtained by projecting the position of the face area in the authentication image onto the imaging surface of the monitoring image is compared with the position of the head area of each person image detected from the monitoring image, and the authentication target person is compared. By identifying the person image from which the head region located in the existence range is detected as the person-to-be-authenticated image, as compared with the case where the person-to-be-authenticated person image is simply identified using the center of gravity position of the entire person image, The person-to-be-authenticated image can be specified with higher accuracy.

  Further, the person-to-be-authenticated identification means 33 may identify the person-to-be-authenticated image based on not only the position of the face area in the authentication image but also the size of the face area. The size of the face of the person to be authenticated Pa in the authentication image changes according to the distance from the position where the person to be authenticated Pa stands to the image pickup position C1 for authentication. Therefore, the distance range from the authentication image capturing position C1 where the face of the person to be authenticated Pa is present in the monitoring image is estimated from the size of the face area in the authentication image, and each person image or each detected from the monitoring image is detected. Compare the position of the head area of the person image with the distance range and specify the person image located in the distance area as the person-to-be-authenticated image, or the person whose head area located in the distance range is detected By specifying the image as the person-to-be-authenticated image, the person-to-be-authenticated image can be specified more accurately.

  Specifically, for example, the authentication image capturing unit 21 captures a person having a standard-sized face from various distances, and records the size of the face area at that time, thereby determining the size of the face area. Formulas, conversion tables, and the like corresponding to the correspondence between the distance from the authentication image capturing unit 21 to the face of the person Pa to be authenticated are determined and stored in the storage unit 12 in advance. The person-to-be-authenticated person existence range calculating means 33a applies a parameter indicating the size of the face area to this mathematical formula or conversion table, and the position where the face of the person to be authenticated Pa is estimated to exist from the authentication image capturing position C1. The estimated distance range is calculated. Using this estimated distance range information, the authenticated person presence range calculation means 33a further limits the authenticated person presence range calculated based on the position of the face area in the authentication image, and narrows down the range. Thus, for example, even if a plurality of persons stand in the same direction as viewed from the authentication image capturing position C1 and the person standing behind among the plurality of persons is the person to be authenticated Pa, it is extracted from the monitoring image. The person-to-be-authenticated person image can be specified with high accuracy from the obtained person image.

  The tracking unit 34 performs tracking processing of a person image as described below every time the person image extraction unit 31 extracts a person image for each of the monitoring images captured by the monitoring image capturing unit 13 at predetermined time intervals. . That is, the tracking unit 34 extracts each of the person images extracted by the person image extracting unit 31 from the newly captured monitoring image by using the person image extracting unit 31 from the monitoring image captured at a past time point from the monitoring image. The movement path in the monitoring space of the person corresponding to the person image is traced by comparing with the person image stored in the storage unit 12. Specifically, the tracking unit 34 uses the position of each person image extracted from the new monitoring image and the amount related to the image feature (for example, a color histogram) as the position of the person image extracted from the previously captured monitoring image. The person images representing the same person are associated with each other by comparing with the amount related to the image feature. And the time series information which shows the position in the monitoring image imaged according to the time series of the person image mutually associated as a person image showing the same person is output as movement path information which shows the movement path | route of the said person.

  Here, in order to distinguish the person image identified as the person-to-be-authenticated image by the person-to-be-authenticated person specifying unit 33, the tracking unit 34, with respect to the movement path information of the person, Associate attribute information indicating that the image is a person to be authenticated. As a specific example, the tracking unit 34 newly adds a person image extracted from a new monitoring image for a person image determined to have no person image indicating the same person in the previously captured monitoring image. First, attribute information indicating that the person is not the person to be authenticated Pa is associated as a person image of the person who has entered the monitoring space. Thereby, each person in the monitoring space to be tracked by the tracking unit 34 is always managed as not initially the person to be authenticated Pa. Then, at the timing when the authentication signal is received from the authentication device 10, for the person image specified as the person to be authenticated by the person to be authenticated specifying unit 33, this attribute information is a value indicating that it is the person to be authenticated Pa. Update to As a result, among the persons to be tracked by the tracking unit 34, the person specified as the person to be authenticated Pa is tracked separately from other persons thereafter.

  The abnormality determination unit 35 determines whether or not there is an unauthorized movement that matches a predetermined condition based on the movement route information of each person in the monitoring space output by the tracking unit 34. As a specific example, movement route information indicating that a person associated with attribute information indicating that the person is not an authenticated person Pa has moved from outside the area corresponding to the door D into the area corresponding to the door D in the monitoring image. Is output by the tracking unit 34, the abnormality determination unit 35 determines that there is an unauthorized passage by a person who is not permitted to pass through the door D, and outputs an abnormality signal indicating detection of abnormality to the output unit 16. Output. Thereby, the monitoring apparatus 1 can sound a warning sound or notify the center when an unauthenticated person tries to pass through the door D.

  Here, a specific example of the process of calculating the epipolar line EP by the to-be-authenticated person presence range calculating unit 33a will be described.

  FIGS. 5A and 5B are examples of a monitoring image captured by the monitoring image capturing unit 13 and an authentication image captured by the authentication image capturing unit 21 when there are a plurality of persons in the monitoring space. FIG. FIG. 5A shows an example of a monitoring image, in which three persons P1, P2, and P3 are shown. Further, the persons P1 and P2 are shown in the authentication image shown in FIG. 5B, and the person P3 is behind the person P1 and is not shown in the authentication image. Here, it is assumed that the authentication apparatus 10 selects the person P1 as the person to be authenticated Pa, detects the face area F, and performs authentication. At this time, the authentication signal transmitted by the authentication signal transmission unit 24 includes position information indicating the position of the face area F of the person to be authenticated Pa in the authentication image. Here, the coordinate of the center position of the face area F in units of pixels is (u1, v1) with the image center of the authentication image as the origin.

  In the following description, it is assumed that parameters described below are stored in the storage unit 12 as parameters relating to the imaging conditions of the authentication image capturing unit 21 and the monitoring image capturing unit 13. That is, the number of pixels from the image center of the authentication image to the edge of the image is defined as horizontal w1 and vertical h1, the horizontal field angle of the authentication image capturing unit 21 is α1, and the vertical field angle is β1. Further, the number of pixels from the image center of the monitoring image to the edge of the image is horizontal w2 and vertical h2, and the horizontal field angle of the monitoring image imaging unit 13 is α2, and the vertical field angle is β2.

  Furthermore, the storage unit 12 also stores information indicating the positional relationship between the authentication image capturing position C1 and the monitoring image capturing position C2, and the image capturing direction of each image capturing unit. As a specific example, FIG. 6 shows a positional relationship between the authentication image capturing position C1 and the monitoring image capturing position C2 in real space. In this figure, the authentication image capturing position C1 is the origin of the coordinate system. Further, the image pickup direction of the authentication image pickup unit 21 is the Y axis positive direction, the vertically upward direction is the Z axis positive direction, the direction orthogonal to both the Y axis and the Z axis, and the direction toward the right side of the authentication image is the X axis positive direction. Yes. Furthermore, the position coordinates of the monitoring image imaging position C2 are (0, C2y, C2z), and the imaging direction of the monitoring image imaging unit 13 is the Z-axis negative direction.

なお、被認証者存在範囲算出手段３３ａは、ｕ１，ｖ１，ｗ１，ｈ１の各値を、記憶部１２に記憶されている認証用画像撮像部２１のレンズ歪みに関するパラメタに基づいて歪み成分を除去した値に換算してから、演算を行ってもよい。 By applying a pinhole camera model using each of these pieces of information, the authenticated person presence range calculating unit 33a can detect the center position of the face of the authenticated person Pa with respect to the authentication image capturing position C1 in the real space (hereinafter, The direction of the face position Fc) is calculated. Specifically, the X-axis direction component θu1 and the Z-axis direction component θv1 of the angle formed by the vector Vf from the origin C1 toward the face position Fc and the imaging direction (Y-axis positive direction) of the authentication image capturing unit 21 are: Each can be calculated by the following formula.

The person-to-be-authenticated person existence range calculating unit 33a removes distortion components from the values of u1, v1, w1, and h1 based on parameters relating to lens distortion of the authentication image capturing unit 21 stored in the storage unit 12. The calculation may be performed after conversion to the calculated value.

  Thus, the vector Vf is expressed as (T · tan θu1, T, T · tan θv) using a predetermined constant T. This vector Vf represents the direction of the face position Fc viewed from the authentication image capturing position C1 in the real space. Here, considering a virtual point Q whose position coordinates are represented by (T · tan θu1, T, T · tan θv1), it is estimated that the face position Fc exists on a straight line connecting the origin C1 and the virtual point Q. . In the following, the position coordinates of the virtual point Q are expressed as (Qx, Qy, Qz).

さらに、このθｕ２及びθｖ２を用いて、仮想点Ｑを監視画像上に射影した位置の位置座標は、監視画像の画像中心を原点として、画素単位で

で表される。この位置座標と、予め記憶部１２に記憶されている監視画像上における認証用画像撮像位置Ｃ１と、を結ぶ直線が、エピポーラ線ＥＰとなる。このとき、監視画像上における仮想点Ｑの位置が認証用画像撮像位置Ｃ１に近いと、算出誤差による影響が大きくなる。そこで、仮想点Ｑが認証用画像撮像位置Ｃ１と所定距離以上離れるように、前述した定数Ｔを定めることとしてもよい。また、被認証者存在範囲算出手段３３ａは、得られた直線を監視画像撮像部１３のレンズ歪みに関するパラメタに基づいて補正した直線を、エピポーラ線ＥＰとしてもよい。 Furthermore, the to-be-authenticated person presence range calculating unit 33a calculates the epipolar line EP on the monitoring image by projecting the virtual point Q onto the imaging surface of the monitoring image imaging unit 13. Specifically, an angle X formed by a vector (Qx, Qy-C2y, Qz-C2z) from the monitoring image imaging position C2 toward the virtual point Q and the imaging direction (Z-axis negative direction) of the monitoring image imaging unit 13 is formed. The axial direction component θu2 and the Y-axis direction component θv2 can be calculated by the following formulas, respectively.

Furthermore, using these θu2 and θv2, the position coordinates of the position where the virtual point Q is projected onto the monitoring image are in pixel units with the image center of the monitoring image as the origin.

It is represented by A straight line connecting the position coordinates and the authentication image capturing position C1 on the monitoring image stored in the storage unit 12 in advance is an epipolar line EP. At this time, if the position of the virtual point Q on the monitoring image is close to the authentication image capturing position C1, the influence of the calculation error increases. Therefore, the above-described constant T may be determined so that the virtual point Q is separated from the authentication image capturing position C1 by a predetermined distance or more. Further, the person-to-be-authenticated person existence range calculating unit 33a may use a straight line obtained by correcting the obtained straight line based on a parameter relating to lens distortion of the monitoring image capturing unit 13 as an epipolar line EP.

  An example of the epipolar line EP calculated by the method described above is shown by a broken line in FIG. In the example of FIG. 5A, for example, when the person image extraction unit 31 extracts a region including a person's shadow as a person image, the person image present at the position closest to the authentication image capturing position C1 is It becomes a person image of the person P2. However, the head region H2 of the person P2 is not on the epipolar line EP, while the head region H1 of the person P1 exists on the epipolar line EP. Therefore, by using the range along the epipolar line EP as the authenticated person existence range, the authenticated person image selecting means 33b specifies the person image of the person P1 instead of the person P2 as the authenticated person image. In the example of FIG. 5A, not only the person P1 but also the head region H3 of the person P3 exists on the epipolar line EP. In this case, for example, as described above, the person-to-be-authenticated image selection means 33b selects the person image of the person P1 based on the information on the distance from the authentication image capturing position C1 to the head area of each person in the monitoring image. It is specified as the person to be authenticated.

  Hereinafter, an example of the flow of processing executed by the control unit 11 of the monitoring device 1 according to the present embodiment will be described based on the flowchart of FIG.

  First, when the power of the monitoring device 1 is turned on, the control unit 11 executes a predetermined initialization process (S1). At this time, the monitoring image captured by the monitoring image capturing unit 13 is stored in the storage unit 12 as a background image. Furthermore, the control unit 11 determines whether or not predetermined parameters necessary for subsequent processing, such as information regarding the imaging conditions of the authentication image capturing unit 21 and the monitoring image capturing unit 13, are stored in the storage unit 12 (S2). ). If not stored, the control unit 11 presents a message prompting the input of these parameters to the administrator via the operation unit 15, and accepts the input of the parameter from the administrator (S3).

  Thereafter, each time the monitoring image capturing unit 13 captures a new monitoring image at a predetermined time interval, the processing described below is repeatedly executed. That is, first, the person image extraction means 31 of the control unit 11 acquires a new monitoring image captured by the monitoring image capturing unit 13 (S4). Then, the monitoring image acquired in S4 is compared with the background image to extract a person image (S5). Further, the control unit 11 determines whether or not one or more person images have been extracted in the process of S5 (S6). If no person image is extracted, the monitoring image acquired in S4 is stored in the storage unit 12 as a new background image (S7), and the process returns to S4 to process the next monitoring image. Here, the reason why the monitoring image is updated when the person image is not extracted is to cope with the case where the background image changes with the passage of time.

  On the other hand, if it is determined in S6 that one or more person images have been extracted, the head detecting means 32 and the person-to-be-authenticated person specifying means 33 of the control unit 11 specify the person-to-be-authenticated image from the extracted person images. Authentication subject identification processing is performed (S8). Details of the authentication-subjected person identification process will be described later.

  Subsequently, the tracking unit 34 of the control unit 11 performs the tracking process of the person image extracted in S5 (S9). Specifically, in this tracking process, for each of the person images extracted in S5, it is determined whether or not the person image indicates the same person as the person image extracted up to the previous time, and according to the determination result, This is a process of adding the position of the newly extracted person image to the movement route information of each person already recorded in the storage unit 12. At this time, the tracking unit 34 associates attribute information indicating that the user is the authenticated person Pa with the movement path information corresponding to the person image identified as the authenticated person image by the process of S8.

  Further, the abnormality determination unit 35 of the control unit 11 refers to the movement route information obtained by the process of S9, and determines whether there is an unauthorized traffic that a person who has not been authenticated by the authentication device 10 tries to pass through the door D. (S10). If unauthorized traffic is detected as a result of the determination, the abnormality determination means 35 outputs an abnormality signal to the output unit 16 (S11). In any case where unauthorized traffic is not detected in S10 and when abnormal traffic is detected and an abnormal signal is output, the process returns to S4 and the process for a new monitoring image is repeated.

  Next, a specific example of the flow of the person-to-be-authenticated identification process of S8 in the flow of FIG.

  First, the control unit 11 determines whether or not the authentication signal receiving unit 14 has newly received an authentication signal from the authentication device 10 (S21). The authentication device 10 operates independently even while the control unit 11 repeatedly executes the flow from S4 to S11 in the flow of FIG. 7 described above. If it is determined that the person is permitted to pass, an authentication signal for notifying the controller 11 of the fact is output. Here, when it is determined that the authentication signal is received in S21, the monitoring image to be processed is an image captured at the timing when the authentication by the authentication device 10 is performed. It is determined that the person to be authenticated Pa is shown. In that case, the control part 11 performs the process after S22 mentioned later. On the other hand, if it is determined in S21 that the authentication signal has not been received, since it is determined that the newly authenticated person Pa is not reflected in the monitoring image, the process ends.

  Next, the head detection means 32 of the control unit 11 detects the head region for each of the person images extracted in the process of S5 in the flow of FIG. 7 (S22). Thereafter, the person-to-be-authenticated person existence range calculating unit 33a of the control unit 11 creates a straight line connecting the face position Fc of the person to be authenticated Pa and the authentication image capturing position C1 based on the position information of the face area included in the authentication signal. The epipolar line EP projected on the monitoring image is calculated as the authentication subject existence range (S23). Further, based on the size information of the face area included in the authentication signal, an estimated distance range from the authentication image capturing position C1 to the position where the face position Fc of the person to be authenticated Pa exists is calculated (S24).

  Subsequently, the to-be-authenticated person image selection unit 33b of the control unit 11 selects a head region that intersects the epipolar line EP calculated in S23 among the head regions detected in S22 (S25). Then, it is determined whether there is only one selected head region (S26). If it is determined in S26 that there are two or more head regions that intersect with the epipolar line EP, the person-to-be-authenticated image selection means 33b further determines that the distance from the authentication image capturing position C1 among these head regions. A head region included in the estimated distance range calculated in S24 is selected (S27). Then, it is determined whether there is only one head region selected again (S28). If it is determined in S28 that there are two or more head regions that meet the conditions, the person-to-be-authenticated image selection means 33b is located in the head region closest to the authentication image capturing position C1. 1 is selected (S29).

  If it is determined in S26 or S28 that only one head region matches the condition, or if one head region is selected by the processing in S29, the authenticated person image selecting means 33b selects this head region. The person image including the head region thus specified is specified as the authentication subject image (S30). Thereby, since the person-to-be-authenticated person image has been specified, the process is terminated.

  In the above flow, the head region intersecting with the epipolar line EP always exists. However, there may be no head region intersecting with the epipolar line EP due to a detection error of the head region. possible. In this case, of the head region having the shortest distance to the epipolar line EP and the head region having a distance to the epipolar line EP of a predetermined value or less, the estimated distance calculated from the authentication image capturing position C1 in S24 A person image including a head region within the range may be specified as an authenticated person image. Also in the processing of S29, the head closest to the threshold of the estimated distance range calculated in S24 is not the head region present at the position closest to the authentication image capturing position C1, but the distance from the authentication image capturing position C1. A partial area may be selected.

  According to the present embodiment described above, the monitoring apparatus 1 uses the information related to the face position of the person to be authenticated Pa when the authentication apparatus 10 authenticates the person to be authenticated Pa to be authenticated in the monitoring image. By specifying the person image, a plurality of persons in the monitoring image are included in the monitoring image as compared to the case where the image of the person to be authenticated is specified based only on the positional relationship between the authentication image capturing unit 21 and the person image in the monitoring image. Even if the person is shown, the person-to-be-authenticated can be identified with high accuracy.

  The embodiment of the present invention is not limited to the above-described embodiment. For example, in the above description, the authentication apparatus 10 performs authentication of the person to be authenticated Pa by analyzing the feature amount of the face image. However, the present invention is not limited to this, and an image including the face of the person to be authenticated Pa is captured. If authentication is performed, other authentication methods may be used. Specifically, for example, the authentication apparatus 10 may include at least a means for detecting a face area, and may authenticate the person to be authenticated Pa based on an image of a partial area of the face such as an iris or retina pattern.

  In the above description, the epipolar line EP in the monitoring image is calculated from the position coordinates in the authentication image as a rule for deriving the correspondence between the position coordinates in the authentication image and the position coordinates in the monitoring image. Although the rule to do was illustrated, you may use rules other than this. Specifically, for example, a region in the monitoring image that has a correspondence relationship with a plurality of preset small regions in the authentication image based on camera parameters is obtained, and the storage unit 12 uses the obtained correspondence table as a rule. Can be remembered. In this case, the person-to-be-authenticated person existence range calculating unit 33a refers to the table to obtain a corresponding area in the monitoring image that has a corresponding relationship with the small area in the authentication image including the position where the face area is detected. Derived as the existence range.

  In the above description, the person-to-be-authenticated specifying unit 33 calculates the position in the monitoring image corresponding to the position of the face area by projecting the position of the face area in the monitoring image. May be used. Specifically, each person image extracted in the monitoring image is projected to the real space, and the position of the face area in the authentication image is projected to the real space, and the person who intersects the position of the face area in the real space The image may be specified as the image of the person to be authenticated. Further, each person image extracted in the monitoring image may be projected into the authentication image, and the person image projected at the position of the face area may be specified as the authentication target person image.

It is a block diagram showing the schematic structure of the monitoring apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the mode of the monitoring space. It is a figure which shows an example of the monitoring image imaged by the monitoring image imaging part. It is a functional block diagram showing the example of a function implement | achieved by the monitoring apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the monitoring image and the image for authentication in case a some person exists in the monitoring space. It is a figure which shows an example of the positional relationship of the image pick-up position for authentication in the real space, and the monitoring image pick-up position. It is a flowchart which shows an example of the flow of the process which the monitoring apparatus which concerns on embodiment of this invention performs. It is a flowchart which shows an example of the flow of a to-be-authenticated person specific process.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Monitoring apparatus, 10 Authentication apparatus, 11 Control part, 12 Storage part, 13 Monitoring image imaging part, 14 Authentication signal receiving part, 15 Operation part, 16 Output part, 21 Authentication image imaging part, 22 Authentication information storage part, 23 Authentication processing unit, 24 authentication signal transmission unit, 31 person image extraction unit, 32 head detection unit, 33 person-to-be-authenticated identification unit, 33a user-existing person existence range calculation unit, 33b user-to-be-authenticated image selection unit, 34 tracking unit, 35 Abnormality determination means.

Claims (5)

An authentication image capturing unit that captures an authentication image including the face of the person to be authenticated;
An authentication unit that detects a face area indicating the face of the person to be authenticated from the authentication image, and authenticates the person to be authenticated based on an image in the face area;
A monitoring image capturing unit that captures a monitoring image including the person to be authenticated from above;
Person image extraction means for extracting one or more person images from the monitoring image;
From the relationship between camera parameters of the authentication image capturing unit and the monitoring image capturing unit, the authentication is performed for each face area detected by the authentication unit among the human images in the monitoring image extracted by the human image extracting unit. the figures corresponding to the position in use image and the authenticated person identifying means for identifying a person image of the person to be authenticated,
A monitoring device comprising:   The monitoring apparatus according to claim 1, wherein the person image extraction unit detects a human head region as a person image. The monitoring device according to claim 1 or 2,
The authenticated person specifying means calculates a range in which the face of the authenticated person is estimated to exist in the monitoring image based on a position where the face area of the authenticated person is detected in the authentication image. And a person image of the person to be authenticated is specified using the calculated range. The monitoring device according to any one of claims 1 to 3,
The authentication-subjected person specifying means further extracts the person image of the person to be authenticated from the extracted person images based on the size of the face area of the person to be authenticated detected in the authentication image. A monitoring device characterized by specifying. The monitoring device according to any one of claims 1 to 4,
The monitoring image capturing unit is installed at a position where the position where the authenticated person is estimated to exist when the authenticating unit authenticates the authenticated person is picked up from substantially vertically above. apparatus.

Images