JP2012237569A - Person management device and person management method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To determine whether person's appearance is suitable or not without requiring manual operation.SOLUTION: A person management device 1 includes: image acquisition means 11 for acquiring a photographed image; storage means 12 for storing information of a detection target; face detection means 13 for detecting a face of a person from the photographed image acquired by the image acquisition means 11; area setting means 14 for setting an area in which the detection target should exist based on the coordinates of the face detected by the face detection means 13; determination means 15 for determining whether or not the detection target exists in the area set by the area setting means 14 and whether or not the detection target is suitably equipped; and control means 16 for performing processing corresponding to a determination result of the determination means 15.

Description

  The present invention relates to a person management apparatus and a person management method.

  When entering or leaving a specific facility or room, it may be necessary to manage the appearance of the entrance / exit. For example, nuclear power plants and nuclear fuel processing facilities are divided into multiple controlled areas, and those who enter and leave are appropriately equipped according to the nature of the controlled area. Is required. For this reason, the entrance / exit of equipment is checked at the entrance / exit of each management area.

  Patent Document 1 discloses a person management device that takes a picture of a person entering and leaving, and compares the photographed image with a reference image acquired in advance to check the appearance of clothes and the like without human intervention. . Here, the reference image is an image obtained by photographing a photographer's appropriate dressing and inappropriate dressing in advance.

Japanese Patent Laying-Open No. 2005-030861

However, in the invention according to Patent Document 1, it is necessary to prepare a reference image for each person to be determined for each check item.
Specifically, if the appropriate dressing is “wearing a helmet, not wearing extravagant clothes, and not bearded”, the person must wear a helmet and wear the appropriate range of clothes. Images of shaving, wearing a helmet and wearing the appropriate range, but bearded, wearing a helmet and shaving in an excessive range, wearing a helmet, wearing an excessive range Dressed but bearded image, not wearing a helmet, shaving the appropriate range of clothing, not wearing a helmet, appropriate range of clothing but bearded image, helmet Like an image that is not worn and shaves with an excessive range of clothes, an image that is not worn with a helmet and is an excessive range of clothes but bearded Image 2 8 (sheets) of 3 (item) squared (street) is required for each registered person.
Therefore, when the number of check items is increased, the necessary reference images increase exponentially, which makes it difficult to prepare images. Further, when the reference image increases, a search is performed from a large number of images at the time of determination, and there is a problem that processing time by software increases.

A person management apparatus according to the present invention is a person management apparatus that checks whether or not a detection target is appropriately equipped based on a photographed image obtained by photographing a person, and that obtains the photographed image. Based on the coordinates of the face detected by the face detection means, the face detection means for detecting the face of the person from the captured image acquired by the image acquisition means, the storage means for storing the information of the detection object A region setting means for setting a region where the detection target object should exist, and a determination is made as to whether or not the detection target object exists in the region set by the region setting means and the detection target object is appropriately equipped Determination means, and control means for performing processing according to the determination result of the determination means.
As a result, the person management device can determine whether or not the detected article is appropriately equipped without preparing a reference image for each combination of the person to be determined and the appearance, and can perform an operation based on the determination result. .

The person management method according to the present invention acquires a captured image including a person, detects a person's face from the acquired captured image, and stores information in advance based on the coordinates of the detected face. An area where the detection target object should be present is set, it is determined whether or not the detection target object exists in the set area, and processing according to the determination result is performed.
Accordingly, it is possible to determine whether or not the detected article is appropriately equipped without preparing a reference image for each combination of the person to be determined and the appearance, and to perform an operation based on the determination result.

  It is possible to determine whether a person is appropriately dressed without human intervention.

It is a figure for every functional block of the person management device concerning Embodiment 1. FIG. It is the figure which showed the structural article of the person management apparatus concerning Embodiment 1. FIG. FIG. 3 is a diagram of a file configuration in the storage device according to the first embodiment. It is a figure of the equipment image database concerning Embodiment 1. FIG. It is a figure of the warning history file concerning Embodiment 1. FIG. FIG. 6 is a diagram of a face detection history file according to the first embodiment. It is a figure of the processing flow of the person management apparatus concerning Embodiment 1. FIG. It is a figure showing the relationship between the face image concerning Embodiment 1, and a helmet detection object area | region. It is a figure of the detection field at the time of helmet wearing concerning Embodiment 1. It is a figure of the detection area at the time of helmet non-wearing concerning Embodiment 1. FIG. It is the figure which showed the structural article of the person management apparatus concerning Embodiment 2. FIG. FIG. 6 is a diagram illustrating an arrangement of photographing apparatuses according to a second embodiment. FIG. 6 is a diagram illustrating a shooting state of the first shooting apparatus according to the second embodiment. FIG. 6 is a diagram illustrating a photographing state by a second photographing apparatus according to the second embodiment.

Embodiment 1
Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram illustrating the person management apparatus according to the first embodiment for each functional block. The person management apparatus 1 includes an image photographing unit 11, a storage unit 12, a face detection unit 13, an area setting unit 14, a determination unit 15, and a control unit 16. The person management apparatus 1 will be described as an entry / exit management apparatus that manages opening / closing of a person by controlling opening / closing of a door (entrance / exit gate).

  FIG. 2 is a diagram showing the constituent articles of the person management device 1. The person management device 1 includes an image capturing device 101, a storage device 102, a CPU (Central Processing Unit) 103, a RAM (Random Access Memory) 104, an input device 105, a display device 106, a communication device 107, A warning device 108 and an entry / exit control device 109 are provided. Each component is connected to each other via a bus line.

  The image photographing means 11 is constituted by an image photographing device 101. The image capturing apparatus 101 is a digital video camera using, for example, a CCD image sensor or the like as an image sensor. Typically, the image capturing apparatus 101 captures the face of a person entering / exiting from the front, and transmits the captured image to the CPU 103 or the RAM 104. In the following description, it is assumed that the image capturing apparatus 101 captures one person from the front. Note that the image acquired by the image capturing apparatus 101 may be either a still image or a moving image.

  The storage unit 12 stores data related to a detection target that is a detection target and data related to a person. The storage unit 12 includes a storage device 102.

The storage device 102 has a program storage area and a data storage area, and is configured by a magnetic, optical, semiconductor memory, or a drive system thereof. For example, the storage device 102 may have a configuration in which a removable storage medium such as a CD-ROM or DVD can be mounted in addition to a fixed memory such as a hard disk.
FIG. 3 shows a list of files provided in the storage device 102. In the storage device 102, an equipment image database 21, a warning history file 22, and a face detection history file 23 are provided.

FIG. 4 is a diagram showing the contents of the equipment image database 21. For example, the equipment image database 21 stores an image ID, equipment location, warning content when not detected, remarks, and the like for each of the captured images acquired by the image capturing apparatus 101. Note that the equipment image database 21 stores information such as a beard and hair as well as information to be detected such as a helmet and glasses. In the equipment image database 21, relative coordinates from the face are set for each equipment.
FIG. 5 is a diagram showing the contents of the warning history file 22. For example, the warning history file 22 stores date and time, warning content, and image file as warning history when the warning device 108 issues a warning.
FIG. 6 is a diagram showing the contents of the face detection history file 23. For example, the face detection history file 23 stores the date and time (Date), the number of faces in the captured image, and the face image file for the recorded face detection history.
Note that what the storage device 102 stores is not limited to these.

The face detection unit 13 detects an area (face area) where a person's face is present from the image acquired by the image capturing unit 11. For example, a face area in the captured image is extracted by face recognition processing. The area setting means sets an area where the detection target object should be present when the target object is appropriately equipped based on the acquired face area. For example, the area setting means 14 sets the face area above the face area when detecting a helmet. The determination unit 15 determines whether or not the detection target is within the region set by the region setting unit 14.
The operations of the face detection unit 13, the region setting unit 14, and the determination unit 15 are realized by the CPU 103, the RAM 104, and the storage device 102 operating in cooperation with each other.

The CPU 103 is a central processing unit that performs operations according to an operating system and various application software stored in the storage device 102 and controls the operation of the person management device 1.
Specifically, the CPU 103 performs a calculation for detecting a face from a captured image acquired by the image capturing device 101 based on a program stored in the storage device 102 or the RAM. Further, the CPU 103 sets an area where the detection target object should exist when the detection target object is appropriately equipped based on the detected face coordinates. Further, the CPU 103 compares and analyzes the captured image captured by the image capturing apparatus 101 and the reference image stored in the storage device 102, and determines whether or not the person's appearance meets the reference.

  The RAM 104 loads programs and data stored in the storage device 102 as necessary, and transmits them to the CPU 103 at high speed. Further, the RAM 104 temporarily stores the calculation result by the CPU 103 and transmits the calculation result to the storage device 102. Typically, the RAM 104 is a main memory that can be directly accessed from the CPU 103.

  The control unit 16 manages the operation of the person management device 1. For example, the control unit 16 includes an input device 105, a display device 106, a communication device 107, a warning device 108, and an entry / exit control device 109.

  The input device 105 is a device for performing input by a user's operation, for example. Typically, the input device 105 is a keyboard, a mouse, a numeric keypad, or the like.

  The display device 106 is, for example, a liquid crystal display. The display device 106 displays the determination result determined by the CPU 103.

  The communication device 107 communicates with a device provided outside the person management device 1. For example, the communication device 107 transmits content stored in the storage device 102 to a person at a remote location and receives data transmitted from the remote location.

  The warning device 108 is, for example, a speaker. If the CPU 103 determines that the person's dress is in violation of the rules, the warning device 108 generates a warning message sound corresponding to the violation content.

  The entrance / exit control device 109 performs door opening / closing control based on the determination result of whether the person's equipment conforms to the rules performed by the determination unit 15.

  Next, the operation of the person management device 1 will be described. FIG. 7 is a diagram of a processing flow of the person management apparatus 1. Here, what performs entrance / exit management depending on whether a person is equipped with a helmet will be described.

  The image photographing apparatus 101 performs photographing and updates one frame image (step S1). At this time, when the image capturing apparatus 101 captures a moving image, a new frame is read. When the image capturing apparatus 101 captures a still image, a captured image is acquired.

  The CPU 103 performs processing (face detection processing) for detecting a person's face from the photographed new image (step S2). If a human face is detected by the face detection process (Yes in step S3), the process proceeds to step S4. If the face cannot be detected by the face detection process (No in step S3), the process returns to step S1.

  The CPU 103 acquires the coordinates of the four corners of the face and the orientation of the face in the image (step S4). Further, the face detection history is updated in the face detection history file 23 of the storage device 102. FIG. 8 is a diagram showing the relationship between the detected face image, the coordinates of the four corners of the acquired face, and the helmet detection target area. The coordinates of the four corners of the detected face are A (Xa, Ya), B (Xb, Yb), C (Xc, Yc), and D (Xd, Yd), respectively. Here, as shown in FIG. 8, the coordinate A is the lower left of the screen, the coordinate B is the lower right of the screen, the coordinate C is the upper left of the screen, and the coordinate D is the upper right of the screen. For example, if the size of the acquired image in the X direction (horizontal direction) is 640 pixels and the size in the Y direction (vertical direction) is 480 pixels, the lower left coordinates of the acquired image are (0,0), and the upper right coordinates are The description will be made assuming that it is (640,480).

  The CPU 103 sets a helmet detection target region from the acquired image (step S5). For example, as shown in FIG. 8, the helmet detection target region is a region surrounded by coordinates E to H determined based on the coordinates A to D. If the face orientation in the image is upward, the coordinate E is (Xa-50, Yc-100), the coordinate F is (Xb + 50, Yd-100), and the coordinate G is (Xa-50, Yc-100). Yc + 200) and the coordinates H are (Xb + 50, Yd + 200).

CPU103 detects a helmet (step S6). 9 and 10 are diagrams of helmet detection when the detected face orientation is upward. FIG. 9 shows a state where a person is equipped with a helmet, and FIG. 10 shows a state where the person is not equipped with a helmet.
For example, the CPU 103 detects a helmet by matching with a helmet image registered in the storage device 102. When the CPU 103 can detect the helmet (Yes in step S7), the CPU 103 acquires the direction of the helmet and the coordinates of the four corners of the helmet, and proceeds to step S8. Here, the coordinates of the four corners detected as the helmet are I (Xi, Yi), J (Xj, Yj), K (Xk, Yk), and L (Xl, Yl), respectively.
If the helmet cannot be detected (No in step S7), the CPU 103 controls the warning device 108 to issue a warning that the helmet equipment is inappropriate with reference to the equipment image database 21, and further. The warning history is recorded in the warning history file 22 of the storage device 102 (step S10). Then, it returns to step S1.

CPU103 confirms the relative position of a person about the direction of a helmet, and the coordinate of a helmet (step S8). More specifically, the CPU 103 determines whether or not the coordinates I to L of the helmet are inside the coordinates E to H that are the helmet detection target areas. As shown in FIG. 8, if a helmet is present in the helmet detection area (Yes in step S <b> 8), the CPU 103 determines that the helmet is properly equipped, and enters / exits the entry / exit control device 109. A control signal indicating permission is output. The entrance / exit control device 109 controls the opening of the door based on the control signal received from the CPU 103 (step S9). Then, it returns to step S1.
As shown in FIG. 10, when there is no helmet in the helmet detection area (No in step S <b> 8), the CPU 103 refers to the equipment image database 21 and the helmet apparatus is not properly equipped. A warning history is recorded in the warning history file 22 of the storage device 102 (step S10). Then, it returns to step S1.

As a result, detection objects such as clothes and equipment are detected at the same time as the face of a person, and the possession of the detection object is checked from the positional relationship between the detected face and the detection object, and the detection object is appropriately equipped. It can be determined whether or not it has been done, and entry / exit management can be performed.
More specifically, the orientation and position coordinates of the face obtained as a result of performing face detection processing on a still image or video shot by a camera and a pre-defined wearing obligation for the same image It can be determined from the relative position to the position coordinates of the equipment in consideration of whether or not the equipment is properly performed. Therefore, not only helmets but also equipment such as name tags and glasses can be determined based on the relative coordinates with the face.
Compared to the invention disclosed in Patent Document 1, the person management apparatus 1 according to the present embodiment requires a smaller number of reference images. Specifically, in the case of making a determination for three equipments, the invention disclosed in Patent Document 1 requires eight reference images, whereas the person management apparatus 1 according to the present embodiment requires There should be at least three reference images. Further, the person management device 1 can reduce the number of reference images prepared in advance as compared with the invention disclosed in Patent Document 1, even if the number of persons to be targeted increases. Therefore, the person management apparatus 1 can execute the comparison process with the reference image at high speed.
Further, when managing a plurality of persons, it is not necessary to register each person's reference image in advance, and it is possible to determine the dressing of unregistered persons.

  In addition, using the fact that some areas in the captured image are set as the detection area of the equipment, matching is performed with the reference image in the detection area instead of the entire image, so that the equipment is appropriate. The determination process of whether or not can be performed at higher speed.

Embodiment 2
FIG. 11 shows the configuration of the person management device 2. The person management apparatus 2 includes the second image photographing apparatus 110 in addition to the person management apparatus 1 shown in the first embodiment. Constituent articles similar to those of the person management device 1 are denoted by the same reference numerals and description thereof is omitted. Note that the image capturing apparatus 101 is the first image capturing apparatus 101.

  The second image capturing device 110 is, for example, a digital video camera. The second image capturing device 110 captures the person entering and exiting from the head direction and transmits the captured image to the CPU 103 and the RAM 104. That is, the second image capturing device 110 captures images from a different angle at the same time as the first image capturing device 101. FIG. 12 is a schematic diagram when a person is photographed from two directions.

  Next, the operation of the person management device 2 will be described.

The first image capturing apparatus 101 captures a person's face. FIG. 13 is a diagram when the first image capturing apparatus 101 captures a person from the front.
The CPU 103 detects the face of a person from the image photographed by the first image photographing device 101 and creates a first helmet detection area above the detected face. Further, the CPU 103 detects the helmet and determines whether or not the coordinates of the detected helmet are inside the first helmet detection area.
If the detected coordinates of the helmet are within the first helmet detection area, the CPU 103 determines that the equipment is appropriately equipped. If the helmet cannot be detected, or if the coordinates of the helmet are outside the first helmet detection area, the CPU 103 determines that the equipment is inappropriate.

The second image capturing device 110 captures an image from above a person. FIG. 14 is a diagram when the second image capturing device 110 captures a person from above, and the upper part of the helmet is captured. Here, it is assumed that when a person approaches the first image capturing apparatus 101, the person moves downward in the captured image of the second image capturing apparatus 110.
The CPU 103 sets a second helmet detection area for determining that the helmet is worn on the image captured by the second image capturing device 110. The CPU 103 may set the second helmet detection area based on the face information photographed by the first image photographing device 101. For example, in the image captured by the first image capturing apparatus 101, when it is determined that there is a person on the right side of the image, the second helmet detection area may be set on the right side of the image. Further, the second helmet detection area may be moved in the vertical direction based on the size of the face photographed by the first image photographing device 101.

  CPU103 confirms whether the helmet was detected in the 1st helmet detection area, when a helmet is detected in the 2nd helmet detection area. If the helmet is detected in both the first helmet detection area and the second helmet detection area, the CPU 103 determines that the helmet is properly equipped and allows the entry / exit control device 109 to enter / exit. A control signal to that effect is output. The entrance / exit control device 109 controls the opening of the door based on the control signal received from the CPU 103.

If a helmet cannot be detected in at least one of the first helmet detection area and the second helmet detection area, it is determined that the helmet is not properly equipped.
If it is determined that the helmet is not suitable, the warning device 108 issues a warning that the helmet is not properly equipped. Further, the warning history file 22 of the storage device 102 records that the helmet equipment is inappropriate. Thereafter, the process returns to the standby state, and the same operation is performed when a new image is acquired.

  By acquiring an overhead image at the same time and adding the detection of the “safety helmet seen from above” image to the determination condition, it is determined that the helmet is being worn reliably, and entry / exit is reliably managed can do.

Note that the present invention is not limited to the above-described embodiment, and can be changed as appropriate without departing from the spirit of the present invention.
Although the second image capturing apparatus 110 has been described as capturing from directly above a person, it may be capturing from another angle.
The CPU 103 may be a plurality of CPUs instead of a single CPU. For example, the first CPU may determine whether or not the person's appearance meets a standard, and the second CPU may control the operations of the warning device 108 and the entry / exit control device 109.
Further, although the inspection object has been described as a helmet, it can also be applied to other equipment such as a measuring machine equipped on the upper arm.
Moreover, although the case where it uses for management of entrance / exit was described, you may use in scenes other than management of entrance / exit. For example, the effect of improving safety can be expected by using it for monitoring the wearing of a safety helmet at a construction site.

DESCRIPTION OF SYMBOLS 1 Person management apparatus 2 Person management apparatus 11 Image photographing means 12 Storage means 13 Face detection means 14 Area setting means 15 Determination means 16 Control means 21 Equipment image database 22 Warning history file 23 Face detection history file 101 Image photographing apparatus 102 Storage apparatus 103 CPU
104 RAM
105 Input Device 106 Display Device 107 Communication Device 108 Warning Device 109 Entry / Exit Control Device 110 Image Shooting Device

Claims (6)

A person management device that checks whether a detection target is appropriately equipped based on a photographed image of a person,
Image acquisition means for acquiring the captured image;
Storage means for storing information of the detection object;
Face detection means for detecting a human face from the captured image acquired by the image acquisition means;
An area setting means for setting an area where the detection object should exist based on the coordinates of the face detected by the face detection means;
Determination means for determining whether or not the detection object is present in the area set by the area setting means;
Control means for performing processing according to the determination result of the determination means,
Person management device. The image acquisition means includes a plurality of photographing devices for photographing a person at different angles,
The determination unit performs determination based on each image acquired by the plurality of imaging devices.
The person management device according to claim 1. The image acquisition means includes a first imaging device that captures images from the front of a person and a second imaging device that captures images from above the person.
The person management device according to claim 2. The control means issues a warning according to the determination result,
The storage means
Equipment image database for storing information on the detection object;
A warning history file for storing the warning history;
A face detection history file that stores a face detection history detected by the face detection means,
The person management device according to any one of claims 1 to 3. The control means controls opening and closing of an entrance / exit gate where a person enters and exits based on a determination result of the determination means.
The person management device according to any one of claims 1 to 4. Get a shot image that includes people
A human face is detected from the acquired image,
Based on the coordinates of the detected face, set a region where the detection target for which information is stored in advance should exist,
Determine whether the detection object is present in the set area;
Processing according to the determination result;
Person management method.

Images