JP5203281B2 - Person detection device, person detection method, and person detection program - Google Patents

Description

The present invention relates to a person detection apparatus, a person detection method, and a person detection program, and in particular, a person detection apparatus for searching for a person at high speed and with high accuracy by selectively using a face detection filter and a human body detection filter in person detection, The present invention relates to a person detection method and a person detection program.

2. Description of the Related Art Conventionally, there is a method for performing image processing or the like on an image or video captured by an imaging unit such as a camera and automatically detecting a specific subject such as a person included in the image or video. Various methods for improving detection accuracy have been proposed (see, for example, Patent Documents 1 and 2).

In the above-described Patent Document 1, a human body detection device that detects a human body based on visual data, an imaging unit that captures an image, a thermal information detection unit that detects information about heat distribution, and an imaging range in the imaging unit, Detection range setting means arranged so as to overlap the heat detection ranges in the heat information detection unit, and a human body that takes out an imaging region corresponding to a region where the amount of heat of the human body is detected in the heat detection range as a human body extraction range from the captured image Further, there is shown a human body detection device that includes an extraction unit, and the human body extraction unit extracts a face region from a captured image based on a range in which the amount of heat of the human body is detected.

Further, in Patent Document 2 described above, the subject reliability for each region is determined based on resolution conversion means for converting the matching pattern into a plurality of different resolution patterns, and local feature amounts in the plurality of local regions of the plurality of resolution patterns. There is shown an image processing apparatus having subject reliability calculation means for calculating and determination means for determining whether or not a matching pattern is a specific subject based on the sum of subject reliability for each region.

JP 2006-59015 A JP 2008-33424 A

However, according to the above-described conventional method, it is necessary to have a configuration for detecting thermal information in addition to performing face analysis based on a photographed image or video, and temperature fluctuations depending on the season, place, clothes worn, etc. Is likely to be inaccurate.

In addition, in the conventional detection method, when detecting a person using face detection from an image, if a small face is detected on the image of a person who is far away, the face area in the image becomes small. The detection filter is also reduced, and accordingly, the processing time for extracting a face region from the entire image frame is increased. In other words, the longer the person who is far from the camera can be detected, the longer the processing time will be.

In addition, a method of identifying the person's area by measuring the person's height from the captured image or video, etc., and detecting the presence of the person can be considered, but when detecting a person using the human body area In some cases, the human body cannot be detected unless the entire human body is visible in the image, from the tip of the head to the tip of the foot. For example, for the person near the camera to be photographed, the human body region is accurately detected. Can not do it.

The present invention has been made in view of the above problems, and a person detection device and person detection for searching for a person with high speed and high accuracy by properly using a face detection filter and a human body detection filter in person detection It is an object to provide a method and a person detection program.

In order to solve the above problems, the present invention employs means for solving the problems having the following characteristics.

In the present invention, in the person detection device for detecting a person included in a video photographed by the imaging means, one or a plurality of time-series images included in the video having a size smaller than a predetermined size from a predetermined image Obtained by a human body detecting means for detecting a human body area for a person, a face detecting means for detecting one or a plurality of face areas having a predetermined size or more from the predetermined image, and the human body detecting means and the face detecting means. And identifying means for performing identification processing on persons included in the plurality of images based on the detection result. Thereby, a person search can be performed at high speed and with high accuracy by using a face detection filter and a human body detection filter properly in person detection.

Furthermore, in the present invention, in the person detection method for detecting a person included in a video taken by an imaging means, one or more of a time series image included in the video having a predetermined size or less from a predetermined image Obtained by a human body detection procedure for detecting a human body region for a plurality of persons, a face detection procedure for detecting one or more face regions having a predetermined size or more from the predetermined image, and the human body detection procedure and the face detection procedure. And an identification procedure for performing identification processing on a person included in a plurality of images based on the detected result. Thereby, a person search can be performed at high speed and with high accuracy by using a face detection filter and a human body detection filter properly in person detection.

Further, according to the present invention, in a person detection program for detecting a person included in a video photographed by an imaging unit, the computer is configured to have a predetermined size or less from a predetermined image among time-series images included in the video. A human body detecting unit for detecting a human body region for one or more persons having a height, a face detecting unit for detecting one or a plurality of face regions having a predetermined size or more from the predetermined image, and the human body detecting unit; Based on the detection result obtained by the face detection means, it functions as an identification means for performing an identification process on a person included in a plurality of images. Thereby, a person search can be performed at high speed and with high accuracy by using a face detection filter and a human body detection filter properly in person detection. Further, by installing the program, the person detection process according to the present invention can be easily realized by a general-purpose personal computer or the like.

According to the present invention, it is possible to detect a person at high speed and with high accuracy by properly using a face detection filter and a human body detection filter in person detection.

It is a figure which shows an example of schematic structure of the person detection apparatus in this embodiment. It is a figure which shows an example of the height estimation method in this embodiment. It is a figure which shows an example of person tracking. It is a figure which shows an example of the person tracking which considered the movement range in the real space in the tracking means. It is a figure which shows an example of the movement result after progress of time. It is a figure which shows an example of the correction process of a human body area | region when a part of human body mentioned above is concealed. It is a figure which shows an example of the hardware constitutions which can implement | achieve the person detection process in this embodiment. It is a flowchart which shows an example of the person detection processing procedure in this embodiment. It is an example flowchart which shows a person identification process procedure. It is a flowchart which shows an example of a tracking process procedure. It is a flowchart which shows an example of a concealment state determination processing procedure.

<Regarding the Present Invention> In the present invention, when performing human detection on an image or video captured by an imaging means such as a camera, human detection is performed by using face detection using a face detection filter and human detection using a human body detection filter. It is intended to perform at high speed and with high accuracy. Specifically, for example, when a person in the image is near the camera, the person detection is mainly performed by the face detection filter, and when the person in the image is far from the camera, the person detection is mainly performed by the human body detection filter. It is possible to speed up human detection by using different detection filters. Furthermore, when a person is detected by both the face detection filter and the person detection filter, identification processing is performed using the detection result from which information suitable for person identification has been obtained, thereby improving the accuracy of person identification. To do.

Hereinafter, embodiments in which a person detection device, a person detection method, and a person detection program according to the present invention are preferably implemented will be described with reference to the drawings.

<Schematic Configuration Example of Person Detection Device> FIG. 1 is a diagram illustrating an example of a schematic configuration of a person detection device according to the present embodiment. 1 includes an input unit 11, an output unit 12, a storage unit 13, a human body detection unit 14, a face detection unit 15, a detection information selection unit 16, an identification unit 17, and a tracking unit. Means 18, suspicious person detection means 19, screen generation means 20, notification means 21, transmission / reception means 22, and control means 23 are configured. Note that the transmission / reception unit 22 is connected to an imaging unit 24 such as a camera that captures a predetermined region, and can acquire each time-series image included in the video captured by the imaging unit 24. The imaging unit 24 may be configured integrally with the person detection device 10.

The input unit 11 accepts various instructions such as a human body detection instruction, a face detection instruction, a detection information selection instruction, an identification instruction, a tracking instruction, a suspicious person detection instruction, a screen generation instruction, a notification instruction, and a transmission / reception instruction from a user or the like. . Note that the input unit 11 includes, for example, a keyboard, a pointing device such as a mouse, a voice input device such as a microphone, and the like.

The output means 12 includes the instruction content input by the input means 11, control data generated based on each instruction content, human body detection means 14, face detection means 15, detection information selection means 16, identification means 17, tracking means. 18. The contents of various information obtained by the progress or result executed by the components such as the suspicious person detection means 19, the screen generation means 20, the notification means 21, and the transmission / reception means 22 are displayed or the sound is output. The output unit 12 has a screen display function such as a display, a sound output function such as a speaker, and the like.

The storage unit 13 can store various information for realizing the above-described embodiment, and reading and writing are performed as necessary. Specifically, the accumulating unit 13 includes a human body detection result in the human body detection unit 14, a face detection result in the face detection unit 15, a selection result in the detection information selection unit, an identification result in the identification unit 17, and a tracking result in the tracking unit 18. The suspicious person detection result in the suspicious person detection means 19, the screen information generated in the screen generation means 20, the contents notified by the notification means 21, the notified contents, the contents transmitted / received by the transmission / reception means 22, and the control by the control means 23 Each information such as recorded information, error information when an error occurs, log information, a program for realizing the present invention, and the like is stored.

The human body detection unit 14 acquires a video captured by the imaging unit 24 such as a camera via the transmission / reception unit 22, and among the time-series images included in the acquired video, a predetermined image (each frame One or a plurality of human body regions are detected from an image or an image having an interval of several frames. Specifically, the human body detection unit 14 compares, for example, successive image frames, and there is a place where the color information (luminance, chromaticity, etc.) changes within a predetermined time, and the area surrounded by the place. Is detected as a human body area if the area is greater than or equal to a predetermined area or the moving range with time is within a predetermined range.

Other human detection methods include, for example, a method of extracting an area that matches the shape of a person using edges or the like, a method of using a thermal image to set a heat source different from the background to a human body region, and a plurality of cameras A method for detecting human-like objects by recognizing shape and size using a human, a method for detecting a human body by learning in advance using silhouette images etc. for human movement (like walking), etc. Can be used. Note that the person detection method in the person detection means 14 is not limited to this in the present invention.

In the present invention, when the speed of human detection is to be increased, the human body detection filter that performs the above-described matching process is used to perform a process that detects only a human body having a predetermined size or less, for example. In other words, the detection of a human body having a predetermined size or larger in the image is omitted, thereby speeding up human detection. For example, a human body detection filter is used so as to detect a human body that is smaller than the size of a human body standing at a position 5 m away from the camera, that is, a person 5 m or more away from the camera.

The human body detecting means 14 has a function of estimating the height when detecting the person's region. This height estimation function estimates one or a plurality of persons included in the image, determines whether or not the person is a new person, and if it is a new person, The position of the person in real space is calculated from the previous point and the vanishing point corresponding to the image, and the height of the actual person is estimated together with the apparent size on the image. The vanishing point may be included in the image, but may not be included in the image depending on the camera angle or the like. In this case, a vanishing point is set on a virtual space and the vanishing point is used.

That is, the human body detection means 14 estimates the position and height of the person by the above-described method, uses the height as the vertical width of the human body region in the image, and then sets the horizontal width from the color difference information of the image, etc. An appropriate human body region is detected by correcting the rectangle. Further, the human body region can be said to be a region including the head to the foot of the person, but for example, a part of the human body (for example, the head or the lower knee) is shielded while being tracked by the tracking means 18 described later. When the image is shielded by an object or the like, the human body region can be corrected by using the above-described height information, toe and head position information, etc. obtained in advance. Thereby, it can grasp | ascertain easily by including the position of the head and a leg which are hidden in a human body area | region, and displaying. The human body region correction process described above will be described later.

In addition, the shape of the human body region described above is not limited to a rectangle in the present invention. For example, the shape is a circle, an ellipse, another polygon, a silhouette shape enlarged from a person's outer shape at a predetermined magnification, or the like. There may be. The human body detection means 14 can also detect color information of the person's face, head (hair, hat, etc.) and body (upper body clothing, lower body clothing, etc.). Further, the human body detection unit 14 can perform control such as performing or not performing person detection by an instruction signal from the detection information selection unit 16.

The face detection unit 15 acquires a video shot by the imaging unit 24 such as a camera, and among the time-series images included in the acquired video, a predetermined image (interval for each frame image or several frames). The face of one or a plurality of persons is detected. Specifically, the face detection means 15 acquires the feature amount of the face from position information such as eyes, nose and mouth in the face included in the photographed image, and is detected as a preset face. A face of a person is detected by performing a matching process using a matching pattern of the feature amount of. Note that the above-described collation pattern is stored in, for example, the storage unit 13 or the like.

Further, the face detection means 15 is not limited to the above-described face detection process, and for example, face detection by edge detection or shape pattern detection, face detection by hue extraction or skin color extraction, or the like can be used. The face detection means 15 detects a rectangular face area having the vertical and horizontal widths of the face in the image after detecting the face, as in the human body area means 14. Note that the shape of the face region is not particularly limited as is the case with the shape of the human body region.

In the present invention, when speeding up human detection, the face detection filter that performs the above-described matching process is used to perform a process that detects only a face having a predetermined size or more. Use. That is, the face detection filter is made larger than a predetermined size to speed up human detection. For example, a face detection filter is used to detect a face that is larger than that of a person standing at a position 5 m away from the camera, that is, a face of a person within 5 m from the camera. .

Further, the face detection means 15 can estimate the gender and age (or age) of the face image detected from the pre-registered reference face information according to conditions and the like in addition to the face detection, Detects face orientation information, or if the detected person wears sunglasses, masks, hats, etc., from the color information of facial features such as eyes and mouth registered in advance. can do. Further, the face detection unit 15 performs control such as performing or not performing face detection based on an instruction signal from the detection information selection unit 16.

The detection information selection means 16 is an identification means based on the human body detection result and the face detection result obtained by the human body detection means 14 and the face detection means 15 described above, and the video imaged by the imaging means 24 synchronized with each detection result. It is selected whether the detection result used in 17 is the above-described human body detection result or face detection result. In addition, when the result of either the human body detection unit 14 or the face detection unit 15 is not obtained, the detection information selection unit 16 selects the other detection result from which the result is obtained.

For example, when a person is detected by the human body detection means 14, the height may not be detected if the person is bent or a part of the body is hidden. Further, when a person is detected by the face detection means 15, the hair color cannot be detected if the person is wearing a hat, and information that can be acquired if the person is wearing a mask or sunglasses is limited. End up. As described above, since the information that can be used for the identification process obtained according to the situation at the time of shooting is different, the information of the human body detection means 14 and the face detection means 15 is not used, but only the information obtained by any of the detection means. The identification process may be performed using information that can be used for the identification process from the information obtained by both sides.

In this case, it is also possible to prioritize which information is used and use the highest priority information among the detected information.

Furthermore, the detection information selection unit 16 selects only the detection result of human detection or face detection from the history of detection information selected so far, and the human body detection unit 14 or face detection unit based on the selection result. Control can be performed so that only one of the 15 processes is continuously performed. Accordingly, for example, when the position where the imaging unit 24 is installed is the ceiling or the like, and there is a high possibility that an image of the entire person is acquired, and only an image that reduces the face area can be acquired, the detection information The selection unit 16 can select only the detection result from the human body detection unit 14 and can control the face detection unit 15 not to perform the face detection process. Thereby, it is possible to realize high-speed person detection by efficiently performing only necessary processing.

Here, the detection information selection unit 16 performs processing for associating the human body region obtained by the human body detection unit 14 with the face region obtained by the face detection unit 15 as belonging to the same person. Specifically, the detection information selection unit 16 acquires the center of gravity coordinates of the face area in the image, and when there is a human body area that includes the acquired coordinates, the face area and the human body area are from the same person. Associating as there is.

Furthermore, when one human body region includes two or more face regions, the detection information selection unit 16 associates the same human body region with all the face regions. As a result, even in the case of tracking processing by the tracking means 18 or the like, even when a plurality of persons temporarily overlap on the screen, it is possible to continuously track without interruption. Persons are managed using identification information such as IDs. Note that the processing for associating the human body region and the face region as those of the same person is not limited to this in the present invention. For example, the posture and orientation of the person are extracted and the extracted information is used. May be associated with each other.

Note that the detection results by the human body detection means 14 and the face detection means 15 described above and the detection information (including history information) selected by the detection information selection means 16 are stored in the storage means 13.

The identification unit 17 performs identification processing as to whether or not the same person is included in one or a plurality of persons detected from two images extracted from videos taken at different times. Specifically, for example, as a method for determining whether the person included in the shooting is a person who has been shot or a new person, the identification unit 17 is, for example, whether there is a person in the previous video. If there is a person, it is determined whether or not the person is included in the movable range set in advance, and further, it is the same depending on the presence or absence of similarity by comparing the front and back face feature amounts It can be determined whether or not a person is included.

Further, the identification means 17 uses the face pattern detected by the face detection means 15 and the personal information (name, gender, age, which celebrity (person) is similar) stored in the storage means 13 in advance. It is possible to determine who the face is, that is, who the person is by comparing with the pattern of the included face.

Note that the identification unit 17 may identify a person using an appropriate parameter by a method for selecting a feature amount according to the state of the image. For example, the identification unit 17 determines the state of the person (stopped, walking, whether there was a change in sunshine, etc.), and based on the result, various feature amounts (step length, walking speed, clothing color, hair color, etc.) Color etc.) can be used for person identification. Furthermore, the identification unit 17 calculates the degree of separation only for the feature amount to be used, performs weighting based on the degree of separation, and changes the contribution rate for each feature amount with respect to person identification. Thereby, highly accurate person determination can be performed using the feature-value according to a condition.

In the identification unit 17, the human body detection result in the human body detection unit 14 may be different from the face detection result in the face detection unit 15. In such a case, the identification means 17 may not be able to determine the same person by identification because the detection information selection means 16 does not match even if the comparison is made when the process is switched halfway. Conceivable. Therefore, the detection information selection unit 16 performs both the human body detection in the human body detection unit 14 and the face detection in the face detection unit 15 on the acquired video before and after switching, for example, and the detection results are described above. In this way, it is associated with whether or not they are the same person. As a result, when the identification unit 17 performs the identification process, the person identification process can be easily performed even if the detection information is switched.

The tracking unit 18 tracks the behavior of the same person identified by the identifying unit 17 over time using the captured video. The tracking means 18 can track a plurality of persons included in the video over time in parallel.

In addition, when tracking the person included in the video over time, the tracking unit 18 can estimate the next movable range from the orientation, posture, action so far, and the like of the person acquired from the image. it can. In this case, the tracking unit 18 generates information for combining the transferable range with the image and outputs the information to the screen generation unit 20.

Here, the tracking means 18 tracks a person from the position of the foot and head of the human body region included in the image and its characteristics, and when the size of the person that can be determined to be the same changes, It is judged whether or not it is hidden by etc. The part of the person to be concealed is, for example, the lower body, the upper body, the head, arms, and the torso. In other words, in this embodiment, when information about the height of the human body from the toes of the feet to the head can be acquired at least once on the screen, if both the head and feet are not simultaneously hidden, The person can be estimated and tracked.

The suspicious person detecting means 19 collates one or more suspicious behavior patterns, behavior data, etc., which are set in advance and stored in the accumulating means 13 with the result of tracking by the tracking means 18 and photographed by the imaging means 24. The person who corresponds to a suspicious person is detected among these persons.

For example, the suspicious person detecting means 19 has at least one suspicious behavior such as the person to be tracked hidden behind a shield, being worried about the camera for more than a predetermined time, scrambled, or staying for a long time. In that case, the person is detected as a suspicious person. Furthermore, the suspicious person detection means 19 detects as a suspicious person even when the person to be tracked is wearing a mask or wearing sunglasses to hide his face. The suspicious person detecting means 19 notifies the notifying means 21 of the detected suspicious person information. Further, the suspicious person detecting means 19 causes the screen generating means 20 to display information relating to the detected suspicious person and an image of a person corresponding to the suspicious person.

The screen generation unit 20 generates a screen that displays the video imaged by the imaging unit 24. The screen generation unit 20 also converts the human body region detected by the human body detection unit 14, the face region detected by the face detection unit 15, movable range information from the tracking unit 18, and the like into a corresponding captured image. Generate a composite screen. At this time, the screen generation means 20 can also display, for example, digitized data such as position information corresponding to the region of the photographed person.

The screen generation means 20 generates and displays a screen so that various data such as a movable range used when tracking a person included in the video and a tracking route obtained by the tracking means 18 can be easily monitored. Can do. Furthermore, when the screen generation means 20 corresponds to the action pattern of a suspicious person set in advance, the screen generation means 20 generates a screen for notifying the content to that effect. Various kinds of information necessary for screen generation can be used by appropriately reading out necessary information from information stored in the storage unit 13 in advance. Further, the screen generation unit 20 can display the generated screen or the like on a display as the output unit 12 or output sound or the like through a speaker or the like.

The notification means 21 generates an image detected as a suspicious person obtained by the suspicious person detection means 19 and information related to the image (detection date and time, detection location, video for a predetermined time, etc.) by the screen generation means 20. To display. In addition, the notification means 21 has an alert function for notifying a manager, a security officer, a supervisor, a representative manager, etc. of the building in response to a problem occurrence signal in such suspicious behavior detection. .

The transmission / reception means 22 receives monitoring video from one or a plurality of imaging means 24 via a communication network such as a LAN (Local Area Network) or the Internet. Note that the transmission / reception unit 22 does not have to receive the monitoring video directly from the imaging unit 24. For example, the video acquired in advance by the imaging unit 24 is temporarily stored somewhere, and the stored information is used. In this embodiment, human body detection may be performed.

The transmission / reception means 22 can be used as a communication interface for transmitting to other terminals constituting the person detection device 10 and receiving various data from other terminals.

The control means 23 controls the entire functional configuration of the person detection device 10. Specifically, the control unit 23 performs human body detection, face detection, selection of detection information, identification processing, etc. based on input information from the user input by the input unit 11, Various controls such as a tracking process, a suspicious person detection process, a screen generation, notification control, and transmission / reception control are performed.

<Example of Height Estimation Processing in Human Body Detection Unit 14> Next, an example of height estimation processing in the human body detection unit 14 of the present embodiment will be specifically described. Here, FIG. 2 is a diagram illustrating an example of the height estimation method in the present embodiment. 2A is a diagram for explaining the distance between the imaging means 24, which is a camera, and the person 30, and FIG. 2B shows the number of pixels between the head, feet, and vanishing points at the time of height estimation. And the relationship of distance. In the present embodiment, as shown in FIG. 2B, the distance from the person's foot position to the camera is calculated, and the distance is also calculated from the position of the head (rectangular upper end of the human body region). As a result, if it is determined from either one of the distance calculation results that the same person has moved, it is assumed that the person detected last time has moved.

As shown in FIGS. 2A and 2B, the distance (5 m, 10 m) from the imaging means 24 such as a camera, the vanishing point, the positions of the head and feet, the number of pixels of the photographed image, and the like are used. The person's height H is estimated. Further, as shown in FIG. 2B, the height estimating means 15 estimates the height of the person 30-1 and the person 30-2 from the relationship between the position, height, and vanishing point.

Specifically, when estimating the height, the human body detection means 14 has a reference length indicating the actual height of the reference object and a reference distance indicating the actual distance from the camera fixed at a predetermined position to the reference object. A reference coordinate indicating the lower end position of the reference object, a first reference length pixel number corresponding to the reference length in the reference object image, and a reference object image in which the reference object is located when the camera is moved to infinity. The vanishing point coordinates indicating the position of the vanishing point that is the position of the person are acquired, a person image including a person is acquired as a subject from the camera, and the number of person length pixels corresponding to the height of the person in the acquired person image is counted. Further, the number of person vanishing point pixels from the lower end of the person in the acquired person image to the vanishing point in the person image is counted.

Next, based on the reference coordinates and vanishing point coordinates held by the holding means described above, the reference vanishing point pixel number from the lower end position of the reference object to the vanishing point position in the reference object image is calculated and specified. Based on the ratio of the number of person vanishing point pixels to the number of reference vanishing point pixels and the reference distance held by the holding unit, the actual person distance from the camera to the person is calculated, the calculated person distance, and the holding unit The second reference length pixel number corresponding to the reference length when the reference object is arranged at the person distance is calculated based on the reference distance held by the first reference length pixel number and the first reference length pixel number.

Further, the actual height of the person is calculated based on the ratio between the second reference length pixel number and the above-described person length pixel number and the reference length held by the holding means.

For example, the human body detection unit 14 may use a technique disclosed in Japanese Patent Application Laid-Open No. 2008-286638 filed by the present applicant. Further, the method for calculating the height of a person from an image is not limited to this in the present invention.

The human body detection means 14 estimates the height by the above-described method, uses the height information during the identification process, and also uses it as the vertical width in the image of the human body region, and then sets the horizontal width from the color difference information of the image. Then, an appropriate human body region is detected.

<Tracking Method in Tracking Unit 18> FIG. 3 is a diagram illustrating an example of person tracking. 3A shows an example in which the foot of the person being tracked is out of frame in the image being shot, and FIG. 3B tracks the person in FIG. 3A over time. Results are shown. 3 is a screen generated by the screen generation unit 20 and displayed on the output unit 12 such as a display.

As shown in FIG. 3, the screen 40 has a video display screen 41 and a tracking display screen 42. On the video display screen 41, a human body region 43 detected by the human body detection unit 14 with respect to an image taken by the imaging unit 24 such as a camera, and a face region 44 detected by the face detection unit 15 are displayed on the screen generation unit 20. Is synthesized and displayed.

Here, as shown in the video display screen 41, when there is a human body region 43 including the barycentric coordinates of the face region 44, in the present embodiment, the face region 44 and the human body region 43 are attributed to the same person 30. Judgment is made and processing is performed.

In addition, the video display screen 41 includes a time information display area 45 for displaying captured time information, person-related information regarding the detected person 30 (whether it is a tracking target, an estimated height, whether a position is identified, Whether the rectangle has been corrected, etc.) 46, a tracking route (which may include the moving direction) 47 tracked so far, and the like are synthesized and displayed by the screen generation means 20.

Note that the person-related information 46 is not limited to the above-described content. For example, when the photographed time or the person can extract and measure the facial feature amount from the screen, the person's designation, gender, estimated age (May be the age) can be displayed.

In the present embodiment, for example, as shown in the video display screen 41 of FIG. 3, when a part of the human body being tracked out of the frame from the foot, the face detected by the face detection unit 15 by the detection information selection unit 16 An area 44 is used, and the identification unit 17 identifies a person using the face area 44.

Further, on the tracking display screen 42 shown in FIG. 3, the tracking route 47 and the current position 49 with respect to the angle of view 48 of the imaging means 24 such as a camera can be moved after a preset time period or a predetermined number of frames from the present time. It is possible to acquire a movable range 50 that can be estimated to be.

In addition, as shown in FIG. 3A, when the person 30 being tracked out of the frame from the foot, for example, when the foot is concealed, the position of the person in the real space is determined from the head coordinates on the image. It can also be estimated and tracked from the head coordinates. Further, as shown in FIG. 3, when the person 30 being tracked is approaching the camera, the vertical movement of the head coordinates accompanying the walking motion appears significantly. Therefore, when position estimation in the real space is performed from the head coordinates, the movable range estimated from the position of the previous frame may be exceeded, resulting in another person. Therefore, in the present embodiment, regarding the position estimation processing from the head coordinates, in the case of a short distance from the camera, the tracking is correctly performed by setting a wider movable range according to the distance (the closer the distance is). It can be performed.

FIG. 4 is a diagram showing an example of person tracking in consideration of the moving range in the real space in the tracking means. Furthermore, FIG. 5 is a diagram illustrating an example of the movement result after the lapse of time. 4 and 5 are screens generated by the above-described screen generation unit 20 and displayed on the output unit 12 such as a display.

In addition to the video display screen 41 and the tracking display screen 42 shown in FIG. 3 described above, person position information 51 is displayed on the screen 40 shown in FIGS. Here, as the person position information 51, for example, the distance (h1, h2) of the position of the person 30 is measured on the basis of the distance and angle from the imaging means 24 such as a camera and the number of pixels in the entire field of view, The person coordinates (x, y, z) obtained from the measurement result are acquired. The time information display area 45 and the person related information 46 described above may be displayed as the person position information 51.

As shown in FIGS. 4 and 5, the person 30 in the video has a small face area and the face is not facing the front of the camera. Can not. In such a case, the person is identified using the information detected by the human body detection means 14 described above.

The tracking display screen 42 shown in FIGS. 4 and 5 includes an angle of view 48 (= field of view) of the image pickup means 24, a person position 49, and an angle 52 (20 ° from the center of the camera image) )) Is displayed, and the distance information (7.2 m) measured from the image as the distance from the camera (5 m, 10 m, etc.) and the person position information 51 are displayed.

In the movement result after the elapse of time shown in FIG. 5, the past person recognition area 53, the positions of the past and present frames (human body area 43, past person recognition area 53), and time information up to that time are used. A movable range 50 in which the person 43 is estimated to move is set in the area, and the area is displayed in a circle or the like. It should be noted that the frames of the human body area 43 and the past person authentication area 53 can be displayed by color-coding each person included on the screen so that a plurality of persons can be distinguished.

Further, the shape of the movable range 50 is not limited to this in the present invention, and may be a sector, an ellipse, a rectangle, or the like. Further, the movable range 50 may be set based on the direction obtained by measuring the orientation of the person 43 by the human body detection means 14 or the like and by the orientation. Thereby, the next prediction range can be easily estimated. The movable range 50 is also displayed on the tracking display screen 42.

In this way, identification processing and tracking processing can be performed with high accuracy by switching between face detection and human body detection according to the size, orientation, movable range, etc. of the person projected in the video.

<Human Body Area Correction Process> FIG. 6 is a diagram illustrating an example of a human body area correction process when a part of the human body described above is concealed. For example, as shown in FIG. 6A, if a human body region 43 and a face region 44 are detected for a person 30, the person related information 46 can be displayed and tracked. Thereafter, as shown in FIG. 6 (b) or FIG. 6 (c) during tracking by the tracking means 18, a part of the person's body (under the knee in the example of FIG. 6) is covered by a shield 61 (for example, cardboard). When concealed, the human body region 43 is normally located up to the knee. Therefore, in the present invention, the human body region 43 is corrected in the case described above. That is, if the tracking means 18 can acquire information about the height of the human body from the toes of the feet to the head at least once on the screen, then, if both the head and feet are not concealed at the same time, the tracking means 18 Can be estimated.

Specifically, the tracking unit 18 tracks a person from the position and characteristics of the foot and head of the human body region included in the image, and when the size of the person that can be determined to be the same changes, a part of the human body is It is determined whether or not it is concealed by a shielding object such as a building. The part of the person to be concealed is, for example, the lower body, the upper body, the head, arms, and the torso.

Further, the tracking means 18 measures the distance between the surveillance camera and the person from the distance between the tip of the foot of the human body region and the vanishing point, calculates the position of the person in the real space, calculates the movable range, and performs tracking. Use. For example, when the foot is concealed, the tracking unit 18 calculates the position of the person in the real space from the position of the head and the height calculated in advance, and continues tracking. As a result, it is possible to correct a part of the human body concealed by a shield or the like and to accurately detect the human body region. Further, it is possible to easily grasp that the feet of the person are concealed.

<Screen Generation Unit 20> Next, a screen generation example in the above-described screen generation unit 20 will be described with reference to the drawings. The screen generation means 20 generates various screens in human body detection in the present embodiment as shown in FIGS. The screen generated by the screen generation means 20 can be displayed on a display screen such as a display.

As the person position information 51 shown in FIGS. 4 and 5 described above, for example, the above-mentioned “distance”, “angle”, “movement speed”, and “person coordinates” can be displayed. It is not particularly limited, and for example, the time of shooting, the facial feature amount of a person included in the video, and the like are extracted, and the extracted gender amount and estimated age (or age) of the person are estimated. Can also be displayed.

<Hardware Configuration Example> Here, the person detection device 10 described above can be controlled by a dedicated device configuration having the above-described functions, but an execution program (person detection) that can cause a computer to execute each function. Program), and the execution program is installed in, for example, a general-purpose personal computer, server, etc., the person detection processing in the present invention can be realized.

A hardware configuration example of a computer capable of realizing the person detection processing in the present embodiment will be described with reference to the drawings. FIG. 7 is a diagram illustrating an example of a hardware configuration capable of realizing the person detection process according to the present embodiment.

7 includes an input device 71, an output device 72, a drive device 73, an auxiliary storage device 74, a memory device 75, a CPU (Central Processing Unit) 76 for performing various controls, and a network connection device. 77, which are connected to each other by a system bus B.

The input device 71 has a pointing device such as a keyboard and a mouse operated by a user or the like, and inputs various operation signals such as execution of a program from the user or the like.

The output device 72 has a monitor that displays various windows and data necessary for operating the computer main body for performing processing in the present invention, and displays the program execution progress and results by the control program of the CPU 76. can do.

The input device 71 and the output device 72 may be integrated input / output means such as a touch panel. In this case, the input device 71 and the output device 72 are predetermined using a finger of a user, a pen-type input device, or the like. The position can be touched to input.

Here, the execution program installed in the computer main body in the present invention is provided by a portable recording medium 78 such as a USB (Universal Serial Bus) memory or a CD-ROM, for example. The recording medium 78 on which the program is recorded can be set in the drive device 73, and the execution program included in the recording medium 78 is installed from the recording medium 78 to the auxiliary storage device 74 via the drive device 73.

The auxiliary storage device 74 is a storage means such as a hard disk, and can store an execution program according to the present invention, a control program provided in a computer, etc., and perform input / output as necessary.

The memory device 75 stores an execution program or the like read from the auxiliary storage device 74 by the CPU 76. The memory device 75 includes a ROM (Read Only Memory), a RAM (Random Access Memory), and the like.

The CPU 76 performs various operations and input / output of data with each hardware component based on a control program such as an OS (Operating System) and an execution program read and stored by the memory device 75. Each process can be realized by controlling the process. Various information necessary during the execution of the program can be acquired from the auxiliary storage device 74, and the execution results and the like can also be stored.

The network connection device 77 obtains an execution program from another terminal connected to the communication network by connecting to a communication network or the like, or an execution result obtained by executing the program, or in the present invention. The execution program itself can be provided to other terminals. With the hardware configuration as described above, the person detection process in the present invention can be executed. Further, by installing the program, the person detection process according to the present invention can be easily realized by a general-purpose personal computer or the like. Next, specific contents of the person detection process will be described.

<Person Detection Process> Next, a person detection process procedure executed by the execution program (person detection program) according to the present invention will be described with reference to a flowchart. FIG. 8 is a flowchart illustrating an example of a person detection processing procedure in the present embodiment.

In FIG. 8, first, an image taken by an imaging means such as a camera is input (S01). Next, a human body region and a face region for one or more persons included in the video are detected (S02, S03). Here, it is selected and searched which detection information is used from the contents of the human body area and the face area for the person included in the video (S04), and identification processing is performed based on the selected information (S05).

Further, the above process is performed over time to track the person (S06). Here, the person's action pattern based on the person tracking result in the process of S06 is compared with the suspicious action pattern accumulated in advance, and if there is a person corresponding to the suspicious person, the person is detected (S07). . A screen is generated (S08), and the generated screen is output (S09).

  Further, it is determined whether or not the tracking is to be ended (S10). If the tracking is not to be ended (NO in S10), the process returns to S01, and the captured image is continuously input to perform the subsequent processing. If the tracking is to be ended (YES in S10), the entire process is ended.

<Person Identification Processing Procedure in Height Estimation> Next, a person identification processing procedure using the above-described height estimation will be described with reference to a flowchart. FIG. 9 is a flowchart illustrating an example of a person identification processing procedure. FIG. 9 shows an outline of tracking by detecting the position of the person. In the person identification process shown in FIG. 9, first, it is determined whether or not one or more persons (in FIG. 9, persons X1,..., Xn) included in the video have been detected (S11). If not (NO in S12), the state is updated according to the state transition diagram (S12). Specifically, in the process of S12, for example, “initial state → initial state”, “collation state → initial state”, “tracking state → disappearing state”, “disappearing state → disappearing state or leaving state or initial state”, “ Any setting such as “removal state → initial state” is performed. Here, the “initial state” indicates that the area for storing the tracking information is a free area. For example, when the number of people that can be tracked is 3, the areas of tracking ID1, ID2, and ID3 are secured, and each holds a tracking state and personal information.

The “collation state” indicates a state in which it is being confirmed whether or not the person is the same as a person detected in the past. In the “verification state”, if it is determined that the person is the same as the new person or the person being tracked, the state shifts to the “tracking state”. Further, in the “collation state”, when it is determined that the region is not stably detected or the region does not move like human (region that seems to be erroneously detected), the process returns to the “initial state”.

The “tracking state” indicates a state in which a person is being tracked. The “tracking state” shifts to the “disappearing state” when the person being tracked cannot be detected. The “disappearance state” indicates a state in which the person being tracked can no longer be detected. The “disappearing state” shifts to the “withdrawal state” after a certain period of time. Further, the “disappearing state” returns to the “tracking state” when the person being tracked is detected again within a predetermined time. Further, the “withdrawal state” indicates a state in which it is determined that the person being tracked is gone. However, when the person returns (when the person who is out of the frame comes into the frame again), the tracking information is held for a certain period of time so that the person can be identified and tracked. In addition, the “withdrawal state” means that after a certain time has elapsed, the information is discarded and the state returns to the “initial state”.

If one or more persons are detected in the process of S11 (YES in S11), it is determined whether or not a person A that is not in an initial state already exists (S13). Here, when there is no person who is not in the initial state (NO in S13), the registration state is updated as “new person to“ initial state → collation state ”(S14). If there is a person who is not in the initial state (YES in S13), it is next determined whether or not the person A is within the moving range (S15). If it is not within the movement range (NO in S15), the registration state is updated as shown in S14 described above.

If it is within the movement range of person A in the process of S15 (YES in S15), it is next determined whether or not there are a plurality of persons within the movement range (S16). Here, when there are a plurality of persons (YES in S16), person identification processing is performed on person A with persons X1,..., Xn (S17). At this time, the person identification process in S17 is performed using a still image selected from the video. Further, it is determined whether or not there is a person identified in the process of S17 (S18). If there is no person identified (NO in S18), the registration state is updated as shown in S14 described above.

Further, if there are not a plurality of persons within the movement range in the process of S16 (NO in S16), or if there is an identified person in the process of S18 (YES in S18), the person A and the person Xi are the same person. The state is updated (S19). This makes it possible to accurately determine whether the fractions included in the video are the same person.

<Tracking Process Procedure> Next, a specific example of the tracking process procedure in the tracking unit 18 and the like described above will be described using a flowchart. FIG. 10 is a flowchart illustrating an example of the tracking processing procedure. As shown in FIG. 10, in the tracking processing procedure, first, the position is estimated from the foot of the person A detected in the current image frame (S21). Next, it is determined whether or not the person A exists within the movement range of the person B (S22). Here, when the person A does not exist within the movement range of the person B (NO in S22), it is assumed that the person B is the same person as the person B detected last time (S23). Next, using the height Hb of the person B, the position of the person A is estimated from the head (S24), and it is determined whether or not the person A exists within the movement range of the person B (S25). Here, when person A exists within the movement range of person B (YES in S25), or when person A exists within the movement range of person B in the process of S22 (YES at S22), It is determined that they are the same person (assumed to be established) (S26). Further, in the process of S25, when the person A exists within the movement range of the person B (NO in S25), it is determined that the person is a different person (assuming not established) (S27). In other words, in the above-described process, if the person A is within the movement range of the person B in the process of S22, the person is immediately determined to be the same person. It is determined whether or not it is established.

<Concealment State Determination Processing Procedure> Next, a specific example of the concealment state determination processing procedure in the tracking unit 18 and the like described above will be described using a flowchart. FIG. 11 is a flowchart illustrating an example of the concealment state determination processing procedure. As shown in FIG. 11, in the concealment state determination processing procedure, first, a position PAf estimated from the foot of the person A and a position PAh estimated from the head are acquired (S31). Next, it is determined whether or not the positions of PAf and PAh are within a certain range (S32). Here, if the positions of PAf and PAh are not within a certain range (NO in S32), it is assumed that the person is the same as person B detected last time (S33).

Next, the position PBah of the person B is estimated from the head using the height Ha of the person A (S34). Thereafter, it is determined whether or not the positions of PAf and PBah are within a certain range (S35). Here, when the positions of PAf and PBah are within a certain range (YES in S35), it is determined that person area A is the foot and person area B is the head (that is, the torso is concealed) (S36). ). In the process of S35, when the positions of PAf and PBah are not within a certain range (NO in S35), it is determined that the feet of the person area A are concealed (S37). In the process of S32, if the positions of PAf and PAh are within a certain range (YES in S32), it is determined that person A has no concealment area (S38).

That is, in the above-described processing, when the position PAf estimated from the foot and the position PAh estimated from the head indicate the same position, it is determined that they are the foot and head of the same person. Therefore, it is determined that the region is clearly extracted from the head to the foot. If the positions are different, it is determined that the foot or head region may be missing due to some influence.

According to the above-described embodiment, even if a part of the body is missing from the captured image or part of the body is shielded by a shielding object, the person is recognized as the same person from the face detection result, etc. Furthermore, it is possible to correct a part of the human body concealed by a shield or the like, and to accurately detect the human body region.

As described above, according to the present invention, it is possible to perform a person search with high speed and high accuracy by properly using a face detection filter and a human body detection filter in person detection. Specifically, when performing human body detection on an image or video captured by an imaging means such as a camera, if the area in the video is small from the camera, detection is performed using a human body detection filter, When the area in the image close to the camera is large, the detection can be performed using the face detection filter, and the speed can be increased while maintaining the accuracy of the person detection process. That is, it is possible to track a person with high accuracy by properly using tracking based on a human body region and tracking based on a face region using preset conditions.

Furthermore, when tracking a person using face detection from a captured image, the body may overlap as long as the face can be detected, so that it is strong against overlap and can be detected even if the person is close to the camera. If it is far from the camera, the detection accuracy is lowered and the person tracking is interrupted. On the other hand, when tracking a person using human body region detection from the captured image, it is possible to detect even if the person is far from the camera, but when the person approaches the camera, the person tends to overlap and the person tracking is interrupted End up. Accordingly, face detection can be performed without interruption by performing face detection when the person is close to the camera and detecting the person using the human body region when the person is far away.

Further, according to the present invention, it is possible to track a moving object in a monitoring image with high accuracy, improve the determination performance as to whether or not the object is a moving object, and enable moving image tracking of a monitoring image that enables efficient image monitoring. Can provide a way.

The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

DESCRIPTION OF SYMBOLS 10 Person detection apparatus 11 Input means 12 Output means 13 Accumulation means 14 Human body detection means 15 Face detection means 16 Detection information selection means 17 Identification means 18 Tracking means 19 Suspicious person detection means 20 Screen generation means 21 Notification means 22 Transmission / reception means 23 Control means 24 imaging means 30 person 40 screen 41 video display screen 42 tracking display screen 43 human body area 44 face area 45 time information display area 46 person related information 47 tracking route 48 angle of view 49 current position 50 movable range 51 person position information 52 angle 53 Past person recognition area 61 Shield 71 Input device 72 Output device 73 Drive device 74 Auxiliary storage device 75 Memory device 76 CPU (Central Processing Unit) 77 Network connection Connecting device 78 Recording medium

Claims (3)

In a person detection device for detecting a person included in an image taken by an imaging means,
Human body detection means for detecting a human body region for one or a plurality of persons having a predetermined size or less from a predetermined image among time-series images included in the video;
Face detecting means for detecting one or a plurality of face regions having a predetermined size or more from the predetermined image;
Based on the detection results obtained by the human body detection means and the face detection means, an identification means for performing identification processing for a person included in a plurality of images;
A person detecting device characterized by comprising: In a person detection method for detecting a person included in an image taken by an imaging means,
A human body detection procedure for detecting a human body region for one or a plurality of persons having a predetermined size or less from a predetermined image among the time-series images included in the video;
A face detection procedure for detecting one or more face regions having a size greater than or equal to a predetermined size from the predetermined image;
Based on the detection result obtained by the human body detection procedure and the face detection procedure, an identification procedure for performing identification processing on a person included in a plurality of images,
A person detection method characterized by comprising: In a person detection program for detecting a person included in an image captured by an imaging means,
Computer
Human body detection means for detecting a human body region for one or more persons having a predetermined size or less from a predetermined image among time-series images included in the video,
Face detecting means for detecting one or a plurality of face regions having a size greater than or equal to a predetermined size from the predetermined image; and
Identification means for performing identification processing on a person included in a plurality of images based on detection results obtained by the human body detection means and the face detection means;
Person detection program to function as

Images