WO2008035411A1

WO2008035411A1 - Mobile body information detection device, mobile body information detection method, and mobile body information detection program

Info

Publication number: WO2008035411A1
Application number: PCT/JP2006/318635
Authority: WO
Inventors: Osafumi Nakayama; Daisuke Abe; Morito Shiohara
Original assignee: Fujitsu Limited
Priority date: 2006-09-20
Filing date: 2006-09-20
Publication date: 2008-03-27
Also published as: JPWO2008035411A1; JP4667508B2

Abstract

A person information detection device for detecting passage information on a person passing a target monitoring area by using a camera (10) placed at the upper part of the area. The device performs image processing for detecting individual persons appearing in the center of the area imaged by the camera (10) by using images of the persons. Then, the device performs image processing for detecting the faces of persons in the periphery of the center of the area imaged by the camera (10) by using images of the persons. After that, the device detects each person's information associated with a detected face and an advance route of the person, and such information is detected as passage information together with the number of passages of the person and a face image of the person.

Description

Specification

Mobile object information detection apparatus, mobile object information detection method, and mobile object information detection program

Technical field

[0001] The present invention relates to a moving body information detecting device, a moving body information detecting method, and a moving body information detecting program for detecting passage information about a moving body that passes through the area using an imaging device that captures a predetermined area. In particular, the present invention relates to a mobile object information detection apparatus, a mobile object information detection method, and a mobile object information detection program capable of simultaneously detecting and recognizing a mobile object without performing pan-tilt and zooming with an imaging device. It is.

Background art

Conventionally, there has been a human information detection device that automatically detects (recognizes) a person by photographing a person who passes through a passage using a camera installed on the top, such as a ceiling, and processing the image. Exists. This person information detection device is used, for example, in an entrance / exit monitoring system for monitoring a person entering / exiting an entrance / exit of a facility or a passing person counting system for counting the number of persons passing through a sidewalk or a passage.

[0003] In a powerful entry / exit monitoring system and passing person counting system, it is hoped that people who enter / exit and pass through can be detected (counted) without omission and that it can be realized with a small number of cameras from the viewpoint of introduction and operation costs. It is rare. Furthermore, in recent years, since security awareness has increased, it is also important to acquire facial images as information for identifying the person who has passed.

[0004] In order to realize such a demand, methods proposed so far include a method using a camera that captures the upper force of an area to be monitored, a method using a plurality of cameras, There is a method using a camera (PTZ camera) having a pan / tilt / zoom function.

[0005] As a method of using a camera that captures an image just below the upward force of an area to be monitored, for example, there is a method described in Patent Document 1 (Japanese Patent Laid-Open No. 6-223157). FIGS. 19 to 22 are diagrams (1) to (4) for explaining the method of Patent Document 1. FIG. In this method, the camera is placed directly on the ceiling (see Fig. 19), and the captured image (see Fig. 20) is changed. Is recognized by image processing and detected as a person who has passed this change. As a result, since the force camera and the passage are facing each other, only the person who has passed can be photographed, and the person can walk.

V, as if shooting in an oblique direction toward the direction of coming (see Fig. 21), correctly distinguishing each passing person who can be seen in the depth direction (see Fig. 22) can do.

[0006] Further, as a method using a plurality of cameras, for example, there is a method described in Patent Document 2 (Japanese Patent Laid-Open No. 9-54894). FIG. 23 is a diagram for explaining the technique of Patent Document 2. In FIG. As shown in the figure, in this method, a camera is installed in the direction in which a person walks just underneath, and the face of a passing person is recognized. As a result, it is possible to acquire and recognize the face image of a passing person by simply counting the number of passing persons.

[0007] Further, as a method using a PTZ camera, for example, there is a method described in Patent Document 3 (Japanese Patent Laid-Open No. 10-188145). FIG. 24 is a diagram for explaining the method of Patent Document 3. In FIG. As shown in the figure, this method detects a person who passes the middle force of a widely captured image, and pans and tilts the camera to the detected position to acquire and recognize the face image of the person. Do. This makes it possible to detect (count) people and acquire and recognize facial images with a single camera.

[0008] As described above, various conventional techniques for detecting a passing person have been proposed, and their features are also different.

Patent Document 1: Japanese Patent Laid-Open No. 6-223157

Patent Document 2: JP-A-9-54894

Patent Document 3: JP-A-10-188145

Disclosure of the invention

Problems to be solved by the invention

However, the conventional techniques represented by Patent Documents 1, 2, and 3 have the following problems.

[0011] First, with the method using a camera that captures the upper force of the area to be monitored, the number of passing people can be counted, but the face image for recognizing the person cannot be obtained. There is a problem that it is not suitable. [0012] The method using a plurality of cameras has the following problems. FIG. 25 is a diagram for explaining the problem of the method using a plurality of force lenses. As shown in the figure, in this method, in order to associate the same person appearing on different cameras with the same person by image processing, it is necessary to install each camera strictly, and the installation position shifts due to secular change or the like. Considering this, there is a problem that the cost of on-site adjustment is large. In addition, since this method uses a plurality of cameras, there are problems of introduction cost and installation space.

[0013] In addition, the method using a PTZ camera has the following problems. FIG. 26 is a diagram for explaining the problems of the method using the PTZ camera. As shown in the figure, this method pans and tilts the camera to focus on one passing person, so if multiple persons pass through the monitoring area at the same time, all passing persons are removed. There is a problem that it cannot be detected. To deal with this problem, using multiple PTZ cameras has the same problem as using multiple force cameras. In addition, pan / tilt / zoom operations cause mechanical wear on the camera, making it difficult to guarantee long-term use.

[0014] The present invention has been made in order to solve the above-described problems caused by the prior art, and it is possible to simultaneously detect and recognize a moving body without performing pan “tilt” and zoom in an imaging apparatus. An object of the present invention is to provide a mobile object information detection apparatus, a mobile object information detection method, and a mobile object information detection program.

Means for solving the problem

In order to solve the above-described problems and achieve the object, the present invention provides a moving body information detection device that detects passage information regarding a moving body that passes through a predetermined region using an imaging device that captures a predetermined region. A central area processing means for performing first image processing using a moving body image projected in a central portion of the area photographed by the imaging device, and a region within the area photographed by the imaging device. Peripheral area processing means for performing second image processing using a moving body image projected on the peripheral portion of the central portion, and first information detected by the first image processing by the central area processing means And passage information detection means for detecting, as the passage information, information in which the second information detected in the second image processing by the peripheral area processing means is associated with each moving object. To do. [0016] Further, according to the present invention, in the above invention, the central area processing means and Z or the peripheral area processing means determine the reliability of the detected first information and Z or second information, respectively. And the passing information detecting means associates the first information with the second information based on the reliability calculated by the central area processing means and Z or the peripheral area processing means. Features.

[0017] Also, in the present invention according to the above-described invention, the passage information detection unit may perform the central region processing when the reliability calculated by the central region processing unit and Z or the peripheral region processing unit is low. And Z or the surrounding area processing means is instructed to redetect the first information and Z or the second information.

[0018] Also, in the above invention, the present invention is the first information detected in the first image processing by the central region processing means and the second information processing detected by z or the peripheral region processing means. In addition, a mobile body collating means for identifying the mobile body by collating the second information with the mobile body specifying information stored in advance is further provided.

[0019] Further, according to the present invention, in the above invention, when the mobile body collating means collates the first information and Z or the second information, the mobile body identifying information is included in the mobile body specifying information. If there is no matching information, the first information and the Z or second information are added to the information for specifying the moving object.

[0020] Further, according to the present invention, in the above invention, the central area processing means detects a traveling path of the moving body in the first image processing, and the peripheral area processing means is the second area processing device. In the image processing, the attribute information of the moving object is detected, and the passing information detecting means determines the traveling path detected by the central area processing means and the attribute information detected by the peripheral area processing means for each moving object. The associated information is detected as the passing information.

[0021] Further, in the present invention according to the above invention, the moving body is a person, and the central area processing unit detects a travel path of the person in the first image processing, and performs the peripheral area processing. The means detects the face information of the person in the second image processing, The passage information detecting means detects, as the passage information, information that associates the face information detected by the peripheral area processing means with the travel path detected by the central area processing means for each person. Features.

[0022] Further, the present invention is characterized in that, in the above invention, the imaging device photographs the predetermined area using a wide-angle lens.

[0023] Further, the present invention is characterized in that, in the above-mentioned invention, the imaging device photographs the predetermined area using a fisheye lens.

[0024] Further, the present invention is a moving body information detection method for detecting passage information relating to a moving body that passes through a predetermined area using an imaging apparatus that captures a predetermined area, and is shot by the imaging apparatus. A central region processing step for performing first image processing using an image of a moving object projected in the central portion of the region, and a peripheral portion of the central portion in the region captured by the imaging device. A peripheral region processing step of performing second image processing using the image of the moving body, a first information detected in the first image processing by the central region processing step, and a first region of the peripheral region processing step. A passage information detection step of detecting, as the passage information, information in which the second information detected in the second image processing is associated with each moving body.

[0025] Further, the present invention provides a moving body information detection program for detecting passage information regarding a moving body that passes through a predetermined area using an imaging apparatus that takes an image of the predetermined area, which is shot by the imaging apparatus. A central area processing procedure for performing a first image processing using an image of a moving object displayed in the central portion of the image area, and a peripheral portion of the central portion in the area photographed by the imaging device. A peripheral region processing procedure for performing the second image processing using the image of the moving body, a first information detected by the first image processing by the central region processing procedure, and a second information by the peripheral region processing procedure. A passage information detection procedure for detecting, as the passage information, information in which the second information detected by the second image processing is associated with each moving object;

Are executed by a computer.

The invention's effect

[0026] According to the present invention, the first image processing is performed using the image of the moving body imaged in the central portion in the area photographed by the imaging device, and the center in the area photographed by the imaging device is performed. The second image processing is performed using the moving body image displayed in the peripheral portion of the portion, and the first information detected by the first image processing and the second information detected by the second image processing are transferred. Since the information associated with each moving object is detected as passing information, the number of moving objects and the traveling route are detected based on the first information, and the moving object is recognized based on the second information. As a result, it is possible to simultaneously detect and recognize a moving body without performing pan / tilt zoom in the imaging apparatus. In addition, since this configuration can be realized with a single imaging apparatus, it is possible to reduce the cost of installation and troublesome position adjustments and installation costs.

[0027] According to the present invention, the reliability of the detected first information and Z or second information is calculated. Based on the calculated reliability, the first information, the second information, Therefore, it is possible to suppress the adverse effect such as a decrease in the detection performance of the moving body due to disturbance factors such as illuminance change and to detect the passage information of the person stably.

[0028] Further, according to the present invention, when the calculated reliability is low, the first information and the Z or second information are instructed to be re-detected. Even when a state in which an error is likely to occur in the detection of a moving body due to a disturbance element occurs, the moving body information can be detected with high accuracy.

[0029] Further, according to the present invention, the first information detected in the first image processing and the second information detected in the second image processing or the second information detected in the second image processing are stored in advance. Since the mobile object is specified by collating with the information, it is possible to specify the mobile object simply by recognizing the mobile object. This makes it possible to count the number of passes for each moving object, or to detect the moving object to be monitored that has been determined in advance, and to improve the function of the moving object detection function. There is an effect that can be.

[0030] Further, according to the present invention, when the first information and the Z or second information are collated, there is no matching information in the mobile object specifying information! Is configured to add the first information and Z or the second information to the mobile object identification information, so it becomes possible to learn and increase the number of mobile objects that can be identified, and the mobile object detection function. If you can improve it automatically! [0031] Further, according to the present invention, the traveling path of the moving body is detected in the first image processing, the attribute information of the moving body is detected in the second image processing, and the detected traveling path and attribute are detected. Since the information associated with each moving object is detected as passing information, it is possible to recognize the moving object based on the attribute information and simultaneously detect the traveling path of the moving object! / , Has an effect.

[0032] According to the present invention, the moving body is a person, the person's path is detected in the first image processing, the person's face information is detected in the second image processing, and the detected progress is detected. Since it is configured to detect the information that correlates the path and face information for each person as passage information, when the person is recognized based on the face information and at the same time the path of the person can be detected at the same time, Has an effect.

[0033] Further, according to the present invention, since the imaging apparatus is configured to capture a predetermined area using a wide-angle lens, the area captured by the imaging apparatus is wider than a normal lens. More passage information can be detected for each moving body, and the accuracy of detecting and recognizing the moving body can be improved.

[0034] Further, according to the present invention, since the imaging apparatus is configured to capture a predetermined area using a fisheye lens, the area captured by the imaging apparatus is wider than a normal lens or a wide-angle lens. As a result, more passing information can be detected for each moving body, and the accuracy of detecting and recognizing the moving body can be improved.

Brief Description of Drawings

FIG. 1 is a diagram (1) for explaining the concept of the person information detecting apparatus according to the first embodiment.

FIG. 2 is a diagram (2) for explaining the concept of the person information detecting apparatus according to the first embodiment.

FIG. 3 is a functional block diagram of the configuration of the person information detecting apparatus according to the first embodiment.

[FIG. 4] FIG. 4 is a diagram (1) for explaining an example of image conversion of a peripheral region by a processed image conversion unit.

[FIG. 5] FIG. 5 is a diagram for explaining an example of image conversion of a peripheral region by the processing image conversion unit. It is figure (2).

FIG. 6 is a diagram (3) for explaining an example of the image conversion of the peripheral area by the processed image conversion unit.

FIG. 7 is a diagram (1) for explaining an example of image conversion of the central region by the processed image conversion unit.

FIG. 8 is a diagram (2) for explaining an example of image conversion of the central region by the processed image conversion unit.

FIG. 9 is a diagram for explaining an example of person detection by the central region processing unit.

FIG. 10 is a diagram for explaining person association by the processing result integration unit.

FIG. 11 is a flowchart showing a processing procedure of the person information detecting apparatus according to the first embodiment.

FIG. 12 is a functional block diagram of the configuration of the person information detecting apparatus according to the second embodiment.

FIG. 13 is a diagram for explaining re-detection by a reliability determination unit.

FIG. 14 is a functional block diagram of the configuration of the person information detecting apparatus according to the third embodiment.

FIG. 15 is a functional block diagram of the configuration of the person information detecting apparatus according to the fourth embodiment.

FIG. 16 is an explanatory diagram for explaining the concept of the package information detecting apparatus according to the fifth embodiment.

FIG. 17 is a functional block diagram illustrating the configuration of the package information detecting apparatus according to the fifth embodiment.

FIG. 18 is a functional block diagram illustrating a configuration of a computer that executes a moving body information detection program according to the present embodiment.

FIG. 19 is a diagram (1) for explaining the method of Patent Document 1.

FIG. 20 is a diagram (2) for explaining the method of Patent Document 1.

21] FIG. 21 is a diagram (3) for explaining the method of Patent Document 1. In FIG.

FIG. 22 is a diagram (4) for explaining the method of Patent Document 1. FIG. 23 is a diagram for explaining the method of Patent Document 2.

FIG. 24 is a diagram for explaining the method of Patent Document 3.

FIG. 25 is a diagram for explaining problems of a method using a plurality of cameras.

[FIG. 26] FIG. 26 is a diagram for explaining the problems of the method using the PTZ camera. Explanation of symbols

100, 200, 300, 400 Person information detection device

500 Baggage information detection device

110 AZD converter

111 Image storage

112 Peripheral area processing result storage

113 Central area processing result storage

114 Image controller

115 Processed image converter

116 Peripheral area processing section

117 Central area processing section

118 Processing result integration section

219 Reliability judgment unit

320 Face image matching unit

521 Side image matching unit

522 Top image matching unit

600 computers

610 RAM

611 Mobile body information detection program

620 CPU

621 Moving object information detection task

630 ROM

640 LAN interface

650 camera interface BEST MODE FOR CARRYING OUT THE INVENTION

[0037] Exemplary embodiments of a mobile object information detection apparatus, a mobile object information detection method, and a mobile object information detection program according to the present invention will be described below in detail with reference to the accompanying drawings. In the present embodiment, the case where the present invention is applied to a person information detecting apparatus that detects passage information relating to a person passing through a passage will be mainly described.

Example 1

[0038] First, the concept of the person information detecting apparatus according to the first embodiment will be described. 1 and 2 are diagrams (1) and (2) for explaining the concept of the person information detecting apparatus according to the first embodiment.

[0039] The human information detection apparatus according to the first embodiment simultaneously performs different image processing for each area in the video, with respect to the video taken by the monitoring camera installed above the area to be monitored, such as the ceiling. By doing so, the detection of the passing person and the face image of the person are acquired and recognized.

[0040] The surveillance camera used here is a camera that performs photographing using a wide-angle lens such as a fisheye lens. By using a camera with a wide-angle lens, the area to be photographed becomes wider than that of a normal lens, and more passing information can be detected for each person, improving the accuracy of person detection and recognition. be able to.

Specifically, as shown in FIG. 1, a camera 10 having a wide-angle lens is installed on the upper part of the monitoring area, and thereby a passing person is photographed. For example, as shown in the figure, it is assumed that the person A passes under the camera 10 between times T1 and T3. In this case, the camera 10 captures an image as shown in FIG. As shown in the figure, at time T1, person A is located far away from the front of camera 10 (peripheral part), so the face is photographed. At time T2, person A is near the front of camera 10 ( Only the head is photographed (time T2).

[0042] Therefore, the person information detecting device has two different processes: a process of detecting the number of people passing using the central part of the video, and a process of detecting and recognizing a face image of a passing person using the peripheral part of the video. Perform image processing simultaneously. Furthermore, by tracking the face detected in the peripheral part and the person detected in the central part, the passing person and the face are matched (matching )

As described above, in the person information detecting apparatus according to the first embodiment, the detection (counting) of the person who has passed without performing the pan “tilt” zoom with the camera and the acquisition (recognition) of the face image at the same time. In addition to being able to do this, it is possible to provide new functions that are more than just solving conventional technologies, such as counting the number of passing people with high accuracy and acquiring movement trajectories of passing people by associating passing people with faces. System can be realized.

Next, the configuration of the person information detecting apparatus according to the first embodiment will be described. FIG. 3 is a functional block diagram of the configuration of the person information detecting apparatus according to the first embodiment. As shown in the figure, the person information detecting device 100 inputs an analog video sent from the camera 10 or a digital video sent from an image recording device (not shown). This is a device that detects a face image of a person who has been detected and outputs these as passage information. The AZD conversion unit 110, the image storage unit 111, the peripheral region processing result storage unit 112, and the central region processing result storage unit 113, an image control unit 114, a processed image conversion unit 115, a peripheral region processing unit 116, a central region processing unit 117, and a processing result integration unit 118.

[0045] The AZD conversion unit 110 is a processing unit that digitizes an analog video sent from the camera 10. Specifically, the AZD conversion unit 110 converts an analog image that can be transmitted by the camera 10 into a digital video, and sends the converted digital video to the image control unit 114. If the input video is a digital video, the AZD conversion unit 110 is not required due to its configuration.

[0046] The image storage unit 111 is a storage unit that stores video data of a digitalized input video. Specifically, the image storage unit 111 stores video data sequentially transmitted from an image control unit 114, which will be described later, for a predetermined period of time in a predetermined time interval.

[0047] The peripheral region processing result storage unit 112 is a storage unit that stores the processing result of the face detection processing and the video data used for the processing. Specifically, the peripheral region processing result storage unit 112 includes a processing result of a face detection process sent from a peripheral region processing unit 116 described later and a processing result of a processing result integration process sent from a processing result integration unit 118 described later. Are stored in time series at predetermined time intervals.

[0048] The central region processing result storage unit 113 is used for the processing result and processing of the person detection processing. A storage unit for storing the received video data. Specifically, the central region processing result storage unit 113 includes a processing result of a person detection process sent from a central region processing unit 117 described later and a processing result of a processing result integration process sent from a processing result integration unit 118 described later. Are stored in chronological order at predetermined time intervals.

[0049] The image control unit 114 is a process for controlling the input of digital video sent from the AZD conversion unit 110 and an image recording device (not shown) and the input / output of the input digital video to the image storage unit 111. Part. Specifically, the image control unit 114 inputs digital video sent from the AZD conversion unit 110 and the image recording device, and sequentially sends them to the image storage unit 111 at predetermined time intervals. Further, the image control unit 114 stores the video data stored in the image storage unit 111 when the video data is stored in the image storage unit 111 and the processed image conversion unit 115 is in a processable state. The images are sequentially extracted and sent to the processed image conversion unit 115 sequentially.

The processed image conversion unit 115 is a processing unit that converts the digital video sequentially sent from the image control unit 114 into an image to be processed by the peripheral region processing unit 116 and the central region processing unit 117. Specifically, the processed image conversion unit 115 divides the digital video sequentially sent from the image control unit 114 into a peripheral image and a central image, converts each image, and then converts the peripheral image. Is sent to the peripheral region processing unit 116, and the central image is sent to the central region processing unit 117. Hereinafter, the image conversion of the peripheral part and the image conversion of the central part will be described.

First, the image conversion of the peripheral part will be described. 4, 5, and 6 are diagrams (1), (2), and (3) for explaining an example of image conversion of the peripheral area by the processed image conversion unit 115. FIG. For example, as shown in FIG. 4, the processed image conversion unit 115 removes the central region from the processing target and forms a rectangular image with the lines PQ and RSS as opposite sides. By expanding the peripheral area, conversion to an image of only the peripheral part is performed. In this case, as shown in the figure, the developed images may be two or one. Alternatively, as shown in FIG. 5, the processed image conversion unit 115 may perform processing that is not subject to processing by painting the center portion without expanding the image.

[0052] Also, as shown in FIG. 6, when the input video is a fish-eye video, the processed image conversion unit 115 normally converts the video from a fish-eye video using a known video conversion technique. You may convert to video and convert it to a rectangular image with no distortion and line TU and line Vw.

[0053] As a known video conversion technique used here, for example, a video described in "Kobayashi et al.," Fisheye lens camera image correction ", 2005 General Conference Proceedings A-4 19, IEICE. Conversion technology and so on.

[0054] Next, image conversion in the central area will be described. 7 and 8 are diagrams (1) and (2) for explaining an example of image conversion of the central region by the processed image conversion unit 115. FIG. For example, as illustrated in FIG. 7, the processed image conversion unit 115 converts the input image into an image of only the central portion, excluding the peripheral region from the processing target. Alternatively, as shown in FIG. 8, the processed image conversion unit 115 may perform processing to be excluded from the processing target by painting the peripheral portion.

[0055] Also, as shown in FIG. 6, when the input video is a fisheye video, the processed image conversion unit 115 performs a known video conversion technique on the video in the same manner as the image conversion of the peripheral area. Using the technique, the fish-eye image power is also converted to normal image.

[0056] The peripheral region processing unit 116 is a processing unit that performs face detection processing on an image of a peripheral part converted by the processing image conversion unit 115. Specifically, the peripheral region processing unit 116 performs face detection processing on the peripheral image sent from the processing image conversion unit 115 using a known face detection technique, and the detection result includes Acquires the position and size of the face, the face image, and the evaluation value in the detection process (for example, the output value of the neural network representing the likelihood of a face). Then, the peripheral area processing unit 116 sends these detection results to the peripheral area processing result storage unit 112.

As a known face detection technique used here, for example, in this face detection process, “H.

A. Rowley et al., Neural network-based face detection, IEEE Trans, on Pattern Analysis and Machine Intelligence Vol. 20, No. 1, pp. 23-38, 1998 ”.

Further, the peripheral region processing unit 116 compares the detection result of the previous time and the detection result of the current time held in the peripheral region processing result storage unit 112, thereby moving the path of the face in the image. Also detect. Specifically, for example, the face image detected at the previous time and the current time are detected. The normalized correlation shown in the following formula (1) is calculated using the value representing the face image that is output, and when a correlation value greater than a certain value is obtained, the face of each face image is Judge that they are the same. In Equation (1), “X” and “Y” indicate the pixel values of the face image detected at the previous time and the current time, respectively. “Χ′— (bar)” and “— (bar)” The average values of the face images detected at the previous time and the current time are shown.

[0059]

... I)

By determining in this way, even when a plurality of persons exist at the same time, it is possible to distinguish each person and detect the traveling path of each person. The peripheral area processing unit 116 sends the progress path detection result to the peripheral area processing result storage unit 112 even if it is detected.

[0061] In addition, in the peripheral portion image, only the back side of the person is photographed and the face may not be reflected. In this case, for example, the peripheral region processing unit 116 learns the image of the back of the head during learning, for example. Thus, the position of the person is estimated based on the person's travel path information detected by the processing result integration unit 118 described later.

The central area processing unit 117 is a processing unit that performs person detection processing on the central partial image converted by the processing image conversion unit 115. Specifically, the central area processing unit 117 performs a person (silhouette) detection process on the central part image sent from the processing image conversion unit 115 using a known person detection technique.

[0063] Well-known person detection techniques used here include, for example, the techniques described in "Sasamura et al.," Research on differential system by automatic background update ", IEICE General Conference, 2003". It is done. This technology calculates the difference between a reference image that does not appear as a detection target, called a background image, and the input image, and detects a person who exists in the central area by detecting the difference area as an object. Is detected.

By performing such person detection processing, the central area processing unit 117 obtains the position and size of the person in the input image, the image of the person (the person's head) as the detection result by the person detection processing. Part) and the evaluation value in the detection process. FIG. 9 is a diagram for explaining an example of person detection by the central area processing unit 117. For example, as shown in the figure, the central area processing unit 117 compares a human model prepared in advance with a background difference result, and uses the degree of coincidence as an evaluation value.

Then, the central area processing unit 117 sends these detection results to the central area processing result storage unit 113. Similarly to the peripheral region processing unit 116, the central region processing unit 117 compares the detection result of the previous time stored in the central region processing result storage unit 113 with the detection result of the current time, thereby The traveling path in the image is also detected. For example, similar to the peripheral region processing unit 116, the central region processing unit 117 compares person videos using the normal correlation value expressed by the equation (1).

[0066] The processing result integration unit 118 integrates the face detection result held in the peripheral region processing result storage unit 112 and the person detection result held in the central region processing result storage unit 113, and passed. It is a processing unit that performs counting of persons and acquisition of face images of persons who have passed.

In addition, the processing result integration unit 118 integrates the processing results from the peripheral region processing unit 116 and the central region processing unit 117, and associates the passed person with the face image. Specifically, the processing result integration unit 118 compares the detection result of the person's (face) travel path in the peripheral image with the detection result of the person's travel path in the central image, Match those that satisfy the movements of

[0068] For example, the processing result integration unit 118 detects the locus of a point defined at a predetermined position of the face (for example, the upper end of the face) detected in the peripheral image and the image of the central part. The trajectory of a point defined at a predetermined position of the person (for example, the top of the head) is analyzed, and the face and the person whose trajectories are most naturally connected are associated with each other as the same person.

FIG. 10 is a diagram for explaining person association by the processing result integration unit 118.

For example, as shown in the figure, during a certain period of time, four persons (peripheral persons 1 to 4) are detected in the peripheral area, and two persons (central persons 1 and 2) are detected in the central area. (See “Input Image and Detection Position” on the left side of the figure). In this case, the processing result integration unit 118 integrates the detection results in the respective regions, and associates the peripheral person 1, the central person 1, and the peripheral person 4 as the same person from the traveling path (see “Integration” on the right side of the figure). "Person" shown in "Result" 1), and the peripheral person 2, the central person 2, and the peripheral person 3 are associated as the same person (see “person 2” shown in “integration result” on the right side of the figure).

[0070] Then, the processing result integration unit 118 notifies the monitoring center or the like of the result of associating the person's travel route in the input image with the face image of the person, and also the peripheral region processing result storage unit 112 and the central region processing result storage unit 113.

Next, a processing procedure of processing performed in the person information detection apparatus 100 according to the first embodiment will be described. FIG. 11 is a flowchart illustrating the processing procedure of the person information detecting apparatus 100 according to the first embodiment. As shown in the figure, in the person information detecting apparatus 100, first, the A / D conversion unit 110 digitizes the analog video transmitted from the camera 10 (step S101), and then the image control unit. 114 divides the input image into a peripheral region and a central region (step S102).

[0072] Then, for the divided input image, the peripheral area processing unit 116 detects information about the image force / face of the peripheral part (step S103), and the central area processing unit 117 detects the image power of the central part and information about the person. Is detected (step S104).

Thereafter, the processing result integration unit 118 associates the face detection result with the person detection result (step S105), and notifies the monitoring center or the like of the processing result (step S106).

[0074] As described above, in the first embodiment, the central area processing unit 117 detects an individual person using an image of a person shown in the central part of the area captured by the camera 10. The image processing is performed, and the peripheral region processing unit 116 performs image processing for detecting a person's face using the image of the person shown in the peripheral part of the central part in the region photographed by the camera 10, and the processing result The integration unit 118 detects, as passage information, information that associates each person detected by the central region processing unit 117 and the face detected by the peripheral region processing unit 116 with each person. In addition to recognizing people, it is possible to detect the number of passages and travel paths based on the individual people detected, and simultaneously detect and recognize people without panning and tilting with the camera 10. be able to. In addition, since this configuration can be realized with a single imaging device, it is possible to reduce the troublesome position adjustment at the installation site and the cost of introduction.

[0075] The passing information detected in this way can be used in various ways. Example For example, log powerful information on passing traffic and track the suspicious person by searching the log information when a crime occurs, or pass through a potential location for a new commercial facility. This can be used when investigating the attributes and number of people.

Example 2

By the way, in the first embodiment, the case where the face detection result in the peripheral image and the person detection result in the central image are simply integrated has been described. Re-detection based on the reliability of the detected detection result, or high reliability! The detection result is low in reliability. Therefore, in the following, a case where control based on strong reliability is performed will be described as a second embodiment.

FIG. 12 is a functional block diagram of the configuration of the person information detecting apparatus according to the second embodiment. Of the functional units shown in the figure, those having the same functions as the functional units included in the person information detecting apparatus 100 shown in FIG. 3 are denoted by the same reference numerals. The detailed explanation is omitted.

As shown in the figure, the person information detecting device 200 inputs an analog video sent from the camera 10 or a digital video sent from an image recording device (not shown), and passes through the input video. A device that detects the number of people and face images of people who have passed through and outputs them as passing information. AZD converter 110, image storage unit 111, peripheral region processing result storage unit 112, and central region processing The result storage unit 113, the image control unit 114, the processing image conversion unit 115, the peripheral region processing unit 116, the central region processing unit 117, the processing result integration unit 118, and the reliability determination unit 219 are included.

[0079] The reliability determination unit 219 determines the reliability of the detection results detected by the peripheral region processing unit 116 and the central region processing unit 117, and re-detects a face or a person when the reliability is low. It is a processing unit that performs control such that each detection result is integrated while complementing the detection result with low reliability using the detection result with high reliability. Note that the reliability here refers to, for example, the evaluation value of the face detection result by the neural network for the peripheral image (see the description of the face detection process by the peripheral region processing unit 116). The degree of coincidence with the person model of the background difference result (see the description of the person detection process by the center area processing unit 117) is used for the image of the center part.

FIG. 13 is a diagram for explaining re-detection by the reliability determination unit 219. For example, as shown in the figure, it is assumed that the reliability of the person detection result in the central area where the reliability of the face detection result in the peripheral area is small (the “input image and detection position” on the left of the figure). ). In this case, the reliability determination unit 219 uses the person detection result in the central region to process the processed image so that the peripheral region processing unit 116 performs face detection again around the position where the person is estimated to have passed. The conversion unit 115 is instructed (see “Redetection by the reliability determination unit” on the right side of the figure).

[0081] Here, since the reliability determination unit 219 re-detects a face or a person when the reliability is low, a state in which an error is likely to occur in the detection of a moving object due to a disturbance element such as an illuminance change occurs. Even when it is generated, it is possible to detect moving body information with high accuracy.

[0082] Further, the reliability determination unit 219 controls the integration of the detection results performed in the processing result integration unit 118 so as to associate the travel route based on the reliability. For example, the reliability determination unit 219, for a certain traveling path detected in the image of the central part, if there are multiple traveling paths that can be associated with the traveling path in the image of the peripheral part, The processing result integration unit 118 is controlled so that the one with the highest reliability is associated with the traveling path in the central region.

As described above, in the second embodiment, the reliability of the detection result detected by the peripheral region processing unit 116 and the central region processing unit 117 is calculated, and the reliability determination unit 219 is calculated. Based on the reliability, control is performed so that the path of travel is associated, so that adverse effects such as deterioration in the detection performance of moving objects due to disturbance factors such as illuminance changes are suppressed, and the passage information of a person is detected stably. be able to.

Example 3

By the way, in the first and second embodiments, the case where the peripheral region processing unit 116 detects the face image and the traveling path of the passing person has been described. However, the detected face image is registered in advance. It is also possible to identify the person who has passed by comparing with the face image of the person who has been sent. Therefore, in the following, a person is identified based on the detected face image. This case will be described as Example 3.

FIG. 14 is a functional block diagram of the configuration of the person information detecting apparatus according to the third embodiment. Of the functional units shown in the figure, those having the same functions as the functional units included in the person information detecting apparatus 100 shown in FIG. 3 are denoted by the same reference numerals. The detailed explanation is omitted.

[0086] As shown in the figure, the person information detecting device 300 inputs an analog video sent from the camera 10 or a digital video sent from an image recording device (not shown), and passes through the input image. A device that detects the number of people and face images of people who have passed through and outputs them as passing information. AZD converter 110, image storage unit 111, peripheral region processing result storage unit 112, and central region processing The result storage unit 113, the image control unit 114, the processing image conversion unit 115, the peripheral region processing unit 116, the central region processing unit 117, the processing result integration unit 118, and the face video matching unit 320 are included.

The face image collating unit 320 is a processing unit that identifies a person who has passed based on the face image detected by the peripheral region processing unit 116. Specifically, the face image collation unit 320 collates the face image detected by the peripheral region processing unit 116 with the face image of a person who has been registered in advance using a known face collation technique. The person who passed is identified.

[0088] As a known face matching technique used here, for example, "Komatsu et al.," Cut and recognize face without orientation using subspace method ", PRU95-191, pp.7-14, 1996. ". This technology compares a vector composed of face part positions such as eyes, nose and mouth with the registered face image and the detected face image, and identifies or resembles a person based on this similarity. The person who enumerates is enumerated.

By performing such face collation processing, the face image collation unit 320 identifies a person who has passed, and acquires information such as which person has passed and how many times. In addition, the face image collation unit 320 adds a new face image to the face image that has not been registered at that time in order to collate the next time it passes. .

[0090] As described above, in the third embodiment, the face video collation unit 320 collates the face video detected by the peripheral region processing unit 116 with the face video of the person who has been registered in advance. Do Therefore, it is possible to identify the person who has passed by simply recognizing the person. As a result, it is possible to count the number of passages for each person, or to detect a person to be monitored that has been determined in advance, so that the person detection function can be enhanced. In addition, since it is possible to improve the tracking accuracy of the person and to identify the person who has passed, it is possible to realize a more sophisticated function for detecting a passing person for security purposes.

[0091] Also, when the face image matching unit 320 collates the face image, if the matching face image does not exist in the face images of the registered human being, Since a new video is added, it becomes possible to learn and increase the number of people who can be identified, and the detection function of people can be automatically improved.

Example 4

By the way, in the above-described Embodiments 2 and 3, the reliability is higher than that of Embodiment 1! In the above description, the reliability determination unit 219 for performing control and the face image collation unit 320 for identifying a person based on the detected face image have been added. The person information detection apparatus may be configured by arbitrarily combining these. Therefore, in the following, a case where the second and third embodiments are combined will be described as a fourth embodiment.

FIG. 15 is a functional block diagram of the configuration of the person information detecting apparatus according to the fourth embodiment. It should be noted that each functional unit included in the person information detecting device 400 shown in the figure includes the person information detecting device 100 shown in FIG. 3, the person information detecting device 200 shown in FIG. 12, or the person information detecting device 300 shown in FIG. Therefore, detailed description thereof is omitted here.

[0094] As described above, in the fourth embodiment, by combining the functional units described in the first, second, and third embodiments, it is possible to detect a practical passing person with reduced influence of disturbance such as illuminance change. In addition to realizing a system, it is possible to increase the functionality of the person detection function, such as recognizing a passing person and calculating the number of times a particular person has passed.

Example 5

So far, the human body information detecting apparatus for detecting a person has been described as an example of the moving body information detecting apparatus according to the present invention. However, the present invention is not limited to this, and for example, a bell The present invention may be applied to a package information detection device that detects packages flowing on a conveyor. Therefore, in the following, the case where the present invention is applied to a package information detection apparatus will be described as Example 5.

First, the concept of the package information detecting apparatus according to the fifth embodiment will be described. FIG. 16 is an explanatory diagram for explaining the concept of the package information detecting apparatus according to the fifth embodiment.

[0097] The package information detection apparatus according to the fifth embodiment performs different image processing for each area in the video, with respect to the video shot by the monitoring camera installed on the upper part of the belt conveyor through which the package passes, such as the ceiling. By carrying out at the same time, the side and top of the package are detected. As the surveillance camera used here, a camera having a wide-angle lens such as a fisheye lens is used as in the embodiments described above.

[0098] Specifically, as shown in FIG. 16, a camera 10 having a wide-angle lens is installed on the upper part of the belt conveyor through which the luggage passes, whereby the luggage flowing on the belt conveyor is photographed.

[0099] As shown in the figure, the cargo flowing on the belt conveyor has labels attached to the side surface and the upper surface, respectively. This label shows the attributes related to the package, such as type and weight, in characters and colors. When the baggage is at a location (peripheral part) farther away from the front of the camera 10, the label attached to the side is photographed. When the baggage is near the front of the camera 10 (center part), it is attached to the upper surface. The label is shot.

[0100] Therefore, the package information detection apparatus uses two different images: a process of counting the package using the central part of the video and a process of detecting and recognizing the attribute indicated by the label on the side of the package using the peripheral part of the video. Process simultaneously. Furthermore, the load flowing on the belt conveyor is specified by associating the attribute indicated by the label on the side of the load detected in the peripheral portion with the attribute indicated by the label on the upper surface of the load detected in the central portion.

As described above, in the baggage information detection apparatus according to the fifth embodiment, the detection (counting) of the baggage flowing on the belt conveyor without performing the pan “tilt” zoom with the camera, and the side and upper surfaces of the baggage are detected. It is possible to detect the characters on the attached label at the same time and to associate the information on the label attached to the side with the information on the label attached to the upper surface.

FIG. 17 is a functional block diagram illustrating the configuration of the package information detection apparatus according to the fifth embodiment. . Of the functional units shown in the figure, those having the same functions as those of the human information detecting device 100 shown in FIG. 3 or the human information detecting device 200 shown in FIG. 12 are the same. The code | symbol is attached | subjected. Specifically, these functional units are detected in the peripheral area of the input image, the target is replaced with the label on the side of the facial load, and the target detected in the central area is replaced with the label on the upper side of the human load. Therefore, detailed explanation is omitted here.

[0103] As shown in the figure, the package information detecting device 500 inputs an analog video sent from the camera 10 or a digital video sent from a camera such as an image recording device (not shown), and inputs the input image. The number of loads flowing on the belt conveyor, the images of the side and top surfaces of the luggage, etc. are detected and output as passage information.The AZD conversion unit 110, the image storage unit 111, and the peripheral Region processing result storage unit 112, central region processing result storage unit 113, image control unit 114, processing image conversion unit 115, peripheral region processing unit 116, central region processing unit 117, and processing result integration unit 118, a reliability determination unit 219, a side image collation unit 521, and a top image collation unit 522.

The side image collating unit 521 is a processing unit that identifies the attribute of the package flowing on the belt conveyor based on the image of the label on the side of the package detected by the peripheral area processing unit 116. Specifically, the side image collation unit 521 detects the character indicated by the label from the image of the label on the side of the package detected by the peripheral region processing unit 116, and registers the detected character and the character in advance. The attributes of the package are specified by collating with the characters using a known character collation technique.

The upper surface image collating unit 522 is a processing unit that identifies the attribute of the load flowing on the belt conveyor based on the image of the label on the upper surface of the load detected by the central area processing unit 117. Specifically, the upper surface image collation unit 522 detects the character indicated by the label from the image of the label on the upper surface of the package detected by the central region processing unit 117, and registers the detected character and the character in advance. The attributes of the package are specified by collating with the characters using a known character collation technique.

For example, “Negishi et al.,” “High-speed allowing deformation for character recognition in scene image” is used as a known character matching technique used in the side image matching unit 521 and the top image matching unit 522. Template matching method described in "Nano Template Matching", PRMU2005-247, ρρ.101-106, 2005 ". This technology collates characters by comparing them with previously registered dictionary images.

As described above, the side image collating unit 521 collates the information indicated on the label on the package side detected by the peripheral region processing unit 116 with the information for identifying the moving body accumulated in advance, The collation unit 522 identifies the baggage by collating the information shown on the label on the upper surface of the baggage detected by the central area processing unit 117 with the information for identifying the mobile object stored in advance. It is also possible to specify the name. As a result, it is possible to detect a package to be detected that has been determined in advance, and to enhance the function of the package detection function.

[0108] Although the case where the package attribute is specified using the characters shown on the label has been described here, the package attribute may be specified using a color other than the characters.

[0109] As described above, in the fifth embodiment, the peripheral area processing unit 116 is displayed on the label on the upper surface of the luggage using the image of the luggage displayed in the peripheral area of the area photographed by the camera 10. The center area processing unit 117 detects the information shown on the label on the side of the luggage using the image of the luggage displayed in the center of the area photographed by the camera 10. The processing result integrating unit 118 uses the information associated with the label information detected by the peripheral region processing unit 116 and the label information detected by the central region processing unit 117 for each package as passing information. Since it is detected, it is possible to simultaneously count the number of packages that pass through and recognize the side and top surfaces of the packages.

In the first to fifth embodiments, the mobile body information detection device (person information detection device and baggage information detection device) has been described. However, the configuration of the mobile body information detection device is realized by software. Thus, a mobile object information detection program having the same function can be obtained. Therefore, a computer that executes the mobile body information detection program will be described.

FIG. 18 is a functional block diagram illustrating a configuration of a computer that executes a mobile object information detection program according to the present embodiment. As shown in the figure, the computer 600 includes a RAM (Random Access Memory) 610, a CPU (Central Processing Unit) 620, a ROM ( It has a Read Only Memory (630), a LAN (Local Area Network) interface 640, and a power interface 650.

[0112] The RAM 610 is a memory that stores a program and a program execution result, and the CPU 620 is a central processing unit that reads a program from the RAM 610 and the ROM 630 and executes the program.

[0113] The ROM 630 is a memory for storing programs and data, and the LAN interface 640 is an interface for connecting the computer 600 to another computer or a terminal device of a monitoring center via the LAN. Is an interface for connecting surveillance cameras.

Then, the moving body information detection program 611 executed in the computer 600 is stored in advance in the ROM 630 and executed as the moving body information detection task 621 by the CPU 620.

[0115] In the present embodiment, the person information detection device for detecting a person and the load information detection device for detecting a load have been described. However, the present invention is not limited to this example. The present invention can also be applied in the same manner when detecting a car that is running.

[0116] In addition, among the processes described in the present embodiment, all or part of the processes described as being performed automatically can be performed manually, or are described as being performed manually. All or part of the processing can be automatically performed by a known method.

[0117] In addition, the processing procedure, control procedure, specific name, information including various data and parameters shown in the above-mentioned document and drawings can be arbitrarily changed unless otherwise specified.

[0118] Each component of each illustrated device is functionally conceptual and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or a part thereof is functionally or physically distributed in an arbitrary unit according to various loads and usage conditions. · Can be integrated and configured.

Industrial applicability As described above, the mobile body information detection apparatus, the mobile body information detection method, and the mobile body information detection program according to the present invention use the imaging device that captures a predetermined area and pass through the mobile body that passes through the area. This is useful for detecting information, and is particularly suitable for simultaneously detecting and recognizing a moving object without performing pan “tilt” zooming with an imaging device.

Claims

The scope of the claims

[1] A moving body information detecting apparatus that detects passing information about a moving body that passes through a predetermined area using an imaging device.

Using the image of the moving body shown in the center of the area photographed by the imaging device

V, central area processing means for performing first image processing;

A peripheral region processing means for performing second image processing using an image of a moving body imaged on a peripheral portion of the central portion in the region captured by the imaging device;

Information in which the first information detected by the first image processing by the central area processing means and the second information detected by the second image processing by the peripheral area processing means are associated with each moving object. Passage information detecting means for detecting the passage information;

A moving body information detection apparatus comprising:

[2] The central area processing means and Z or the peripheral area processing means calculate the reliability of the detected first information and Z or the second information, respectively.

The passage information detecting means associates the first information with the second information based on the reliability calculated by the central area processing means and Z or the peripheral area processing means. Item 4. The moving body information detection device according to Item 1.

[3] The passage information detection unit is configured to detect the central region processing unit and Z or the peripheral region processing unit when the reliability calculated by the central region processing unit and Z or the peripheral region processing unit is low. 3. The mobile body information detection apparatus according to claim 2, wherein the mobile information detection apparatus instructs to perform redetection of the first information and Z or the second information.

[4] The first information detected in the first image processing by the central region processing means and the second information detected in the second image processing by Z or the peripheral region processing means are stored in advance. 2. The moving body information detecting apparatus according to claim 1, further comprising moving body collating means for identifying the moving body by collating with the moving body specifying information.

[5] When the mobile body collating means collates the first information and Z or the second information, there is no matching information in the mobile body identifying information! / Case 5. The moving body information detecting apparatus according to claim 4, wherein the first information and z or second information are added to the moving body specifying information.

[6] The central region processing means detects a traveling path of the moving body in the first image processing,

The peripheral region processing means detects the attribute information of the moving object in the second image processing,

The passage information detection unit detects, as the passage information, information in which a traveling path detected by the central region processing unit and attribute information detected by the peripheral region processing unit are associated with each moving object. The moving body information detecting device according to claim 1.

[7] The moving body is a person,

In the first image processing, the central area processing means detects a travel path of the person,

The peripheral area processing means detects the face information of the person in the second image processing,

The passage information detection unit detects, as the passage information, information associating the travel path detected by the central region processing unit with the face information detected by the peripheral region processing unit for each person. The moving body information detecting device according to claim 1

8. The moving body information detecting apparatus according to any one of claims 1 to 7, wherein the imaging apparatus captures the predetermined area using a wide-angle lens.

9. The moving body information detecting device according to any one of claims 1 to 7, wherein the imaging device images the predetermined area using a fisheye lens.

[10] A moving body information detection method for detecting passing information about a moving body passing through a predetermined area using an imaging device that captures a predetermined area,

A central area processing step for performing first image processing using an image of a moving object projected on a central portion in an area captured by the imaging device; and

The transition projected on the peripheral part of the central part in the area photographed by the imaging device. A peripheral region processing step of performing second image processing using an image of a moving object;

Information in which the first information detected by the first image processing by the central region processing step and the second information detected by the second image processing by the peripheral region processing step are associated with each moving object. A passage information detection step for detecting the passage information;

A moving body information detection method comprising:

A moving body information detection program for detecting passing information relating to a moving body that passes through a predetermined area using an imaging device.

A central region processing procedure for performing a first image processing using an image of a moving object projected on a central portion in a region captured by the imaging device;

A peripheral region processing procedure for performing second image processing using an image of a moving body imaged on a peripheral portion of the central portion in the region captured by the imaging device;

Information that associates the first information detected by the first image processing by the central region processing procedure and the second information detected by the second image processing by the peripheral region processing procedure for each moving object. A passage information detection procedure to detect as the passage information;

A program for detecting moving body information, which causes a computer to execute.