JP4667508B2 - Mobile object information detection apparatus, mobile object information detection method, and mobile object information detection program - Google Patents

Description

  The present invention relates to a moving body information detecting device, a moving body information detecting method, and a moving body information detecting program for detecting passing information about a moving body that passes through a predetermined area using an imaging device that captures a predetermined area. The present invention relates to a moving body information detecting apparatus, a moving body information detecting method, and a moving body information detecting program capable of simultaneously detecting and recognizing a moving body without performing pan / tilt / zoom in an imaging apparatus.

  2. Description of the Related Art Conventionally, there is a human information detection device that automatically detects (recognizes) a person by photographing a person passing through a passage using a camera installed on an upper part such as a ceiling and processing the image. This person information detection device is used, for example, in an entrance / exit monitoring system for monitoring a person entering / exiting an entrance / exit of a facility and a passing person counting system for counting the number of persons passing through a sidewalk or a passage.

  In such an entry / exit monitoring system and a passing person counting system, it is desired that persons who enter / exit and pass through can be detected (counted) without omission and that it can be realized with a small number of cameras from the viewpoint of introduction / operation costs. Furthermore, in recent years, since security awareness has increased, it is also important to acquire a face image as information for specifying a person who has passed.

  In order to realize such demands, methods proposed so far include a method of using a camera that takes a picture from directly above an area to be monitored, a method of using a plurality of cameras, a pan / tilt / zoom method, and the like. There is a method using a camera having a function (PTZ camera).

  An example of a method using a camera that captures an image directly below an area to be monitored is described in Patent Document 1 (Japanese Patent Laid-Open No. 6-223157). 19 to 22 are diagrams (1) to (4) for explaining the method of Patent Document 1. FIG. In this method, a camera is placed directly on the ceiling (see FIG. 19), a change in a captured image (see FIG. 20) is recognized by image processing, and a person who has passed this change is detected. . Thereby, since the camera and the passage are facing each other, only the person who has passed can be photographed, and the depth can be obtained as in the case of photographing obliquely in the direction in which the person walks (see FIG. 21). No other person appears in the direction (see FIG. 22), and the passing person can be distinguished and detected correctly one by one.

  As a method using a plurality of cameras, for example, there is a method described in Patent Document 2 (Japanese Patent Laid-Open No. 9-54894). FIG. 23 is a diagram for explaining the technique of Patent Document 2. In FIG. As shown in the figure, in this method, not only directly below but also in the direction in which the person walks, a camera is installed to recognize the face of the passing person. Thereby, it is possible to acquire and recognize not only the number of passing persons but also the face images of the passing persons.

  As a method using a PTZ camera, for example, there is a method described in Patent Document 3 (Japanese Patent Laid-Open No. 10-188145). FIG. 24 is a diagram for explaining the method of Patent Document 3. As shown in the figure, in this method, a passing person is detected from widely captured images, and the camera is panned, tilted, and zoomed at the detected position to acquire and recognize a person's face image. . This makes it possible to detect (count) a person and acquire and recognize a face image with one camera.

  As described above, various conventional techniques for detecting a passing person have been proposed, and their features are also different.

JP-A-6-223157 JP-A-9-54894 Japanese Patent Laid-Open No. 10-188145

  However, the conventional techniques represented by Patent Documents 1, 2, and 3 have the following problems.

  First of all, the method using a camera that captures the image from directly above the area to be monitored can count the number of people passing through, but cannot acquire facial images for recognizing people, and is not suitable for security purposes. is there.

  Further, the method using a plurality of cameras has the following problems. FIG. 25 is a diagram for explaining the problem of the method using a plurality of cameras. As shown in the figure, in this method, in order to associate the same person appearing in different cameras with the same person by image processing, it is necessary to strictly install each camera, and the installation position shifts due to secular change or the like. Considering this, there is a problem that the cost of on-site adjustment is large. Further, since this method uses a plurality of cameras, there is a problem of introduction cost and installation space.

  Further, the method using the PTZ camera has the following problems. FIG. 26 is a diagram for explaining a problem of the method using the PTZ camera. As shown in the figure, this method pans, tilts, and zooms the camera so as to focus on one passing person, so if multiple persons pass through the monitoring area at the same time, There is a problem that it cannot be detected. In contrast to this problem, when a plurality of PTZ cameras are used, there is the same problem as a method using a plurality of cameras. In addition, the pan / tilt / zoom operation has a problem that it is difficult to guarantee a long-time use because the camera is mechanically worn.

  The present invention has been made to solve the above-described problems caused by the prior art, and is capable of simultaneously detecting and recognizing a moving object without performing pan / tilt / zoom in an imaging apparatus. An object of the present invention is to provide an information detection apparatus, a mobile object information detection method, and a mobile object information detection program.

  In order to solve the above-described problems and achieve the object, the present invention provides a moving body information detection apparatus that detects passing information about a moving body that passes through a predetermined area using an imaging device that captures a predetermined area. A central area processing means for performing first image processing using a moving body image projected on a central portion in the area photographed by the imaging device; and the central portion in the area photographed by the imaging device. Peripheral area processing means for performing second image processing using a moving body image projected on the peripheral part of the first information, the first information detected in the first image processing by the central area processing means, and the peripheral area Passage information detecting means for detecting, as the passage information, information in which second information detected by the second image processing by the processing means is associated with each moving object.

Further, the present invention is the above invention, wherein at least one of the central area processing means and the peripheral area processing means calculates the reliability of at least one of the detected first information and second information, respectively. The passage information detecting means associates the first information with the second information based on the reliability calculated by at least one of the central area processing means and the peripheral area processing means. To do.

The present invention, in the above invention, the passage information detection means, wherein, when the central region processing means and reliability calculated by at least one of said peripheral region processing means is low, the central region processing means and said It is characterized in that at least one of the peripheral area processing means is instructed to redetect the first information and / or the second information.

Further, the present invention provides the first information detected in the first image processing by the central region processing means and the second information detected in the second image processing by the peripheral region processing means in the above invention. It is further characterized by further comprising mobile body collating means for identifying the mobile body by collating at least one of the above with the mobile body specifying information stored in advance.

Also, in the present invention according to the above-mentioned invention, the mobile body collating means matches information in the mobile body specifying information when collating at least one of the first information and the second information. Is not present , at least one of the first information and the second information is added to the mobile object specifying information.

  Further, the present invention is the above invention, wherein the central region processing means detects a traveling path of the moving body in the first image processing, and the peripheral region processing means is the second image processing. Detecting the attribute information of the moving object, the passage information detecting means associates the traveling path detected by the central area processing means with the attribute information detected by the peripheral area processing means for each moving object. It is detected as the passage information.

  Further, the present invention is the above invention, wherein the moving body is a person, the central area processing means detects a travel route of the person in the first image processing, and the peripheral area processing means In the second image processing, the face information of the person is detected, and the passage information detecting means determines the face information detected by the peripheral area processing means and the travel path detected by the central area processing means for each person. The associated information is detected as the passing information.

  In addition, the present invention is characterized in that, in the above-mentioned invention, the imaging device photographs the predetermined area using a wide-angle lens.

  In addition, the present invention is characterized in that, in the above invention, the imaging device photographs the predetermined area using a fisheye lens.

  The present invention also provides a moving body information detection method for detecting passage information relating to a moving body that passes through a predetermined area using an imaging apparatus that captures a predetermined area, the area being captured by the imaging apparatus. A central area processing step of performing first image processing using an image of a moving body projected on the central portion, and an image of the moving body projected on a peripheral portion of the central portion in the area photographed by the imaging device A peripheral region processing step for performing second image processing using the first information detected by the first image processing by the central region processing step and the second image processing by the peripheral region processing step. And a passage information detection step of detecting information associating the second information for each moving body as the passage information.

  Further, the present invention is a moving body information detection program for detecting passage information regarding a moving body that passes through a predetermined area using an imaging apparatus that captures a predetermined area, wherein the information is stored in the area captured by the imaging apparatus. A central area processing procedure for performing first image processing using an image of a moving object projected on the central portion, and an image of the moving object projected on a peripheral portion of the central portion in the area photographed by the imaging device The peripheral area processing procedure for performing the second image processing using the first information detected by the first image processing by the central area processing procedure and the second image processing by the peripheral area processing procedure. And a passage information detection procedure for detecting, as the passage information, information in which the second information is associated with each moving body.

  According to the present invention, the first image processing is performed using the image of the moving body projected on the central portion in the region photographed by the imaging device, and the peripheral portion of the central portion in the region photographed by the imaging device The second image processing is performed using the image of the moving object displayed on the screen, and the first information detected by the first image processing and the second information detected by the second image processing are handled for each moving object. Since the attached information is detected as passing information, it is possible to detect the passing number of the moving body and the traveling route based on the first information and to recognize the moving body based on the second information. Thus, the moving object can be detected and recognized at the same time without performing pan / tilt / zoom in the imaging apparatus. In addition, since this configuration can be realized even with a single image pickup apparatus, there is an effect that it is possible to perform complicated position adjustment at the installation site and reduce the cost for introduction.

According to the present invention, the reliability of at least one of the detected first information and second information is calculated, and the first information and the second information are associated with each other based on the calculated reliability. Since it comprised in this way, it has the effect that the bad influence of the detection performance of a moving body falling by disturbance elements, such as an illumination intensity change, can be suppressed, and the passage information of a person can be detected stably.

In addition, according to the present invention, when the calculated reliability is low, it is configured to instruct to perform redetection of at least one of the first information and the second information. Even when a state in which an error is likely to occur in the detection of the moving body occurs, the moving body information can be detected with high accuracy.

According to the present invention, the first information detected in the first image processing and the second information detected in at least one of the second image processing are used as the mobile object specifying information accumulated in advance. Since the mobile object is specified by collating, not only the mobile object can be recognized but also specified. As a result, it is possible to count the number of passes for each moving object, or to detect a moving object to be monitored, which has been determined in advance, and an effect that the moving object detection function can be enhanced. Play.

According to the present invention, when at least one of the first information and the second information is collated, if there is no matching information in the mobile object specifying information, the first information Since at least one of the information and the second information is added to the mobile object specifying information, it is possible to learn and increase the number of mobile bodies that can be specified, and automatically improve the detection function of the mobile body There is an effect that can be.

  Further, according to the present invention, the traveling path of the moving body is detected in the first image processing, the attribute information of the moving body is detected in the second image processing, and the detected traveling path and attribute information are detected for each moving body. Since the information associated with is detected as passing information, the moving body is recognized based on the attribute information, and at the same time, the traveling path of the moving body can be detected at the same time.

  Further, according to the present invention, the moving body is a person, the person's travel path is detected in the first image processing, the person's face information is detected in the second image processing, and the detected travel path and face information are detected. Is associated with each person and is detected as passage information, so that the person can be recognized based on the face information, and at the same time, the travel path of the person can be detected at the same time.

  According to the present invention, since the imaging device is configured to capture a predetermined area using a wide-angle lens, the area captured by the imaging device is wider than that of a normal lens, and more passing information is obtained. Can be detected for each moving body, and the accuracy of detecting and recognizing the moving body can be improved.

  In addition, according to the present invention, the imaging apparatus is configured to capture a predetermined area using a fisheye lens, so that an area captured by the imaging apparatus is further wider than a normal lens or a wide-angle lens, and more The passage information can be detected for each moving body, and the accuracy of detecting and recognizing the moving body can be improved.

  Exemplary embodiments of a mobile object information detection apparatus, a mobile object information detection method, and a mobile object information detection program according to the present invention will be explained below in detail with reference to the accompanying drawings. In the present embodiment, the case where the present invention is applied to a person information detecting apparatus that detects passage information relating to a person passing through a passage will be mainly described.

  First, the concept of the person information detecting apparatus according to the first embodiment will be described. 1 and 2 are diagrams (1) and (2) for explaining the concept of the human information detecting apparatus according to the first embodiment.

  The person information detection apparatus according to the first embodiment performs passing image processing on a video captured by a monitoring camera installed above an area to be monitored, such as a ceiling, by simultaneously performing different image processing for each area in the video. And detecting and recognizing the person's face image.

  As the surveillance camera used here, a camera that performs photographing using a wide-angle lens such as a fisheye lens is used. By using a camera having a wide-angle lens, the area to be photographed becomes wider than that of a normal lens, and more passage information can be detected for each person, thereby improving the accuracy of person detection and recognition. be able to.

  Specifically, as shown in FIG. 1, a camera 10 having a wide-angle lens is installed on the upper part of the monitoring area, and thereby a passing person is photographed. For example, as shown in the figure, it is assumed that the person A passes under the camera 10 between times T1 and T3. In this case, the camera 10 captures an image as shown in FIG. As shown in the figure, at time T1, since the person A is in a place (peripheral part) far from the front of the camera 10, the face is photographed. At time T2, the person A is near the front of the camera 10 ( Only the head is photographed (time T2).

  Therefore, the person information detection apparatus simultaneously performs two different image processes, that is, a process of detecting the number of people passing using the central part of the video and a process of detecting and recognizing the face image of the passing person using the peripheral part of the video. Do. Further, by tracking the face detected in the peripheral part and the person detected in the central part, the passing person and the face are matched (associated).

  As described above, the person information detecting apparatus according to the first embodiment can simultaneously detect (count) passing persons and acquire (recognize) face images without performing pan / tilt / zoom with the camera. A system that not only solves the conventional technology but also provides new functions, such as counting the number of passing people with high accuracy and acquiring the moving trajectory of passing people by associating passing people with faces. realizable.

  Next, the configuration of the person information detection apparatus according to the first embodiment will be described. FIG. 3 is a functional block diagram of the configuration of the person information detecting apparatus according to the first embodiment. As shown in the figure, the person information detecting device 100 inputs an analog video sent from the camera 10 or a digital video sent from an image recording device (not shown). This is a device that detects a face image of a person and outputs them as passage information, and includes an A / D conversion unit 110, an image storage unit 111, a peripheral region processing result storage unit 112, and a central region processing result storage. Unit 113, image control unit 114, processed image conversion unit 115, peripheral region processing unit 116, central region processing unit 117, and processing result integration unit 118.

  The A / D conversion unit 110 is a processing unit that digitizes an analog video sent from the camera 10. Specifically, the A / D conversion unit 110 converts an analog video sent from the camera 10 into a digital video, and sends the converted digital video to the image control unit 114. When the input video is a digital video, the A / D conversion unit 110 is not necessary due to the configuration.

  The image storage unit 111 is a storage unit that stores the digitized video data of the input video. Specifically, the image storage unit 111 stores video data sequentially transmitted from an image control unit 114, which will be described later, for a predetermined period of time in a predetermined time interval.

  The peripheral region processing result storage unit 112 is a storage unit that stores the processing result of the face detection processing and the video data used for the processing. Specifically, the peripheral region processing result storage unit 112 includes a processing result of a face detection process sent from a peripheral region processing unit 116 described later and a processing result of a processing result integration process sent from a processing result integration unit 118 described later. Are stored in time series at predetermined time intervals.

  The central area processing result storage unit 113 is a storage unit that stores a processing result of the person detection processing and video data used for the processing. Specifically, the central region processing result storage unit 113 includes a processing result of a person detection process sent from a central region processing unit 117 described later and a processing result of a processing result integration process sent from a processing result integration unit 118 described later. Are stored in time series at predetermined time intervals.

  The image control unit 114 is a processing unit that controls input of digital video sent from the A / D conversion unit 110 and an image recording apparatus (not shown) and input / output of the input digital video to the image storage unit 111. . Specifically, the image control unit 114 inputs digital video sent from the A / D conversion unit 110 and the image recording apparatus, and sequentially sends them to the image storage unit 111 at predetermined time intervals. The image control unit 114 sequentially stores the video data stored in the image storage unit 111 when the video data is stored in the image storage unit 111 and the processed image conversion unit 115 is in a processable state. The image data is taken out and sequentially sent to the processed image conversion unit 115.

  The processed image conversion unit 115 is a processing unit that converts the digital video sequentially transmitted from the image control unit 114 into an image to be processed by the peripheral region processing unit 116 and the central region processing unit 117. Specifically, the processed image conversion unit 115 divides the digital video sequentially sent from the image control unit 114 into a peripheral part image and a central part image, converts each, and then converts the peripheral part image into the peripheral part image. The image of the central portion is sent to the peripheral area processing unit 116 to the central area processing unit 117. Hereinafter, image conversion of the peripheral part and image conversion of the central part will be described.

  First, the image conversion of the peripheral part will be described. 4, 5 and 6 are diagrams (1), (2) and (3) for explaining an example of image conversion of the peripheral region by the processed image conversion unit 115. FIG. For example, as illustrated in FIG. 4, the processed image conversion unit 115 makes the input image a rectangular image that excludes the central region from the processing target and has the line PQ and the line RS as opposite sides. By expanding the peripheral area, conversion to an image of only the peripheral part is performed. In this case, as shown in the figure, the developed images may be two or one. Alternatively, as illustrated in FIG. 5, the processing image conversion unit 115 may perform processing to exclude the processing target by painting the central portion without expanding the image.

  Also, as shown in FIG. 6, when the input video is a fish-eye video, the processed image conversion unit 115 converts the video from a fish-eye video to a normal video using a known video conversion technique. A conversion process may be performed to convert the line TU and the line VW into a rectangular image without distortion having the opposite sides.

  Known video conversion techniques used here include, for example, video conversion techniques described in “Kobayashi et al.,“ Correction of Fisheye Camera Image ”, 2005 General Conference Proceedings A-4-19, The Institute of Electronics, Information and Communication Engineers”, etc. Is mentioned.

  Subsequently, the image conversion of the central area will be described. 7 and 8 are diagrams (1) and (2) for explaining an example of image conversion of the central area by the processed image conversion unit 115. FIG. For example, as illustrated in FIG. 7, the processed image conversion unit 115 converts the input image into an image of only the central portion, excluding the peripheral region from the processing target. Alternatively, the processed image conversion unit 115 may perform processing that is excluded from the processing target by painting the peripheral portion as shown in FIG.

  Also, as shown in FIG. 6, when the input video is a fish-eye video, the processed image conversion unit 115 uses a known video conversion technique for the video in the same manner as the image conversion of the peripheral area. Alternatively, a conversion process from fisheye video to normal video may be performed.

  The peripheral region processing unit 116 is a processing unit that performs face detection processing on the peripheral image converted by the processing image conversion unit 115. Specifically, the peripheral region processing unit 116 performs face detection processing using a known face detection technique on the peripheral portion image sent from the processing image conversion unit 115, and the detection result includes The position and size of the face, the face image, and the evaluation value in the detection process (for example, the output value of the neural network representing the likelihood of a face) are acquired. Then, the surrounding area processing unit 116 sends these detection results to the surrounding area processing result storage unit 112.

  As a known face detection technique used here, for example, this face detection processing includes “HARowley et al.,“ Neural network-based face detection ”, IEEE Trans. On Pattern Analysis and Machine Intelligence, Vol. 20, No. 1, pp. 23-38, 1998 ”.

  Further, the surrounding area processing unit 116 also detects the advancing path of the face in the image by comparing the detection result of the previous time held in the surrounding area processing result storage unit 112 with the detection result of the current time. Specifically, for example, using the values representing the face image detected at the previous time and the face image detected at the current time, the normalized correlation represented by the following equation (1) is calculated, and constant When the above correlation values are obtained, it is determined that the faces of the face images are the same. In Expression (1), “X” and “Y” indicate pixel values of the face image detected at the previous time and the current time, respectively, “X ^ − (bar)” and “Y ^ − (bar)”. Indicates the average values of the face images detected at the previous time and the current time, respectively.

  By determining in this way, even when a plurality of persons are present at the same time, it is possible to distinguish each person and detect the traveling path of each person. The peripheral area processing unit 116 also sends the detection result of the traveling route to the peripheral area processing result storage unit 112.

  Further, in the peripheral image, there are cases where only the back side of the person is photographed and the face is not reflected. In this case, for example, the peripheral region processing unit 116 learns the image of the back of the head during learning. Then, it is determined that the face detection result is a backward face, or the position of the person is estimated based on the person's travel path information detected by the processing result integration unit 118 described later.

  The central area processing unit 117 is a processing unit that performs person detection processing on the central partial image converted by the processing image conversion unit 115. Specifically, the central area processing unit 117 performs a person (silhouette) detection process on the central portion image sent from the processed image conversion unit 115 using a known person detection technique.

  Examples of the known person detection technique used here include techniques described in “Masamura et al.,“ Research on differential system by automatic background update ”, IEICE General Conference, 2003”. The technology calculates a difference between a reference image, which is called a background image, where the detection target is not reflected and the input image, and detects a person existing in the central region by using a background subtraction method that detects a region having a difference as an object. It is to detect.

  By performing such person detection processing, the central region processing unit 117 detects the position and size of the person in the input image, the video of the person (the head part of the person), and detection processing as the detection result by the person detection processing. Get evaluation value at. FIG. 9 is a diagram for explaining an example of person detection by the central area processing unit 117. For example, as shown in the figure, the central region processing unit 117 compares a human model prepared in advance with a background difference result, and uses the degree of coincidence as an evaluation value.

  Then, the central area processing unit 117 sends these detection results to the central area processing result storage unit 113. Similarly to the peripheral region processing unit 116, the central region processing unit 117 compares the detection result of the previous time held in the central region processing result storage unit 113 with the detection result of the current time, thereby The traveling path in the image is also detected. For example, the central area processing unit 117 compares the person videos using the normalized correlation value represented by the expression (1), similarly to the peripheral area processing unit 116.

  The processing result integration unit 118 integrates the face detection result held in the peripheral region processing result storage unit 112 and the person detection result held in the central region processing result storage unit 113, and counts and passes the passing person It is a processing unit that acquires the face image of the person who has performed.

  Further, the processing result integration unit 118 integrates the processing results obtained by the peripheral region processing unit 116 and the central region processing unit 117, and associates the person who has passed with the face image. Specifically, the processing result integration unit 118 compares the detection result of the person's (face) travel path in the peripheral image with the detection result of the person's travel path in the central image. Corresponding things that satisfy the movement of

  For example, the processing result integration unit 118 detects the locus of a point defined at a predetermined position of the face (for example, the upper end portion of the face) detected in the peripheral image and the person detected in the central image. The trajectory of the point defined at a predetermined position (for example, the top of the head) is analyzed, and the face and the person whose trajectories are most naturally connected are associated with each other as the same person.

  FIG. 10 is a diagram for explaining person association by the processing result integration unit 118. For example, as shown in the figure, four persons (peripheral persons 1 to 4) are detected in the peripheral area and two persons (central persons 1 and 2) are detected in the central area during a certain period of time. (See “Input Image and Detection Position” on the left side of the figure). In this case, the processing result integration unit 118 integrates the detection results in the respective regions, and associates the peripheral person 1, the central person 1, and the peripheral person 4 as the same person from the traveling path ("Integration" on the right side of the figure). “Person 1” shown in “Result”), the peripheral person 2, the central person 2, and the peripheral person 3 are associated as the same person (see “Person 2” shown in “Integration result” on the right side of the figure).

  Then, the processing result integration unit 118 notifies the monitoring center or the like of the result of associating the person's travel path in the input image with the face image of the person, and also includes the peripheral region processing result storage unit 112 and the central region. The result is sent to the processing result storage unit 113.

  Next, a processing procedure of processing performed in the person information detection apparatus 100 according to the first embodiment will be described. FIG. 11 is a flowchart illustrating the processing procedure of the person information detecting apparatus 100 according to the first embodiment. As shown in the figure, in the person information detection apparatus 100, first, the A / D conversion unit 110 digitizes the analog video transmitted from the camera 10 (step S101), and then the image control unit 114 The input image is divided into a peripheral area and a central area (step S102).

  Then, with respect to the divided input image, the peripheral area processing unit 116 detects information about the face from the image of the peripheral part (step S103), and the central area processing unit 117 detects information about the person from the image of the central part. (Step S104).

  Thereafter, the processing result integration unit 118 associates the face detection result with the person detection result (step S105), and notifies the monitoring result to the monitoring center (step S106).

  As described above, in the first embodiment, the central area processing unit 117 performs image processing for detecting individual persons using an image of a person shown in the central portion of the area captured by the camera 10. The peripheral region processing unit 116 performs image processing for detecting a person's face using an image of the person shown in the peripheral part of the central portion in the region captured by the camera 10, and the processing result integrating unit 118 Since the information associated with each person detected by the central area processing unit 117 and the face detected by the peripheral area processing unit 116 is detected as passing information, the person is recognized based on the face, It is possible to detect the number of passages and the traveling route based on the person detected at the same time, and detect and recognize the person at the same time without performing pan / tilt / zoom with the camera 10. It is possible. In addition, since this configuration can be realized even with a single image pickup apparatus, it is possible to reduce complicated position adjustment and installation costs at the installation site.

  The passing information detected in this way can be used in various ways. For example, such passing information is recorded as log information, and a suspicious person is tracked by searching the log information when a crime occurs, or a person who goes through a candidate site for a location when a new commercial facility is constructed. This can be used when investigating attributes and numbers.

  By the way, in the first embodiment, the case where the detection result of the face in the peripheral image and the detection result of the person in the central image are simply integrated has been described, but the detection detected by each detection process Re-detection may be performed based on the reliability of the result, or detection results with high reliability and detection results with low reliability may be complemented. Therefore, hereinafter, a case where control based on the reliability is performed will be described as a second embodiment.

  FIG. 12 is a functional block diagram of the configuration of the person information detection apparatus according to the second embodiment. Of the functional units shown in the figure, those having the same functions as the functional units included in the person information detecting apparatus 100 shown in FIG. 3 are denoted by the same reference numerals. Description is omitted.

  As shown in the figure, the person information detecting device 200 inputs an analog video sent from the camera 10 or a digital video sent from an image recording device (not shown). This is a device that detects a face image of a person and outputs them as passage information, and includes an A / D conversion unit 110, an image storage unit 111, a peripheral region processing result storage unit 112, and a central region processing result storage. Unit 113, image control unit 114, processed image conversion unit 115, peripheral region processing unit 116, central region processing unit 117, processing result integration unit 118, and reliability determination unit 219.

  The reliability determination unit 219 determines the reliability of the detection results detected by the peripheral region processing unit 116 and the central region processing unit 117, and performs re-detection of a face or a person when the reliability is low, It is a processing unit that controls to integrate each detection result while complementing the detection result with low reliability using the detection result with high reliability. For the reliability here, for example, for the peripheral image, the evaluation value of the face detection result by the neural network (see the description of the face detection processing by the peripheral region processing unit 116) is used, and the reliability of the central portion is used. For the image, the degree of coincidence of the background difference result with the person model (see the description of the person detection process by the central area processing unit 117) is used.

  FIG. 13 is a diagram for explaining redetection by the reliability determination unit 219. For example, as shown in the figure, it is assumed that the reliability of the face detection result in the peripheral area is low and the reliability of the person detection result in the central area is high (see “Input Image and Detection Position” on the left of the figure). ). In this case, the reliability determination unit 219 uses the human detection result in the central region to process the processed image so that the peripheral region processing unit 116 performs face detection again around the position where the person is estimated to have passed. The conversion unit 115 is instructed (see “re-detection by the reliability determination unit” on the right side of the figure).

  Here, since the reliability determination unit 219 performs face and person redetection when the reliability is low, even when a state in which an error is likely to occur in the detection of a moving object due to a disturbance element such as an illuminance change occurs, Mobile object information can be detected with high accuracy.

  In addition, the reliability determination unit 219 controls the integration of the detection results performed in the processing result integration unit 118 so as to associate the travel route based on the reliability. For example, the reliability determination unit 219, when there is a plurality of travel paths that can be associated with the travel path for a certain travel path detected in the image of the central part, Then, the processing result integration unit 118 is controlled so that the one having the highest reliability is associated with the travel route in the central region.

  As described above, in the second embodiment, the reliability of the detection result detected by the peripheral region processing unit 116 and the central region processing unit 117 is calculated, and the reliability determination unit 219 is based on the calculated reliability. Therefore, since the control is performed so that the travel paths are associated with each other, it is possible to suppress a bad influence such as a decrease in detection performance of the moving body due to a disturbance element such as an illuminance change, and to detect passage information of a person stably.

  By the way, in the first and second embodiments, the case where the peripheral area processing unit 116 detects the face image and the traveling path of the passing person has been described, but the face image detected here is the face of the person registered in advance. You may make it identify the person who passed by comparing with an image | video. Thus, hereinafter, a case where a person is specified based on the detected face image will be described as a third embodiment.

  FIG. 14 is a functional block diagram illustrating the configuration of the person information detecting apparatus according to the third embodiment. Of the functional units shown in the figure, those having the same functions as the functional units included in the person information detecting apparatus 100 shown in FIG. 3 are denoted by the same reference numerals. Description is omitted.

  As shown in the figure, the person information detecting device 300 inputs an analog video sent from the camera 10 or a digital video sent from an image recording device (not shown), and from the inputted image, the number of people passing and the number of passing This is a device that detects a face image of a person and outputs them as passage information, and includes an A / D conversion unit 110, an image storage unit 111, a peripheral region processing result storage unit 112, and a central region processing result storage. Unit 113, image control unit 114, processed image conversion unit 115, peripheral region processing unit 116, central region processing unit 117, processing result integration unit 118, and face video collation unit 320.

  The face video collation unit 320 is a processing unit that identifies a person who has passed based on the face video detected by the peripheral region processing unit 116. Specifically, the face image collation unit 320 collates the face image detected by the peripheral region processing unit 116 with a face image of a person registered in advance using a known face collation technique. Identify the person who passed.

  Known face matching techniques used here include, for example, “Komatsu et al.,“ Cut and recognize faces without orientation using subspace method ”, PRU95-191, pp.7-14, 1996”. It is done. This technology compares a vector composed of face part positions such as eyes, nose, and mouth with a registered face image and a detected face image, and identifies or resembles a person based on this similarity. It enumerates the person who is.

  By performing such face collation processing, the face image collation unit 320 identifies a person who has passed, and acquires information such as which person has passed, when and how many times. In addition, when the detected face image is a face image that is not registered at that time, the face image collation unit 320 newly registers the face image for collation when it passes next time. .

  As described above, in the third embodiment, the face video collation unit 320 has passed by collating the face video detected by the peripheral area processing unit 116 with the face video of a person registered in advance. Since the person is specified, not only the person but also the person can be specified. As a result, it is possible to count the number of passages for each person, or to detect a person to be monitored that has been determined in advance, so that the person detection function can be enhanced. In addition, since the tracking accuracy of the person can be improved and the person who has passed through can be identified, it is possible to realize a passing person detection system with higher functionality and for security purposes.

  In addition, when the face video collation unit 320 collates the face video, if there is no matching face video in the face video of the person registered in advance, the face video is newly added. Therefore, it becomes possible to learn and increase the number of persons who can be identified, and to automatically improve the person detection function.

  By the way, in the said Example 2 and 3, the reliability determination part 219 for performing control based on reliability with respect to Example 1, and the face image for specifying a person based on the detected face image Although the case where the collation unit 320 is added has been described, the person information detection apparatus may be configured by arbitrarily combining these functional units. Therefore, hereinafter, a case where the first, second, and third embodiments are combined will be described as a fourth embodiment.

  FIG. 15 is a functional block diagram of the configuration of the person information detecting apparatus according to the fourth embodiment. It should be noted that each functional unit included in the person information detecting apparatus 400 shown in the figure includes the person information detecting apparatus 100 shown in FIG. 3, the person information detecting apparatus 200 shown in FIG. 12, or the person information detecting apparatus 300 shown in FIG. Therefore, detailed description thereof is omitted here.

  As described above, in the fourth embodiment, by combining the functional units described in the first, second, and third embodiments, a practical passing person detection system that suppresses the influence of disturbance such as illuminance change is realized. In addition, it is possible to improve the function of the person detection function such as recognition of a passing person and calculation of the number of times a specific person has passed.

  So far, the human information detecting device for detecting a person has been described as an example of the moving body information detecting device according to the present invention. However, the present invention is not limited to this, and for example, a load flowing on a belt conveyor is detected. You may apply to a package information detection apparatus. Thus, hereinafter, a case where the present invention is applied to a package information detection apparatus will be described as a fifth embodiment.

  First, the concept of the package information detection apparatus according to the fifth embodiment will be described. FIG. 16 is an explanatory diagram for explaining the concept of the package information detecting apparatus according to the fifth embodiment.

  The baggage information detection apparatus according to the fifth embodiment performs simultaneous image processing different for each region in the video on the video taken by the monitoring camera installed on the upper part of the belt conveyor such as the ceiling. The side and the upper surface of the luggage are detected. As the surveillance camera used here, a camera having a wide-angle lens such as a fisheye lens is used as in the embodiments described so far.

  Specifically, as shown in FIG. 16, a camera 10 having a wide-angle lens is installed on the upper part of the belt conveyor through which the luggage passes, and thereby the luggage flowing on the belt conveyor is photographed.

  As shown in the figure, a label is affixed to the side surface and the upper surface of the load flowing on the belt conveyor. In this label, attributes relating to the luggage such as type and weight are indicated by characters or colors. When the baggage is in a place (peripheral part) far from the front of the camera 10, the label attached to the side is photographed, and when the baggage is in the vicinity of the front of the camera 10 (center part), it is attached to the upper surface. The label is shot.

  Therefore, the package information detection apparatus simultaneously performs two different image processes: a process of counting the package using the central part of the video and a process of detecting and recognizing the attribute indicated by the label on the side of the package using the peripheral part of the video. Do. Further, the load flowing on the belt conveyor is specified by associating the attribute indicated by the label on the side of the load detected in the peripheral portion with the attribute indicated by the label on the upper surface of the load detected in the central portion.

  As described above, in the package information detecting apparatus according to the fifth embodiment, the baggage flowing on the belt conveyor is detected (counted) without being panned, tilted, or zoomed by the camera, and is attached to the side surface and the upper surface of the package. The label characters can be detected at the same time, and the label information affixed to the side surface can be associated with the label information affixed to the top surface.

  FIG. 17 is a functional block diagram illustrating the configuration of the package information detection apparatus according to the fifth embodiment. Of the functional units shown in the figure, those having the same functions as the functional units of the human information detecting device 100 shown in FIG. 3 or the human information detecting device 200 shown in FIG. It is attached. For these functional units, specifically, the object detected in the peripheral area of the input image is changed from the face to the label on the side of the luggage, and the object detected in the central area is changed from the person to the label on the upper surface of the luggage. Therefore, detailed description is omitted here.

  As shown in the figure, the baggage information detection device 500 inputs an analog video sent from the camera 10 or a digital video sent from an image recording device (not shown), etc. Is a device that detects the number of luggage flowing through the vehicle, images of the side and top surfaces of the luggage, and outputs these as passing information, and includes an A / D conversion unit 110, an image storage unit 111, and a peripheral region processing result storage Unit 112, central region processing result storage unit 113, image control unit 114, processed image conversion unit 115, peripheral region processing unit 116, central region processing unit 117, processing result integration unit 118, and reliability determination Unit 219, side image collation unit 521, and top image collation unit 522.

  The side image collating unit 521 is a processing unit that identifies the attribute of the load flowing on the belt conveyor based on the image of the label on the side surface of the load detected by the peripheral region processing unit 116. Specifically, the side image collating unit 521 detects the character indicated by the label from the image of the label on the side of the luggage detected by the peripheral region processing unit 116, and the detected character is registered in advance. The attributes of the package are specified by collating the existing characters with a known character collation technique.

  The upper surface image collating unit 522 is a processing unit that identifies the attribute of the load flowing on the belt conveyor based on the image of the label on the upper surface of the load detected by the central region processing unit 117. Specifically, the upper surface image collating unit 522 detects the character indicated by the label from the image of the label on the upper surface of the luggage detected by the central region processing unit 117, and the detected character and the registered character are registered in advance. The attributes of the package are specified by collating the existing characters with a known character collation technique.

  For example, “Negishi et al.,“ High-speed template matching that allows deformation for character recognition in a scene image ”, PRMU2005-247, is known publicly used in the side image matching unit 521 and the top image matching unit 522. , Pp. 101-106, 2005 ”. The technique is to collate characters by comparing with a dictionary image registered in advance.

  As described above, the side image collating unit 521 collates the information indicated on the label on the side surface of the luggage detected by the peripheral region processing unit 116 with the information for identifying the moving body accumulated in advance, and the upper surface image collating unit 522 The package is identified by collating the information indicated on the label on the upper surface of the package detected by the central area processing unit 117 with the information for identifying the moving body stored in advance, so that not only the package is recognized but also identified It is also possible to do. As a result, it is possible to detect a package to be detected, which has been determined in advance, so that the package detection function can be enhanced.

  Although the case where the package attribute is specified using the characters indicated on the label has been described here, the package attribute may be specified using a color instead of the characters.

  As described above, in the fifth embodiment, the peripheral area processing unit 116 uses the image of the baggage displayed in the peripheral part of the area captured by the camera 10 to display the information indicated on the label on the upper surface of the baggage. The image processing to detect is performed, and the center region processing unit 117 performs image processing to detect the information indicated on the label on the side of the baggage by using the baggage image displayed in the central portion of the region shot by the camera 10. The processing result integration unit 118 detects the information associated with the label information detected by the peripheral region processing unit 116 and the label information detected by the central region processing unit 117 for each package as passage information. It is possible to simultaneously count the number of packages passing through and recognize the side and top surfaces of the packages.

  By the way, in the said Examples 1-5, although the mobile body information detection apparatus (person information detection apparatus and package information detection apparatus) was demonstrated, the structure which a mobile body information detection apparatus has is implement | achieved by software, and the same A moving body information detection program having a function can be obtained. Therefore, a computer that executes the mobile body information detection program will be described.

  FIG. 18 is a functional block diagram illustrating a configuration of a computer that executes a moving body information detection program according to the present embodiment. As shown in the figure, this computer 600 includes a RAM (Random Access Memory) 610, a CPU (Central Processing Unit) 620, a ROM (Read Only Memory) 630, a LAN (Local Area Network) interface 640, a camera. Interface 650.

  The RAM 610 is a memory that stores programs and results of program execution, and the CPU 620 is a central processing unit that reads and executes programs from the RAM 610 and ROM 630.

  The ROM 630 is a memory for storing programs and data, the LAN interface 640 is an interface for connecting the computer 600 to another computer or a terminal device of a monitoring center via the LAN, and the camera interface 650 is a monitoring camera. Is an interface for connecting

  A moving body information detection program 611 executed in the computer 600 is stored in the ROM 630 in advance, and is executed as a moving body information detection task 621 by the CPU 620.

  In the present embodiment, the person information detecting device for detecting a person and the baggage information detecting device for detecting a baggage have been described. However, the present invention is not limited to this, for example, a vehicle traveling on a road The present invention can be applied in the same manner to the case where it is detected.

  In addition, among the processes described in the present embodiment, all or part of the processes described as being automatically performed can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method.

  In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

  Each component of each illustrated device is functionally conceptual and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured.

  As described above, the mobile object information detection device, the mobile object information detection method, and the mobile object information detection program according to the present invention use the imaging device that captures a predetermined area, and pass information about the mobile object that passes through the area. This is particularly useful when detecting and recognizing a moving object at the same time without performing pan / tilt / zoom in an imaging apparatus.

FIG. 1 is a diagram (1) for explaining the concept of the person information detecting apparatus according to the first embodiment. FIG. 2 is a diagram (2) for explaining the concept of the person information detecting apparatus according to the first embodiment. FIG. 3 is a functional block diagram of the configuration of the person information detecting apparatus according to the first embodiment. FIG. 4 is a diagram (1) for explaining an example of the image conversion of the peripheral area by the processed image conversion unit. FIG. 5 is a diagram (2) for explaining an example of the image conversion of the peripheral area by the processed image conversion unit. FIG. 6 is a diagram (3) for explaining an example of the image conversion of the peripheral area by the processed image conversion unit. FIG. 7 is a diagram (1) for explaining an example of image conversion of the central region by the processed image conversion unit. FIG. 8 is a diagram (2) for explaining an example of the image conversion of the central area by the processed image conversion unit. FIG. 9 is a diagram for explaining an example of person detection by the central region processing unit. FIG. 10 is a diagram for explaining person association by the processing result integration unit. FIG. 11 is a flowchart illustrating the processing procedure of the person information detecting apparatus according to the first embodiment. FIG. 12 is a functional block diagram of the configuration of the person information detection apparatus according to the second embodiment. FIG. 13 is a diagram for explaining re-detection by the reliability determination unit. FIG. 14 is a functional block diagram illustrating the configuration of the person information detecting apparatus according to the third embodiment. FIG. 15 is a functional block diagram of the configuration of the person information detecting apparatus according to the fourth embodiment. FIG. 16 is an explanatory diagram for explaining the concept of the package information detecting apparatus according to the fifth embodiment. FIG. 17 is a functional block diagram illustrating the configuration of the package information detection apparatus according to the fifth embodiment. FIG. 18 is a functional block diagram illustrating a configuration of a computer that executes a moving body information detection program according to the present embodiment. FIG. 19 is a diagram (1) for explaining the method of Patent Document 1. FIG. FIG. 20 is a diagram (2) for explaining the method of Patent Document 1. FIG. FIG. 21 is a diagram (3) for explaining the method of Patent Document 1. FIG. FIG. 22 is a diagram (4) for explaining the method of Patent Document 1. FIG. FIG. 23 is a diagram for explaining the method of Patent Document 2. In FIG. FIG. 24 is a diagram for explaining the method of Patent Document 3. FIG. 25 is a diagram for explaining the problem of the method using a plurality of cameras. FIG. 26 is a diagram for explaining a problem of the method using the PTZ camera.

Explanation of symbols

100, 200, 300, 400 Person information detection device 500 Baggage information detection device 110 A / D conversion unit 111 Image storage unit 112 Peripheral region processing result storage unit 113 Central region processing result storage unit 114 Image control unit 115 Processing image conversion unit 116 Peripheral region processing unit 117 Central region processing unit 118 Processing result integration unit 219 Reliability determination unit 320 Face image matching unit 521 Side image matching unit 522 Top surface image matching unit 600 Computer 610 RAM
611 Moving object information detection program 620 CPU
621 Mobile object information detection task 630 ROM
640 LAN interface 650 Camera interface

Claims (8)

A moving body information detecting device that detects passage information about a moving body that passes through a predetermined area using an imaging device that captures a predetermined area,
A central area processing means for performing first image processing using an image of a moving body projected on a central portion in an area captured by the imaging device;
Peripheral area processing means for performing second image processing using an image of a moving body projected on the peripheral part of the central part in the area captured by the imaging device;
Information that associates the first information detected by the first image processing by the central region processing means and the second information detected by the second image processing by the peripheral region processing means for each moving object Passage information detecting means for detecting passage information;
Equipped with a,
At least one of the central area processing means and the peripheral area processing means calculates the reliability of at least one of the detected first information and the second information,
The passage information detecting means associates the first information with the second information based on the reliability calculated by at least one of the central area processing means and the peripheral area processing means. Mobile body information detection device. The passage information detection means, wherein when the reliability calculated by the at least one central region processing means and the peripheral region processing means is low, the relative at least one of said central region processing means and the peripheral region processing means The mobile body information detection apparatus according to claim 1 , wherein an instruction is given to redetect at least one of the first information and the second information. A moving body in which at least one of the first information detected in the first image processing by the central region processing means and the second information detected in the second image processing by the peripheral region processing means is stored in advance. The mobile body information detecting apparatus according to claim 1, further comprising a mobile body collating unit that identifies the mobile body by collating with identification information. When the mobile body collating means collates at least one of the first information and the second information, and there is no matching information in the mobile body specifying information, The mobile body information detecting apparatus according to claim 3 , wherein at least one of the first information and the second information is added to the mobile body specifying information. The central area processing means detects a traveling path of the moving body in the first image processing,
The peripheral area processing means detects attribute information of the moving body in the second image processing,
The passage information detection unit detects, as the passage information, information in which a traveling path detected by the central region processing unit and attribute information detected by the peripheral region processing unit are associated with each moving object. The moving body information detection apparatus according to claim 1. The moving body is a person,
The central area processing means detects a travel path of the person in the first image processing,
The peripheral area processing means detects the face information of the person in the second image processing,
The passage information detection means detects information associating a travel path detected by the central area processing means with face information detected by the peripheral area processing means for each person as the passage information. The moving body information detection apparatus according to claim 1. A moving body information detection method performed by a moving body information detection apparatus that detects passage information regarding a moving body that passes through a predetermined area using an imaging device that captures a predetermined area,
A central area processing step in which the mobile body information detection device performs first image processing using an image of the mobile body projected on a central portion in the area captured by the imaging device;
A peripheral area processing step in which the mobile body information detection device performs second image processing using an image of the mobile body projected on the peripheral portion of the central portion in the area captured by the imaging device;
The mobile body information detection device is configured to detect, for each mobile body, the first information detected by the first image processing by the central region processing step and the second information detected by the second image processing by the peripheral region processing step. A passage information detection step of detecting information associated with the passage information as the passage information;
It includes,
In at least one of the central region processing step and the peripheral region processing step, the mobile body information detection device calculates the reliability of at least one of the detected first information and the second information, respectively.
In the passage information detection step, the mobile body information detection device is configured to determine the first information and the second information based on the reliability calculated by at least one of the central region processing step and the peripheral region processing step. The moving body information detection method characterized by matching. A moving body information detection program for detecting passage information relating to a moving body that passes through a predetermined area using an imaging device that captures a predetermined area,
A central region processing procedure for performing first image processing using an image of a moving body projected on a central portion in a region captured by the imaging device;
A peripheral region processing procedure for performing second image processing using an image of a moving body projected on a peripheral portion of the central portion in the region captured by the imaging device;
Information that associates the first information detected by the first image processing by the central region processing procedure and the second information detected by the second image processing by the peripheral region processing procedure for each moving object Passage information detection procedure to detect as passage information;
To the computer ,
At least one of the central area processing procedure and the peripheral area processing procedure calculates the reliability of at least one of the detected first information and the second information, respectively.
The passing information detection procedure associates the first information with the second information based on the reliability calculated by at least one of the central region processing procedure and the peripheral region processing procedure. Mobile object information detection program.

Images