JP6080572B2 - Traffic object detection device - Google Patents

Description

  The present invention relates to a passing object detection device that detects a passing object region of a passerby or the like from an image of a space including a door.

  In recent years, in order to monitor illegal traffic such as co-entrance, research has been conducted to track a passerby by processing an image taken near the door. At this time, if the both sides of the door are photographed by separate monitoring cameras and the tracking of the passers-by passing through the doors is taken over between the monitoring cameras, the behavior of the passers-by can be strictly monitored and security is improved.

  For example, Patent Document 1 discloses a traffic in which a photographing unit is installed in each of an open area outside a door and a standby area inside the door, and a traffic object area is tracked based on a change area extracted by background difference processing. A control device is disclosed. In this traffic control device, a door area is set between two areas, and a passerby who disappears from one of the open area and the standby area to the door area and a passerby who appears from the door area in the other of the open area and the standby area. Are taken over by the same passerby.

JP 2010-157149 A

  However, in the prior art, since tracking blank period and blank area occur in the door area, there is a case where tracking cannot be succeeded correctly. For example, when two people enter the door area and one of them returns after making a U-turn, or when the person who opened the door passes the following person first, there are multiple people in the door area. Sometimes a problem of erroneous takeover is likely to occur. Therefore, it is desired to continue tracking while the person is present in the door area. In order to continue tracking, it is necessary to continue extracting the passing object area of the person.

  However, when the door is opened, the other side of the door that was previously shielded by the doorboard is reflected in the opening and the background changes. For this reason, if background difference processing is performed on the image at this time, the entire opening becomes a change region, and it is difficult to extract a passing object region of a person whose part or whole is reflected in the opening. Furthermore, since the area shape of the opening changes from moment to moment, and a passerby hides a part of the opening, it is difficult to accurately obtain each pixel value of the opening at the latest time as background information.

  Further, since the door plate is displaced when the door is opened and closed, the change region includes the door plate itself and the shadow of the door plate in addition to the opening. Also in the area of the doorboard, as with the opening, since the time change of the area shape and concealment by passers-by occur, it is difficult to accurately obtain background information at the latest time in pixel units. Thus, the background changes differently in the three areas of the opening area, the doorboard area, and the door outside area by opening and closing the door, and the area of the passerby can extend over these three areas. Therefore, it is difficult to extract a passing object region near the door.

  The present invention has been made to solve the above problem, and an object of the present invention is to provide a passing object detection device capable of continuously detecting a passing object region even when a door is opened and closed.

  One aspect of the present invention is a passing object detection device that detects a passing object from a captured image obtained by photographing a space including a door, and includes at least a shape pattern of the door including a shape characteristic of an opening generated by opening the door. And a door pattern storage unit that stores in advance for each opening of the door, and the captured image is compared with the shape pattern to calculate a similarity, and the captured image matches the shape pattern with the highest similarity. An area dividing unit that divides a door area and an outside door area other than the door area, and a background storage unit that stores background information of the space that does not include the passing object in each area divided by the area dividing unit; By calculating the difference between each pixel of the captured image and the background information of the area for each area divided by the area dividing unit, a group of pixels whose difference is equal to or greater than a predetermined reference is calculated. And passing the object region extraction unit that extracts as a row object area, a traffic object detection apparatus comprising: a.

  Here, the background storage unit stores distribution data of the pixel value of the door region as the background information in the door region, and the pixel value of each pixel of the door outer region as the background information in the door outer region. It is preferable to memorize.

  Here, the door pattern storage unit stores the shape pattern further including a shape characteristic of a door plate constituting the door, and the region dividing unit further opens the door region and the opening region where the opening portion appears. It is preferable that the background storage unit stores pixel value distribution data of each region as the background information in each of the opening region and the door plate region. .

  Further, the passing object region extraction unit is configured to pass the region to which the passing object model is applied when the sum of the differences between pixels included in the passing object model imitating the shape of the passing object is equal to or greater than the predetermined reference. It is preferable to detect the object area.

  In addition, the passing object region extracting unit extracts pixels in which the degree of difference of each pixel of the captured image is equal to or greater than the predetermined reference, and the extracted pixels are gathered to be equal to or larger than a preset passing object size. Is preferably detected as the passing object region.

  Further, an image feature of the passing object is extracted from the passing object region of the captured image, and the passing object region extraction unit obtains a similarity between the passing object region of the door region and the image feature of the passing object, and It is preferable that the higher the similarity is, the higher the degree of difference between the pixels of the passing object region or the lower the predetermined reference.

  Further, the image feature of the passing object is extracted from the passing object region of the photographed image, and the passing object region extraction unit obtains the similarity of the image feature of the passing object for each pixel of the door region, and the similarity It is preferable to perform correction to increase the degree of difference of the pixel or to decrease the predetermined reference as the value is higher.

  In addition, the passing object area extraction unit inputs the captured image of the door area to the classifier that has learned the image characteristics of the passing object in advance, and the output of the classifier is a value indicating the presence of the passing object. It is preferable to perform correction to increase the degree of difference or to decrease the predetermined reference.

  According to the present invention, it is possible to continue to accurately detect a passing object even when the door is opened and closed.

It is a figure which shows the structure and arrangement | positioning of the passing object detection apparatus in embodiment of this invention. It is a figure explaining each area | region in the image in embodiment of this invention. It is a functional block diagram of the passing object detection apparatus in the embodiment of the present invention. It is a flowchart of the passing object detection process in the embodiment of the present invention. It is a figure which shows the example of the door pattern in embodiment of this invention. It is a flowchart of the passing object area | region detection process in embodiment of this invention. It is a figure explaining the passing object determination by the discriminator in embodiment of this invention. It is a flowchart of the door outside / whole background information generation processing in the embodiment of the present invention. It is a flowchart of the opening part background information generation process in embodiment of this invention. It is a flowchart of the opening part background information generation process in embodiment of this invention. It is a flowchart of the doorboard background information generation process in the embodiment of the present invention.

<Configuration of the passing object detection device>
As shown in FIG. 1, the passing object detection device 100 includes an imaging unit 2 (first imaging unit 2 a and second imaging unit 2 b), a control unit 3, and a display unit 4.

  The passing object detection device 100 is installed in the room 6 (the first room 6a and the second room 6b), and generates background information from captured images obtained by photographing the first room 6a and the second room 6b. The process which detects a passing object using is performed. In the present embodiment, a description will be given assuming that a passing object is a person, that is, a detection target is a passer.

  The first room 6a and the second room 6b are partitioned by a door plate 15 of the door 5 and a wall. The door plate 15 is installed so as to be displaceable by a hinge or the like. When the door 5 is in the closed state, the door plate 15 remains in the steady position, and the door 5 is opened by the displacement of the door plate 15 from the steady position. In addition, the application range of the passing object detection device 100 of the present embodiment is not limited to the case where the door plate 15 is a hinged door, and may be a sliding door.

  The first imaging unit 2a is installed in the first room 6a. The first photographing unit 2 a photographs a space including the door 5 from the first room 6 a every predetermined time, and outputs a photographed image (first image) to the control unit 3. The second imaging unit 2b is installed in the second room 6b. The second photographing unit 2b photographs a space including the door 5 from the second room 6b every predetermined time, and outputs a photographed image (second image) to the control unit 3. In the present embodiment, the first photographing unit 2a and the second photographing unit 2b are disposed downward on the ceilings of the first room 6a and the second room 6b, respectively. As will be described later, from the viewpoint of the accuracy and stability of acquisition of background information, the background of the second room 6b included in the field of view (view angle) of the first photographing unit 2a is the floor 9 when the door 5 is open. It is preferable to install the first photographing unit 2a. For example, the first photographing unit 2a may be installed on the ceiling near the door 5 so that the depression angle is large.

  Here, it is preferable that the shooting timings of the first shooting unit 2a and the second shooting unit 2b are synchronized.

  When the door 5 is closed, the space of the second room 6b including the shielded space 7 is not photographed from the viewpoint (first viewpoint) of the first photographing unit 2a. That is, in the closed state, the shielding space 7 is shielded from the first viewpoint by the door plate 15 and is not photographed. However, even when the door 5 is in the closed position, the shielding space 7 is imaged from the second viewpoint by the second imaging unit 2b different from the first imaging unit 2a. On the other hand, when the opening 8 is generated by opening the door 5, a part or all of the shielding space 7 is imaged from the first imaging unit 2 a through the opening 8. As a result, on the first image photographed by the first photographing unit 2a, when the door 5 is in the closed state, the floor 9 of the second room 6b appears as a background in the region where the door plate 15 is the background. . As described above, when the door 5 is opened, the shielded space 7 suddenly appears on the first image as the background, so that it is difficult to detect a passing object. Incidentally, when the door 5 is opened, the shielding space 7 has a common field of view common to the field of view of the first photographing unit 2a and the field of view of the second photographing unit 2b.

  FIG. 2 shows an example of a first image taken when the door 5 is in the open state. The open area 11 is an area where a part of the second room 6b is photographed in an open state. The door board area | region 12 is an area | region where the door board 15 is image | photographed. When the door 5 is a hinged door, a region obtained by combining the opening region 11 and the door plate region 12 is a door region. If the door 5 is a sliding door and the shadow is hardly changed even if the door plate 15 is displaced, the door plate region 12 is not set and only the opening region 11 may be set as the door region. The outside door area 13 is an area excluding the opening area 11 and the doorboard area 12 in the first image. The door 5, the floor 9, the wall, and the like are backgrounds that are not detected, and the passerby 10 that is the detection target is the foreground. A passer-by area (passing object area) 14 in which the passer-by 10 appears can overlap over one or a plurality of areas of the opening area 11, the doorboard area 12, and the door outside area 13.

  The shape of the area of the open area 11 changes from moment to moment as the door 5 is opened and closed. Further, the shape of the region of the door panel region 12 changes every moment as the door 5 is opened and closed, and the shadow of the door plate 15 changes every moment. Furthermore, a part of the opening area 11 and the doorboard area 12 can be concealed by the passerby 10. When the door 5 is in the closed state, the opening area 11 does not appear.

  The second viewpoint background acquisition region 311 is a region in the second image corresponding to the opening region 11 in the first image. That is, the second viewpoint background acquisition region 311 in the second image is a region for acquiring background information of the opening region 11 in the first image. Back projection of the opening area 11 of the first image onto the floor, which is a reference plane of a common coordinate system (world coordinate system) common to the first photographing unit 2a and the second photographing unit 2b, by pre-calculation using the camera parameter 310 Further, the second viewpoint background acquisition region 311 is obtained by projecting the back projection of the opening region 11 onto the second image and stored in the storage unit 3. The area of the floor 9 in FIG. 1 is an example of back projection of the opening area 11, and the second viewpoint background acquisition area 311 in FIG. 2 is an example of projecting back projection of the opening area 11 onto the second image.

  The camera parameters 310 are internal parameters and external parameters of the first photographing unit 2a and the second photographing unit 2b. These values measured by the prior calibration (calibration) work are stored in advance in the storage unit 31 in association with the imaging unit IDs of the first imaging unit 2a and the second imaging unit 2b. The internal parameters are information such as the focal length and the optical center position of the lens. External parameters are information such as position and orientation in the world coordinate system. By applying a pinhole camera model or the like to the camera parameter 310, coordinate conversion between corresponding coordinates in the common field of view of the first imaging unit 2a and the second imaging unit 2b becomes possible.

  When the door plate 15 is displaced, an opening area appears on the second image. The first viewpoint background acquisition area in the second image corresponding to the opening area in the second image is also obtained in advance and stored in the storage unit 3.

  The passing object detection device 100 is realized by a computer that is the control unit 3 executing a passing object detection program. The control unit 3 is connected to the first imaging unit 2a, the second imaging unit 2b, and the display unit 4, processes the captured images sequentially input from the first imaging unit 2a and the second imaging unit 2b, and appears in the captured image. The passerby area 14 is extracted, and the position of the passerby 10 is tracked from the extraction result. The extraction result and the tracking result are displayed on the display unit 4. The display unit 4 includes a monitor such as a liquid crystal display or a CRT, and a communication interface that receives these data.

  FIG. 3 is a functional block diagram of each function realized by the control unit 3. The control unit 3 functions as an image acquisition unit 30, a storage unit 31, an image processing unit 32, and an output unit 33.

  The image acquisition unit 30 includes a communication interface that sequentially receives the captured images captured by the first imaging unit 2 a and the second imaging unit 2 b and outputs them to the image processing unit 32. The storage unit 31 includes a memory device such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The storage unit 31 stores various data and images including a camera parameter 310, a second viewpoint background acquisition region 311, a door pattern 312, outside / overall background information 313, opening background information 314, doorboard background information 315, and a color conversion function. A passing object detection program that defines the processing of the processing unit 32 is stored. The image processing unit 32 includes an arithmetic device such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an MCU (Micro Control Unit). The image processing unit 32 reads out and executes a passing object detection program from the storage unit 31 to execute processing in each part of the door state detection unit 320, the background information generation unit 321, the passing object region extraction unit 322, and the passing object tracking unit 323. Execute. The output unit 33 includes a communication interface that outputs information processed by the image processing unit 32 to the display unit 4.

  Hereinafter, the passing object detection process realized by the passing object detection program will be described with reference to the flowchart of FIG. The passing object detection process is started after confirming that no passerby exists in the monitoring space. In the passing object detection process, the following steps S1 to S9 are repeated every time an image is acquired.

  In step S1, the first image and the second image are acquired from the first photographing unit 2a and the second photographing unit 2b. The first imaging unit 2a and the second imaging unit 2b capture the first image and the second image at a predetermined cycle and output them to the image acquisition unit 30. The image acquisition unit 30 receives the first image and the second image and stores them in the storage unit 31.

  In step S2, the state of the door 5 is detected. The door state detection unit 320 detects the opening degree of the door 5 by performing pattern matching between the first image and the door pattern 312.

  The door pattern 312 is a shape pattern that represents the shape characteristics that the opening region 11 and the door plate region 12 can take on the captured image for each opening degree of the door 5. The door pattern 312 is created and registered in advance in accordance with the monitoring space (stored in the storage unit 31). Specifically, the door pattern 312 includes the shape pattern indicating the position of the contour pixel in the opening region 11 at each opening angle of the door 5, the position of the contour pixel in the door plate region 12, the pixels belonging to the opening region 11, and the door plate region 12. Including an attribute pattern indicating pixels belonging to. For example, as shown in FIG. 5, 13-step patterns are registered every 10 degrees from 0 degrees (steady position) to 120 degrees (fully open). The door pattern 312 is created in advance by a simulation using a three-dimensional model imitating the door 5 and registered in advance. Further, the pattern image of the opening area 11 and the pattern image of the doorboard area 12 may be separately registered. The door state detection unit 320 calculates the similarity by comparing the shape pattern of each opening angle and the first image of the current frame, and detects the opening angle having the highest similarity as the door state. Specifically, the door state detection unit 320 generates an edge image from the first image, performs pattern matching between the door pattern 312 at each opening angle and the edge image, and detects the opening angle having the maximum similarity. . Thereby, the door state corresponding to the shape of the door 5 appearing in the first image of the current frame can be detected. The door state detection unit 320 outputs the detected opening angle to the background information generation unit 321 and the passing object region extraction unit 322 and stores the detected opening angle in the storage unit 31 for processing in the next frame.

  Further, the door state may be detected by pattern matching between the second image and the door pattern 312. In this case, the door pattern 312 may be registered in advance at each opening angle of the door 5 with respect to the second image. Since the opening angle in the first image and the second image is common, the process in step S2 may be performed on either one.

  Moreover, it is good also as a process which evaluates a similarity higher in the area | region of the upper part of the door 5 than the area | region of the lower part of the door 5. FIG. That is, it is highly possible that the upper area of the door 5 does not overlap with a passerby, and the area in the pattern matching with the door pattern 312 is highly reliable. Therefore, the matching in the upper area of the door 5 is more important. It is preferable to perform processing. For example, the similarity in the region above the average height of the person in the door region may be multiplied by a coefficient larger than 1 to evaluate higher than the similarity in the region below the average height. Alternatively, the upper half of the door region may be simply the upper region and the lower half may be the lower region.

  In step S3, a region attribute is set for each pixel of the first image and the second image. The door state detection unit 320 sets the opening area 11, the doorboard area 12, and the other outside door area 13 on the first image and the second image according to the attribute pattern set in the detected door pattern 312 having the opening angle. To do. The processing in steps S2 and S3 corresponds to the processing in the area division unit included in the door state detection unit 320.

  For example, if the opening angle is other than 0 degrees, the door state detection unit 320 sets one of the three region attributes to each pixel of the first image and the second image, and each of the first image and the second image. Are divided into three regions whose contour shapes and positions coincide with the shapes and positions of the opening region 11, the door plate region 12 and the door outside region 13 in the door pattern 312 having the highest degree of similarity. If the opening angle is 0 degree, the door state detection unit 320 displays each of the first image and the second image in the shape and position of the door plate region 12 and the door outside region 13 in the door pattern 312 in the closed state. Divide into two matching areas. In the example of the sliding door that does not set the door panel region 12 described above, the door state detection unit 320 opens the opening region in the door pattern 312 having the highest degree of similarity between the first image and the second image in the open state. It is divided into two regions whose contour shape and position coincide with the shape and position of 11l and the outside door region 13.

  Thus, the door state detection unit 320 compares the opening area 11 in which the opening 8 appears in the photographed image by opening and closing the door 5 with the shape pattern (door pattern 312) of the opening area 11 registered in advance. To detect. Thereby, the shape change of the opening area | region 11 which arises with the displacement of the doorplate 15 in a picked-up image can be tracked. Therefore, since the opening area 11 to which the opening background information 314 is to be applied is accurately obtained in the captured image, it is possible to generate highly accurate background information and detect a highly accurate passing object area.

  Moreover, the door state detection part 320 detects the doorboard area | region 12 in which the doorboard 15 currently displaced in the picked-up image appears from the comparison with the shape pattern (door pattern 312) of the doorboard area | region 12 registered previously. Thereby, the shape change of the door board area | region 12 which arises in a picked-up image can be tracked. Therefore, the door panel background information 315 is acquired in the captured image, and the door panel area 12 to be applied can be accurately obtained. Therefore, it is possible to generate highly accurate background information and detect a highly accurate passing object area.

  In step S4, a passer-by area is extracted from the first image and the second image. First, the passing object region extraction unit 322 processes the first image according to the flowchart shown in FIG. In steps S30 to S40, each pixel is sequentially set as a target pixel, and a loop process is performed on all pixels of the first image.

  In step S30, the target pixel is set in the first image. That is, the passing object region extraction unit 322 sequentially selects one pixel included in the first image and sets it as a target pixel each time the processing returns to step S30.

  In step S31, it is confirmed whether or not the state of the door 5 is an open state. If the opening angle detected by the door state detection unit 320 is not 0 degree, the passing object region extraction unit 322 determines that the state of the door 5 is an open state (YES in step S31), and shifts the process to step S32. Otherwise, the process proceeds to step S38 (NO in step S31).

  In step S <b> 32, it is determined whether or not the target pixel is the opening area 11. The passing object region extraction unit 322 refers to the region attribute of each pixel determined by the door state detection unit 320 in step S4. If the region attribute of the target pixel is the opening region 11, the process proceeds to step S33. (YES in step S32), otherwise, the process proceeds to step S36 (NO in step S32).

  In steps S <b> 33 to S <b> 35, the passing object region extraction unit 322 compares each pixel of the captured image in the opening region 11 with the opening background information 314 and is an index that represents the degree of difference of the pixel with respect to the opening background information 314. The passer-by likelihood (the value increases as the difference is calculated) is calculated.

  The opening background information 314 is background information of the opening area 11 in the first image and the second image. The opening background information 314 of each pixel is information including two types of color histogram (opening color histogram) of the opening area 11 and a background pixel value (opening background pixel value) corresponding to each pixel of the opening area 11. And stored in the storage unit 31. The opening color histogram is, for example, a three-dimensional array of 8 bins each of YUV. The opening background information 314 is generated by the background information generation unit 321 and updated as appropriate. The generation process of the opening background information 314 will be described later.

  In step S33, it is determined whether or not the background pixel value of the target pixel has already been generated. If the background pixel value is already stored in the opening background information 314, the passing object region extraction unit 322 causes the process to proceed to step S35 (YES in step S33), and if not, the process proceeds to step S34. (NO in step S33).

In step S34, the passer likelihood of the pixel of interest for the opening color histogram is calculated. The passing object region extraction unit 322 reads the opening color histogram H _{0 of} the first image from the opening background information 314 and the color histogram H _P of the passerby near the door 5 from the passing object information 316, respectively. In accordance with (2), the passer-by likelihood L _O (x) of the pixel of interest x is calculated.

Here, x is the target pixel, P _O (x) is the passer probability of the target pixel x, c _x is the color of the target pixel x, and H _O (c _x ) is the frequency of bins of the color c _x in the aperture color histogram. , H _P (c _x ) is the frequency of bins of the color c _x in the color histogram read from the passing object information 316, L _O (x) is the _passer likelihood of the pixel of interest x, and β _O1 is appropriate through previous experiments. It is a constant adjusted to. The passer likelihood L _O (x) monotonously increases with respect to the passer probability P _O (x). Therefore, the lower the frequency of the pixel value of the target pixel in the aperture color histogram, the higher the passer likelihood.

  As described above, the passing object region extracting unit 322 calculates the passer likelihood of the pixel as the pixel of the opening region 11 in the first image is different from the opening color histogram. In addition, the passing object region extraction unit 322 calculates a higher passer likelihood of the pixel as the pixels of the opening region 11 in the first image are more similar to the passing object information 316.

  Here, the passing object information 316 is information indicating a history of image features and position of the passerby area 14 of each passerby (that is, a passerby being tracked) existing in the monitoring space. Each passerby is assigned a passer ID for uniquely identifying the passer-by, and the passer ID, the image feature amount, and the position history are associated with the passer ID and stored in the storage unit 31. Here, the image feature amount is a color histogram, and is represented by, for example, a three-dimensional array of 8 bins of YUV. Information on the passer-by area is stored in the form of a three-dimensional shape model in the common coordinate system and its three-dimensional position (for example, the head center of gravity). When the passing object is a person, the three-dimensional shape model can be data obtained by approximating the head, torso and foot with a spheroid. By projecting the three-dimensional shape model onto the coordinate system of the captured image using the camera parameter 310, a two-dimensional shape model representing the passer-by area is obtained.

ここで、β_O2は予めの実験を通じ適切に調整した定数である。 In step S35, the passer likelihood of the pixel of interest with respect to the aperture background pixel value is calculated. The passer-by likelihood is calculated using Equation (3). Specifically, the passing object region extraction unit 322 calculates a background difference between the pixel value I (x) of the target pixel and the background pixel value B _O (x), and calculates the difference value (I (x) −B _O ( x)) to passer likelihood L _O (x).

Here, β _O2 is a constant adjusted appropriately through previous experiments.

The passer-by likelihood L _O (x) increases monotonously with respect to the difference value (I (x) −B ₀ (x)). Therefore, the passer likelihood becomes higher as the difference between the pixel value of the target pixel and the aperture background pixel value is larger.

  As described above, the passing object region extracting unit 322 calculates the passer likelihood of the pixel so high that each pixel of the opening region 11 in the first image is different from the opening background information 314.

  In steps S <b> 36 and S <b> 37, the passing object region extraction unit 322 compares each pixel of the first image in the doorboard region with the doorboard background information 315, and a passerby is an index indicating the degree to which the pixel differs from the background information. Calculate the likelihood.

  Here, the doorboard background information 315 is background information of the doorboard region 12 in the first image and the second image, and is represented by a color histogram (doorboard color histogram). The door panel color histogram of each image is expressed in, for example, a three-dimensional array of 8 bins of YUV and stored in the storage unit 31. The door section background information 315 is generated by the background information generation section 321 and updated appropriately while the opening / closing of the door 5 is detected.

  In step S <b> 36, it is determined whether or not the target pixel is the door panel region 12. If the region attribute of the pixel of interest is the door panel region 12 (YES in step S36), the passing object region extraction unit 322 moves the process to step S37, and otherwise (NO in step S36). The process proceeds to S38.

In step S37, the passer-by likelihood of the pixel of interest for the door panel color histogram is calculated. The passing object region extraction unit 322 compares the color of the pixel of interest x, that is, the pixel value, with the door panel color histogram of the first image and the color histogram of the passerby near the door 5, and follows equations (4) and (5). A passer likelihood L _D (x) of the pixel of interest x is calculated.

Here, P _D (x) is the passer probability of the target pixel x, H _D (cx) is the bin frequency of the color c _x in the door panel color histogram, and L _D (x) is the passer likelihood of the target pixel x. , Β _D is a constant adjusted appropriately through previous experiments. The passer likelihood L _D (x) monotonously increases with respect to the passer probability P _D (x).

  That is, the passing object region extraction unit 322 calculates the passer likelihood of the pixel so high that each pixel of the doorboard region 12 in the first image is different from the doorboard background information 315. In addition, the passing object region extraction unit 322 calculates the passer likelihood of the pixel so high that the pixels constituting the door board region 12 of the first image are similar to the passing object information 316.

In step S38, the passer-by likelihood for the pixel outside the door / overall background pixel value is calculated for the pixel of interest that is neither the opening area 11 nor the door panel area 12 being displaced. The passing object region extraction unit 322 reads the background luminance value B _G (x) corresponding to the target pixel x from the outside / overall background information 313, and the pixel value I (x) and the background pixel value B _G ( A difference value (I (x) −B _G (x)) from x) is calculated and converted into a passer likelihood L _G (x). Specifically, the passer-by likelihood L _G (x) is calculated by Equation (6).

Here, L _G (x) is a passer likelihood of the pixel of interest x, and β _G is a constant adjusted appropriately through a prior experiment. The passer-by likelihood L _G (x) monotonously increases with respect to the difference value (I (x) −B _G (x)). Thus, the greater the difference between the pixel value of the pixel of interest and the overall background pixel value or the outside door background pixel value, the higher the passer likelihood.

  Thus, when the door 5 is in the closed state, the passing object region extraction unit 322 calculates the passer likelihood of the pixel as high as different from the overall background pixel value for all the pixels of the first image, When the door 5 is in the open state, the passer likelihood of each pixel of the first image in the outside door region 13 is calculated so high that the pixel differs from the outside door background pixel value.

  Here, the outside / overall background information 313 is background information in the entire first image and background information in the entire second image. As the outside / overall background information 313, a background pixel value corresponding to each pixel of the first image and a background pixel value corresponding to each pixel of the second image are stored in the storage unit 31. The outside / overall background information 313 is generated by the background information generation unit 321 and updated as appropriate. When the door 5 is in the closed state, the background pixel values of all the pixels are used as the entire background pixel value, and when the door 5 is in the open state, the background pixel value corresponding to the pixel in the outside door region 13 is used as the outside door background pixel value. Used.

  In step S40, a passer likelihood correction process using a discriminator is performed. First, the passer-by area extraction unit 322 inputs a photographed image in the detection window area A to the discriminator while gradually shifting the detection window area A in the opening area 11 and the doorboard area 12, and detects from the score output by the discriminator. It is determined whether or not a passerby is shown in the window area A. For example, the discriminator is trained in advance to output a score greater than 0 and less than 1 if a passerby appears in the detection window area A, and to output a score greater than -1 and less than 0 if no passer is reflected. In this case, the passer-by area extraction unit 322 sets 0 as a threshold value, determines that a passer-by appears if the score is greater than the threshold, and if the score is equal to or lower than the threshold, the passer-by is indicated. Judge that there is no. At this time, the passer-by area extraction unit 322 performs determination while reducing the captured images of the opening area 11 and the doorboard area 12 in order to correspond to different heights (individual differences of passing objects) for each passer-by. The reduction ratio may be stored in advance in the storage unit 31 in consideration of the size of the passerby on the image.

  The classifiers used here are feature quantities of a large number of "passerby" images that each show a person, and feature quantities of a large number of "passerby" images that show only the background without any human beings. Is a program in which parameters for determination criteria are previously learned by the AdaBoost method or the support vector machine (SVM) using the learning data consisting of the above, and are stored in the storage unit 31 in advance. Histograms of Oriented Gradients (HOG), Haar-like features, etc. can be used as the feature quantities. The discriminator extracts a feature amount from the captured image in the input detection window region A, and outputs a score for each detection window region in light of the extracted feature amount in accordance with a criterion.

  The passing object region extraction unit 313 corrects the passer likelihood based on the determination result thus obtained. For example, when the score output from the discriminator exceeds a predetermined threshold, the passer likelihood of each pixel that has already been calculated in the detection window area A is multiplied by a certain value as a passerby is detected. In order to increase the likelihood of the passerby. Further, in the detection window region A where the passer-by is detected by the classifier, the passer-likelihood of each pixel already calculated is multiplied by the score output from the classifier so that the passer-likelihood becomes higher. It may be processed. For example, when the score Ds is represented by a numerical value of 0 to 1, (1 + Ds) is multiplied. Further, when the score output from the discriminator does not exceed a predetermined threshold, a certain value is added to the passer likelihood of each pixel already calculated in the detection window region A, and the passer likelihood is increased. You may raise the degree.

  Further, a humanoid passing object model may be set in accordance with the size of the rectangle of the detection window area A, and the correction process may be performed only on the area of the passing object model in the detection window area A. Alternatively, a multiplication value or an addition value may be set so that the passer likelihood becomes higher as it is closer to the center of the area of the passing object model.

  In addition, when multiplying or adding a multi-value score such as (1 + Ds) to the passer likelihood, when there are a plurality of passersby in the opening region 11 and the doorboard region 12, etc., in a plurality of detection window regions It may be determined that a passerby is shown, and some of these detection window areas may overlap. In this case, what is necessary is just to multiply or add the maximum value of the score output in the several detection window area | region in each pixel of an overlap area | region.

  Further, as the classifier, one type of classifier learned from learning data in which a passer-by image is composed of a human whole body image can be used, but a plurality of types of classifiers may be used. For example, a discriminator that learns a passer image by learning data composed only of a whole body image of a person and a plurality of identifications that are learned by a learning data that consists of a passer image composed of images for each part such as a leg and a head. Can be used. In this case, for example, the positional relationship between the detection window region of the whole body classifier and the detection window region of the part classifier with respect to the whole body is managed in association with each other in advance, and the score of the part classifier is associated with the corresponding whole body detection window region. Can be used for correction processing as a score for the detection window region of the whole body.

  The passer likelihood image obtained in this way is information representing the passer-by area. That is, a region where pixels with high passer-like likelihood are concentrated in a human shape is obtained as the passer-by region 14. In step S41, processing for determining the passer-by area 14 from the passer-likelihood image is performed.

  Specifically, a 3D human model is projected on each position on the passer likelihood image to generate a pass object model that imitates a 2D shape, and the passer likelihood set for the pixels in the pass object model When the total value is equal to or greater than the detection threshold T1, the area indicated by the passing object model at the position is extracted as the passer-by area 14. Alternatively, each pixel of the passer likelihood image is compared with a predetermined detection threshold value T2, and pixels whose passer likelihood is equal to or higher than the detection threshold value T2 are adjacent to each other in the range of the passer size (passing object size) in the door area. You may extract as the passerby area 14 the area which is doing.

  In the above subroutine, only the extraction process of the passer-by area 14 included in the first image has been described, but the passer-by area 14 included in the second image is obtained in the same manner. That is, the passing object region extraction unit 322 calculates a passer likelihood for each pixel of the second image and generates a passer likelihood image corresponding to the second image, and a pixel having a high passer likelihood has a human shape. A dense area is determined as a traveling area. At this time, if the door 5 is in the open state, the passing object region extraction unit 322 calculates the passer likelihood by comparing the pixel of the second image in the opening region 11 with the opening background information 314, and the door plate region 12. The passer likelihood is calculated by comparing the pixels of the second image with the door panel background information 315 and comparing the pixels of the second image in the outside door region 13 with the outside / overall background information 313. Is calculated. In addition, when the door 5 is in the closed state, the passing object region extraction unit 322 compares the all pixels of the second image with the outside / overall background information 313 to calculate the passer likelihood.

  When the process in the passing object region extraction unit 322 is completed, the process returns to step S5 of the main routine.

  In step S5, a passer-by tracking process is performed. The passing object tracking unit 323 extracts the position of each passer-by area 14 and the color histogram of the first image in the passer-by area 14 as the feature quantity of the passer-by and stored in the extracted feature quantity and the passing object information 316. Check against the characteristics of each passerby. The passing object tracking unit 323 determines that the passer-by area obtained by this collation is the same as the passer-by of the passing object information 316 and associates it. Similarly, the passerby area extracted from the second image is identified with the passing object information 316. Further, the passing object tracking unit 323 selects a passing person area corresponding to a position in the world coordinate system among the passing person area 14 extracted from the first image and the passing person area 14 extracted from the second image using the camera parameter 310. Associate with the same passerby. The passing object tracking unit 323 updates the color histogram of the passing object information 316 with the color histogram of the corresponding passer's area 14, and the passerby who has obtained a match in the position history of the passer's passing object information 316. The position of the area 14 is added. In addition, a new passer's passing object information 316 is generated for a passer's area for which no correspondence has been obtained, and the passing object information 316 for which no correspondence has been obtained is deleted because it has disappeared.

  The passerby tracking process is not limited to these, and various existing image tracking processes can be applied.

In step S <b> 6, a passer's color histogram used for calculating the passer likelihood is generated from the pixels of the passer's area 14. The histograms H _{1 to} H _K of the _K passers who exist in the vicinity of the door 5 are added together, and the normalized passer's color histogram H _P is included in the pass object information 316 and stored in the storage unit 31.

  In step S <b> 7, a process for generating outside / overall background information 313 is performed according to the flowchart of FIG. 8. The background information generation unit 321 sequentially sets each pixel of the first image of the current frame as a target pixel (Step S50), and executes the loop processing of Steps S50 to S55.

  In step S51, it is confirmed whether the state of the door 5 is an open state. If the opening angle detected by the door state detection unit 320 is other than 0 degrees, the background information generation unit 321 determines that the door 5 is in the open state (YES in step S51). The process proceeds to S53 (NO in step S51).

  In step S52, it is confirmed whether or not the region attribute of the target pixel is the outside door region 13. If the region attribute set for the pixel of interest by the door state detection unit 320 is the door outside region 13, the background information generation unit 321 moves the process to step S53 (YES in step S52), otherwise the step The process proceeds to S55 (NO in step S52).

  In step S53, it is confirmed whether or not the target pixel is a pixel in the passer-by area 14. The background information generation unit 321 refers to the passer-by area 14 extracted from the first image by the passing object area extraction unit 322, and shifts the processing to step S55 if the target pixel is a pixel in the passer-by area 14 (step S55). If YES in S53), the process proceeds to Step S54 if not (NO in Step S53).

  In step S54, the background information generation unit 321 updates the background pixel value of the target pixel stored in the outside door / overall background information 313 with the pixel value of the target pixel of the current frame.

ただし０＜α_G＜１であり、例えばα_G＝０．３と設定する Specifically, the background information generation unit 321 determines the pixel value I (x) of the target pixel of the current frame and the background pixel value B _G (x, t− of the outside / overall background information 313) according to Expression (8). A background pixel value B _G (x, t) obtained by weighting and averaging 1) at a ratio of α _G : (1−α _G ) is obtained and written in the outside / overall background information 313.

However, 0 <α _G <1, and for example, α _G = 0.3 is set.

  In step S55, the background information generation unit 321 checks whether all pixels of the first image have been processed as the target pixel. If all the pixels have not been processed, the process returns to step S50 to process the next pixel (NO in step S55). When all the pixels are processed as the target pixel, the processing in FIG. 8 ends.

  As described above, the background information generation unit 321 uses the first image of the current frame for the area excluding the passer-by area 14, the opening area 11, and the moving doorboard area 12, and the outside of the door at the first viewpoint. / Generate / update overall background information 313.

  The background information generation unit 321 also generates the outside / overall background information 313 for the second image in the same manner. That is, if the door 5 is in the open state, the background information generation unit 321 corresponds to the pixel value of the outside door area 13 in which the passerby is not detected in the second image of the current frame and the outside / overall background information 313. The weighted average value with the background pixel value is written in the storage unit 31 as a new background pixel value of the opening area 11 in the second image. In addition, if the door 5 is in the closed state, the background information generation unit 321 will detect the pixel value of the area where no passerby is detected in the second image of the current frame and the corresponding background pixel value of the outside / overall background information 313. Is written in the storage unit 31 as a new background pixel value for the second image.

  When the outside / overall background information 313 is generated, the process proceeds to step S8 in FIG. 4, and the background information generating unit 321 generates the opening background information 314.

  In step S8, opening background information generation processing is performed according to the flowcharts shown in FIGS.

  First, the background information generation unit 321 generates an opening color histogram in the first image using the second image according to the flowchart of FIG.

  In the loop processing of steps S60 to S64, a color histogram in the second viewpoint background acquisition region 311 of the current frame is generated. In step S60, each pixel of the second image of the current frame is sequentially set as the target pixel. That is, the background information generation unit 321 sequentially selects pixels included in the second image and sets them as the target pixel every time the process returns to step S60. At the start of the loop processing, the background information generation unit 321 prepares a storage area for storing the color histogram of the current frame, and initializes the frequency of each bin to zero.

  In step S61, it is confirmed whether or not the target pixel is a pixel in the second viewpoint background acquisition region 311. If the target pixel is a pixel in the second viewpoint background acquisition region 311 set for the second image, the background information generation unit 321 moves the process to step S62 (YES in step S61), otherwise. The process proceeds to step S66 (NO in step S61).

  In step S62, it is confirmed whether or not the target pixel is a pixel in the door plate region 12. The background information generation unit 321 shifts the process to step S66 if the pixel of interest is in the doorboard region 12 (YES in step S62), otherwise shifts the process to step S63 (NO in step S62). . With this determination, the background information generation unit 321 excludes the overlapping doorboard area 12 from the second viewpoint background acquisition area 311 when the door 5 is opened forward as viewed from the second viewpoint. By this exclusion, the door plate 15 is not mixed with the background, so that the accuracy of the opening color histogram can be maintained high.

  In step S63, it is confirmed whether or not the target pixel is a pixel in the passer-by area 14. The background information generation unit 321 refers to the passer-by area 14 extracted from the second image by the passing object area extraction unit 322, and if the target pixel is in the passer-by area 14, shifts the processing to step S66 (step S63). Otherwise, the process proceeds to step S64 (NO in step S63).

  In step S64, a color conversion process is performed. The background information generation unit 321 uses a color conversion function set in advance so that the average luminance value and variance of the second image match the average luminance value and variance of the first image, and the pixel value of the target pixel of the second image Convert.

  In step S65, the value of the target pixel is accumulated in the color histogram of the current frame. The background information generation unit 321 increases the frequency of the bin corresponding to the color of the pixel of interest in the current frame color histogram prepared in the storage unit 31 by one.

  In step S66, the background information generation unit 321 confirms whether all the pixels of the second image have been processed as the target pixel. If all the pixels have not been processed, the process returns to step S60 to process the next pixel (NO in step 66), and if the process has been completed, the process proceeds to step S67.

ただし０＜α_O1＜１であり、例えばα_O1＝０．３と設定する。 In step S67, the background information generation unit 321 updates the color histogram stored in the opening background information 314 using the color histogram of the current frame. Specifically, the background information generation unit 321 normalizes the color histogram of the current frame with the number of pixels to be processed, and the value of each bin of the color histogram H of the current frame and the opening background information 314 according to Expression (9). the color histogram H _O (t-1) corresponding to the value of the bin alpha _O1: write to background information generation unit 321 obtains the (1-α _O1) color histogram H _O (t) weighted average rate.

However, 0 <α _O1 <1 and, for example, α _O1 = 0.3 is set.

If the color histogram is not stored in the opening background information 314, the color histogram H of the current frame is stored as H 2 _O as it is.

  In this way, the background information of the opening area is not pixel-by-pixel information, but the pixel value distribution data (color histogram), which is information of the area alone, makes it visible and hidden due to changes in the shape of the opening area and the passage of passing objects The opening background information that can be continuously used can be generated. Therefore, even when the door is opened and closed, the passing object region can be detected with high accuracy. In addition, errors in the camera parameter 310 and the color conversion function are diluted, so that a reduction in accuracy can be prevented.

  Next, the background information generation unit 321 generates an opening background pixel value in the first image according to the flowchart of FIG.

  In step S70, it is confirmed whether or not the door 5 is in an open state. If the opening angle detected by the door state detection unit 320 is other than 0 degrees, the background information generation unit 321 shifts the process to step S71 assuming that the door 5 is in the open state (YES in step S70). If so, the process proceeds to step S76 (NO in step S70).

  In step S71, the background information generation unit 321 sequentially sets each pixel of the first image of the current frame as a target pixel. In the loop processing in steps S71 to S75, the opening background pixel value is generated / updated from the first image of the current frame.

  In step S72, it is confirmed whether or not the target pixel is a pixel in the opening area 11. If the region attribute of the target pixel set for the first image by the door state detection unit 320 is the opening region 11, the background information generation unit 321 moves the process to step S73 (YES in step S72). If not, the process proceeds to step S75 (NO in step S72).

  In step S <b> 73, it is confirmed whether or not the target pixel is a pixel in the passer-by area 14. The background information generation unit 321 refers to the passer-by area 14 extracted from the first image by the passing object area extraction unit 322, and shifts the processing to step S75 if the pixel of interest is in the passer-by area 14 (step S75). Otherwise, the process proceeds to step S74 (NO in step S73).

ただし０＜α_O2＜１であり、例えばα_O2＝０．３と設定する。 In step S74, the background pixel value of the target pixel is updated. The background information generation unit 321 updates the background pixel value of the target pixel stored in the aperture background information 314 with the pixel value of the target pixel of the current frame. Specifically, the background information generation unit 321 calculates the pixel value I (x) of the pixel of interest in the current frame and the background pixel value B _O (x, t−1) of the opening background information 314 according to Expression (10). A background pixel value B _O (x, t) obtained by weighted averaging at a ratio of α _O2 : (1−α _O2 ) is obtained and written in the opening background information 314.

However, 0 <α _O2 <1 and, for example, α _O2 = 0.3 is set.

  In step S75, it is confirmed whether all pixels of the first image have been processed as the target pixel. If all the pixels have not been processed, background information generation unit 321 returns the process to step S71 to process the next pixel (NO in step S75); otherwise, the process proceeds to step S9 in FIG. Return (YES in step S75). Background pixel values are generated in portions where no passersby appear in the opening area 11 by the processing in steps S71 to S75, and can be used instead of the color histogram.

  In step S76, it is confirmed whether or not the door 5 has changed from the open state to the closed state. If the opening angle of the previous frame stored in the storage unit 31 is other than 0 degree and the opening angle of the current frame is 0 degree, the background information generation unit 321 is in the open state in the current frame. If it is changed from the closed state to the closed state, the process proceeds to step S77 (YES in step S76). Otherwise, the process proceeds to step S9 in FIG. 4 (NO in step S76).

  In step S77, the aperture background pixel value is cleared. The background information generation unit 321 clears all the background pixel values of the opening portion background information 314. That is, when it is determined in step S70 that the door 5 is in the closed state, the latest background pixel value in the opening region 11 cannot be obtained from the first image. The information generation unit 321 does not generate the background pixel value in the opening area 11, and further clears the opening background pixel value so that the opening background pixel value can be used only while the door 5 is in the open state.

  The background information generation unit 321 also generates the opening background information 314 in the same manner for the opening area 11 in the second image. That is, if the door 5 is in the open state, the background information generation unit 321 calculates a color histogram of the current frame from the pixels of the opening area 11 where no passerby has been detected in the first image of the current frame, and The weighted average value of the histogram and the color histogram stored in the aperture background information 314 is written in the storage unit 31 as a new color histogram of the aperture region 11 in the second image. In addition, if the door 5 is in the open state, the background information generation unit 321 will detect the pixel value of the opening area 11 where no passerby has been detected in the second image of the current frame and the corresponding background pixel value of the opening background information 314. Is written in the storage unit 31 as a new background pixel value of the opening area 11 in the second image. Further, the background information generation unit 321 clears the background pixel value of the opening background information 314 when the door 5 changes from the open state to the closed state in the current frame.

  When the opening background information 314 is generated, the process proceeds to step S9 in FIG. 4, and the background information generation unit 321 generates the doorboard background information 315. The background information generation unit 321 executes the loop processing of steps S80 to S84 along the flowchart shown in FIG. 11 to generate a color histogram in the doorboard region 12 of the current frame.

  In step S80, the background information generation unit 321 sequentially sets each pixel of the first image of the current frame as a target pixel. Further, the background information generation unit 321 prepares a storage area for storing the color histogram of the current frame, and initializes the frequency of each bin to zero.

  In step S81, the background information generation unit 321 confirms whether or not the target pixel is a pixel of the doorboard region 12 set for the first image. If the pixel of interest is within door panel region 12, the process proceeds to step S82 (YES in step S81), otherwise the process proceeds to step S84 (NO in step S81).

  In step S <b> 82, the background information generation unit 321 refers to the passer-by area 14 extracted from the first image by the passable object area extraction unit 322, and confirms whether the pixel of interest is a pixel in the passer-by area 14. If the pixel of interest is in passerby area 14, the process proceeds to step S84 (YES in step S82), otherwise the process proceeds to step S83 (NO in step S82).

  In step S <b> 83, the background information generation unit 321 accumulates the color histogram H of the current frame in the storage unit 31 by increasing the frequency of the bin corresponding to the color of the target pixel by 1 by setting the target pixel as the target of color histogram generation. To do.

  In step S84, the background information generation unit 321 checks whether all pixels of the first image have been processed as the target pixel. If all the pixels have not been processed, the process returns to step S80 to process the next pixel (NO in step S84). If not, the process proceeds to step S85 (YES in step S84).

ただし０＜α_D＜１であり、例えばα_D＝０．３と設定する。 In step S85, the background information generation unit 321 updates the color histogram stored in the doorboard background information 315 using the color histogram of the current frame. Specifically, the background information generation unit 321 normalizes the color histogram of the current frame with the number of pixels targeted for processing, and the value of each bin of the color histogram H of the current frame and the doorboard background information according to the equation (11). A color histogram H _D (t) obtained by weighting and averaging the bin values corresponding to the color histogram H _D (t−1) 315 at a ratio of α _D1 : (1−α _D1 ) is written in the door panel background information 315. . If the color histogram is not stored in the door panel background information 315 immediately after the start-up, the color histogram H of the current frame is stored as _HD as it is.

However, 0 <α _D <1 and, for example, α _D = 0.3 is set.

  In addition, the background information generation unit 321 similarly generates the door panel background information 315 for the door panel region in the second image. That is, the background information generation unit 321 calculates a color histogram of the current frame from the pixels in the door panel area 12 where no passerby was detected in the second image of the current frame, and stores the color histogram in the color histogram and the door panel background information 315. The weighted average value with the color histogram is written into the storage unit 31 as a new color histogram of the doorboard region 12 in the second image. In this way, the background information of the doorboard area is not pixel-by-pixel information, but pixel-value distribution data (color histogram), which is area-by-area information, so that it can be seen and hidden due to changes in the shape of the doorboard area and the passage of passing objects. In contrast, it is possible to generate door panel background information that can be continuously used. Therefore, even when the door is opened and closed, the passing object region can be detected with high accuracy.

  When the above process is completed, the process returns to step S1 in FIG. 4 to process the next frame.

  As described above, according to the present embodiment, when a passing object is detected in the space where the door is installed, the door area whose shape changes with the opening and closing of the door, particularly the opening area and the door plate area, can be accurately determined. Can be recognized. As a result, the background information of each area can be accurately generated and applied, so that when the background difference process is performed, the entire opening can be prevented from becoming a change area, and a part or the whole is reflected in each area. It is possible to appropriately detect a passing object.

  In addition, the background information of the opening area and the doorboard area is used as the distribution data of the pixel value in each area, so that the background information of the hidden part can be obtained even if part of the background is hidden or hidden due to the change of the area shape or the passage of passing objects Since it can compensate, it becomes possible to continue detecting a passing object. As described above, the detection of the passing object region can be continued even when the door is opened and closed, and the tracking process of the passing object can be reliably performed.

  In this embodiment, an example of a hinged door that opens to the front is shown, but the present invention can be applied to various doors such as a hinged door, a sliding door, and a folding door that are opened to the back by adapting the door pattern 312. Moreover, in this Embodiment, although the opening part color histogram was produced | generated from the 2nd image, it can also produce | generate from a 1st image. That is, the background information generation unit 321 generates and updates a color histogram using the pixel values of the opening area 11 of the first image when the door 5 is in the open state, and opens the color histogram when the door 5 is in the closed state. And the color histogram is retained. In this case, an upper color histogram is separately generated from the first image at the upper part of the opening region to compensate for illumination variation during the closed period, and is stored in the storage unit 31, and the upper color at the time of the current opening relative to the upper color histogram at the time of the previous opening. The variation amount of the histogram is obtained, and the held opening color histogram is corrected with the variation amount.

  In the present embodiment, the passing object is a person, but the present invention is not limited to this. For example, a site entrance where a gate is installed or a garage with a shutter can be photographed, and a vehicle passing through a door such as a gate door or shutter can be detected as a passing object.

In addition, the passing object area extraction unit 322 generates one passer likelihood image in common for all passers-by who are being tracked, but instead of this, one passer likelihood image is generated for each passer. It may be generated. In this case, the passer-by object region extraction unit 322 creates a passer-by likelihood image L _k of the passer-by using the color histogram H _k of the passer-by of interest _k , and a passer-by from the passer-by likelihood image L _k. The process of determining one passer-by area is repeated for all passers-by who are passing. In this way, it is possible to generate a passer likelihood image suitable for the color difference for each passerby, and to improve the extraction accuracy when there are a plurality of passers with different colors.

  Further, in the present embodiment, the mode in which the function of each unit of the passing object detection device is realized by one computer has been described, but the present invention is not limited to this. The functions of each part of the passing object detection device can be realized by controlling a general computer by a program, and the functions of these devices may be appropriately combined and processed by one computer. Distributed processing may be performed by a plurality of computers connected via a network or the like.

  2 imaging unit, 2a first imaging unit, 2a second imaging unit, 3 control unit, 4 display unit, 5 door, 6 rooms, 6a first room, 6b second room, 7 opening, 7 shielding space, 8 opening part, 9 floor, 10 passing object, 11 opening area, 12 doorboard area, 13 outside door area, 14 passerby area, 30 image acquisition part, 31 storage part, 32 image processing part, 33 output part, 100 passing object Detection device, 310 camera parameters, 311 second viewpoint background acquisition region, 312 door pattern, 313 outside / overall background information, 314 opening background information, 315 doorboard background information, 316 passing object information, 320 door state detection unit, 321 Background information generation unit, 322 passing object region extraction unit, 323 passing object tracking unit, 401, 402, 403, 411, 412, 413 histogram.

Claims (8)

A passing object detection device that detects a passing object from a captured image of a space including a door,
A door pattern storage unit that stores in advance a shape pattern of the door including at least a shape characteristic of an opening generated by opening the door;
An area that compares the photographed image with the shape pattern to calculate a similarity, and divides the photographed image into a door area that matches the shape pattern with the highest similarity and an outside door area other than the door area A section,
A background storage unit that stores background information of the space that does not include the passing object in each region divided by the region dividing unit;
The degree of difference between each pixel of the captured image and the background information of the area is calculated for each area divided by the area dividing unit, and a group of pixels whose difference is equal to or greater than a predetermined reference is extracted as a passing object area A passing object region extraction unit,
A passing object detection apparatus comprising: The passing object detection device according to claim 1,
The background storage unit stores distribution data of pixel values of the door region as the background information in the door region, and stores pixel values of each pixel in the door outer region as the background information in the door outer region. A passing object detection device characterized by the above. The passing object detection device according to claim 2,
The door pattern storage unit stores the shape pattern further including a shape characteristic of a door plate constituting the door,
The area dividing section further divides the door area into an opening area where the opening appears and a door plate area where the door plate appears,
The background storage unit stores distribution data of pixel values of each area as the background information in each of the opening area and the doorboard area;
A passing object detection device characterized by the above. The passing object detection device according to any one of claims 1 to 3,
The passing object region extraction unit is configured to apply a region to which the passing object model is applied when a sum of the differences between pixels included in the passing object model imitating a shape of the passing object is equal to or greater than the predetermined reference. Detecting as a passing object detecting device. The passing object detection device according to any one of claims 1 to 3,
The passing object region extracting unit extracts pixels in which the degree of difference of each pixel of the captured image is equal to or greater than the predetermined reference, and extracts regions where the extracted pixels are gathered to be equal to or larger than a preset passing object size. A traffic object detecting apparatus, wherein the traffic object area is detected as a traffic object area. The passing object detection device according to any one of claims 1 to 5,
Extracting an image feature of a passing object from the passing object region of the captured image;
The passing object area extraction unit obtains the similarity between the passing object area of the door area and the image feature of the passing object, and the higher the similarity, the higher the difference between the pixels of the passing object area or A passing object detection apparatus that performs correction to lower the predetermined reference. The passing object detection device according to any one of claims 1 to 5,
Extracting an image feature of a passing object from the passing object region of the captured image;
The passing object region extraction unit obtains the similarity of the image feature of the passing object for each pixel of the door region, and the higher the similarity, the higher the difference of the pixel or the lower the predetermined criterion A passing object detection apparatus characterized by performing: The passing object detection device according to claim 6,
The passing object region extraction unit inputs the captured image of the door region to the classifier that has learned the image characteristics of the passing object in advance, and the difference is obtained when the output of the classifier is a value indicating the presence of the passing object. A passing object detection apparatus which performs correction to increase the degree or to lower the predetermined reference.

Images