JP2011185664A - Object detector - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To more reduce processing load in detection of an object by using a distance image, while reducing the amount of information keeping information for determining whether it is the object or not. <P>SOLUTION: A distance image is created by an active-type distance image sensor 1. A background acquisition means 2 stores a background distance image, and a difference image creation means 3 creates a distance difference image from the distance image and the background distance image. A pixel-of-interest extraction means 4 extracts pixels whose pixel values in the distance difference image are a presence threshold or above from the distance image as pixels of interest, and a density measuring means 5 maps the pixels of interest in a three-dimensional virtual space, and finds the number of pixels of interest in each block of a unit volume as a density. An effective block extraction means 6 extracts blocks whose densities are an object threshold or above as effective blocks, and a region integration means 7 integrates adjoining effective blocks and forms a group. An object determination means 8 determines the group as an object, if the size of the group is in a proper range. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an object detection apparatus that detects an object using a distance image sensor.

  Conventionally, a technique for detecting an object using a distance image has been proposed. For example, Patent Document 1 describes a technique for detecting an object existing in front of an automobile using a distance image generated by stereo image processing.

  In Patent Document 1, a plurality of sections are set on a distance image, a histogram of distance values is generated for each section, and a distance value whose appearance frequency is equal to or greater than a threshold is obtained as a distance value representing the section. It is described. Furthermore, in the processing space composed of sections and distances, when the positions occupied by the representative values of the distances in each section are adjacent, they are regarded as the same object and grouped.

  In the technique described in Patent Document 1, it is possible to separate objects having different distances as different groups by performing grouping in the processing space.

JP 2008-64628 A

  By the way, if the technique described in Patent Document 1 is adopted, even if a plurality of objects occupying adjacent areas in the distance image can be separated by performing grouping based on the distance difference. Moreover, by performing grouping, there is a possibility that the processing load can be reduced as compared with the case where the pixels of the distance image are handled as they are.

  However, since information about the shape of the object is almost lost during grouping, there is a problem that the type of the object cannot be identified. That is, the technique described in Patent Document 1 cannot be used in applications that recognize objects using distance images.

  The present invention has been made in view of the above-mentioned reasons, and the object of the present invention is to make it possible to reduce the amount of information while leaving information for determining whether or not the object is an object in the distance image, and to use the distance image. An object of the present invention is to provide an object detection apparatus capable of reducing the processing load in detection of an object as compared with the conventional technique.

  In order to achieve the above object, the present invention generates an active distance image sensor that generates a distance image having a pixel value as a distance for a visual field region, and a distance image sensor in a state where no object exists in the visual field region. A background acquisition unit that stores a distance image as a background distance image, and a distance difference image that is a difference image between a distance image generated by a distance image sensor and a background distance image stored in the background acquisition unit in a state of monitoring a visual field region Among the pixels included in the distance image generated by the difference image generating means and the distance image sensor, the pixel whose pixel value in the distance difference image generated by the difference image generating means is equal to or greater than the specified existence threshold is set as the target pixel. The target pixel extracting means for extracting and the target pixel extracted by the target pixel extracting means are mapped to a three-dimensional virtual space. Density measuring means for obtaining the number of pixels of interest as a density for each block of a unit volume defined in the virtual space, and an effective block for extracting a block whose density obtained by the density measuring means is equal to or more than a prescribed target threshold as an effective block Extraction means, area integration means for forming a group by integrating effective blocks adjacent to the effective block extracted by the effective block extraction means, and using knowledge about the object for the group formed by the area integration means And an object discriminating means for discriminating whether or not the object is an object.

  It is desirable that the effective block extraction unit sets the target threshold value smaller as the distance to the block is larger.

  In this case, the effective block extraction unit uses a target threshold value at a specified reference distance as a reference target threshold value, and uses a value obtained by multiplying the reference target threshold by 1 / square of the ratio of the distance to the block with respect to the reference distance as the target threshold value. It is desirable.

  The object discriminating means includes a feature extracting means for extracting the size as a feature from the group formed by the area integrating means, and the group using the knowledge that the size extracted by the feature amount extracting means is within a prescribed appropriate range. It is desirable to include size determining means for determining whether the object is an object.

  In addition, when the size extracted as a group feature by the feature extraction unit exceeds the upper limit of the specified appropriate range, the object discrimination unit performs clustering of the effective blocks forming the group, thereby performing a plurality of the groups. It is desirable to provide clustering means for dividing the data into groups, and to perform discrimination by the size discrimination means for the group after division by the clustering means.

  The effective block extraction means obtains the frequency distribution of the pixels included in the effective block on the coordinate plane intersecting the center line of the visual field region of the distance image sensor among the coordinate planes of the orthogonal coordinate system defining the virtual space, It is desirable not to employ a pixel having a coordinate value whose frequency is smaller than the specified noise threshold as the target pixel.

  The center line in the visual field area of the distance image sensor is set obliquely downward, and the density measuring means performs correction according to the depression angle of the central line in the visual field area of the distance image sensor when mapping the target pixel in the virtual space. The configuration to be performed can be adopted.

  The pixel-of-interest extracting means may adopt a configuration in which a pixel located above a threshold plane defined at a predetermined height position in the visual field region of the distance image sensor is extracted as the pixel of interest.

  The background acquisition unit may update the newly generated background distance image when a state in which the target is not detected by the target determination unit reaches a predetermined update time.

  The distance image sensor has a function of generating a moving image based on a distance image, and when the object discriminating means detects the object, the object in the visual field region is tracked by using adjacent distance images in time series. You may employ | adopt the structure provided with an object tracking means.

  According to the configuration of the present invention, it is possible to reduce the amount of information while leaving the information for determining whether or not the object is present by treating the distance image as a block including a plurality of pixels that are candidates for the object. Thus, it is possible to reduce the processing load in the detection of the object using the distance image as compared with the conventional case.

It is a block diagram which shows embodiment. It is a figure which shows the example of the image in the same as the above. It is explanatory drawing of the block in the same as the above. It is a figure which shows the process sequence same as the above. It is a figure which shows the example which needs clustering in the same as the above. It is a figure which shows the example which needs clustering in the same as the above. It is a figure which shows the example of the effective block before clustering same as the above. It is a figure which shows the procedure of the clustering same as the above. It is a figure which shows the processing procedure of the clustering same as the above. It is a figure which shows the example of generation | occurrence | production of the abnormal value in the same as the above. It is the schematic of the example of generation | occurrence | production of the abnormal value in the same as the above. It is the schematic which shows the generation example and removal example of an abnormal value in the same as the above. It is explanatory drawing of the operation | movement which detects the abnormal value in the same as the above. It is a figure which shows the example of a setting of the threshold value surface in the same as the above. It is a block diagram which shows the structural example of the distance image sensor used for the same as the above. FIG. 16 is an operation explanatory diagram of the configuration shown in FIG. 15. It is a conceptual explanatory drawing of the coordinate transformation same as the above. It is a conceptual explanatory drawing of the affine transformation in the same as the above.

(Basic configuration)
The object detection apparatus used in the embodiments described below employs a configuration that detects the presence / absence / movement of an object using a distance image in which the values of pixels constituting the image are distance values. A distance image sensor that generates a distance image emits light such as infrared rays to the visual field area and receives reflected light from an object (including a person) existing in the visual field area. It has an active configuration for measuring the distance to an object.

  Active distance image sensors include technologies that measure the time from light projection to light reception, as well as technologies that use the principle of triangulation, and various types of technologies are known for each technology. . In the embodiments described below, any of these techniques can be used, but the time-of-flight method (Time) that detects the distance to an object by using information corresponding to the time difference from light projection to light reception. The explanation is based on an example using the principle of Of Flight.

  The distance image sensor used in the embodiment emits modulated light whose intensity changes with time, from a light source, and calculates the phase difference between the modulated light reflected by the object and received by the light receiving element and the projected modulated light. It is used as information corresponding to the time difference from light projection to light reception.

  As the modulation waveform of the modulated light, a sine wave, a triangular wave, a sawtooth wave, a square wave, or the like can be used. When a sine wave, a triangular wave, or a sawtooth wave is used, the period of the modulated light is set to a constant period. In addition, when using a square wave, the period of the modulated light is set to a fixed period, and the ratio between the ON period (light emitting source light emitting period) and the OFF period (light emitting source non-light emitting period) is changed randomly. It is also possible to adopt a technique for making them. Regardless of which waveform is used as the modulation waveform, the phase difference between light projection and light reception is not obtained in one cycle (in the case of a square wave, the period including the on period and the off period once), but a large number of periods The phase difference is obtained from the average value (or integrated value) of.

  As the light receiving element, an imaging element in which a plurality of light receiving parts (photoelectric conversion parts) are two-dimensionally arranged is used. As the imaging device, a well-known configuration for imaging a grayscale image provided as a CCD image sensor or a CMOS image sensor can be used. However, the imaging device is designed specifically to have a structure suitable for a distance image sensor. It is desirable to use an element.

  The latter image sensor has a basic structure similar to an image sensor having a well-known configuration for capturing gray images. However, it has a function of controlling the light receiving sensitivity in units of light receiving units. The reason why the light receiving sensitivity is controlled in units of light receiving units is to detect information including a phase difference between light projection and light reception by controlling the timing of receiving the modulated light. This imaging element has an accumulation region for accumulating charges generated in each light receiving unit over a period that is an integral multiple of the modulation period of the modulated light. Furthermore, there is a case where a function of reducing components of ambient light or ambient light that is not modulated light among charges generated in the light receiving unit is provided.

  In the following description, the following configuration is assumed as one configuration example of the distance image sensor. However, this configuration is not intended to limit the present invention, and the modulation waveform of the modulated light, the configuration of the imaging device, the control of the imaging device, etc. The configurations provided in various known active distance image sensors can be used.

  As shown in FIG. 15, the distance image sensor 1 used in the following description includes a light emitting source 11 that emits light (desirably using near infrared rays) and an image sensor 12 that images a visual field region. In the illustrated example, an area image sensor in which light receiving units are two-dimensionally arranged is assumed as the image sensor 12.

  The light emitting source 11 is a light emitting element such as a light emitting diode or a laser diode that can obtain a light output proportional to the instantaneous value of the input. Further, in order to ensure the amount of light emitted from the light source 11, the light source 11 is configured using an appropriate number of light emitting elements. The modulated light emitted from the light source 11 is projected through the light projecting optical system 13. The image sensor 12 receives light from the visual field region through the light receiving optical system 14. The light projecting optical system 13 and the light receiving optical system 14 are arranged close to each other with the light projecting and receiving directions parallel to each other. That is, the distance between the light projecting optical system 13 and the light receiving optical system 14 can be substantially ignored with respect to the visual field region.

  In the distance image sensor 1, a modulation signal generation unit 15 that outputs a modulation signal for driving the light source 11, and a light reception timing at the image sensor 12 based on the modulation signal output from the modulation signal generation unit 15 are defined. A timing control unit 16 that generates a received light timing signal, and an arithmetic processing unit 17 that obtains a distance to the object Ob existing in the visual field region using the received light signal output from the image sensor 12.

  The modulation signal generation unit 15 generates a modulation signal whose output voltage changes with a sine waveform having a constant frequency (for example, 20 MHz), and gives the modulation signal to the light emission source 11, as shown in FIGS. Then, modulated light whose light output changes in a sine wave form is emitted from the light emitting source 11. When a light-emitting diode is used as the light-emitting source 11, by applying a signal voltage of a modulation signal to the light-emitting diode via a current limiting resistor, the current flowing through the light-emitting diode is changed to emit modulated light.

  On the other hand, the image pickup device 12 can generate an electric charge according to the received light intensity only during a period synchronized with the received light timing signal by using the electronic shutter technique. Further, the charge generated in the light receiving unit is accumulated over an accumulation period corresponding to a plurality of periods (for example, 10000 periods) of the modulation signal, and then taken out as a light reception output to the outside of the image sensor 12.

  That is, the operation of generating charge at the timing synchronized with the light reception timing signal and the operation of storing the generated charge are repeatedly performed in a plurality of periods of the modulation signal that is an accumulation period. Accordingly, the charges generated during the accumulation period are accumulated, and the accumulated charges are taken out as a light receiving output. With the charge for one period of the modulation signal, the amount of generated charge is small, and the amount of charge tends to vary due to the influence of ambient light and internal noise, but this kind of charge can be obtained by accumulating charge during the accumulation period. Variation in charge amount due to the cause is suppressed.

  The timing control unit 16 generates a light reception timing signal synchronized with the modulation signal. Here, four different phases are defined in one period of the modulation signal, and four types of light reception timing signals for setting a light reception period having a certain time width for each phase are generated. By supplying each of the four types of light reception timing signals among the four types of light reception timing signals to the imaging element 12 for each accumulation period, the charges received during the period corresponding to the phase set by the light reception timing signals are accumulated.

  That is, the charge generated for each light receiving unit in the light receiving period defined by one type of light receiving timing signal is accumulated in one accumulation period, and the accumulated charge is taken out to the outside of the image sensor 12 as a light receiving output four times. Repeatedly, the light receiving outputs corresponding to the four types of light receiving timing signals are taken out of the image sensor 12 in four accumulation periods.

Now, as shown in FIG. 16 (c), it is assumed that the light reception timing signal is defined by a phase different by 90 degrees in one cycle of the modulation signal. In this case, when the light reception outputs (charge amounts) corresponding to the respective light reception timing signals are A0, A1, A2, and A3, the phase difference ψ [rad] is expressed by the following equation.
ψ = (A0−A2) / (A1−A3)
If the frequency of the modulation signal is f [Hz], the time difference Δt from light projection to light reception is expressed as Δt = ψ / 2π · f using the phase difference ψ, so the speed of light is c [m / s]. Then, the distance to the object can be expressed as c · ψ / 4π · f.

  That is, the distance to the object can be obtained from the four types of light reception outputs (charge amounts) A0 to A3. The time width of the light receiving period can be appropriately set so that an appropriate amount of received light can be obtained in the light receiving section (for example, a time width corresponding to a quarter cycle of the modulation signal can be set). ). However, the time width of each light receiving period needs to be equal to each other.

  In the arithmetic processing unit 17, in addition to the above-described processing for obtaining the phase difference ψ based on the light reception outputs (charge amounts) A <b> 0 to A <b> 3 and converting it to the distance, processing described in the following embodiments can also be performed. The arithmetic processing part 17 is comprised using a computer (microcomputer), and the process mentioned above is implement | achieved by running a program with a computer. Further, not only the arithmetic processing unit 17 but also the configuration excluding the light emission source 11 and the image sensor 12 is realized using a computer.

  In the above-described operation example, four types of light reception timing signals are used. However, the phase difference ψ can be obtained using three types of light reception timing signals, and two types of light reception timing signals can be obtained in an environment where there is no ambient light or ambient light. It is possible to obtain the phase difference ψ even with the light reception timing signal.

  Further, in the above-described operation, four accumulation periods are required to take out four types of light reception outputs (charge amounts) A0 to A3 from the image pickup device 12, but one of the modulation signals is obtained using two light reception units. It is also possible to generate charges corresponding to two types of light reception timing signals in a cycle. With this configuration, it is possible to read out light reception outputs corresponding to two types of light reception timing signals from the image sensor 12 at a time. Similarly, if four light receiving units are used to generate charges corresponding to four types of light reception timing signals in one cycle of the modulation signal, light reception outputs corresponding to the four types of light reception timing signals can be read out once. It becomes possible.

  Since the above-described distance image sensor 1 uses an image pickup element in which a plurality of light receiving parts are two-dimensionally arranged as a light receiving element for receiving light from the field of view, based on the amount of light received by each light receiving part. A distance image is generated by obtaining the distance value. As described above, when one light receiving unit is associated with four types of light receiving timings, one light receiving unit corresponds to one pixel of the distance image. However, when two light receiving units are used in association with two types of light receiving timing signals, the two light receiving units correspond to one pixel, and four light receiving units are used in association with four types of light receiving timing signals. In this case, four light receiving units correspond to one pixel.

  As described above, the light receiving surface of the image sensor is defined as a virtual projection surface obtained by projecting the visual field region of the distance image sensor 1, and a plurality of light receiving units arranged two-dimensionally on the projection surface are pixels of the distance image. Is associated with. Therefore, the position of the light receiving unit corresponds to the position of the pixel of the distance image. The generated distance image is stored in the memory of the computer.

  In the distance image sensor 1 described above, light is emitted from the light emission source 11 to the visual field region, and the visual field region is picked up by the single image sensor 12, so the visual field region is separated from the distance image sensor 1 with the distance image sensor 1 as a vertex. Will spread.

  For example, when the light projecting optical system 13 and the light receiving optical system 14 are formed isotropic around the optical axis, the visual field region has a pyramid shape with the distance image sensor 1 as a vertex. When a lens formed anisotropically, such as a lens having an elliptical outer periphery or a cylindrical lens, is used as the light projecting optical system 13, the visual field region is formed in a fan shape having the distance image sensor 1 as a vertex. Will be. Furthermore, it is possible to employ a configuration in which a plurality of light emitting sources 11 are provided and two or more light projecting optical systems 13 are provided, and the light projecting directions are different for each light projecting optical system 13.

  In the following description, it is assumed that the visual field region is formed in a pyramid shape having the distance image sensor 1 as a vertex. Accordingly, the positions of the pixels arranged on the virtual projection plane described above correspond to the direction in which the visual field area is viewed from the distance image sensor 1, and the pixel value of each pixel indicates the distance to the object existing in the direction. Will represent. If the position of each pixel of the distance image is expressed by the coordinate value of the two-dimensional orthogonal coordinate system (that is, the position of the pixel position in the horizontal direction and the vertical direction), this coordinate position indicates the azimuth angle, depression angle and elevation angle at which the object is viewed. Corresponding to In other words, the information that each pixel of the distance image has is equivalent to information that represents the position of the object in polar coordinates. Therefore, hereinafter, the distance image output from the distance image sensor 1 is referred to as a polar coordinate system distance image.

  The distance image of the polar coordinate system described above is highly convenient when information on the distance from the distance image sensor 1 is necessary, but the correspondence with each position in the real space that is the visual field region is difficult to understand and exists in the real space. It is inconvenient to specify a region based on the object to be used. Therefore, an image conversion unit 8 is provided to map each pixel of the distance image in the polar coordinate system to a three-dimensional virtual space associated with the real space. In the virtual space, coordinate axes of an orthogonal coordinate system are set so as to correspond to the real space.

  Now, it is assumed that the distance image sensor 1 is arranged above the visual field area and the center line in the visual field area of the distance image sensor 1 is set obliquely downward. In addition, it is assumed that the floor surface in the real space is associated with the lower surface of the virtual space. The coordinate axis of the orthogonal coordinate system set in the virtual space defines the extension direction of a straight line formed by projecting the center line in the visual field region of the distance image sensor 1 on the floor surface as the positive direction of the z-axis, and the distance image The vertical downward direction from the sensor 1 is defined as the positive direction of the y-axis. Also, a right-handed orthogonal coordinate system is adopted, and the right direction from the origin to the yz plane is the positive direction of the x axis. Also, the position of the origin is set to a predetermined height position from the floor surface on the line of sight (that is, the center line of the visual field area) from the center pixel of the image sensor 12 provided in the distance image sensor 1. To do.

  The image conversion unit 8 performs coordinate conversion from a polar coordinate distance image to a virtual space, which is a three-dimensional orthogonal coordinate system, in the following procedure. First, the coordinate conversion from the distance image of the polar coordinate system to the orthogonal coordinate system is performed on the assumption that the light receiving surface of the image sensor 12 provided in the distance image sensor 1 is parallel to a plane orthogonal to any coordinate axis of the virtual space. Thereafter, by performing affine transformation on the obtained orthogonal coordinate system, coordinate transformation for correcting the inclination of the center line of the visual field region of the distance image sensor 1 with respect to the coordinate axis of the virtual space is performed.

  Now, the coordinate position of the pixel on the light receiving surface of the image sensor 12 is represented by (u, v), and the pixel value of the pixel at the coordinate position (u, v) (that is, the distance value to the object) is r. The coordinate position in the orthogonal coordinate system in the real space is represented by (X, Y, Z). That is, as shown in FIG. 17, the origin O and the coordinate system XYZ are defined in the field of view, and the coordinate position (X, Y, Z) in the field of view is represented by the coordinate position (u of the pixel on the light receiving surface of the image sensor 12. , V) and the distance value r. Here, the unit of the coordinate position of the pixel is [pixel].

  First, it is assumed that the u-axis direction and the X-axis direction are parallel, and the v-axis direction and the Y-axis direction are parallel. That is, it is assumed that the center line of the visual field area of the distance image sensor 1 coincides with the Z axis. In this state, if the X axis coincides with the x axis of the virtual space, the Y axis rotates about the x axis with respect to the y axis of the virtual space.

The origin of the Cartesian coordinate system XYZ is assumed to be located on the center line of the visual field region. First, conversion for expressing the reference position in the distance image by the distance from the center pixel (uc, vc) on the light receiving surface of the image sensor 12 is performed. I do.
u1 = su (u-uc)
v1 = sv (v−vc)
However, su, sv [m / pixel] is the pixel size of the image sensor 12. Here, the following equation is corrected for the coordinate position (u1, v1) in consideration of lens distortion.
u2 = a · u1
v2 = a · v1
However, a = 1 / {1 + κ (u1 ² + v1 ² )}, κ is a lens distortion parameter [1 / m ² ]. When the corrected coordinate position (u2, v2) is used, the coordinate position of the orthogonal coordinate system XYZ is expressed by the following equation.
X = u2 · r / b
Y = v2 · r / b
Z = fr / b
However, b = (u2 ² + v2 ² + f ² ) ^1/2 and f [m] are focal lengths of the light receiving optical system 14.

  By the way, since the distance image sensor 1 is arranged so as to look down on the visual field region, in order to correspond to the orthogonal coordinate system of the virtual space, the above-described orthogonal coordinate system xyz defined in the virtual space is further used from the orthogonal coordinate system XYZ. It is necessary to perform affine transformation to. That is, the coordinate position (X, Y, Z) obtained by performing the conversion from the polar coordinate system distance image to the orthogonal coordinate system XYZ as described above is the distance image sensor 1 as shown in FIG. Is set to the Z axis. On the other hand, the orthogonal coordinate system xyz in the virtual space rotates around the X axis with respect to the orthogonal coordinate system XYZ as shown in FIG. Therefore, by rotating the coordinate position (X, Y, Z), coordinate conversion to the coordinate position (x, y, z) of the orthogonal coordinate system xyz defined in the virtual space is performed.

  Now, the center line of the visual field area of the distance image sensor 1 (the Z axis described above) is included in the yz plane defined in the virtual space, and is rotated by an angle θ in the positive direction of the y axis with respect to the z axis. Shall. Further, it is assumed that the X axis coincides with the x axis of the virtual space. In other words, the depression angle of the distance image sensor 1 is set to θ.

In this case, if affine transformation is performed to rotate the orthogonal coordinate system XYZ around the X axis by an angle θ so that the Z axis coincides with the z axis, coordinate transformation from the orthogonal coordinate system XYZ to the orthogonal coordinate system xyz is performed. be able to. That is, affine transformation in the yz plane is performed as in the following equation.
x = X
y = Y · cos θ−Z · sin θ
z = Y · sin θ + Z · cos θ
By performing the coordinate transformation from the polar coordinate system distance image to the Cartesian coordinate system according to the above-described procedure, and further performing the affine transformation, the polar coordinate system distance image information obtained by setting the visual field region obliquely downward is obtained in the virtual space. Can be associated with the coordinate position of the Cartesian coordinate system defined in the above. Note that the depression angle θ used for performing the affine transformation is an angle formed by the light receiving surface of the image sensor 12 with respect to the vertical plane. Therefore, the depression angle θ is given by manual input after the distance image sensor 1 is installed. Further, by providing a gyro sensor that detects an angle of the distance image sensor 1 with respect to the gravity direction, the depression angle θ can be automatically obtained.

  As described above, the distance image of the polar coordinate system can be mapped to the coordinate position (x, y, z) of the orthogonal coordinate system defined in the virtual space. In the embodiment described below, a technique for detecting an object on the assumption of a technique for mapping from a polar coordinate system distance image generated by the distance image sensor 1 to a three-dimensional virtual space represented by an orthogonal coordinate system will be described. To do.

(Embodiment)
Hereinafter, the configuration of the present embodiment will be described with reference to FIG. Further, images generated by the respective configurations shown in FIG. 1 will be described with reference to FIG.

  In the present embodiment, it is assumed that a person is detected as an object in the visual field region of the distance image sensor 1. When detecting a person in the field of view of the distance image sensor 1, mapping is performed from a distance image in the polar coordinate system to a three-dimensional virtual space represented by an orthogonal coordinate system.

  In the distance image sensor 1, a background distance image P1 (FIG. 2 (a)) as a distance image (polar coordinate system) in a state where no person exists in the visual field region is generated in advance, and the background distance image P1 is used as the background acquisition means 2. Remember me.

  In order to detect the presence / absence of an object in the field of view of the distance image sensor 1, first, the difference image generation means 3 obtains the background image P2 (FIG. 2B) of the polar coordinate system generated by the distance image sensor 1 and the background. A difference image from the background distance image P1 stored in the means 2 is generated. Hereinafter, this difference image is referred to as a distance difference image P3 (FIG. 2C). In the distance difference image P3, a relatively large value is obtained only for the pixel in which the change from the background distance image P1 occurs.

  The distance difference image generated by the difference image generation means 3 is given to the attention pixel extraction means 4, and the attention pixel extraction means 4 binarizes the distance difference image P3 as appropriate with a specified existence threshold. When the distance difference image P3 is binarized by the presence threshold, a binary image P4 as shown in FIG. 2D is obtained. When pixels included in the binary image P4 are extracted from the distance image P2, an image P5 obtained by extracting the target pixel from the distance image P2 is obtained as shown in FIG. That is, the pixel-of-interest extraction unit 4 extracts, as pixels of interest, a pixel whose pixel value in the distance difference image P3 is equal to or greater than a predetermined existence threshold among the pixels included in the distance image P2.

  The presence threshold value is set to a relatively small value because it is necessary to detect the target pixel when the target object to be detected exists in the visual field region. However, if the presence threshold is too small, a noise component generated by fluctuations in the distance image P2 may be included in the pixel of interest, so the size is set to such a level that the noise component can be removed. Actually, if the existence threshold is set to about several tens of centimeters, the pixel of interest corresponding to the object can be extracted while removing the noise component.

  The pixel of interest extracted from the distance image in the polar coordinate system by the pixel-of-interest extraction unit 4 is mapped in a three-dimensional virtual space by the density measurement unit 5. Further, the density measuring means 5 obtains the number of pixels of interest as the density for each block of the unit volume defined in the virtual space. As shown in FIG. 3B, the block B is set as a unit volume cube. The density measuring unit 5 measures the number of target pixels Px included in the block B as the density.

  Here, as described as the basic configuration of the virtual space, the vertical downward direction from the distance image sensor 1 is the y-axis direction, and the extension direction of the straight line in which the center line of the visual field area of the distance image sensor 1 is projected on the floor surface is The z-axis direction is assumed. When the positive direction of the y-axis is directed downward, the positive direction of the z-axis is directed away from the distance image sensor 1, and a right-handed orthogonal coordinate system is adopted, when the visual field region is viewed from the distance image sensor 1, the horizontal direction The right direction is the positive direction of the x axis. Each side of the cube that forms the block B is set to be parallel to the x-axis direction, the y-axis direction, and the z-axis direction.

  Since the density obtained by the density measuring means 5 is expected to increase on the surface of the object, effective block extraction is performed to extract a block B having a density equal to or higher than a predetermined object threshold as an effective block that is a candidate for the object. Means 6 are provided.

  By the way, as described in the basic configuration, it is set in a pyramid shape having the distance image sensor 1 as an apex, and as shown in FIG. 3A, the spatial region included in the visual field region VF is further away from the distance image sensor 1. The range becomes wider. Therefore, as shown in FIG. 3B, even when the object is the same, the block from the distance image sensor 1 (indicated by the distance in the Z-axis direction in the illustrated example) increases as the distance increases. The number of target pixels included in B decreases. In other words, if the target threshold value is set to be constant regardless of the position in the visual field region, the probability that the block B is determined to be an effective block is small when the distance from the distance image sensor 1 is small, and the greater the distance, the more effective The probability of being determined as a block is reduced.

  In order to prevent the probability that the block is determined to be an effective block depending on the distance to the block B, the effective block extracting unit 6 desirably sets the target threshold value smaller as the distance to the block B increases. Specifically, the density of the block B is inversely proportional to the square of the distance. Therefore, the effective block extraction unit 6 sets the target threshold value at a specified reference distance (set as appropriate) as the reference target threshold value, and multiplies the reference target threshold value by 1 / square of the ratio of the distance to the block B with respect to the reference distance. It is desirable to use this value as the target threshold value. Note that the relationship between the distance and the target threshold value may be defined by other criteria.

  The effective blocks extracted by the effective block extracting unit 6 as described above are given to the region integrating unit 7, and in the region integrating unit 7, when the effective blocks are adjacent, the effective blocks belonging to the same object Integrate effective blocks. That is, a group of effective blocks is formed by integrating adjacent effective blocks. This process corresponds to a labeling process in a two-dimensional image.

  The area integration means 7 basically determines that both effective blocks are adjacent effective blocks when one of the cubes of unit volume is shared by the two effective blocks. However, even when one vertex of a cube of unit volume is shared by two effective blocks, they may be included in adjacent effective blocks.

  FIGS. 2F, 2G, and 2H show examples of images obtained by projecting the effective blocks by the region integration unit 7 in the z-axis direction, the x-axis direction, and the y-axis direction, respectively. In terms of the projection of the third trigonometric method, FIG. 2 (f) corresponds to a front view, FIG. 2 (g) corresponds to a right side view, and FIG. 2 (h) corresponds to a plan view.

  For the group of effective blocks integrated by the area integration unit 7, the object discrimination unit 8 determines whether the object is an object by applying knowledge about the object. It is simple and effective to use the size of the object as knowledge for determining whether or not it is an object. Further, as the knowledge about the object, it is also possible to use the shape of the group, the position of the center of gravity of the group, the direction of the inertial main axis of the group, and the like. However, since the structural unit of the group is block B, it is difficult to determine only the shape because information on the details of the shape is lost, and the position of the center of gravity and the direction of the inertial spindle can also be determined independently. It is difficult to use. If these pieces of information are used in combination with other knowledge, the reliability of the determination result can be improved.

  Hereinafter, an example will be described in which the object discriminating unit 8 determines whether or not the object is an object by using the group size as a feature. As shown in FIGS. 2F, 2G, and 2H, the size of the group is set to a circumscribed rectangle R for a figure in which the group is projected in one of the coordinate axes of the virtual space, and the sides of the circumscribed rectangle R are set. What is necessary is just to obtain | require the length of as a size.

  When projected in the z-axis direction as shown in FIG. 2 (f), the dimension of the lateral side of the circumscribed rectangle R is the width dimension W and the dimension of the vertical side is the height dimension H. When projected in the x-axis direction as shown in FIG. 2G, the dimension of the lateral side of the circumscribed rectangle R is the depth dimension D and the dimension of the vertical side is the height dimension H. When projected in the y-axis direction as shown in FIG. 2H, the dimension of the lateral side of the circumscribed rectangle R is the width dimension W, and the dimension of the vertical side is the depth dimension D.

  In this embodiment, it is judged whether it is a target object using all the width dimension W, the height dimension H, and the depth dimension D of a target object. Therefore, the object discriminating means 8 has the feature extracting means 81 for obtaining the width dimension W, the height dimension H, and the depth dimension D by projecting the group in the direction of all coordinate axes and setting the circumscribed rectangle R. Prepare. Further, the object discriminating means 8 sets appropriate ranges with respect to the width dimension W, height dimension H and depth dimension D of the group obtained by the feature extracting means 81, and when all are within the appropriate range, Size discriminating means 82 for discriminating as an object.

  The above operations are summarized as shown in FIG. That is, the background distance image P1 is stored in advance (S1), the distance image sensor 1 generates the distance image P2 (S2), and the difference image between the background distance image P1 and the distance image P2 is generated as the distance difference image P3. (S3). Furthermore, a pixel that is equal to or greater than the presence threshold in the distance difference image P3 is extracted as a target pixel from the distance image P2 (S4). Thereafter, coordinate conversion is performed for mapping each pixel of the distance image in the polar coordinate system to the three-dimensional orthogonal coordinate system for the target pixel (S5), and the density of each block B is evaluated to extract an effective block (S6). Further, adjacent effective blocks are integrated (S7), and the size is extracted by the feature extraction means 81 (S8). Thereafter, the size determining means 82 determines whether or not the size is in an appropriate range (S9). In the example illustrated in FIG. 4, a process for tracking an object is described as step S <b> 10, but this process will be described later.

  By the way, when using the size as the feature of the object in the object discriminating means 8, there is a possibility that the width dimension or the depth dimension deviates from the appropriate range when a plurality of people overlap and it is determined that the object is not the object. is there.

  For example, as shown in FIG. 5 or FIG. 6, when two persons M who are objects exist in the visual field region of the distance image sensor 1 and are lined up in the x-axis direction as shown in FIG. In the case where the groups are arranged in the z-axis direction as in FIG. 6, when a group of effective blocks is formed, it may be regarded as one group. In these cases, when three types of information of the width dimension W, the height dimension H, and the depth dimension D are used as the feature amount extracted from the group, any information deviates from the appropriate range. become. That is, the group is not discriminated as an object.

  In order to prevent such misjudgment, the object discriminating unit 8 performs clustering of effective blocks forming the group when the size extracted as the group feature by the feature extracting unit 81 exceeds the upper limit of the appropriate range. Clustering means 83 for performing is provided. Hereinafter, a clustering procedure by the clustering unit 83 will be described.

  Here, as in the example of FIG. 5, a case where the person M who is the object is aligned in the x-axis direction is illustrated. In this case, there is no gap between the effective blocks B0 to be separated as the groups G1 and G2 corresponding to each object, and the effective block B0 is a group of groups G that are continuous as shown in FIG. Therefore, when the size determination means 82 determines that the width dimension deviates from the upper limit of the appropriate range (actually, it is larger than the width dimension for one person and smaller than the width dimension for two persons). If it is determined that the set upper limit threshold is exceeded, the clustering means 83 is activated.

  The clustering unit 83 performs the clustering process described below assuming that the group G delivered from the size determining unit 82 can be divided into two.

  In the clustering process, as shown in FIG. 8A, first, attention is paid to appropriate two effective blocks B1 and B2 in the group G, and both effective blocks B1 and B2 and each effective block B0 in the group G are focused. Find the distance to each. It is desirable to set the effective blocks B1 and B2 to be focused on in the separated groups G1 and G2, respectively. Therefore, the effective blocks B1 and B2 to be focused are set so as to be relatively far apart in the direction (in this example, the x-axis direction) that deviates from the upper limit of the appropriate range in the size determination unit 82. That is, it is considered that the effective block B1 belongs to the group G1, and the effective block B2 belongs to the group G2.

  As shown in FIG. 8B, each effective block B0 of the group G is determined to belong to the same groups G1 and G2 as the effective blocks B1 and B2 having the shorter distances d1 and d2 among the effective blocks B1 and B2 of interest. To do. That is, if d1 <d2 for the effective block B0, it is determined that the effective block B0 belongs to the same group G1 as the effective block B1.

  When the distances between the effective blocks B1 and B2 of interest are obtained for all the effective blocks B0 of the group G, all the effective blocks B0 of the group G are divided into two groups G1 and G2. Determine the coordinate position of the center of gravity. Next, the block including the obtained coordinate position of the center of gravity is used as new effective blocks B1 and B2, and the above processing is repeated. As shown in FIG. 8C, when the positions of the effective blocks B1 and B2 including the coordinate positions of the centroids of the divided groups G1 and G2 do not change, it is determined that the centroid positions have converged, and the above-described processing ends. To do.

  The two groups G1 and G2 obtained in the above-described procedure are regarded as object candidates, and the object determination unit 8 re-applies the feature extraction unit 81 and the size determination unit 82 for each group G1 and G2, respectively. It is determined whether or not it is an object.

  FIG. 9 shows the processing of the object discrimination means 8 including the clustering means 83. In the illustrated example, first, the size is determined by the size determining means 82 (S11). Here, if the size is in an appropriate range (S12: N), group division is unnecessary. On the other hand, when the size exceeds the upper limit of the appropriate range (S12: Y), the effective blocks B1, B2 are set at appropriate positions as the temporary center positions of the groups G1, G2 (S13). The distance between the set effective blocks B1 and B2 and each effective block B0 in the group G is obtained (S14), and the groups G1 and G2 are grouped according to the distance between the effective blocks B1 and B2 as temporary centroid positions. Separation is performed (S15). After separation into groups G1 and G2, the centroid positions of the groups G1 and G2 are obtained (S16), and the above-described processing is repeated until the obtained centroid positions converge (S17).

  By the way, the distance image sensor 1 of the present embodiment projects modulated light whose intensity changes, and obtains the distance to the object based on the received light intensity at a plurality of phases of the modulated signal that generates the modulated light. Further, in order to reduce the error, the distance is obtained on the basis of the electric charge accumulated in the period over the multiple cycles of the modulated light. Therefore, when the object moves within the time until the charge necessary for obtaining the distance is obtained, the distance may not be obtained accurately. Since the time for accumulating the electric charge necessary for obtaining the distance is short, it is possible to measure the distance for many parts of the object. However, when the position of the object is not fixed, there is a possibility that the position of the periphery of the object fluctuates and the distance cannot be obtained accurately.

  Now, it is assumed that a pixel of interest is extracted by the pixel-of-interest extraction means 4 and an image P5 as shown in FIG. 2 (e) is obtained. Since the image P5 includes three-dimensional information, it can be viewed from the viewpoint in the y-axis direction as shown in FIG. 10 instead of the viewpoint in the z-axis direction as shown in FIG. According to FIG. 10, it can be seen that the distance value of the pixel of interest extracted at the periphery of the object has a distribution with a tail. As described above, this phenomenon is considered to be caused by fluctuations at the peripheral edge of the object. As described above, if a distance value that is an abnormal value is generated with respect to the distance value of the object, the density cannot be accurately measured when the density measuring unit 5 measures the density related to the block B. It may not be possible to accurately determine whether or not the object is an object.

  Therefore, in order not to employ a pixel having an abnormal value that occurs in the peripheral portion of the object as a target pixel, it is desirable to employ the following processing. The following processing is performed by the effective block extracting unit 6, but can also be performed by the density measuring unit 5.

  If the state in which the above-described abnormal value of the distance value is generated is simplified, as illustrated in FIG. 11, the distribution of the abnormal value on the peripheral portion of the object Ob in the direction in which the modulated light is projected from the distance image sensor 1. It can be said that the region A1 occurs. Therefore, the abnormal value distribution area A1 exists not only in the viewpoint of the y-axis direction as shown in FIG. 12A but also in the viewpoint of the x-axis direction as shown in FIG.

  From such knowledge, the frequency on the xz plane is obtained for the pixels included in the effective block B0 extracted as the block B having a density equal to or higher than the target threshold. As shown in FIG. 13, the frequency on the xz plane is large for the normal value, and the frequency on the xz plane is small for the abnormal value. Accordingly, an appropriate interval (for example, an interval of several cm squares) is set on the xz plane to obtain a frequency distribution, and coordinate values whose frequency is smaller than the prescribed noise threshold Th are not adopted as abnormal values. . In other words, only the pixel whose frequency on the xz plane is equal to or higher than the noise threshold Th is adopted as the target pixel corresponding to the object.

  By performing this processing, in the example shown in FIGS. 12A and 12B, the abnormal value distribution area A1 is removed as shown in FIG. 12C. In the above example, the distance image sensor 1 measures the distance to the object on the floor from an obliquely upper side, and thus the frequency on the xz plane is obtained, but the positional relationship between the distance image sensor 1 and the object. Depending on the case, the frequency on the xy plane may be obtained. That is, the frequency may be obtained on a coordinate plane that intersects the center line of the visual field region of the distance image sensor 1 in the coordinate plane of the virtual space.

  As described above, a pixel having a coordinate value whose frequency is smaller than the noise threshold Th is not adopted as a target pixel, so that a pixel (including an effective block B0) having a pixel value that is an abnormal value is determined from the object determination. Can be removed. As a result, the accuracy of discriminating the object increases.

  In the above-described operation, the timing at which the background acquisition unit 2 acquires the background distance image is not particularly mentioned. However, when the article arranged in the visual field region changes, it is also necessary to update the background distance image. In order to automate the update of the background distance image, when the state in which the object is not detected by the object discrimination means 8 reaches a predetermined update time, the new distance image obtained by the distance image sensor 1 is used as the background distance image. It is desirable to store in the background acquisition means 2.

  Since it is desirable to update the new background distance image at a time when it is estimated that the object does not exist, the new background distance image may be updated at a fixed time such as at night after the state in which the object is not detected reaches the update time. Alternatively, the distance image at the time when the update time is reached may be used as a new background distance image.

  Furthermore, in the above-described operation, since the movement of the object is not taken into consideration, the time interval at which the distance image sensor 1 generates the distance image can be made relatively large (than the time during which the object passes through the visual field region. Need to be short). For example, it may be used for the purpose of confirming an object by the distance image sensor 1 after detecting that an object has entered the visual field region by another sensor.

  On the other hand, when an object passing through the visual field region is to be detected, it is necessary to generate a moving image of the distance image by the distance image sensor 1. That is, it is necessary to generate a distance image at a relatively short time interval (for example, one tenth of a second or less), and determine the presence / absence of the object in the object determination means 8 from each distance image. In this case, after the object discrimination means 8 detects the object, the group determined as the object by the object discrimination means 8 is delivered to the object tracking means 9. The object tracking means 9 tracks the position of the representative point of the group as the object while the object is detected in the visual field region of the distance image sensor 1. That is, the position of an object is tracked using distance images adjacent in time series. As the representative point of the group, the center of gravity position or the upper end position may be used.

  Since the position of the moving object can be tracked by providing the object tracking means 9, it can be used for the purpose of tracking the behavior of the intruder with the intruder into the field of view as the object. It is.

  In the configuration example described above, the target pixel extraction unit 4 extracts the target pixel from the difference image between the background distance image, which is a polar coordinate system distance image, and the polar coordinate system distance image generated by the distance image sensor 1. Further, the extracted target pixel is mapped in the three-dimensional virtual space by the density measuring means 5. On the other hand, when the object is a person, it is only necessary to determine the presence or absence of the object in the space area above the floor surface, and information about the floor surface is not necessary.

  In view of this, in the pixel-of-interest extraction means 4, as shown in FIG. 14, a threshold surface PL1 is defined at a predetermined height position (above the floor surface) from a reference surface PL0 such as a floor surface, and is more than the threshold surface PL1. It is desirable to extract only the pixel existing above as a target pixel that is a candidate for the object Ob. In this case, the pixel of interest is extracted after mapping the polar coordinate system distance image generated by the distance image sensor 1 to the virtual space of the orthogonal coordinate system.

  In addition, after mapping the polar coordinate system distance image generated by the distance image sensor 1 to the virtual space of the orthogonal coordinate system, the pixel existing above the threshold plane set at a predetermined height position from the reference plane is set as the target pixel. You may employ | adopt the structure to extract. In this case, it is possible to extract a target pixel that is a candidate for an object without using a background distance image. Further, since the extracted target pixel is mapped in the virtual space, the density is evaluated as it is. Is possible. However, if this procedure is adopted, an object placed on the floor is also extracted as a candidate for the target, and it is necessary to separately remove information other than the target.

DESCRIPTION OF SYMBOLS 1 Distance image sensor 2 Background acquisition means 3 Difference image generation means 4 Interesting pixel extraction means 5 Density measurement means 6 Effective block extraction means 7 Area integration means 8 Object discrimination means 81 Feature extraction means 82 Size discrimination means 83 Clustering means 9 Object Tracking means B block B0 effective block B1, B2 effective block G group G1, G2 group Ob object P1 background distance image P2 distance image P3 distance difference image P4 binary image P5 image

Claims (10)

  An active distance image sensor that generates a distance image whose pixel value is a distance for the visual field region, and a background acquisition that stores the distance image generated by the distance image sensor as a background distance image when no object is present in the visual field region A difference image generation means for generating a distance difference image that is a difference image between a distance image generated by the distance image sensor and a background distance image stored in the background acquisition means in a state of monitoring a visual field region; Pixel-of-interest extracting means for extracting, as pixels of interest, a pixel whose pixel value in the distance difference image generated by the difference image generating means is greater than or equal to a predetermined existence threshold among the pixels included in the distance image generated by the distance image sensor; , Mapping the target pixel extracted by the target pixel extraction means into a three-dimensional virtual space Density measuring means for obtaining the number of pixels of interest as a density for each block of the unit volume defined in the above, and effective block extracting means for extracting, as an effective block, a block whose density obtained by the density measuring means is equal to or greater than a prescribed target threshold value; By using the area integration unit that forms a group by integrating adjacent effective blocks with respect to the effective block extracted by the effective block extraction unit, and by using knowledge about the object for the group formed by the area integration unit An object detection apparatus comprising: an object determination unit that determines whether the object is an object.   The target detection apparatus according to claim 1, wherein the effective block extraction unit sets the target threshold value smaller as the distance to the block increases.   The effective block extraction unit uses a target threshold value at a specified reference distance as a reference target threshold value, and uses a value obtained by multiplying the reference target threshold by one-quarter of the ratio of the distance to the block with respect to the reference distance as the target threshold value. The object detection apparatus according to claim 2, wherein:   The object discriminating unit uses the feature extraction unit that extracts a size as a feature from the group formed by the region integration unit, and the knowledge that the size extracted by the feature amount extraction unit is within a prescribed appropriate range. The object detection apparatus according to claim 1, further comprising a size determination unit that determines whether or not the group is an object.   When the size extracted as a group feature by the feature extraction unit exceeds the upper limit of the appropriate range, the target object discrimination unit performs clustering of the effective blocks that form the group, thereby grouping the group into a plurality of groups. The object detection apparatus according to claim 4, further comprising: a clustering unit that divides the data into groups, and wherein the size determination unit determines the group after the division by the clustering unit.   The effective block extracting means includes a frequency of pixels included in the effective block on a coordinate plane intersecting a center line of a visual field area of the distance image sensor among coordinate planes of an orthogonal coordinate system defining the virtual space. 6. The object detection device according to claim 1, wherein a pixel having a coordinate value smaller than a noise threshold value obtained by obtaining a distribution is not adopted as a target pixel.   The center line in the visual field region of the distance image sensor is set obliquely downward, and the density measuring unit, when mapping the pixel of interest in the virtual space, has a depression angle of the central line in the visual field region of the distance image sensor. The object detection apparatus according to any one of claims 1 to 6, wherein correction is performed in accordance with the object.   8. The pixel of interest according to claim 1, wherein the pixel-of-interest extracting means extracts a pixel located above a threshold plane defined as a predetermined height position within the visual field region of the distance image sensor as a pixel of interest. The target object detection apparatus of any one of Claims.   9. The background acquisition unit updates a newly generated background distance image when a state in which the target is not detected by the target determination unit reaches a predetermined update time. The object detection apparatus of Claim 1.   The distance image sensor has a function of generating a moving image based on a distance image, and when the object determination unit detects the object, the object in the visual field region is used by using adjacent distance images in time series. The object detection device according to claim 1, further comprising an object tracking unit that tracks the object.