JP2014025804A - Information acquisition device and object detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information acquisition device capable of appropriately acquiring distance information, and an object detection device.SOLUTION: The information acquisition device searches for a displacement position on an actual measurement dot pattern, of a segment area on the basis of first determination indexes S103 and S105 and acquires distance information for the segment area on the basis of the searched displacement position (S104), wherein the first determination index has a threshold Tβ1 and a threshold Tβ2 larger than the threshold Tβ1 in order to evaluate the probability of the displacement position. When a parameter value (Av/Bt1) relating to the displacement position is between the threshold Tβ1 and the threshold Tβ2, the information acquisition device determines whether acquisition of the distance information based on the displacement position is suitable or not on the basis of the second determination index S107.

Description

  The present invention relates to an object detection apparatus that detects an object in a target area based on the state of reflected light when light is projected onto the target area, and an information acquisition apparatus suitable for use in the object detection apparatus.

  Conventionally, object detection devices using light have been developed in various fields. An object detection apparatus using a so-called distance image sensor can detect not only a planar image on a two-dimensional plane but also the shape and movement of the detection target object in the depth direction. In such an object detection apparatus, light in a predetermined wavelength band is projected from a laser light source or an LED (Light Emitting Diode) onto a target area, and the reflected light is received by a light receiving element such as a CMOS image sensor. Various types of distance image sensors are known.

  In a distance image sensor of a type that irradiates a target region with laser light having a predetermined dot pattern, reflected light from the target region of laser light having a dot pattern is received by a light receiving element. Then, based on the light receiving position of the dot on the light receiving element, the distance to each part of the detection target object (irradiation position of each dot on the detection target object) is detected using triangulation (for example, Patent Literature 1, Non-Patent Document 1).

JP 2012-32379 A

19th Annual Conference of the Robotics Society of Japan (September 18-20, 2001) Proceedings, P1279-1280

  In the object detection apparatus, a reference dot pattern is stored in a memory in advance, and this reference dot pattern is collated with an actually measured dot pattern captured during actual operation. Specifically, the position of the dot in the reference dot pattern on the measured dot pattern is searched. In this search, the degree of matching between the dots in the reference dot pattern and the dots in the measured dot pattern is obtained. At this time, when the matching degree is lower than a predetermined threshold, error processing is performed, and when the matching degree is equal to or greater than the threshold, dot position detection is performed.

  However, such an error determination is normally performed based on one threshold value, and therefore there is a possibility that the error determination is uncertain when the matching degree is in the vicinity of the threshold value. For this reason, inaccurate information is mixed in the acquired distance information, such as when an incorrect distance is acquired for a dot that should be regarded as an error, or a dot whose distance should be acquired is determined as an error. Can happen.

  The present invention has been made in view of this point, and an object thereof is to provide an information acquisition device and an object detection device that can appropriately acquire distance information.

A first aspect of the present invention relates to an information acquisition device. The information acquisition apparatus according to this aspect is configured to project a laser beam emitted from a laser light source onto a target area with a predetermined dot pattern, and to be arranged laterally apart from the projection optical system by a predetermined distance. A light receiving optical system that images the target area with an image sensor, a reference dot pattern that is imaged by the light receiving optical system when the laser beam is irradiated onto a reference surface, and an image that is captured by the image sensor during measurement. A distance acquisition unit that acquires distance information related to a distance to an object included in the target area based on the measured dot pattern. The distance acquisition unit searches for a displacement position on the measured dot pattern of a predetermined segment area set in the reference dot pattern based on a first determination index, and based on the searched displacement position, Obtain distance information for the segment region. The first determination index includes a first threshold value and a second threshold value that is larger than the first threshold value in order to evaluate the likelihood of the displacement position. The distance acquisition unit determines whether or not to acquire the distance information based on the displacement position when the first parameter value related to the displacement position is between the first threshold value and the second threshold value. The determination is made based on the second determination index.

  A 2nd aspect of this invention is related with an object detection apparatus. The object detection device according to this aspect includes the information acquisition device according to the first aspect and an object detection unit that detects a predetermined target object based on the distance information.

  ADVANTAGE OF THE INVENTION According to this invention, the information acquisition apparatus and object detection apparatus which can acquire distance information appropriately can be provided.

  The effects and significance of the present invention will become more apparent from the following description of embodiments. However, the embodiment described below is merely an example when the present invention is implemented, and the present invention is not limited to the following embodiment.

It is a figure which shows the structure of the object detection apparatus which concerns on embodiment. It is a figure which shows the structure of the information acquisition apparatus and information processing apparatus which concern on embodiment. It is a figure which shows the irradiation state of the laser beam with respect to the target area | region which concerns on embodiment, and the light reception state of the laser beam on an image sensor. It is a figure explaining the production | generation method of the reference pattern which concerns on embodiment. It is a figure explaining the distance detection method which concerns on embodiment. It is a figure explaining the error determination parameter | index which concerns on a comparative example. It is a figure which shows the verification result which performed distance detection using the error determination parameter | index which concerns on a comparative example. It is a figure explaining the error determination parameter | index which concerns on embodiment. 6 is a flowchart according to an error determination process according to the first embodiment. It is a figure which shows the classification | category of the error determination result which concerns on Example 1. FIG. 10 is a flowchart according to an error determination process according to the second embodiment. It is a figure explaining the tendency of the ambiguous value which concerns on Example 2, and the surrounding error determination result. 12 is a flowchart according to an ambiguous value correction process according to the second embodiment. 12 is a flowchart according to an ambiguous value correction process according to the second embodiment. It is a figure which shows the example of the ambiguous value correction process which concerns on Example 2. FIG. It is a figure which shows the verification result which performed distance detection using the ambiguous value correction process which concerns on Example 2. FIG. It is a figure which shows the verification result which performed distance detection using the process of the noise removal filter which concerns on the example of a change. It is a flowchart concerning the ambiguous value correction process which concerns on the example of a change. It is a figure which shows the ambiguous value correction process and error determination parameter | index which concern on the example of a change.

  Embodiments of the present invention will be described below with reference to the drawings. In this embodiment, the present invention is applied to an information acquisition apparatus of a type that irradiates a target area with laser light having a predetermined dot pattern.

  First, FIG. 1 shows a schematic configuration of an object detection apparatus 1 according to the present embodiment. As shown in the figure, the object detection device 1 includes an information acquisition device 2 and an information processing device 3. The television 4 is controlled by a signal from the information processing device 3.

  The information acquisition device 2 projects infrared light over the entire target area, and receives the reflected light with a CMOS image sensor, whereby the distance between each part of the object in the target area (hereinafter referred to as “three-dimensional distance information”). To get. The acquired three-dimensional distance information is sent to the information processing apparatus 3 via the cable 5.

  The information processing device 3 is, for example, a television control controller, a game machine, a personal computer, or the like. The information processing device 3 detects an object in the target area based on the three-dimensional distance information received from the information acquisition device 2, and controls the television 4 based on the detection result.

  For example, the information processing device 3 detects a person based on the received three-dimensional distance information and detects the movement of the person from the change in the three-dimensional distance information. For example, when the information processing device 3 is a television control controller, the information processing device 3 detects the person's gesture from the received three-dimensional distance information and outputs a control signal to the television 4 in accordance with the gesture. The application program to be installed is installed.

  For example, when the information processing device 3 is a game machine, the information processing device 3 detects the person's movement from the received three-dimensional distance information, and displays a character on the television screen according to the detected movement. An application program that operates and changes the game battle situation is installed.

  FIG. 2 is a diagram illustrating the configuration of the information acquisition device 2 and the information processing device 3.

  The information acquisition apparatus 2 includes a projection optical system 100 and a light receiving optical system 200 as a configuration of the optical unit. The projection optical system 100 and the light receiving optical system 200 are arranged in the information acquisition device 2 so as to be aligned in the X-axis direction.

  The projection optical system 100 includes a laser light source 110, a collimator lens 120, a mirror 130, and a diffractive optical element (DOE) 140. The light receiving optical system 200 includes an aperture 210, an imaging lens 220, a filter 230, and a CMOS image sensor 240. In addition, the information acquisition device 2 includes a CPU (Central Processing Unit) 21, a laser drive circuit 22, an imaging signal processing circuit 23, an input / output circuit 24, a memory 25, and a buffer memory 26 as circuit units. It has.

  The laser light source 110 outputs laser light in a narrow wavelength band having a wavelength of about 830 nm in a direction away from the light receiving optical system 200 (X-axis negative direction). The collimator lens 120 converts the laser light emitted from the laser light source 110 into substantially parallel light.

  The mirror 130 reflects the laser light incident from the collimator lens 120 side in the direction toward the DOE 140 (Z-axis direction).

  The DOE 140 has a diffraction pattern on the incident surface. Due to the diffractive action of this diffraction pattern, the laser light incident on the DOE 140 is converted into laser light having a predetermined dot pattern and irradiated onto the target area.

  The diffraction pattern of the DOE 140 has, for example, a structure in which a step type diffraction hologram is formed in a predetermined pattern. The diffraction hologram is adjusted in pattern and pitch so as to convert the laser light that has been made substantially parallel light by the collimator lens 120 into a laser light having a dot pattern. The DOE 140 irradiates the target region with the laser beam incident from the mirror 130 as a laser beam having a dot pattern that spreads radially.

  The laser light reflected from the target area enters the imaging lens 220 via the aperture 210.

  The aperture 210 stops the light from the outside so as to match the F number of the imaging lens 220. The imaging lens 220 collects the light incident through the aperture 210 on the CMOS image sensor 240. The filter 230 is a band-pass filter that transmits light in the infrared wavelength band including the emission wavelength (about 830 nm) of the laser light source 110 and cuts the wavelength band of visible light.

  The CMOS image sensor 240 receives the light collected by the imaging lens 220 and outputs a signal (charge) corresponding to the amount of received light to the imaging signal processing circuit 23 for each pixel. Here, in the CMOS image sensor 240, the output speed of the signal is increased so that the signal (charge) of the pixel can be output to the imaging signal processing circuit 23 with high response from the light reception in each pixel.

  The CPU 21 controls each unit according to a control program stored in the memory 25. With this control program, the CPU 21 is provided with the functions of a laser control unit 21a for controlling the laser light source 110 and a distance acquisition unit 21b for generating three-dimensional distance information.

  The laser drive circuit 22 drives the laser light source 110 according to a control signal from the CPU 21.

  The imaging signal processing circuit 23 controls the CMOS image sensor 240 to sequentially take in the signal (charge) of each pixel generated by the CMOS image sensor 240 for each line at a predetermined imaging interval. Then, the captured signals are sequentially output to the CPU 21. Based on the signal (imaging signal) supplied from the imaging signal processing circuit 23, the CPU 21 calculates the distance from the information acquisition device 2 to each part of the detection target by processing by the distance acquisition unit 21b. The distance information calculated by the distance acquisition unit 21 b is temporarily stored in the buffer memory 26. The buffer memory 26 is a line memory that can hold distance information for one line. The input / output circuit 24 reads the distance information stored in the buffer memory 26 and controls data communication with the information processing device 3.

  The information processing apparatus 3 includes a CPU 31, an input / output circuit 32, and a memory 33. In addition to the configuration shown in the figure, the information processing apparatus 3 is configured to communicate with the television 4 and to read information stored in an external memory such as a CD-ROM and install it in the memory 33. However, the configuration of these peripheral circuits is not shown for the sake of convenience.

The CPU 31 controls each unit according to a control program (application program) stored in the memory 33. With this control program, the CPU 31 is provided with the functions of an object detection unit 31a for detecting an object in the image and a function control unit 31b for controlling the function of the television 4 according to the movement of the object. . Such a control program is read from a CD-ROM by a drive device (not shown) and installed in the memory 33, for example.

  The object detection unit 31a extracts the shape of the object in the image from the three-dimensional distance information supplied from the information acquisition device 2, and detects the movement of the extracted object shape. Here, the object detection unit 31a generates a binarized image in order to extract only the planar shape of the object from the three-dimensional distance information supplied from the information acquisition device 2 based on the threshold regarding the distance. Thereafter, the generated binarized image is compared with the object shape extraction template for extracting the object shape stored in the memory 33, and the object shape to be extracted from the motion and the object shape in the three-dimensional distance information are compared. Specify the center position of. In this way, the object detection unit 31a detects the movement of the object by tracking the distance information of the center position of the specified object shape every predetermined time.

  For example, when the control program is a game program, the function control unit 31b executes a process for operating a character on the television screen in accordance with a human movement (gesture) detected by the object detection unit 31a. When the control program is a program for controlling the function of the television 4, the function control unit 31 b performs the function (channel switching) of the television 4 based on a signal from the object detection unit 31 a according to a person's movement (gesture). And volume adjustment, etc.) are executed.

  The input / output circuit 32 controls data communication with the information acquisition device 2.

  The projection optical system 100 and the light receiving optical system 200 are installed side by side with a predetermined distance in the X axis direction so that the projection center of the projection optical system 100 and the imaging center of the light receiving optical system 200 are aligned on a straight line parallel to the X axis. Is done. The installation interval between the projection optical system 100 and the light receiving optical system 200 is set according to the distance between the information acquisition device 2 and the reference plane of the target area.

  Next, a method for acquiring three-dimensional distance information by the information acquisition device 2 will be described.

  FIG. 3A is a diagram schematically showing the irradiation state of the laser light on the target region, and FIG. 3B is a diagram schematically showing the light receiving state of the laser light in the CMOS image sensor 240. For convenience, FIG. 3B shows a flat surface (screen) in the target area and a light receiving state when a person is present in front of the screen.

  As shown in FIG. 3A, the projection optical system 100 irradiates a target region with laser light having a dot pattern (hereinafter, the entire laser light having this pattern is referred to as “DP light”). . In FIG. 3A, the light flux region of DP light is indicated by a solid line frame. In the light flux of DP light, dot regions (hereinafter simply referred to as “dots”) generated by the diffraction action by the DOE 140 are scattered according to the dot pattern by the diffraction action by the DOE 140.

  When a flat surface (screen) exists in the target area, DP light reflected thereby is distributed on the CMOS image sensor 240 as shown in FIG.

In FIG. 3B, the entire DP light receiving area on the CMOS image sensor 240 is indicated by a broken line frame, and the DP light receiving area incident on the imaging effective area of the CMOS image sensor 240 is indicated by a solid line frame. Has been. The effective imaging area of the CMOS image sensor 240 is an area where the CMOS image sensor 240 receives a DP light and outputs a signal as a sensor, and has a size of, for example, VGA (horizontal 640 pixels × vertical 480 pixels). . Further, the light of Dt0 on the target area shown in FIG. 9A enters the position of Dt0 ′ shown in FIG. An image of a person in front of the screen is taken upside down on the CMOS image sensor 240 in the vertical and horizontal directions.

  Here, the distance detection method will be described with reference to FIGS.

  FIG. 4 is a diagram for explaining a reference pattern setting method used in the distance detection method.

  As shown in FIG. 4A, a flat reflection plane RS perpendicular to the Z-axis direction is disposed at a position at a predetermined distance Ls from the projection optical system 100. The emitted DP light is reflected by the reflection plane RS and enters the CMOS image sensor 240 of the light receiving optical system 200. Thereby, an electrical signal for each pixel in the effective imaging area is output from the CMOS image sensor 240. The output electric signal value (pixel value) for each pixel is developed on the memory 25 of FIG.

  Hereinafter, an image including all pixel values obtained by reflection from the reflection surface RS is referred to as a “reference image”, and the reflection surface RS is referred to as a “reference surface”. Then, as shown in FIG. 4B, a “reference pattern region” is set on the standard image. FIG. 4B shows a state in which the light receiving surface is seen through in the positive direction of the Z axis from the back side of the CMOS image sensor 240. The same applies to the drawings after FIG.

  A plurality of segment areas having a predetermined size are set for the reference pattern area set in this way. The size of the segment area is determined in consideration of the contour extraction accuracy of the object based on the obtained distance information, the load of the calculation amount of distance detection for the CPU 21, and the error occurrence rate by the distance detection method described later.

  With reference to FIG.4 (c), the segment area | region set to a reference pattern area | region is demonstrated. In FIG. 4C, for the sake of convenience, the size of each segment area is indicated by 9 pixels wide × 9 pixels high, and the center pixel of each segment area is indicated by a cross.

  As shown in FIG. 4C, the segment areas are set such that adjacent segment areas are arranged at intervals of one pixel in the X-axis direction and the Y-axis direction with respect to the reference pattern area. That is, a certain segment area is set at a position shifted by one pixel with respect to a segment area adjacent to the segment area in the X-axis direction and the Y-axis direction. At this time, each segment area is dotted with dots in a unique pattern. Therefore, the pattern of pixel values in the segment area is different for each segment area. The narrower the interval between adjacent segment areas, the greater the number of segment areas included in the reference pattern area, and the resolution of distance detection in the in-plane direction (XY plane direction) of the target area is enhanced.

  Thus, information on the position of the reference pattern area on the CMOS image sensor 240, pixel values (reference patterns) of all pixels included in the reference pattern area, and segment area information set for the reference pattern area are shown in FIG. 2 memory 25. These pieces of information stored in the memory 25 are hereinafter referred to as “reference templates”.

  The CPU 21 in FIG. 2 refers to the reference template when calculating the distance from the projection optical system 100 to each part of the detection target object. When calculating the distance, the CPU 21 calculates the distance to each part of the object based on the shift amount of the dot pattern in each segment area obtained from the reference template.

For example, as shown in FIG. 4A, when an object is present at a position closer than the distance Ls, DP light (DPn) corresponding to a predetermined segment area Sn on the reference pattern is reflected by the object, and the segment area Sn. It is incident on a different region Sn ′. Since the projection optical system 100 and the light receiving optical system 200 are adjacent to each other in the X-axis direction, the displacement direction of the region Sn ′ with respect to the segment region Sn is parallel to the X-axis. In the case of FIG. 4A, since the object is at a position closer than the distance Ls, the region Sn ′ is displaced in the positive direction of the X axis with respect to the segment region Sn. If the object is at a position farther than the distance Ls, the region Sn ′ is displaced in the negative X-axis direction with respect to the segment region Sn.

  Based on the displacement direction and displacement amount of the region Sn ′ with respect to the segment region Sn, the distance Lr from the projection optical system 100 to the part of the object irradiated with DP light (DPn) is triangulated using the distance Ls. Calculated based on Similarly, the distance from the projection optical system 100 is calculated for the part of the object corresponding to another segment area. The details of this calculation method are described in, for example, Non-Patent Document 1 (The 19th Annual Conference of the Robotics Society of Japan (September 18-20, 2001) Proceedings, P1279-1280).

  In this distance calculation, it is detected to which position the segment area Sn of the reference template has been displaced during actual measurement. This detection is performed by collating the dot pattern obtained from the DP light irradiated onto the CMOS image sensor 240 at the time of actual measurement with the dot pattern included in the segment region Sn. Hereinafter, an image made up of all the pixel values obtained from the DP light irradiated to the imaging effective area on the CMOS image sensor 240 at the time of actual measurement will be referred to as “measured image”. The effective imaging area of the CMOS image sensor 240 at the time of actual measurement is, for example, the size of VGA (horizontal 640 pixels × vertical 480 pixels), as in the case of acquiring the reference image.

  FIGS. 5A to 5E are diagrams for explaining such a distance detection method. FIG. 5A is a diagram showing a reference pattern area set in a standard image on the CMOS image sensor 240, and FIG. 5B is a diagram showing an actually measured image on the CMOS image sensor 240 at the time of actual measurement. FIG. 5C to FIG. 5E are diagrams for explaining a collation method between the DP light dot pattern included in the actual measurement image and the dot pattern included in the segment area of the reference template. For convenience, FIGS. 5 (a) and 5 (b) show only a part of the segment areas, and FIGS. 5 (c) to 5 (e) show the size of each segment area. It is shown by pixel × 9 pixels vertically. 5B, for the sake of convenience, as shown in FIG. 4B, there is a person in front of the reference plane as a detection target object, and an image of the person is reflected. It is shown.

  When searching for the displacement position at the time of actual measurement of the segment area Si in FIG. 5A, as shown in FIG. 5B, the search area Ri is set for the segment area Si on the actual measurement image. The search area Ri has a predetermined width in the X-axis direction. The segment area Si is sent one pixel at a time in the search area Ri in the X-axis direction, and the dot pattern of the segment area Si is compared with the dot pattern on the measured image at each feed position. Hereinafter, a region corresponding to each feed position on the actually measured image is referred to as a “comparison region”. A plurality of comparison areas having the same size as the segment area Si are set in the search area Ri, and the comparison areas adjacent in the X-axis direction are shifted by one pixel from each other.

  In FIG. 5D, the pixel position (center pixel position) on the measured image corresponding to the pixel position of the segment region Si on the reference image is shifted from the position shifted by x pixels in the X axis negative direction to x in the X axis positive direction. The search area Ri is set so that the segment area Si is sent in a pixel shifted range (hereinafter referred to as “search range L0”).

At the time of distance detection, the segment area Si is fed one pixel at a time in the X-axis direction in the search area Ri, and at each feed position, the dot pattern of the segment area Si stored in the reference template and the dot pattern of the DP light of the measured image The degree of matching is required. As described above, the segment area Si is sent only in the X-axis direction in the search area Ri as described above. Normally, the dot pattern of the segment area set by the reference template is a predetermined value in the X-axis direction at the time of actual measurement. This is because the displacement occurs only within the range.

  At the time of actual measurement, depending on the position of the detection target object, the dot pattern corresponding to the segment area may protrude from the actual measurement image in the X-axis direction. In this case, since the dot pattern corresponding to the segment area is not within the effective imaging area of the CMOS image sensor 240, the segment area cannot be properly matched. However, since it is possible to perform matching appropriately in areas other than the end segment areas, there is little influence on object distance detection.

  In addition, when matching is performed appropriately for the end region, the effective imaging region of the CMOS image sensor 240 at the time of actual measurement may be made larger than the effective imaging region of the CMOS image sensor 240 at the time of acquiring the reference image. What can be used should be used. As a result, the actually measured image becomes larger than the reference image, but matching can be appropriately performed for the end segment area.

  When detecting the matching degree, first, the number of gradations of the pixel value of each pixel in the reference pattern area and the pixel value of each pixel in each segment area of the actually measured image is reduced. For example, when the pixel values of the reference image and the actually measured image are 8-bit gradation, the pixel values of 0 to 255 are converted into the pixel values of 0 to 12. Then, the converted pixel value is held in the memory 25. Thereafter, the similarity between the comparison region and the segment region Si is obtained. That is, the difference between the pixel value of each pixel in the segment area Si and the pixel value of the corresponding pixel in the comparison area is obtained. A value Rsad obtained by adding the obtained difference to all the pixels in the comparison region is acquired as a value indicating the similarity.

  For example, as shown in FIG. 5C, when pixels in n columns × m rows are included in one segment area, the pixel values T (i, j) of the pixels in i columns and j rows in the segment area. ) And the pixel value I (i, j) of the pixel in the comparison area i column and j row. Then, the difference is obtained for all the pixels in the segment area, and the value Rsad of the equation shown in FIG. 5C is obtained from the sum of the differences. The smaller the value Rsad, the higher the degree of similarity between the segment area and the comparison area.

  Thus, as shown in FIG. 5D, the value Rsad is obtained for all the comparison regions of the search region Ri for the segment region Si. FIG. 5E is a graph schematically showing the value Rsad at each feed position in the search area Ri.

  When the value Rsad is obtained for all the comparison regions of the search region Ri for the segment region Si, the minimum value Bt1 of the value Rsad is referred to, and the position of the comparison region Ci corresponding to the minimum value Bt1 is the position of the segment region Si. The displacement position is determined. However, at the time of such determination, a predetermined error determination index is applied to determine whether the displacement position is appropriate. If this error determination index is not cleared, the search for the displacement position is an error.

  Thus, when the displacement position of each segment region is searched from the dot pattern of DP light acquired at the time of actual measurement, detection corresponding to each segment region is performed by triangulation based on the displacement position as described above. The distance to the part of the target object is obtained.

  FIG. 6A to FIG. 6C are diagrams showing comparative examples of error determination indexes for determining the suitability of the displacement position. The graphs of FIGS. 6A to 6C are graphs showing examples of the distribution of the value Rsad. 6A to 6C show different error determination indexes.

  In the graph of FIG. 6A, the value Rsad falls significantly at the position of the pixel shift amount 10. In the graph of FIG. 6B, the value Rsad is the smallest at the position of the pixel deviation amount 10, and the value Rsad is somewhat reduced even at the position of the pixel deviation amount −5. Further, in the graph of FIG. 6C, the average value Av of the value Rsad is small.

  In the error determination index of FIG. 6A, the suitability of the displacement position search is determined based on the magnitude of the minimum value Bt1. Specifically, when the minimum value Bt1 is equal to or greater than the threshold value Ta, the search for the segment area is regarded as an error. On the other hand, when the minimum value Bt1 is less than the threshold value Ta, the position of the comparison area corresponding to the minimum value Bt1 is determined as the displacement position of the segment area.

  In the error determination index of FIG. 6B, the suitability of the search for the displacement position is determined by the ratio of the second smallest value Bt2 to the minimum value Bt1 among the values Rsad. Specifically, when the ratio of Bt2 to Bt1 (Bt2 / Bt1) is less than a predetermined threshold value Tα, the search for the segment area is regarded as an error. On the other hand, when the ratio of Bt2 and Bt1 is equal to or greater than a predetermined threshold value Tα, the position of the comparison area corresponding to the minimum value Bt1 is determined as the displacement position of the segment area.

  In the error determination index of FIG. 6C, the suitability of the search for the displacement position is determined by the ratio between the average value Av of the value Rsad and the minimum value Bt1. Specifically, when the ratio (Av / Bt1) between the average value Av and the minimum value Bt1 is less than a predetermined threshold value Tβ, the search for the segment area is regarded as an error. On the other hand, when the ratio between the average value Av and the minimum value Bt1 is equal to or greater than a predetermined threshold value Tβ, the position of the comparison area corresponding to the minimum value Bt1 is determined as the displacement position of the segment area.

  With the error determination index of FIG. 6A, when the dot pattern included in the segment area is not included in the search range, the search for the displacement position is appropriately determined as an error. That is, in such a case, since the value Rsad increases as a whole, the minimum value Bt1 is unlikely to be less than the threshold value Ta, and therefore, the search for the displacement position is likely to be determined as an error. For example, in the example of FIG. 6A, since the minimum value Bt1 is less than the threshold value Ta, it is determined that the position of the pixel shift amount 10 corresponding to the minimum value Bt1 is appropriate as the displacement position.

  In the error determination index of FIG. 6A, the suitability of the displacement position is determined only by the magnitude of the minimum value Bt1, and therefore, for example, as shown in the graph of FIG. 6B, it is close to the minimum value Bt1. Even when the value Bt2 exists, it is determined that the position corresponding to the minimum value Bt1 is appropriate as the displacement position. However, in the case of FIG. 6B, there is a possibility that the position corresponding to the value Bt2 also becomes the displacement position. Therefore, in this case, if the position corresponding to the minimum value Bt1 is determined as the displacement position, As a result, the determination result may be erroneously determined.

  On the other hand, in the error determination index of FIG. 6B, when the ratio (Bt2 / Bt1) between the second smallest value Bt2 and the minimum value Bt1 is less than a predetermined threshold value Tα, the search for the segment area is performed. Since it is considered as an error, such erroneous determination can be suppressed.

  Further, in the error determination index of FIG. 6A, when the value Rsad is small as a whole, even when the difference between the minimum value Bt1 and other values is small, the minimum value Bt1 is less than the threshold value. For example, it is determined that the position corresponding to the minimum value Bt1 is appropriate as the displacement position. However, in such a case, since the difference between the minimum value Bt1 and other values is small, it is considered that the probability that the position corresponding to the minimum value Bt1 is appropriate as the displacement position is low. For this reason, in the error determination index of FIG. 6A, in such a case, the determination result is likely to be erroneously determined.

On the other hand, in the error determination index of FIG. 6C, since an error is determined when the difference between the minimum value Bt1 and another value is small, such erroneous determination can be suppressed. In this error determination index, as shown in FIG. 6C, if the average value Av is large to some extent and the difference between the average value Av and the minimum value Bt1 is large, the position corresponding to the minimum value Bt1 is appropriate as the displacement position. Determined.

  As described above, various error determination indexes for determining the suitability of the displacement position can be used. Moreover, the result obtained by each parameter | index differs. In the present embodiment, the suitability of the displacement position is determined by further using an index different from the above three error determination indices. Such an error determination index will be described later with reference to FIG.

  The distance information corresponding to each segment area acquired in this way is output to the buffer memory 26 in FIG. Then, the distance information stored in the buffer memory 26 is output to the information processing apparatus 3 by the line buffer method for each line. In this way, for all the segment areas from the segment area S1 to the segment area Sn, the same segment area search is performed, and the distance information is acquired.

  The information acquisition device 2 outputs to the buffer memory 26 as an image in which the distance to the detection target object corresponding to each segment area acquired as described above is assigned to pixel values expressed in white to black gradations. To do. Hereinafter, an image in which a distance corresponding to each segment area is expressed by a pixel value is referred to as a “distance image”.

  As described above, in the present embodiment, since the segment area is set at an interval of one pixel with respect to the reference pattern area of the standard image, the distance image has a resolution (substantially approximately the same VGA size as the standard image). 640 × approximately 480). For example, the gradation of the pixel of the distance image is an 8-bit gradation, and a gradation closer to black (pixel value 0) is assigned to the short distance and a gradation closer to white (pixel value 255) is assigned to the far distance. If the segment area search results in an error, a gray scale of black (pixel value 0) is assigned.

  FIG. 7A to FIG. 7D are diagrams illustrating verification results obtained by performing distance detection using the error determination index of the comparative example illustrated in FIG. FIG. 7A and FIG. 7B are diagrams showing actual measurement images used for verification, and FIG. 7C and FIG. 7D are diagrams showing distance images as verification results. 7C is a distance image acquired from the actual measurement image of FIG. 7A, and FIG. 7D is a distance image acquired from the actual measurement image of FIG. 7B. In this verification, the size of the segment area is 15 pixels × 15 pixels, and the search range is −30 to 30 pixels.

  In the actually measured images of FIGS. 7A and 7B, the person and the desk are positioned in front of the reference plane, and the dots irradiated to the person and the desk are imaged. Further, in FIG. 7B, the target area is imaged with the hand protruding forward compared to FIG. 7A. In FIG. 7B, the hand is positioned at a distance closer to the distance detectable range of the information acquisition device 2, and therefore, the dot pattern is irradiated to the hand with a much stronger light amount than the reference image. . In FIGS. 7A and 7B, the background portion of the person such as the ceiling does not exist within the range where the distance can be detected. Therefore, the dots irradiated to these portions are actually measured. It is not reflected in the image.

  Referring to the distance image shown in FIG. 7C, the hand has a gradation value slightly closer to black than the gradation value of the human torso, and the hand is closer to the person than the person. It turns out that it can detect appropriately. In addition, it can be seen that the distance is appropriately obtained for a desk and an object at a distance.

On the other hand, for background parts such as the ceiling where dots are not imaged, black and other gradation values are mixed, and it can be seen that various distance values are acquired in addition to errors. Further, FIG.
Similarly, in the hand area of (), black and other gradation values are mixed, and it can be seen that various distance values are acquired in addition to errors.

  Thus, in the comparative example, even in a region where dots are not imaged or a region where dots cannot be detected properly, errors do not occur uniformly, and various distance values are mixed. That is, it can be seen that even in a region where dots cannot be detected properly, the probability that the distribution of the value Rsad does not meet the error condition of FIG. Such a phenomenon is similarly caused when only one threshold is set as in the error determination index in FIGS. 6A and 6B in addition to the error determination index in FIG. Can happen. Note that even in a region where dots are appropriately irradiated on the object, it may be possible to meet the error conditions of FIGS. 6A to 6C due to differences in the reflectance and shape of the object. Can be.

  If inaccurate distance information or errors are mixed in a region where an error should continue or a region where an object exists in this way, for example, it is erroneously recognized that there is an object on the ceiling portion in FIG. Alternatively, there is a possibility that the object detection process may be adversely affected, such as the position of a person's hand being unable to be detected.

  Therefore, the present embodiment is configured to provide a plurality of threshold values instead of a single threshold value for the above error determination index, and more precisely determine whether or not the search position of the segment area is appropriate. ing.

  FIG. 8A to FIG. 8C are diagrams each illustrating an example using an error determination index provided with a plurality of threshold values. 8A to 8C (distribution example of the value Rsad) are the same as those in FIGS. 6A to 6C. FIGS. 8A to 8C show different error determination indexes.

  In the error determination indexes of FIGS. 8A to 8C, two threshold values Ta1, Ta2, threshold values Tα1, Tα2, and threshold values Tβ1, Tβ2 are used, respectively. The threshold Ta1 is larger than the threshold Ta in FIG. 6A, and the threshold Ta2 is smaller than the threshold Ta in FIG. The threshold value Tα1 is smaller than the threshold value Tα in FIG. 6B, and the threshold value Tα2 is larger than the threshold value Tα in FIG. Furthermore, the threshold Tβ1 is smaller than the threshold Tβ in FIG. 6C, and the threshold Tβ2 is larger than the threshold Tβ in FIG.

  In the error determination index of FIG. 8A, when the minimum value Bt1 is equal to or greater than the threshold value Ta1, the search for the segment area is regarded as an error. When the minimum value Bt1 is less than the threshold value Ta1 and is equal to or greater than the threshold value Ta2, the position corresponding to the minimum value Bt1 is considered uncertain as the displacement position. Hereinafter, such a displacement position is referred to as an “ambiguous displacement position”. Furthermore, when the minimum value Bt1 is less than the threshold value Ta2, it is determined that the position corresponding to the minimum value Bt1 is an appropriate displacement position.

  In the case of FIG. 8B, when the ratio (Bt2 / Bt1) between the second smallest value Bt2 and the minimum value Bt1 is less than a predetermined threshold value Tα1, the search for the segment area is determined to be an error. Further, when the ratio between the second smallest value Bt2 and the minimum value Bt1 is equal to or greater than the predetermined threshold value Tα1 and less than the predetermined threshold value Tα2, the position corresponding to the minimum value Bt1 is an ambiguous displacement position. . Further, when the ratio between the second smallest value Bt2 and the minimum value Bt1 is equal to or greater than a predetermined threshold value Tα2, it is determined that the position corresponding to the minimum value Bt1 is an appropriate displacement position.

Further, in the case of FIG. 8C, if the ratio (Av / Bt1) between the average value Av and the minimum value Bt1 is less than a predetermined threshold value Tβ1, the search for the segment area is determined to be an error. Further, when the ratio between the average value Av and the minimum value Bt1 is equal to or greater than the predetermined threshold value Tβ1 and less than the predetermined threshold value Tβ2, the position of the minimum value Bt1 is regarded as an ambiguous displacement position. When the ratio between the average value Av and the minimum value Bt1 is equal to or greater than a predetermined threshold value Tβ2, the position corresponding to the minimum value Bt1 is an appropriate displacement position.

  As described above, by performing error determination based on a plurality of threshold values, the segment area search results are classified into three, that is, an appropriate displacement position, an error, and an ambiguous displacement position. By classifying in this way, it is possible to apply various processes for determining whether an ambiguous displacement position should be treated as an appropriate displacement position or an error. Can be achieved.

  Hereinafter, a specific example of processing applied to a search determined to be an ambiguous displacement position will be described.

<Example 1>
The first embodiment is an example of a configuration described in claims 3 and 4. In FIG. 9 referred to in the first embodiment, S103 and S105 are examples of the first determination index according to claim 1 to which claim 3 is dependent, and S107 is the second determination index according to claim 3. This is an example of the determination index.

  In the first embodiment, first, based on the first error determination index shown in FIG. 8C, the segment area search results are classified into appropriate displacement positions, errors, and ambiguous displacement positions. When the search result is an ambiguous displacement position, the second error determination index of FIG. 6A is further applied to determine the displacement position search result for the segment area.

  FIG. 9 is a flowchart illustrating the process according to the first embodiment. This process is performed by the function of the distance acquisition unit 21b of the CPU 21 in FIG.

  Referring to FIG. 9, CPU 21 first determines whether or not the search for a predetermined segment area is completed and acquisition of value Rsad is completed (S101). When the acquisition of the value Rsad has not been completed (S101: NO), the CPU 21 returns the process to S101 unless the shutdown is performed (S102: NO), and waits for the process until the acquisition of the value Rsad is completed.

  When the acquisition of the value Rsad is completed for the predetermined segment area (S101: YES), the CPU 21 determines whether the ratio (Av / Bt1) between the average value Av and the minimum value Bt1 is equal to or greater than a predetermined threshold Tβ2. (S103). When the ratio between the average value Av and the minimum value Bt1 is equal to or greater than the predetermined threshold Tβ2 (S103: YES), the CPU 21 determines that the position corresponding to the minimum value Bt1 is an appropriate displacement position, and the pixel shift amount of the minimum value Bt1. Is output to the buffer memory 26 (S104).

  When the ratio between the average value Av and the minimum value Bt1 is less than the predetermined threshold value Tβ2 (S103: NO), the CPU 21 determines whether or not the ratio between the average value Av and the minimum value Bt1 is equal to or greater than the predetermined threshold value Tβ1. (S105). When the ratio between the average value Av and the minimum value Bt1 is less than the predetermined threshold Tβ1 (S105: NO), the CPU 21 determines that the segment area search is an error, and outputs distance information indicating the error value to the buffer memory 26. (S106). As the error value, for example, a value 0 that can be distinguished from normal distance information is used.

  When the ratio between the average value Av and the minimum value Bt1 is less than the predetermined threshold Tβ2 (S103: NO) and is equal to or higher than the predetermined threshold Tβ1 (S105: YES), the CPU 21 applies the second error determination index. Then, it is determined whether or not the minimum value Bt1 is less than a predetermined threshold Ta (S107).

  When the minimum value Bt1 is less than the predetermined threshold Ta (S107: YES), the CPU 21 determines that the position corresponding to the minimum value Bt1 is an appropriate displacement position, and provides distance information corresponding to the pixel shift amount of the minimum value Bt1. The data is output to the buffer memory 26 (S104). On the other hand, when the minimum value Bt1 is greater than or equal to the predetermined threshold Ta (S107: NO), the CPU 21 determines that the segment area search is an error, and outputs distance information indicating the error value to the buffer memory 26 (S106).

  Thus, when the process for the segment area is completed, the CPU 21 returns the process to S101, and executes the distance information acquisition process for the next segment area. And CPU21 performs the process based on the 1st error determination parameter | index and the 2nd error determination parameter | index about all the segment area | regions set to the reference pattern area | region.

  FIG. 10A is a diagram schematically illustrating classification based on the first error determination index, and FIG. 10B is a diagram schematically illustrating classification based on the second error determination index. 10A and 10B are graphs showing distribution examples of the same value Rsad as in FIGS. 6C and 8C.

  Referring to FIG. 10A, in the first error determination index, for example, when minimum value Bt1 is 300, threshold value Tβ1 is 2.5, and threshold value Tβ2 is 3.5, average value Av is Tc1 (750). ) Or more, and when it is less than Tc2 (1050), the position corresponding to the minimum value Bt1 is an ambiguous displacement position. For example, when the average value Av is 1000, the determinations in S103 and S105 in FIG. 9 are “NO” and “YES”, respectively, and the position corresponding to the minimum value Bt1 is determined as an ambiguous displacement position.

  When it is determined that the position corresponding to the minimum value Bt1 is an ambiguous displacement position, the determination for the minimum value Bt1 is further performed using the second error determination index as shown in FIG. For example, when the threshold value Ta is 600, the minimum value Bt1 is approximately 300. Therefore, YES is determined in S107 of FIG. 9, and the position corresponding to the minimum value Bt1 is determined as an appropriate displacement position.

  As described above, by applying the second error determination index to the area determined as the ambiguous displacement position by the first error determination index using a plurality of thresholds, the determination can be optimized. Accordingly, in the distance images shown in FIGS. 7C and 7D, an uncertain distance value is included in the area to be an error, or an error value (0) is present in the area where the object exists. Inclusion can be suppressed.

  In the present embodiment, the error determination index of FIG. 8C is applied as the first error determination index, and the error determination index of FIG. 6A is applied as the second error determination index. A combination of other error determination indices may be used, such as replacing the first error determination index with the error determination index of FIG. However, the error determination index in FIG. 8A and the error determination index in FIG. 6A are merely different in the number of thresholds, and the basic principle of the determination index is the same. This combination is not preferable. That is, it is desirable to combine error determination indexes of different principles as the first error determination index and the second error determination index.

<Effect of Example 1>
According to the first embodiment, the following effects can be achieved.

Error determination is performed with the first error determination index using a plurality of threshold values, and appropriate distance information is obtained by further performing error determination with the second error determination index for the area that is an ambiguous displacement position. Can be acquired. Thereby, the detection accuracy of an object can be improved.

  In addition, since it is possible to evaluate the ambiguous displacement position with another index simply by switching the error determination index of the calculation result of the value Rsad, it is possible to acquire appropriate distance information while suppressing an increase in the amount of calculation. .

<Example 2>
The second embodiment is an example of a configuration according to claims 5 to 10. In FIG. 11 referred to in the second embodiment, S103 and S105 are examples of the first determination index according to claim 1 to which claim 5 is dependent, and FIG. 13 and FIG. It is an example of the 2nd determination parameter | index of description.

  In the first embodiment, the attribute of the position corresponding to the minimum value Bt1 is classified by the error determination index using a plurality of threshold values, and when the classification is “ambiguous displacement position”, the classification is determined as another error determination. Although evaluated by the index, in the second embodiment, the classification for the target segment area is corrected based on the classification applied to the segment areas around the target segment area.

  FIG. 11 is a flowchart illustrating the processing of the second embodiment. This process is performed by the function of the distance acquisition unit 21b of the CPU 21 in FIG. In FIG. 11, the same processes as those in FIG. 9 are given the same numbers, and detailed descriptions thereof are omitted.

  Referring to FIG. 11, in the same manner as in the first embodiment, the error determination index using a plurality of thresholds shown in FIG. 8C indicates that the search result is an appropriate displacement position, an ambiguous displacement position, and an error. being classified. If the ratio of the average value Av to the minimum value Bt1 is equal to or greater than the predetermined threshold value Tβ2 (S103: YES), the distance information acquired based on the minimum value Bt1 is stored in the buffer memory 26 in S104 as in the first embodiment. Is output. If the ratio between the average value Av and the minimum value Bt1 is less than the predetermined threshold Tβ1 (S105: NO), an error value is output to the buffer memory 26 as distance information in S106, as in the first embodiment.

  Furthermore, in the second embodiment, when the ratio between the average value Av and the minimum value Bt1 is less than the predetermined threshold Tβ2 (S103: NO) and is equal to or higher than the predetermined threshold Tβ1 (S105: YES), the CPU 21 The distance information acquired based on the value Bt1 is output to the buffer memory 26 (S108).

  In addition, in the second embodiment, the classification of each distance information output to the buffer memory 26 is held in the buffer memory 26. Here, the classification of the distance information indicates whether the distance information is acquired based on “appropriate displacement position”, “ambiguous displacement position”, or “error”. In the second embodiment, the distance information values based on “proper displacement position”, “ambiguous displacement position”, and “error” are respectively “normal value”, “ambiguity value”, and “error value”. Called.

  In the second embodiment, the buffer memory 26 is provided with an area for holding distance information (line memory) and an area for holding a flag indicating the classification of each distance information. In S104, S106, and S108, the CPU 21 writes the distance information in the corresponding area of the buffer memory 26 and writes a flag indicating the classification of the distance information in the corresponding area of the buffer memory 26.

In this way, the error information is classified into a normal value, an ambiguous value, and an error value according to an error determination index using a plurality of threshold values, and the value of the distance information is stored in the buffer memory 26.

  In the verification results of the comparative example shown in FIGS. 7C and 7D, various distance values are mixed in addition to error values in a region such as a ceiling where the dot pattern is not imaged. These distance values are considered to have been acquired because the ratio between the minimum value Bt1 and the average value Av is the same as or slightly exceeds the threshold value Tβ in the error determination index of FIG. On the other hand, in the processing flow of FIG. 11, when the ratio between the minimum value Bt1 and the average value Av slightly exceeds the threshold value Tβ, the determination in S105 is YES, so many of such distance values are It is assumed to be determined as an ambiguous value.

  Further, as described above, even in a region where an object such as a person is present, the obtained distance information may be determined to be an ambiguous value for a segment region where the accuracy of the value Rsad is lowered.

  FIG. 12 is a diagram schematically illustrating a tendency that an ambiguous value is generated in a distance image. FIG. 12 shows a distance image when a dot pattern is irradiated onto a target area where three objects are present.

  Referring to FIG. 12, in this example, normal values are acquired for three objects, and error values are acquired for other regions. Further, an ambiguous value is included in each of the regions corresponding to the three objects and the other regions. From the verification results of the comparative examples of FIGS. 7C and 7D, it is assumed that the appearance frequency of the ambiguous value is high when the surrounding is an error value and low when the surrounding is a normal value. . Further, when the error value is in the surrounding area, the ambiguous value is likely to be an error value, and when the normal value is in the surrounding area, the ambiguous value is likely to be a normal value.

  Therefore, in the second embodiment, based on the above tendency, when an ambiguous value is surrounded by an error value, the ambiguous value is corrected to an error value, and the ambiguous value is surrounded by a normal value. The ambiguous value is corrected to a normal value. Also, if the area around the ambiguous value is surrounded by the normal value and the error value to the same extent, it may be within the boundary between the object and the background or within the object, so the ambiguous value is corrected to the normal value. .

  FIG. 13 and FIG. 14 are flowcharts showing processing for correcting an ambiguity value based on surrounding error determination results. This process is performed by the function of the distance acquisition unit 21b of the CPU 21 in FIG.

  Referring to FIG. 13, the CPU 21 first sets 1 to a variable k indicating the position on the buffer memory 26 of the value output to the buffer memory 26 (S201). Then, the CPU 21 determines whether or not a value is output to the buffer memory 26 (S202). When the value is not output to the buffer memory 26 (S202: NO), the CPU 21 returns the process to S202 unless the shutdown is performed (S203: NO), and performs the process until the value is output to the buffer memory 26. stand by.

  In the flow of FIG. 11, when the error determination result for the predetermined segment area is completed and a value is output to the buffer memory 26 (S202: YES), the CPU 21 is written in the kth storage location of the buffer memory 26. It is determined whether the obtained value is an ambiguous value (S204). When the k-th value is not an ambiguous value (S204: NO), the CPU 21 determines whether or not the variable k is a variable e indicating the end of one line (S205). The variable e is the number of segment areas shown in FIG. 4A in the X-axis direction, that is, the number of distance information in the X-axis direction. For example, 640 is set in e.

  When the variable k is not the end of one line (S205: NO), the CPU 21 adds 1 to the variable k (S206), and returns the process to S202. As a result, the processes of S202 to S206 are repeated until the variable k reaches the final position of one line, and it is determined whether or not the value written in each storage position is an ambiguous value. When the variable k reaches the final position of one line (k = e) (S205: YES), the CPU 21 returns the process to S201, sets the variable k to 1, and then sets the value of the buffer memory 26 again. to correct. The buffer memory 26 stores values for one line (for example, 640 values), and outputs values for one line to the information processing device 3 at the timing when the correction process is completed. Thereafter, the distance information value for the segment area of the next line is overwritten again from the head (k = 1) of the buffer memory 26.

  If the kth value output to the buffer memory 26 is an ambiguous value (S204: YES), the CPU 21 determines whether k is 1 (S211). Thereby, it is determined whether or not the storage position of the ambiguous value is the first in the buffer memory 26. When the storage position of the ambiguous value is the first in the buffer memory 26 (S211: YES), the CPU 21 corrects the kth (k = 1) ambiguous value to an error value (S212). Specifically, the first distance value stored in the buffer memory 26 is replaced with 0. At the same time, the classification of the first stored value is changed from an ambiguous value to an error value.

  FIG. 15A to FIG. 15I are schematic diagrams conceptually showing the classification of values stored in the buffer memory 26 before and after ambiguous value correction. Note that the numbers 1 to 640 are attached to the buffer memory 26 in FIGS. 15A to 15G from the beginning to the end. These numbers correspond to the variable k in the flowcharts of FIGS.

  Referring to FIG. 15A, when an ambiguous value is output at the head (k = 1) of the buffer memory 26, the ambiguous value cannot be properly evaluated from the surrounding error determination result. Correct the error value. Such an end region is likely to have an error value because the amount of light of the irradiated dot pattern tends to be small. In addition, since it is unlikely that a detection target object such as a person is positioned in such an end region, if an ambiguous value is output in such an end region, the ambiguous value is corrected to an error value. It is desirable to do.

  Returning to FIG. 13, when the value held at the kth storage position of the buffer memory 26 is an ambiguous value (S204: YES) and the variable k is not 1 (S211: NO), the CPU 21 determines that the variable k is a variable. It is determined whether it is e (S214). Thereby, it is determined whether or not the storage position of the ambiguous value is at the end of the buffer memory 26. When the variable k is the variable e (S214: YES), the CPU 21 corrects the e-th ambiguous value in the buffer memory 26 to an error value (S215).

  For example, as shown in FIGS. 15B and 15C, when an ambiguous value is stored at the end of the buffer memory 26, the value stored immediately before the end is the normal value or error value. In either case, the error value is corrected. This is because, when an ambiguous value is output at the end of the buffer memory 26, the ambiguous value can be evaluated only from the error determination result for the previous value, and the ambiguous value is appropriately determined from the surrounding error determination result. This is because it cannot be evaluated. In addition, since such an end area is likely to be an error value as in the case of the top (k = 1) of the buffer memory 26, the ambiguous value output to such an end area is corrected to an error value. It is desirable to do.

  Returning to FIG. 13, when the value held at the kth storage position of the buffer memory 26 is an ambiguous value (S204: YES) and the variable k is greater than 1 and less than the variable e (S211: NO, S214: NO), CPU21 advances a process to S221 of FIG.

  Referring to FIG. 14, CPU 21 sets 1 to variable p for counting the number of consecutive ambiguous values (S221). Then, the CPU 21 determines whether or not a value has been output to the k + pth storage position of the buffer memory 26 (S222). When a value is not output to the k + pth storage location (S222: NO), the CPU 21 returns the process to S222 and outputs a value to the k + pth storage location unless shutdown is performed (S223: NO). Wait until processing is complete. For example, when p = 1, the process waits until a value is output to the storage position next to the storage position of the ambiguous value to be processed.

  When a value is output to the k + pth storage location of the buffer memory 26 (S222: YES), the CPU 21 determines whether or not the value of the k + pth storage location is an ambiguous value (S224). When the value of the k + p-th storage position is an ambiguous value (S224: YES), the CPU 21 determines whether k + p is equal to a variable e indicating the end of one line (S225).

  When k + p is not equal to the variable e (S225: NO), the CPU 21 adds 1 to the variable p (S226), and returns the process to S222. Thereby, it is further determined whether or not the value held at the next storage position is an ambiguous value (S224). If the value held at this storage position is an ambiguous value (S224: YES), the process again proceeds to S225, and the same process as described above is performed. Thus, the storage location of the reference object is sent one by one until a value that is not an ambiguous value is acquired.

  When the k + p-th output value of the buffer memory 26 is not an ambiguous value (S224: NO), the CPU 21 stores the data immediately before the k-th storage position (k-1) from which the ambiguous value was output first. It is determined whether or not the position value is an error value (S231). When the k-1th value is an error value (S231: YES), the CPU 21 determines whether or not the k + pth value in which the ambiguous value is not continuous is an error value (S232).

  When the k−1th value is a normal value (S231: NO), the CPU 21 determines the kth to k + p−1th ambiguous values regardless of whether the k + pth value is an error value or a normal value. Then, it is corrected to a normal value (S233). Similarly, when the k−1th value is an error value (S231: YES) and the k + pth value is a normal value (S232: NO), the CPU 21 similarly determines the kth to k + p−1th ambiguity. The value is corrected to a normal value (S233). When the k−1th value is an error value (S231: YES) and the k + pth value is also an error value (S232: YES), the CPU 21 sets the kth to k + p−1th ambiguous value as an error. The value is corrected (S234).

  For example, as shown in FIG. 15D, when the first is a normal value, the second to sixth are ambiguous values, and the seventh is a normal value, the second to sixth ambiguous values are corrected to normal values. Is done. Also, as shown in FIG. 15E, when the first is a normal value, the second to sixth are ambiguous values, and the seventh is an error value, the second to sixth ambiguous values are corrected to normal values. Is done. Further, as shown in FIG. 15F, when the first is an error value, the second to sixth are ambiguous values, and the seventh is a normal value, the second to sixth ambiguous values are corrected to normal values. Is done. Further, as shown in FIG. 15G, when the first is an error value, the second to sixth are ambiguous values, and the seventh is an error value, the second to sixth ambiguous values are corrected to error values. Is done.

  When the ambiguous value is corrected to the error value, the distance value output to the corresponding storage position is replaced with 0 as described above. At the same time, the classification of the value is changed from an ambiguous value to an error value. When the ambiguous value is corrected to the normal value, the distance value output to the corresponding storage position is not changed, and the classification of the distance value is corrected from the ambiguous value to the normal value.

  As described above, according to the processing of S231 to S234 in FIG. 14, the ambiguous value in which the front and rear are sandwiched by the error value is corrected to the error value according to the above tendency, and the ambiguous value in which the front and rear are sandwiched by the normal value The ambiguous value between the normal value and the error value is corrected to the normal value.

  Returning to FIG. 14, when the correction process of S233 or S234 is completed, the CPU 21 adds the variable p and 1 counted up until the ambiguity value is not continued to the variable k (S235), and the process proceeds to S202 of FIG. return. Then, the CPU 21 performs processing for the value output to the next storage position.

  When the ambiguous value continues to the end of one line by the processing of S222 to S225, the CPU 21 advances the processing to S241. That is, when the k + p-th output value of the buffer memory 26 is an ambiguous value (S224: YES) and k + p is a variable e indicating the end of one line (S225: YES), the CPU 21 performs the same as S215. The e-th ambiguous value is corrected to an error value (S241). Thereafter, the CPU 21 determines whether or not the (k−1) -th value is an error value (S242). When the k−1th value is a normal value (S242: NO), the CPU 21 corrects the kth to k + p−1th ambiguity value to a normal value (S243). When the k−1th value is an error value (S242: YES), the CPU 21 corrects the kth to k + p−1th ambiguity values to error values (S244). In this way, the ambiguous value is corrected according to the same conditions as in S231 to S234.

  For example, as shown in FIG. 15 (h), when the ambiguous value continues from the 631st to the end and the 630th value is a normal value, the ambiguous value at the end (640th) is corrected to an error value. The 639th ambiguous value is corrected to a normal value. As shown in FIG. 15 (i), when the ambiguity value continues from the 631st to the end and the 630th is an error value, the ambiguity value from the 631st to the end (640th) is corrected to an error value. .

  Returning to FIG. 14, when the correction process of S243 or S244 is completed, the CPU 21 returns the process to S201 of FIG. Then, the CPU 21 returns the variable k to 1 (S201), and waits for processing until the next value is output to the buffer memory 26 (S202: NO).

  In this way, the ambiguous value output to the buffer memory 26 is corrected to a normal value or an error value based on the surrounding error determination result.

  FIGS. 16A and 16B are diagrams illustrating verification results obtained by performing distance detection by performing the ambiguous value correction process according to the second embodiment. FIG. 16A and FIG. 16B are diagrams showing distance images as verification results. The actual measurement image used for the verification, the size of the segment area, and the search range are the same as those in the comparative example of FIGS. 7C and 7D.

  In the case of the comparative examples shown in FIGS. 7C and 7D, various distances are also obtained for regions that should have uniform error values, such as a background portion such as a ceiling and a hand portion that is extremely short. The value was being retrieved. On the other hand, in the verification result of the second embodiment, the background portion such as the ceiling in FIG. 16A and the hand portion in FIG. 16B are almost uniformly black. It can be seen that many of the distance values acquired in this way are appropriately corrected to error values. In addition, a normal value is appropriately acquired for a portion such as a person.

  As described above, in the second embodiment, based on the above tendency, an ambiguity value is determined using a plurality of threshold values, and the ambiguity value is corrected based on a surrounding error determination result, thereby the background and the like. An error value is appropriately acquired in a region where error values are continuous, and a distance value is appropriately acquired for a region where distance information of an object such as a person is continuous. Therefore, the accuracy of object detection can be increased.

  Further, in the distance images shown in FIGS. 16A and 16B, compared to the distance images shown in FIGS. 7C and 7D, the sharpness of the outline of the person can be obtained almost inferior. You can see that Therefore, in the second embodiment, even when the ambiguous value is corrected based on the surrounding error determination result, the person's contour extraction accuracy can be maintained appropriately.

  Further, since the correction processing of the second embodiment performs ambiguous value correction processing in units of one line in the horizontal direction, the buffer memory 26 has a line memory of at least one line to hold the value of distance information. It should be. Therefore, ambiguous value correction processing can be performed without increasing the hardware resources of the information acquisition device 2. Even if ambiguous value correction processing is performed only in units of one line in the horizontal direction, as shown in FIGS. 16A and 16B, uncertain distance information can be effectively suppressed. .

<Effect of Example 2>
According to the second embodiment, the following effects can be achieved.

  By correcting the ambiguous value extracted by the error determination index using a plurality of threshold values based on the surrounding error determination result, the distance information can be optimized. Thereby, the accuracy of object detection can be improved.

  Further, since the ambiguous value correction process is performed in units of one line in the horizontal direction, the ambiguous value correction process can be appropriately performed while suppressing an increase in hardware resources.

  In the second embodiment, the ambiguous value is determined by the processing flow of FIG. 11 based on the error determination index of FIG. 8C. However, the error determination of FIGS. 8A and 8B is performed. An ambiguous value may be determined based on the index.

  Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications can be made to the embodiment of the present invention in addition to the above. .

  For example, in the above embodiment, the buffer memory 26 uses a line memory for one line in order to hold the value of distance information. However, the buffer memory 26 holds the value of distance information in a plurality of line memories or in units of frames. A frame memory may be used. In this case, the distance image generated by the information acquisition device 2 is output to the information processing device 3 for each frame. One frame is a unit of one distance image.

  In such a case, as shown in FIGS. 15A and 15B, after performing ambiguous value correction processing in units of one line in the horizontal direction, a noise removal filter such as a median filter is further applied. Thus, a distance image may be generated.

  17 (a) and 17 (b) are correction examples in the case where a median filter is applied by dividing the area into 5 × 5 pixels. FIGS. 17 (c) and 17 (d) are illustrated in FIG. It is a figure which shows the distance image of a verification result at the time of applying a median filter of 5 pixels x 5 pixels to the verification result of a) and FIG.16 (b) further.

  FIG. 17A is a diagram illustrating an example of pixel values (distance values) stored in the buffer memory 26 after ambiguous value correction is performed according to the processing flows of FIGS. 13 and 14.

When the ambiguous value is corrected as described above, and the distance image value for one frame is accumulated in the buffer memory 26, the CPU 21 of the information acquisition device 2 converts the distance image on the buffer memory 26 into 5 pixels × The area is divided into 5 pixels, and the center pixel of interest is replaced with the median value in this area. For example, in the case of FIG. 17A, since the values 0, 10, 30, 100, and 110 are stored in the area, the pixel value of the center pixel of interest is as shown in FIG. To 30. Similarly, by applying the median filter to all the pixels, the distance images shown in FIGS. 17C and 17D are generated. In the example of FIG. 17A, when all numerical values are arranged in ascending order and the median value (13th numerical value) is taken, 100 is obtained.

  In the verification results shown in FIGS. 16A and 16B, the ambiguous value is corrected based only on the error determination result in the horizontal direction, so that a slight amount of horizontal noise is generated in the outline of the person. On the other hand, in FIGS. 17 (c) and 17 (d), it can be seen that noise can be properly removed even in portions such as the outline of a person.

  Thus, when a plurality of line memories or frame memories are used as the buffer memory 26, in addition to the ambiguous value correction processing of the second embodiment, by applying a noise removal filter such as a median filter, Inappropriate distance information can be further suppressed. Thereby, the accuracy of object detection can be further increased.

  In addition to the median filter, various types of noise removal filters can be used. For example, when the mode filter is used, the pixel of interest in the center of FIG. 17A is corrected to 0, which has the highest appearance frequency in the region.

  Further, when a frame memory or the like is used as the buffer memory 26, for example, when an ambiguous value is surrounded by a normal value and an error value, it is not treated as a normal value, and further, an ambiguous value is determined according to the vertical error determination result. The value may be corrected.

  FIGS. 18A and 18B are flowcharts showing the flow of the ambiguous value correction process in this case. This process is performed by the function of the distance acquisition unit 21b of the CPU 21 in FIG. Note that FIG. 18B partially changes the flowchart of FIG. 14, and the same number is assigned to a process that does not change.

  The process flow shown in FIG. 18 is another example of the configuration described in claims 5 to 10.

  Referring to FIG. 18A, the CPU 21 first determines whether or not the distance information value (normal value, ambiguous value, error value) for one frame is output to the buffer memory 26 (S301). When the value for one frame is output to the buffer memory 26 (S301: YES), the CPU 21 sets 1 to a variable r indicating the position of the line in the horizontal direction (S303). Then, the CPU 21 performs the ambiguous value correction process in the horizontal direction for the r-th line as described in the second embodiment (S304).

  As shown in FIG. 14, in the second embodiment, when the k-1th value is an error value (S231: YES) and the k + pth value is a normal value (S232: NO), the CPU 21 Then, the process proceeds to S233, and the ambiguous value is corrected to the normal value. However, in this modified example, as shown in FIG. 18B, the k−1th value is an error value (S231: YES), When the k + p-th value is a normal value (S251: NO), the CPU 21 advances the process to S235 without correcting the ambiguous value. Similarly, if NO is determined in S242 of FIG. 14, the ambiguous value is not corrected and the process returns to S201. As described above, in the present modification example, the ambiguous value sandwiched between the normal value and the error value is not corrected in the ambiguous value correction process in the horizontal direction.

  Returning to FIG. 18A, the CPU 21 determines whether or not there is a value for which the ambiguous value is not corrected in the horizontal ambiguous value correcting process (S305). When there is a value in which the ambiguous value is not corrected in the horizontal ambiguous value correction process (S305: YES), the CPU 21 further performs the ambiguous value correction process in the vertical direction for the ambiguous value of the r-th line (S306). . The vertical ambiguity value correction process is performed according to the same rules as the horizontal ambiguity value correction process shown in the flowcharts of FIGS.

  For example, as shown in FIG. 18C, when there is an ambiguous value surrounded by a normal value and an error value on the r-th line, when the normal value is surrounded in the vertical direction, the ambiguous value becomes a normal value. It is corrected.

  When the vertical ambiguity correction process for the r-th line is completed, the CPU 21 determines whether or not the variable r is a variable L indicating the final line (S307). If it is not the last line (S307: NO), the CPU 21 adds 1 to the variable r (S308), and returns the process to S304. Thereby, the ambiguous value correction process in the horizontal direction and the vertical direction is performed for the next line. When the ambiguous value correction process is completed up to the final line (S307: YES), the CPU 21 returns the process to S301 and waits until the next frame is output.

  In this way, the ambiguous value correction process in the horizontal direction and the vertical direction is performed for each frame.

  In the case of this modified example, the ambiguous value is corrected in consideration of not only the horizontal but also the vertical error determination result, so that the ambiguous value is corrected more accurately. Therefore, in this case, it is assumed that the lateral noise shown in FIGS. 16A and 16B is reduced.

  In this modification, the ambiguous value is corrected based on the error determination result in the horizontal direction and the vertical direction. However, as shown in FIG. 19A, the ambiguous value correction is performed based on the error determination result in the oblique direction. Alternatively, as shown in FIG. 19B, the region may be divided around the ambiguous value, and the ambiguous value may be corrected based on the error determination result in the region. In the case of FIG. 19B, for example, the ambiguous value may be corrected to the classification having the highest frequency in the region. In that case, in the example of FIG. 19B, the ambiguous value is corrected to a normal value.

  Moreover, in the said Example 1, 2, although the ambiguous value was determined using two threshold values, as shown to Fig.8 (a)-FIG.8 (c), as shown in FIG.19 (c), for example. In addition, three threshold values may be used, and four or more threshold values may be used.

  In this case, the appropriateness of the distance information obtained by the value Rsad can be classified more finely. For example, when three threshold values are used as shown in FIG. 19C, the degree of ambiguity can be divided into two levels, and the processing for the ambiguity value may be changed according to the degree of ambiguity. For example, as shown in FIG. 15E and FIG. 15F, when an ambiguous value is sandwiched between a normal value and an error value, if the ambiguous value is the ambiguous value A in FIG. If the ambiguity value is corrected to an error value if the ambiguity value is high, if the ambiguity value is the ambiguity value B in FIG. 19 (c), the ambiguity value is corrected to a normal value because the probability of being a normal value is high. You may do it. Further, the processing of the first embodiment and the second embodiment may be distributed according to the degree of ambiguity. For example, the ambiguous value A region close to the error value is subjected to the ambiguous value correction process in consideration of the horizontal direction and the vertical direction shown in FIG. 18, and the ambiguous value B region close to the normal value is shown in FIG. The ambiguous value correction process considering only the horizontal direction shown in FIG. 14 may be performed. In addition, various processes can be applied to a region determined to be an ambiguous value. For example, a predetermined filter process such as noise removal may be applied only to an ambiguous value.

  In the first and second embodiments, the ambiguous value is determined based on the error determination index shown in FIG. 8C. However, the ambiguous value is determined based on the error determination index shown in FIGS. 8A and 8B. May be. In the first embodiment, the error determination index of FIG. 6 (a) is used after the ambiguous value is determined by the error determination index of FIG. 8 (c), but the error determination index of FIG. 6 (b) is used. May be used. Furthermore, after the error determination index of FIG. 8C, the error determination index of FIG. 8B may be applied, and then the error determination index of FIG. 6A may be applied. In addition, any error determination index may be used as long as an ambiguous value can be determined using a plurality of threshold values.

  In the above embodiment, the sum of absolute values SAD (Sum of Absolute Difference) shown in FIG. 5C is used to calculate the matching degree of the dot pattern. Thus, a sum of squared differences (SSD) of the sum of the squares of the difference values may be used.

  In the above embodiment, the segment areas are set so that the adjacent segment areas overlap with each other. However, the segment areas adjacent to each other on the left and right may not overlap with each other, and the segment areas adjacent to each other on the upper and lower sides. However, they do not have to overlap each other. Further, the shift amount of the segment areas adjacent in the vertical and horizontal directions is not limited to one pixel, and the shift amount may be set to another number of pixels. Furthermore, in the said embodiment, although the segment area | region was set to square shape, a rectangle may be sufficient.

  In the above embodiment, the pixel values of the pixels included in the segment area and the comparison area are reduced to 0 to 12 before calculating the matching ratio between the segment area and the comparison area. It may be reduced and binarized, or matching may be performed using the pixel value obtained by the CMOS image sensor 240 as it is.

  In the above embodiment, the distance information is obtained by using the triangulation method and stored in the memory 25. However, the displacement amount (pixel displacement amount) of the segment area is used as the distance information without calculation by the triangulation method. You may acquire and distance information may be acquired by another method.

  Further, in the above embodiment, the filter 230 is disposed to remove light in a wavelength band other than the wavelength band of the laser light irradiated to the target region. For example, light other than the laser light irradiated to the target region is used. In the case where a circuit configuration for removing the signal component from the signal output from the CMOS image sensor 240 is arranged, the filter 230 may be omitted. In the above embodiment, the CMOS image sensor 240 is used as the light receiving element, but a CCD image sensor may be used instead. Furthermore, the configurations of the projection optical system 100 and the light receiving optical system 200 can be changed as appropriate. In addition, the present invention can be applied to information acquisition forms other than control input such as a television using the right hand.

  The embodiments of the present invention can be appropriately modified in various ways within the scope of the technical idea shown in the claims.

1 ... Object detection device
2 ... Information acquisition device 21b ... Distance acquisition unit 23 ... Imaging signal processing circuit (distance acquisition unit)
DESCRIPTION OF SYMBOLS 3 ... Information processing apparatus 31a ... Object detection part 100 ... Projection optical system 110 ... Laser light source 200 ... Light reception optical system 240 ... CMOS image sensor (image sensor)

Claims (11)

A projection optical system that projects laser light emitted from a laser light source onto a target area with a predetermined dot pattern;
A light receiving optical system that is arranged so as to be laterally separated by a predetermined distance with respect to the projection optical system, and that captures the target area by an image sensor;
Based on the reference dot pattern imaged by the light receiving optical system when the laser beam is irradiated on the reference surface and the measured dot pattern imaged by the image sensor at the time of actual measurement, up to the object included in the target area A distance acquisition unit that acquires distance information about the distance,
The distance acquisition unit searches for a displacement position on the measured dot pattern of a predetermined segment area set in the reference dot pattern based on a first determination index, and based on the searched displacement position, Obtain distance information for the segment area,
The first determination index has a first threshold value and a second threshold value larger than the first threshold value in order to evaluate the likelihood of the displacement position.
The distance acquisition unit determines whether or not to acquire the distance information based on the displacement position when the first parameter value related to the displacement position is between the first threshold value and the second threshold value. Determining based on the second determination index;
An information acquisition apparatus characterized by that. The information acquisition device according to claim 1,
The first parameter value is a value generated based on a matching degree between a dot included in the segment area and a dot on the measured dot pattern.
An information acquisition apparatus characterized by that. In the information acquisition device according to claim 1 or 2,
The second determination index includes a third threshold value applied to a second parameter value different from the first parameter value,
The distance acquisition unit compares the second parameter corresponding to the displacement position with the third threshold value to determine whether the displacement position is appropriate, and when the displacement position is appropriate in the determination The distance information acquired based on the displacement position is the distance information for the segment area, and when the displacement position is inappropriate in the determination, the distance information for the segment area is set to information based on an error.
An information acquisition apparatus characterized by that. In the information acquisition device according to claim 3,
The second parameter value is a value generated based on a matching degree between a dot included in the segment area and a dot on the measured dot pattern.
An information acquisition apparatus characterized by that. In the information acquisition device according to claim 1 or 2,
The second determination index is based on the displacement position based on a relationship between a first segment area from which the distance information is acquired and a second segment area around the first segment area. Including a determination condition for determining the suitability of the distance information acquired in
An information acquisition apparatus characterized by that. The information acquisition device according to claim 5,
The determination condition includes a classification relating to the suitability of acquisition of the distance information for the second segment region,
An information acquisition apparatus characterized by that. The information acquisition apparatus according to claim 6,
The distance acquisition unit is configured such that the number of the second segment areas classified as appropriate for acquiring the distance information is the number of the second segment areas classified as inappropriate for acquisition of the distance information. If the number exceeds, the distance information acquired based on the displacement position is used as the distance information with respect to the first segment area, and the distance information acquired with respect to the first segment area is acquired. If the number of the second segment areas classified as inappropriate is greater than the number of the second segment areas classified as appropriate for obtaining the distance information, Information acquisition apparatus characterized by improper acquisition of distance information for a segment area and setting the distance information for the first segment area to information based on an error In the information acquisition device according to claim 6 or 7,
The distance acquisition unit acquires distance information for the first segment area when the first segment area is sandwiched between the second segment areas classified as appropriate for acquiring the distance information. Using distance information acquired based on the displacement position as the distance information for the first segment region,
An information acquisition apparatus characterized by that. In the information acquisition device according to any one of claims 6 to 8,
The distance acquisition unit, when the first segment area is sandwiched between the second segment areas classified as inappropriate acquisition of the distance information, the distance information of the first segment area The acquisition is inappropriate, and the distance information for the first segment area is set to information based on an error.
An information acquisition apparatus characterized by that. In the information acquisition device according to any one of claims 6 to 9,
The distance acquisition unit is configured such that the first segment area is classified as inappropriate for acquisition of the distance information and the second segment area classified as appropriate for acquisition of the distance information. When it is sandwiched between two segment areas, it is appropriate to acquire distance information for the first segment area, and distance information acquired based on the displacement position is used for the distance information for the first segment area.
An information acquisition apparatus characterized by that. The information acquisition device according to any one of claims 1 to 10,
An object detection unit that detects a predetermined target object based on the distance information,
An object detection apparatus characterized by that.