JP2013024653A - Distance measuring apparatus and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable the distance to a specific object to be accurately measured by using a plurality of image pickup means differing from each other in wavelength characteristics.SOLUTION: A prescribed area is photographed from different viewpoints with a first image pickup device 12 and a second image pickup device 14 differing from each other in the wavelength band of sensitivity. A human skin area is extracted with an object extracting unit 32 from each of the photographed images on the basis of pre-determined conditions regarding the reflection intensity of each of optical wavelength bands regarding human skins. Human skin areas extracted from the photographed images are aligned in position by an aligning unit 34 to calculate the parallax quantities in the human skin areas. The distance to the person is measured by a distance measuring unit 36 on the basis of the calculated parallax quantities, and spectroscopic data of the human skin areas are generated by a spectroscopic data generating unit 38.

Description

  The present invention relates to a distance measuring device and a program, and more particularly, to a distance measuring device and a program for measuring a distance from an image captured by a plurality of imaging means to a specific object.

  Conventionally, systems aimed at detecting and recognizing the position and orientation of complex three-dimensional objects have been developed. For example, the three-dimensional shape of an object is restored using edges extracted from grayscale images obtained by binocular vision as basic elements. A three-dimensional vision system is known (Non-Patent Document 1). In this three-dimensional vision system, an edge detection process is performed on images acquired by two identical cameras arranged so that parallax appears, and the similarity of the detected edges is measured to measure the three-dimensional shape of the target. Is restoring.

  Further, a spectroscopic measurement method that can measure a large number of bands and does not cause a positional shift of spectral images of each band is known (Patent Document 1). In this spectroscopic measurement method, an object to be measured is photographed using a standard camera and a reference camera, and the reference image data created by the reference camera is projectively converted as if photographed from the position of the standard camera, and the object in the image is then converted. Create converted image data in which the coordinates indicating an arbitrary point on the measured object indicate the coordinates on the same point on the measured object of the reference image data captured by the reference camera, and the reference captured by the reference camera By sharing the coordinates of the image data and the converted image data, spectral data of an arbitrary point on the measurement object on the coordinates is created.

Takashi Naito, Yoshikatsu Kimura, “Development of Stereo Vision System”, Toyota Central R & D R & D Review, Vol. 31, no. 2, June 1996

JP 2007-147507 A

  However, in the technique described in Non-Patent Document 1, when two cameras having different wavelength characteristics are used, since the edge images of the two cameras are different, it is impossible to accurately calculate the degree of similarity. There is a problem that the shape cannot be obtained with high accuracy. This is because there is a difference in images to be acquired because the wavelength having sensitivity differs among cameras, and the same applies to other stereo camera techniques that calculate the degree of similarity using luminance.

  Further, as described above, when images are taken with cameras having different wavelength characteristics, it is difficult to align the images. However, in the technique described in Patent Document 1, a reference point is provided on the surface of the object to be measured. Aligning multiple cameras is realized. However, in the technique described in Patent Document 1 described above, only the area of the object having the projected reference point on the surface can be aligned, and the area of the object existing outside the range of the light projection can be detected. There is a problem that it is impossible to perform alignment.

  The present invention has been made in order to solve the above problems, and a distance measuring device capable of accurately measuring a distance to a specific object using a plurality of imaging means having different wavelength characteristics and The purpose is to provide a program.

  In order to achieve the above object, a distance measuring device according to the present invention includes a plurality of imaging means for imaging a predetermined region from different viewpoints, each having a different wavelength range of light having sensitivity, and the plurality of imaging means. A region extracting means for extracting a region representing the specific object from each of the captured images picked up by the method based on a predetermined condition regarding the reflection intensity of each of the light wavelength bands of the specific object; A parallax amount calculating unit that calculates a parallax amount in the region by aligning the regions extracted from each of the captured images by the region extracting unit; and the parallax amount calculated by the parallax amount calculating unit. And a distance measuring means for measuring the distance to the specific object.

  The program according to the present invention allows a computer to specify a specific object from each of captured images captured by a plurality of imaging units that capture different areas from different viewpoints and have different sensitivity wavelength bands. Based on a predetermined condition relating to the reflection intensity of each of the light wavelength bands, region extraction means for extracting a region representing the specific object, the region extraction unit extracted from each of the photographed images A parallax amount calculating unit that calculates a parallax amount in the region by performing alignment of the region, and a distance to the specific object is measured based on the parallax amount calculated by the parallax amount calculating unit. It is a program for functioning as a distance measuring means.

  According to the present invention, the predetermined area is imaged from different viewpoints by the plurality of imaging means. The specific object based on a predetermined condition relating to the reflection intensity of each of the wavelength bands of light for the specific object from each of the captured images captured by the plurality of image capturing means by the region extracting unit. An area representing is extracted.

  Then, the parallax amount calculation means performs alignment of the areas extracted from each of the captured images by the area extraction means, and calculates the parallax amount in the areas. A distance measuring unit measures a distance to the specific object based on the parallax amount calculated by the parallax amount calculating unit.

  In this way, based on the conditions relating to the reflection intensity of each of the wavelength bands of light for a specific object, an area representing the specific object is extracted from each of the captured images, and the area is aligned, By calculating the amount of parallax, it is possible to accurately measure the distance to a specific object using a plurality of imaging means having different wavelength characteristics.

  The distance measuring device according to the present invention represents the reflection intensity of each of the wavelength bands of light in the region representing the specific object based on the region extracted from each of the captured images by the region extraction means. It is possible to further include spectral data generating means for generating spectral data.

  Each of the imaging means according to the present invention can be sensitive to a plurality of light wavelength bands.

  The parallax amount calculation means according to the present invention aligns the regions so that the overlapping area when the regions extracted from each of the captured images are overlapped by the region extraction unit is maximized, The amount of parallax in the region can be calculated based on the region alignment result.

  As described above, according to the distance measuring device and the program of the present invention, a specific object is represented from each of the captured images based on the conditions regarding the reflection intensity of each of the light wavelength bands for the specific object. By extracting a region, aligning the region, and calculating the amount of parallax, it is possible to accurately measure the distance to a specific object using a plurality of imaging units having different wavelength characteristics. The effect of is obtained.

It is the schematic which shows the structure of the distance measuring device which concerns on the 1st Embodiment of this invention. It is a figure which shows the structure of an imaging device. It is a figure which shows the wavelength range which a 1st imaging device has a sensitivity. It is a figure which shows the wavelength band which a 2nd imaging device has a sensitivity. It is a figure which shows a mode that the 1st imaging device and the 2nd imaging device have been arrange | positioned. It is a figure which shows the example of a visible light image. It is a figure which shows the example of a near-infrared image. It is a figure which shows the result of having extracted the human skin area | region from the visible light image. It is a figure which shows the result of having extracted the human skin area | region from the near-infrared image. It is a figure which shows the edge information detected from a visible light image. It is a figure which shows the edge information detected from a near-infrared image. It is a figure which shows the result of having superimposed the human skin area | region extracted from the visible light image, and the human skin area | region extracted from the near-infrared image. It is a graph which shows the relationship between the amount of parallel movement of an image, and the human skin area | region superimposed. It is a figure which shows the superimposition area | region in the result of having aligned the human skin area | region extracted from the visible light image, and the human skin area | region extracted from the near-infrared image. (A) A graph showing spectral reflection characteristics of human skin, and (B) a graph showing spectral data. It is a flowchart which shows the content of the distance measurement process routine in the computer of the distance measurement apparatus which concerns on the 1st Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. An example will be described in which the present invention is applied to a distance measuring device mounted on a vehicle and measuring the distance to a person as an object and outputting spectral data of a human skin region.

  As shown in FIG. 1, the distance measuring device 10 according to the first embodiment is attached to a vehicle (not shown) and captures the front of the vehicle and generates an image. A computer 16 that measures the distance to the person and outputs spectral data of the area of the person based on the captured images obtained from each of the two imaging devices 14 and the first imaging device 12 and the second imaging device 14; And an output device 18 for outputting a processing result in the computer 16.

  Each of the first imaging device 12 and the second imaging device 14 is installed so as to image a target area in front of the vehicle from different viewpoints. As shown in FIG. 2, each of the first imaging device 12 and the second imaging device 14 separates incident light transmitted through the lens 20 with a prism 22. Narrow band-pass filters 24A, 24B, and 24C are deposited on the prism 22, and light of a predetermined wavelength passes through the band-pass filters 24A, 24B, and 24C, and three high-sensitivity imaging units 26A, 26B, and 26C. Is incident on. That is, the three high-sensitivity imaging units 26A, 26B, and 26C can acquire images with the same optical axis.

  Each of the high-sensitivity imaging units 26A, 26B, and 26C is configured by arranging imaging elements that constitute pixels, and generates an image signal.

  The band-pass filters 24A, 24B, and 24C of the first imaging device 12 are configured to transmit the three wavelength bands R, G, and B in the visible region as shown in FIG. 3, and the first imaging device 12 has the visible region. The picked-up image (visible light image) produced | generated from the image signal of these 3 wavelength bands R, G, B is output.

  The band-pass filters 24A, 24B, 24C of the second imaging device 14 are configured to transmit the near-infrared three-wavelength bands IR1, IR2, IR3 (IR1 <IR2 <IR3) as shown in FIG. The second imaging device 14 outputs a captured image (near infrared image) generated from the image signals of the three wavelength bands IR1, IR2, and IR3 in the near infrared region.

  In addition, each of the first imaging device 12 and the second imaging device 14 further includes an A / D converter (A / D converter) that converts an image signal that is an analog signal generated by the high-sensitivity imaging units 26A, 26B, and 26C into a digital signal. And an image memory (not shown) for temporarily storing the A / D converted image signal.

  As shown in FIG. 5, the first imaging device 12 and the second imaging device 14 are installed side by side, and a visible light image and a near-infrared image are acquired simultaneously. For example, a visible light image as shown in FIG. 6 is acquired by the first imaging device 12, and a near-infrared image as shown in FIG. 7 is acquired by the second imaging device 14. In the near-infrared image of FIG. 7, the IR1 filter is expressed using a pseudo color with the B component, the IR2 filter as the G component, and the IR3 filter as the R component.

  The computer 16 includes a CPU that controls the entire distance measuring apparatus 10, a ROM as a storage medium that stores a program for a distance measurement processing routine that will be described later, a RAM that temporarily stores data as a work area, and a bus that connects these. It is configured to include. In the case of such a configuration, a program for realizing the function of each component is stored in the ROM, and the CPU executes the program so that each function is realized.

  When the computer 16 is described with function blocks divided for each function realizing means determined based on hardware and software, as shown in FIG. 1, the visible light image captured by the first imaging device 12 and the second image An image acquisition unit 30 that acquires a near-infrared image captured by the imaging device 14, an object extraction unit 32 that extracts a region representing human skin from each of the visible light image and the near-infrared image, and the extracted Alignment of the skin area to calculate the parallax amount in the skin area, a distance measurement section 36 that measures the distance to the person based on the parallax amount, and spectral data of the skin area are generated And a spectral data generation unit 38. The object extraction unit 32 is an example of a region extraction unit, and the alignment unit 34 is an example of a parallax amount calculation unit.

  The object extraction unit 32 extracts the skin color space pixels in the YCrCb color space according to the conditional expression shown in the following expression (1) using the method of extracting the skin color region from the acquired visible light image, and FIG. The human skin region is extracted as shown in FIG. In FIG. 8, the white area is an area determined as human skin.

  Note that R is the R component of the target pixel, G is the G component of the target pixel, and B is the B component of the target pixel.

  Further, the object extraction unit 32 converts the acquired near-infrared image into light of 870 nm and 1050 nm according to the conditional expression shown in the following equation (2) using the characteristic that human skin absorbs light of 970 nm. On the other hand, a pixel having a low reflection intensity of 970 nm light is extracted, and a human skin region is extracted as shown in FIG. In FIG. 9, the white area is an area determined as human skin.

  However, IR1 is the IR1 component of the target pixel, IR2 is the IR2 component of the target pixel, and IR3 is the IR3 component of the target pixel.

  Next, the principle of performing region alignment will be described.

  The method of obtaining the distance data to the object by matching the positions of the objects represented by the images of the two cameras is called a stereo matching technique, and usually calculates the brightness of each image and the similarity of the edge information to 2 Find the correspondence of the object in one camera. However, in the configuration of the imaging device according to the present embodiment, the wavelength band of light having sensitivity differs in each imaging device, and thus images with different luminances are acquired as shown in FIGS. In such images with different luminances, it is difficult to obtain the similarity using luminance. 10 and 11 show images obtained by extracting edge information from the images shown in FIGS. Here, as in the case of luminance, since different edge information is acquired by the two imaging devices, it is difficult to calculate the degree of similarity using the edge information.

  Therefore, in the present embodiment, a material determination technique using spectral spectrum characteristics (reflection intensity characteristics with respect to wavelength) is applied to the visible light image and near-infrared image obtained by the object extraction unit 32. Then, the human skin region is extracted, and each of the visible light image and the near-infrared image is placed at a position where the overlapping region of the human skin region extracted by the object extraction unit 32 is maximized by the alignment unit 34. Is moved to align the skin region in the obtained visible light image and near-infrared image.

  Here, FIG. 12 shows an image obtained by superimposing the human skin and the extracted region from the images in FIGS. Since the two image pickup devices are installed in parallel, the optical axes of the image pickup devices are different from each other, and as shown in FIG. However, if the two imaging devices are calibrated in advance and only one image is moved in the horizontal direction so that the object can be aligned in the horizontal direction, several pixels are moved in the horizontal direction. Thus, the human skin region can be correctly expressed in the superimposed image at any position.

  In this embodiment, in order to determine that the position is the correct overlay position, the area of the human skin overlay region in the overlay image is used. FIG. 13 is a graph showing the relationship between the image movement amount in the horizontal direction and the area of the overlapping area of human skin in the superimposed image.

  As shown in FIG. 13 described above, the position where the area of the human skin overlapping area is maximized is the best overlapping position. In other words, the object (person) in the image can be aligned by analyzing the area of the human skin overlapping region while shifting the target image in the horizontal direction.

  FIG. 14 shows the result of superimposing images at a position where the area of the superposed region of human skin shown in FIGS. 8 and 9 is maximized.

  As described above, the alignment unit 34 moves the visible light image and the near-infrared image in parallel to the position where the region obtained by superimposing the human skin regions extracted by the object extraction unit 32 is maximized. Then, alignment is performed, and the amount of parallax in the human skin region is calculated based on the result of the alignment.

  The distance measuring unit 36 calculates the distance Zp to the person according to the following equation (3) based on the calculated parallax amount d in the human skin region, and outputs the distance measurement result by the output device 18 as a distance measurement result. .

  Here, B is the distance (base length) between the first imaging device 12 and the second imaging device 14, and f is the focal length of the first imaging device 12 and the second imaging device 14.

  The spectral data generation unit 38 generates spectral data in the superimposed human skin region based on the calculated amount of parallax and the human skin region extracted from each of the visible light image and the near-infrared image. . For example, spectral data as shown in FIG. 15B is generated for human skin having spectral reflection characteristics as shown in FIG. The spectral data is data representing the spectral reflection characteristics of the object, and each point in FIG. 15B indicates the reflection intensity in each wavelength band.

  Further, the spectral data generation unit 38 causes the output device 18 to output the generated spectral data.

  Next, the operation of the distance measuring apparatus 10 according to the present embodiment will be described. When a predetermined area in front of the vehicle is imaged by the first imaging device 12 and the second imaging device 14 while the vehicle equipped with the distance measuring device 10 is traveling, a distance measurement processing routine shown in FIG. Executed.

  In step 100, a visible light image captured by the first imaging device 12 and a near-infrared image captured by the second imaging device 14 are acquired. In the next step 102, pixels that satisfy the condition of the above expression (1) are extracted from the visible light image acquired in step 100 to extract a human skin region, and in step 104, the proximity image acquired in step 100 is extracted. From the infrared image, pixels satisfying the condition (2) are extracted to extract a human skin region.

  In step 106, the visible light image and the near-infrared image are translated in parallel so that the region obtained by superimposing the human skin regions extracted in steps 102 and 104 is maximized. Based on the result of the alignment, the amount of parallax in the human skin region is calculated.

  In the next step 108, based on the parallax amount calculated in step 106, the distance to the person to be measured is measured according to the above equation (3). In step 110, based on the parallax amount calculated in step 106 and the human skin area extracted in steps 102 and 104, spectral data of the human skin area superimposed is generated.

  In step 112, the distance to the target person measured in step 108 and the spectral data generated in step 110 are output by the output device 18, and the distance measurement processing routine ends.

  As described above, according to the distance measuring apparatus according to the first embodiment, two images captured from different viewpoints based on the conditions relating to the reflection intensity of each of the light wavelength bands of human skin. By extracting a human skin region from each of the images, aligning the human skin region, and calculating a parallax amount in the human skin region, using a plurality of imaging devices having different wavelength characteristics, The distance to the person can be measured with high accuracy.

  In addition, distance data is obtained by calculating the amount of parallax in a plurality of imaging devices installed at different positions, and target spectral data is obtained by using a plurality of imaging devices having sensitivity in different wavelength bands. By doing so, the target distance data and spectral data can be acquired simultaneously.

  Moreover, in order to achieve the alignment of the target object in a plurality of images without using the reference point, the target distance data and the spectral data can be obtained without additional devices other than the imaging device, such as projection of the reference point. Can be acquired at the same time.

  Next, a second embodiment will be described. In addition, about the part which becomes the structure similar to 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  The second embodiment is different from the first embodiment in that each imaging device is configured to have sensitivity in one wavelength band.

  The distance measuring device according to the second embodiment includes a plurality of imaging devices each having sensitivity in one different wavelength band. For example, as a plurality of imaging devices, there are bandpass filters that transmit an R wavelength band, an imaging device that captures an R component image, and a bandpass filter that transmits a G wavelength band, and an image of a G component An imaging device that picks up an image, an image pickup device that has a bandpass filter that transmits the B wavelength band, and that picks up an image of the G component, and an image pickup device that picks up an image of the IR1 component An imaging device that has a bandpass filter that transmits the IR2 wavelength band and captures an IR2 component image, and an imaging device that has a bandpass filter that transmits the IR3 wavelength band and captures an IR3 component image Is provided.

  Further, the range of values that human skin can take in each filter region is determined. Therefore, the object extraction unit 32 of the distance measurement device extracts a human skin region by performing simple threshold processing on luminance for each of the images captured by each imaging device. It is assumed that the white balance has been adjusted.

  The alignment unit 34 superimposes the skin regions extracted from the captured images, moves the captured images in parallel so that the area of the superimposed region is maximized, performs alignment, and performs human skin. For a region, the amount of parallax between captured images is calculated.

  The distance measuring unit 36 measures the distance to the person based on the calculated amount of parallax according to the above equation (3). The spectroscopic data generation unit 38 generates spectroscopic data for the superimposed skin regions.

  In addition, about the other structure and effect | action of the distance measuring apparatus which concern on 2nd Embodiment, since it is the same as that of 1st Embodiment, description is abbreviate | omitted.

  As described above, according to the distance measuring device according to the second embodiment, the distance to a person is accurately measured using a plurality of imaging devices having sensitivity in one different wavelength band, and the human skin is also measured. Spectral data of the area can be acquired.

  In the first embodiment and the second embodiment described above, the case where the parallax amount is obtained by moving the image to the position where the region where the human skin region is superimposed is maximized has been described. However, the present invention is not limited to this. For example, human skinness may be defined as one energy function, and the parallax amount may be obtained by moving the image to a position where the value of the energy function is maximized. Here, it can be seen that human skin has a high Cr value, a low Cb value, and a high value of (IR1-IR2) and (IR3-IR1). From such a tendency, an energy function E as shown in the following equation (4) is defined, and the image is moved to a position where the energy function E is maximized to obtain the amount of parallax in the human skin region. May be.

  Moreover, although the case where the specific target object is human skin has been described as an example, the present invention is not limited to this. For example, when a specific object is a tree, the tree has a stronger reflection intensity of light in the green wavelength band than in the blue or red wavelength band, and a near infrared (about 800 nm) in the red wavelength band. There is a feature that the reflection intensity of light is very strong. Therefore, a stereo camera is constructed with the first imaging device as an imaging device having sensitivity to blue and green light and the second imaging device as an imaging device having sensitivity to red and near-infrared light. That's fine. In addition, a region of a tree is extracted by extracting a pixel having a high intensity obtained by subtracting blue light from green from the captured image of the first imaging device, and red light from near infrared is extracted from the captured image of the second captured image. It is only necessary to extract a pixel having a very large intensity and extract a region of a tree that is a specific object to perform alignment.

  In addition, when there are multiple specific objects, distance data for each specific object is obtained by performing region extraction using material determination technology and alignment of the extracted regions for each specific object. And spectral data may be acquired.

  Further, the case where each unit of the image signal processing apparatus according to the present embodiment is realized by a computer has been described as an example. However, the present invention is not limited to this, and a plurality of computers that realize the function of each unit, or one or more The electronic circuit may be configured as follows.

  In the specification of the present application, the embodiment has been described in which the program is installed in advance. However, the program may be provided by being stored in a storage medium such as a CDROM.

DESCRIPTION OF SYMBOLS 10 Distance measuring device 12 1st imaging device 14 2nd imaging device 16 Computer 32 Object extraction part 34 Positioning part 36 Distance measuring part 38 Spectral data generation part

Claims (5)

A plurality of imaging means for imaging a predetermined region from different viewpoints, each having a different wavelength band of light having sensitivity;
An area representing the specific object is extracted from each of the picked-up images picked up by the plurality of image pickup means on the basis of a predetermined condition relating to the reflection intensity of each light wavelength band of the specific object. Region extracting means to perform,
Parallax amount calculating means for calculating the amount of parallax in the region by aligning the regions extracted from each of the captured images by the region extracting unit;
A distance measuring unit that measures a distance to the specific object based on the parallax amount calculated by the parallax amount calculating unit;
Distance measuring device including   Spectral data generation means for generating spectral data representing the reflection intensity of each of the wavelength bands of light in the area representing the specific object based on the area extracted from each of the photographed images by the area extraction means. The distance measuring device according to claim 1, further comprising:   The distance measuring apparatus according to claim 1, wherein each of the imaging units has sensitivity in a plurality of light wavelength bands.   The parallax amount calculating means aligns the areas so that the overlapping area is maximized when the areas extracted from each of the captured images by the area extracting means are overlapped, and the position of the areas The distance measuring device according to any one of claims 1 to 3, wherein a parallax amount in the region is calculated based on a matching result. Computer
Reflection of each of the wavelength bands of light for a specific object from each of the picked-up images picked up by a plurality of image pickup means for picking up a predetermined area from different viewpoints, each having a different light wavelength band having sensitivity. An area extracting means for extracting an area representing the specific object based on a predetermined condition relating to strength;
The regions extracted from each of the captured images by the region extraction unit are aligned, and a parallax amount calculation unit that calculates a parallax amount in the region, and the parallax amount calculated by the parallax amount calculation unit A program for functioning as a distance measuring means for measuring a distance to the specific object based on the program.