JP2012194872A - Passage detection device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a passage detection device capable of detecting a passage more accurately than conventional devices by increasing detection accuracy of a wheel rut.SOLUTION: Provided is a passage detection device 1 which detects a passage area included in an input image which is an image obtained by imaging a predetermined imaging area. The device includes: a central block extraction section 31 which extracts a central area of the passage area by subjecting the input image to predetermined image processing; a vegetation detection section 32 which detects whether or not an object to be imaged is a plant by using an infrared image obtained by imaging the imaging area; a central block correction section 33 which corrects a position of the central area extracted by the central block extraction section 31 based on the detection result by the vegetation detection section 32; and a passage area detection section 34 which detects the passage area by using as a reference the central area corrected in position by the central block correction section 33.

Description

  The present invention relates to a path detection device, a method, and a program for detecting a path such as a traveling path of a vehicle.

  Conventionally, image processing is performed on images taken with an in-vehicle camera, etc., and a traveling path is automatically detected based on a white line on a paved traveling path, and the driving state of the vehicle is automatically detected based on the detection result. Control technology has been developed. Also, in recent years, such technology has been developed, and not only paved travel paths, but also vehicles such as disaster rescue vehicles and construction vehicles that can run unmanned on rough roads and unpaved roads such as forest roads, disaster sites or construction sites. Is also being actively developed.

  Patent Document 1 below discloses an automatic steering device for a vehicle that detects a saddle on a road and runs while avoiding the saddle. This automatic steering device for a vehicle processes a captured image in front of the vehicle obtained by scanning the front of the vehicle with a laser beam to detect white lines and wrinkles on the road, and performs target steering based on the detection result of the white line and the detected vehicle speed. The angle is calculated, the target steering angle is corrected based on the detection result of the road tack, and the steering is driven and controlled so that the detected steering angle value becomes the corrected target steering angle.

  Further, in Patent Document 2 below, a traveling control method for an autonomous traveling body that autonomously travels, using an image captured by an imaging device, in a place where there is no clear road boundary, such as rough terrain and an unpaved road, etc. An example is disclosed. Specifically, the image captured by the imaging device mounted on the traveling body is divided into a plurality of areas, and the feature amount in the divided area obtained for each divided area is compared with a predetermined threshold value to be followed. A region is extracted, and steering control is performed by calculating a steering angle of the traveling body to go to the gazing point set on the tracking target region.

  Moreover, there are Patent Documents 3 to 5 as other techniques disclosing the background of the present invention.

Japanese Patent No. 35582449 JP 2002-132343 A Japanese Patent No. 2836949 JP 2011-8439 A JP 2009-289311 A

  Conventionally, wrinkles are detected based on the luminance of an image or a change in luminance of the image. For example, a region where the luminance of the image is equal to or less than a predetermined threshold is detected as a wrinkle, or a place where the change rate of luminance is large is regarded as a wrinkle edge and divided into a region that is a wrinkle and other regions. However, the method based on the luminance has a problem that it is impossible to detect a wrinkle whose brightness is almost the same as that of the surroundings and only the road surface roughness is different. In addition, the method for detecting a hail based on a change in luminance has a problem that it cannot cope with an unpaved road where the hail boundary is ambiguous due to, for example, grass.

  The present invention has been made in view of the above circumstances, and it is an object of the present invention to provide a path detection device, method, and program capable of detecting a path more accurately than in the past by increasing the detection accuracy of wrinkles. And

  In order to solve the above-described problems, the present invention is a passage detection device that detects a passage region included in an input image that is an image obtained by photographing a predetermined photographing region, and performs predetermined image processing on the input image. Extracting means for extracting the center area of the passage area, vegetation detecting means for detecting whether or not the photographing object is a plant using an infrared image obtained by photographing the photographing area, and detection by the vegetation detecting means Based on the result, correction means for correcting the position of the central area extracted by the extraction means, and passage area detection means for detecting the passage area on the basis of the central area whose position has been corrected by the correction means; It is characterized by providing.

  According to the present invention, based on the detection result of the vegetation detection unit that detects whether or not the photographing target is a plant using an infrared image, the position of the central region extracted by the extraction unit using image processing is It is corrected by the correcting means. Then, the passage area detection means detects the passage area with reference to the center area whose position is corrected by the correction means. Therefore, when a plant is present in the central area extracted by the extracting means, the position of the central area can be easily corrected to a position where the plant is relatively absent. Therefore, the passage area detection means can detect the passage area on the basis of the central area where plants are relatively absent. Therefore, it can suppress that the accuracy of passage detection falls by the influence by the existence of a plant, and can detect the unpaved road where the ridge and the boundary became ambiguous by the plant growing. .

It is a block diagram which shows the principal part structure of the channel | path detection apparatus by one Embodiment of this invention. It is a figure which shows an example of the vehicle-mounted camera. It is a figure which shows an example of the reflectance in a dry state. It is a figure which shows an example of the reflectance in a wet state. It is a flowchart which shows operation | movement of the plant detection part 32 of FIG. It is a figure which shows an example of the center block before correction | amendment. It is explanatory drawing regarding the correction process of a center block. It is a figure which shows an example at the time of correct | amending the center block shown in FIG. It is a figure which shows an example of the result of having detected the channel | path using the center block before correction | amendment shown in FIG. It is a figure which shows an example of the result of having detected the channel | path using the center block after correction | amendment shown in FIG. It is a figure which shows an example of distribution of a feature-value. It is a flowchart which shows the whole flow of detection operation | movement. It is a flowchart which shows the detail of the process which extracts a center block. It is a figure which shows an example of the input image obtained from a vehicle-mounted camera. It is explanatory drawing regarding the filtering process of the small area | region formation part 31a. It is explanatory drawing regarding the texture direction specific process of the small area formation part 31a. It is explanatory drawing regarding the voting process of the small area | region formation part 31a. It is explanatory drawing regarding the voting process of the small area | region formation part 31a. It is explanatory drawing regarding the voting process of the small area | region formation part 31a. It is explanatory drawing regarding the vanishing point specific process of the small area | region formation part 31a. It is explanatory drawing regarding the vanishing point specific process of the small area | region formation part 31a. It is explanatory drawing regarding the small region division | segmentation process of the small region formation part 31a. It is explanatory drawing regarding the small region division | segmentation process of the small region formation part 31a. It is explanatory drawing regarding the noise filtering process of the small area selection part 31b. It is explanatory drawing regarding the noise filtering process of the small area selection part 31b. It is explanatory drawing regarding the noise filtering process of the small area selection part 31b. It is explanatory drawing regarding the clustering process of the channel | path area estimation part 31c. It is explanatory drawing regarding the clustering process of the channel | path area estimation part 31c. It is explanatory drawing regarding the optimal cluster selection process of the channel | path area estimation part 30c. It is explanatory drawing regarding the optimal cluster selection process of the channel | path area estimation part 30c. It is a flowchart showing the centerline estimation process of the centerline estimation part 30d in the center block extraction part 31. It is explanatory drawing regarding the centerline estimation process of the centerline estimation part 31d. It is explanatory drawing regarding the centerline estimation process of the centerline estimation part 31d. It is explanatory drawing regarding the centerline estimation process of the centerline estimation part 31d. It is a figure which shows a mode that a center block is extracted from the estimated center line. It is a figure which shows an example of the correct | amended center block. It is a figure for demonstrating a reference | standard block. It is a figure which shows an example of the detected channel | path area | region.

  Hereinafter, a passage detection device, method, and program according to an embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a main configuration of a path detection device according to an embodiment of the present invention. As shown in FIG. 1, the path detection device 1 of this embodiment includes an A / D (analog / digital) converter 10, a storage device 20, an image processing device 30, and a RAM (Random Access Memory) 40. From the image signal input from, the edge of the unpaved road is detected as a passage area. 1 is realized by a computer such as a personal computer, for example, and is mounted on a special vehicle such as a disaster rescue vehicle or a construction vehicle, and is mounted on an in-vehicle camera (not shown) that captures the traveling direction of the vehicle.轍 is detected as a passage area from the input image signal.

  The A / D converter 10 converts an image signal input from a vehicle-mounted camera (not shown) into digital data, and outputs it to the image processing apparatus 30 as input image data indicating an input image. In this embodiment, it is assumed that an input image signal is supplied from the in-vehicle camera 100 as shown in FIG.

  2 includes a lens 111, half mirrors 112a and 112b, a water absorption wavelength band transmission filter 113, a lens 114, an InSb (Indium Antimonide) semiconductor light receiving element 115, a near-infrared transmission filter 116, a lens 117, and a CCD. (Charge-coupled device) A light receiving element 118, an infrared cutoff filter (that is, a near infrared cut filter) 119, a lens 120, and a CCD light receiving element 121 are provided. The half mirrors (or dichroic mirrors) 112a and 112b reflect a part of incident light and transmit a part thereof.

  Here, the wavelength of the water absorption wavelength band is a wavelength in a wavelength band that is easily absorbed by moisture, and is, for example, water in short wave infrared (Short Wave InfraRed, hereinafter referred to as “SWIR” as appropriate). This corresponds to the absorption wavelength (1450 nm ± 50 nm or 1940 nm ± 100 nm). The InSb semiconductor light receiving element 115 is an image sensor in which photodiodes are formed in an array, and converts shortwave infrared light into an electrical signal. The InSb semiconductor light receiving element 115 is incident with short-wave infrared light that is transmitted through the lens 111, reflected by the half mirror 112 a, further transmitted through the half mirror 112 b and the water absorption wavelength band transmission filter 113, and converged by the lens 114.

  On the other hand, the CCD light receiving element 118 is transmitted through the lens 111, reflected by the half mirror 112a, reflected by the half mirror 112b, further transmitted through the near infrared transmission filter 116, and converged by the lens 117 (Near). −visible Infra Red, hereinafter referred to as “NIR” as appropriate). As the near-infrared transmission filter 116, for example, a long pass filter that transmits only light having a wavelength of 800 nm or more can be used. In addition, as a wavelength of NIR used in the present embodiment, an arbitrary band of 800 nm to 1300 nm can be used. However, in this example, a band between the lower limit wavelength of 800 nm by the near-infrared transmission filter 116 and the upper limit of the light receiving wavelength of the CCD light receiving element, from 900 nm to about 1100 nm, is used.

  Further, visible light converged by the lens 120 is incident on the CCD light receiving element 121 through the lens 111, through the half mirror 112 a, and further through the infrared cutoff filter 120.

  In the present embodiment, as an example, the CCD light receiving element 121 has a visual field of about 60 ° and a resolution of VGA (Video Graphics Array; horizontal 640 pixels × vertical 480 pixels), and outputs a monochrome image signal. I will do it. In other words, the CCD light receiving element 121 outputs an image signal composed of luminance data indicating the luminance of each of the pixels of horizontal 640 × vertical 480. The image signal output from the CCD light receiving element 121 is a signal representing a visible light image, and is input to the central block extraction unit 31 via the A / D converter 10. However, when a digitized image signal is output from the in-vehicle camera 100, the A / D converter 10 can be omitted.

  On the other hand, the InSb semiconductor light receiving element 115 and the CCD light receiving element 118 also output image signals composed of luminance data having a visual field of about 60 ° and indicating the luminance of each of the horizontal 640 × vertical 480 pixels. Here, the image signal output from the InSb semiconductor light receiving element 115 is a signal representing a far-infrared image, and is input to the vegetation detection unit 32 via the A / D converter 10. The image signal output from the CCD light receiving element 118 is a signal representing a near-infrared image, and is input to the vegetation detection unit 32 via the A / D converter 10.

  The in-vehicle camera 100 shown in FIG. 2 includes a half mirror or the like in a single camera and can output a plurality of types of image signals including a visible light image and an infrared image by performing spectroscopy with the half mirror or the like. It is a thing. However, the image signal input to the passage detection apparatus of the present invention is not limited to one captured with a single camera, but may be one captured with a plurality of cameras. That is, a configuration in which a plurality of cameras are used and the plurality of cameras capture images of different frequency bands in the same direction can be used.

  In the following description, it is assumed that the fields of view of the InSb semiconductor light receiving element 115, the CCD light receiving element 118, and the CCD light receiving element 121 are the same for the sake of simplicity. That is, pixels having the same coordinates (that is, pixels corresponding to each other in the horizontal 640 pixels × vertical 480 pixels) in the imaging elements of the InSb semiconductor light receiving element 115, the CCD light receiving element 118, and the CCD light receiving element 121 photograph the same photographing object ( That is, it is assumed that light reflected by the same subject is photoelectrically converted. However, in actuality, since the field of view is different for each lens and the pixel interval may be different between the InSb semiconductor light-receiving element and the CCD light-receiving element, the coordinates for each pixel that captures the same image are coordinated. In some cases, conversion or interpolation processing for correcting a difference in pixel interval may be required.

  The storage device 20 is realized by, for example, a hard disk, and stores a passage detection program PG, a plurality of types of Gabor filters F1 to F30 (that is, data representing parameters of the Gabor filters F1 to F30), and a vegetation determination threshold value TH. The storage device 20 outputs the passage detection program PG, the Gabor filters F1 to F30, and the vegetation determination threshold value TH to the image processing device 30 in response to a read request from the image processing device 30.

  Here, the passage detection program PG is a program for realizing, in the passage detection device 1, processing for detecting a pavement on an unpaved road as a passage area from input image data. The passage detection program PG is read and executed by the image processing device 30, whereby the central block extraction unit 31, the vegetation detection unit 32, the central block correction unit 33, and the passage region detection unit 34 in the image processing device 30. Each function is realized.

Gabor filters F1 to F30 are spatial frequency filters using a two-dimensional Gabor function obtained by multiplying a two-dimensional Gaussian function and a sine wave function propagating in one direction on a two-dimensional plane, and are two-dimensional wavelet filters. It is one form. A two-dimensional Gabor function GB (x, y) in a two-dimensional orthogonal coordinate system composed of an X axis and a Y axis orthogonal to each other is expressed by the following equation (1). In the following equation (1), u ₀ is a parameter indicating the angular frequency of the wave, and σ is a standard deviation of the Gaussian function (gauss window width). The two-dimensional Gabor function GB (x, y) has a declination φ of an angular frequency u ₀ as a parameter (direction parameter) indicating a wave direction.

By filtering the input image data using a Gabor filter using the above two-dimensional Gabor function GB (x, y), the frequency characteristics and texture (pattern) direction characteristics of the input image can be extracted. Specifically, in the input image, a region having a texture direction parallel to the direction parameter (deflection angle φ) and having frequency characteristics close to the frequency parameter (angular frequency u ₀ ) is extracted with high sensitivity.

In the present embodiment, 30 types of Gabor filters F1 to F30 having different directional parameters (deflection angle φ of the angular frequency u ₀ ) and frequency parameters (angular frequency u ₀ ) are stored in the storage device 20. Here, the frequency parameter is set in accordance with the width of the eyelid to be extracted. For example, the angular frequency u ₀ is set so that the wave wavelength (period) is about 1.5 to 4 degrees (25 to 50 cm) on the visual field. Such setting facilitates extraction of soot with high sensitivity. In the present embodiment, an example in which 30 types of Gabor filters F1 to F30 are stored in the storage device 20 will be described, but the number and type thereof depend on the resolution of the input image and the processing capability of the image processing device 30. It can be changed as appropriate.

  The image processing device 30 is, for example, a CPU (Central Processing Unit), and performs predetermined image processing on input image data in accordance with a passage detection program PG read from the storage device 20, thereby passing a coral on an unpaved road. Detect as a region. In the image processing apparatus 30, the passage detection program PG is executed to thereby execute a center block extraction unit 31 (extraction unit), a vegetation detection unit 32 (vegetation detection unit), a center block correction unit 33 (correction unit), and a passage. Each function of the area detection unit 34 (detection means) is realized.

  The center block extracting unit 31 includes a small region forming unit 31a, a small region selecting unit 31b, a passage region estimating unit 31c, and a center line estimating unit 31d, and the visible light image data output from the A / D converter 10 is obtained. The block is divided into a plurality of blocks, and one or a plurality of blocks (center blocks) indicating the approximate position of the coral as a passage area are extracted. Note that this central block can also be said to be a block that may contain a wrinkle image.

  The small area forming unit 31a has a plurality of small candidate areas given path candidate parameters indicating the possibility of being part of the path area as path estimation information necessary for estimating the path area included in the input image. Regions are formed in the input image. Specifically, by performing filtering processing, texture (pattern) direction specifying processing, strong edge region extraction processing, voting processing, vanishing point specifying processing, and small region dividing processing described later on the input image data, The small area is formed in the input image.

  Here, the filtering process refers to each direction (φ1 to φ30) for the path detection target area by filtering the path detection target area in the input image using a plurality of Gabor filters F1 to F30 read from the storage device 20. It is the process which produces | generates intensity distribution data. The texture direction specifying process is a process of specifying the texture direction of each pixel in the path detection target area based on the intensity distribution data generated by the filtering process. The strong edge area extraction process is a process of extracting a strong edge area existing in the path detection target area by performing an edge detection process of the path detection target area in the input image.

  In the voting process, for each divided area obtained by dividing the path detection target area in the input image along the depth direction, each pixel in the divided area is selected as a voting source pixel, and the voting source pixel is selected. This is a process of voting on a pixel that exists on a straight line parallel to the texture direction as a starting point, or on a pixel included in a voting target area that is set along the texture direction starting from a voting source pixel. The vanishing point specifying process is a process of specifying, for each divided area, a pixel having the largest number of votes as a vanishing point based on the voting result of the voting process.

  The small area dividing process is to divide the area having the vanishing point specified in the vanishing point specifying process into a plurality of small areas for each divided area, and for each small area, Is a process of giving a path candidate parameter determined based on the ratio of the voting source pixels of vanishing points included in the. The details of the filtering process, the texture direction specifying process, the strong edge area extracting process, the voting process, the vanishing point specifying process, and the small area dividing process will be described later.

  Based on the path candidate parameters given to each of the plurality of small areas formed by the small area forming section 31a, the small area selection unit 31b selects these small areas as path candidate areas and other (noise components). Are selected as non-passage candidate areas. Specifically, the small area selection unit 31b sets a small area that is likely to be a part of the path area as a path candidate area based on the path candidate parameter given to each of the plurality of small areas. A small area that is unlikely to be a part of the area is set as a non-passage candidate area, while a non-passage candidate area that is partially or wholly surrounded by the path candidate area is increased by increasing the value of the path candidate parameter. Change the path candidate area to a path candidate area, and change the path candidate area to a non-path candidate area by lowering the value of the path candidate parameter of the path candidate area existing at a position isolated from other path candidate areas. To do. The details of the small area sorting process (noise filtering process) by the small area sorting unit 31b will be described later.

  The passage area estimation unit 31c inputs, as an input image, a region group that seems to be the most passage among area groups obtained by grouping adjacent passage candidate areas based on the small area selection result by the small area selection unit 31b. Is selected as a passage area included in. Specifically, the passage area estimation unit 31c sets the value of the evaluation function f (F, H, C, A) represented by the following equation (2) for each of the area groups obtained by grouping the passage candidate areas. And a group of regions having the largest value is selected as a passage region included in the input image.

However, the first variable F in the equation (2) is a variable indicating whether or not the region group includes a region on the near side in the depth direction in the input image, and the second variable H is the depth direction of the region group. The third variable C is a variable indicating the number of path candidate areas having the largest path candidate parameter value included in the area group, and the fourth variable A is a variable indicating the total area of the area group. It is. In addition, the first weight coefficient k ₁ to the fourth weight coefficient k ₄ in the equation (2) are coefficients that are multiplied by the first variable F to the fourth variable A, respectively. The details of the passage area estimation processing by the passage area estimation unit 31c will be described later.

  The center line estimation unit 31d has a function of estimating the center line of the passage region selected by the passage region estimation unit 31c. Specifically, the centerline estimation unit 31d obtains a centroid for each of the divided areas obtained by dividing the input image into a plurality along the depth direction, and each divided area starts from the centroid of the nearest divided area. A temporary center line is created by drawing a straight line toward the vanishing point every time, and the sum of squares of the distances between the temporary center line and each center of gravity is calculated. Further, a plurality of temporary centerlines are created while shifting the starting point along the horizontal direction of the input image, and the sum of squares of the distances between the respective center of gravity for each of the plurality of temporary centerlines is calculated. The temporary center line with the smallest sum of multiplication values is selected as the center line of the passage area. Then, the block that is selected and passes through the center line is set as the center block. The details of the center line estimation processing by the center line estimation unit 31d will be described later.

  The vegetation detection unit 32 includes a reflectance ratio calculation unit 32a and a determination unit 32b. Based on the infrared image data output from the A / D converter 10, each pixel constituting the infrared image is a vegetation region (or a single unit). It is detected whether or not the light reflected by the plant is taken.

  The reflectance ratio calculation unit 32a, the pixel value of each pixel obtained by converting the outputs of the InSb semiconductor light receiving element 115 and the CCD light receiving element 118 using the A / D converter 10, the exposure time at the time of shooting, the gain, and the like Based on this data, the ratio (reflectance ratio) between the reflectance in the water absorption wavelength band and the reflectance in the near-infrared band is calculated.

  The determination unit 32b compares the reflectance ratio calculated by the reflectance ratio calculation unit 32a with the vegetation determination threshold value TH stored in the storage unit 20, and determines whether the observation target is a plant for each pixel. . About the reflectance ratio in short wave infrared (SWIR) and near infrared (NIR), as shown in FIG.3 and FIG.4, it is clear about the value which a plant leaf has by each material, and the value which others have There is a difference. Therefore, the determination part 32b determines whether it is vegetation using this property.

  FIG. 5 is a diagram showing a flow of processing in the vegetation detection unit 32. As shown in FIG. 5, the vegetation detection unit 32 includes (1) a reflectance calculated based on image data related to the SWIR wavelength band and a reflectance calculated based on image data related to the NIR wavelength band. The ratio (reflectance ratio) is calculated (step S41), and (2) it is determined whether or not the observation object is a plant by comparing the reflectance ratio with a predetermined threshold (step S42).

  Note that the vegetation determination threshold value TH is, for example, a value between 2.0 and 3.0 when it is fine, and a value between 2.5 and 4.0 when it is raining. A value between 2.5 and 3.0 can be set in common.

  The center block correcting unit 33 includes a vegetation ratio calculating unit 33a and a center block changing unit 33b, and corrects the position of the center block extracted by the center block extracting unit 31 based on the output of the vegetation detecting unit 32.

  The vegetation ratio calculation unit 33a calculates the ratio of plants in each of the blocks around the center block (for example, blocks in a horizontal row).

  Based on the calculation result of the plant ratio calculated by the vegetation ratio calculating unit 33a, the center block changing unit 33b sets the block having the smallest plant ratio as the “corrected center block”.

  Here, with reference to FIG. 6 to FIG. 8, the processing content by the central block correction unit 33 will be described with a specific example. FIG. 6 is a diagram illustrating an example of the image 50 that is processed by the central block extraction unit 31 and an example of the center line 51 and the central block 52 that are extracted from the image 50 by the central block extraction unit 31. Regions 53 and 54 are surrounding (left and right) regions that are targets for calculating the proportion of plants with the central block 52 as the center. In this example, as shown in FIG. 7, the left and right 5 blocks (blocks 531 to 535 and blocks 541 to 545) of the central block 52 are to be calculated.

  Based on the result of determination by the vegetation detection unit 32, the ratio of plants is calculated by adding the number of pixels determined to be plants by the vegetation ratio calculation unit 33a for each block. For example, for the central block 52 in FIG. 7, a plurality of pixels constituting each of the central block 52, blocks 531 to 535 and blocks 541 to 545 (n × n if each block is n pixels × n pixels) Of these, the number of pixels determined to be a plant is calculated for each block. Then, the central block changing unit 33b selects a block having the smallest number of pixels determined to be a plant among the central block 52, the blocks 531 to 535, and the blocks 541 to 545 as a new central block 52 ( (See FIG. 8). In the center block correction unit 33, correction processing is performed on each center block extracted by the center block extraction unit 31.

  9 and 10 are diagrams illustrating an example of a result when the passage area 55 is detected by the passage area detection unit 34 for the images illustrated in FIGS. 6 and 8. When the correction process of the central block is not performed (in the case of FIG. 9), the grassland portion is the central block, and thus the passage area 55 is extracted in the vegetation area having the same feature amount as that block. . On the other hand, when the correction process of the center block 52 is performed (in the case of FIG. 10), the center block 52 is moved from the grassland part to the part without the plant, so that the passage area 55 is an area other than the vegetation area ( In this example, it can be extracted in the cocoon area.

  The passage area detector 34 includes a first calculator 34a (first calculator), a second calculator 34b (second calculator), a weight determiner 34c (weight determiner), and an extractor 34d (block extractor). The passage area is detected by extracting an area (block) having the same feature quantity as the feature quantity of the central block from the input image. Specifically, a block having the same feature quantity as the feature quantity of the central block is extracted by performing weighting of the feature quantity as described below.

That is, as the feature amount used by the passage area detection unit 34, for example, the following characteristics can be used.
(A) Average value of luminance: 1 type (b) Standard deviation of luminance: 1 type (c) Feature amount obtained by Gabor filter: 30 types

  The first calculation unit 34a uses the center block corrected by the center block correction unit 33 and a block adjacent to the center block in the left-right direction as a reference block, and uses the average value and standard deviation of the feature values of the reference block as features. Calculate for each type of quantity. The second calculation unit 34b obtains an average value of feature values for all blocks for each type of feature value. When the second calculation unit 34b calculates the average value, it is not always necessary to use all the blocks. For example, it is desirable to exclude blocks containing an image that is not involved in the extraction of wrinkles at all (for example, an image above the horizon) and obtain an average value of feature values for the remaining blocks.

  The weight determination unit 34c applies a weight to a feature amount having a large difference between the average value for the reference block obtained by the first calculation unit 34a and the average value for all of the plurality of blocks obtained by the second calculation unit 34b. The weight for each feature quantity is determined so that becomes heavy. Further, the weight for each feature amount is determined so that the feature amount having a small standard deviation for the reference block obtained by the first calculation unit 34a becomes heavy.

Here, the average value and standard deviation of the i-th feature amount (i is an integer satisfying 1 ≦ i ≦ 37) for the reference block obtained by the first calculation unit 34a are μ ^ _i and σ ^ _i , respectively. 2 The average value of the feature values for all the plurality of blocks obtained by the calculation unit 34b is μ _i, and a, b, and c are arbitrary constants. For convenience of description, the symbol “μ ^” means that a hat symbol “^” is added above the character “μ”. Similarly, the symbol “σ ^” means that the hat symbol “^” is added to the upper part of the character “σ”. The weight determination unit 34c determines the weight w ′ _i for each i-th feature amount using the following equation (3).

  Referring to the above expression (3), the first term on the right side represents a bias with respect to the entire feature quantity in the reference block, and the second term on the right side represents the spread of the feature quantity in the reference block. When the first term on the right side is large and the second term on the right side is large, the value of the feature amount differs greatly between the reference block and the other blocks, but becomes a close value between the reference blocks. For this reason, since such a feature amount is considered to well represent a feature of a pavement on an unpaved road, the weight is increased.

Further, referring to the above equation (3), the weight w ′ _i may take a negative value. When the obtained weight w ′ _i is a negative value, the weight determination unit 34c determines the value to be zero. In addition, for any image, a feature amount that is nearly zero in weight is subjected to a deletion process in order to reduce the time required for the process while maintaining accuracy. Furthermore, the weight determination unit 34c obtains a weight w _i normalized using the following equation (4).

  In addition, as shown in FIG. 11, the distribution of the feature amount of each block is often not a symmetric but a biased distribution with respect to the average value μ. FIG. 11 is a diagram illustrating an example of the distribution of feature amounts. Specifically, the distribution of the feature amount is wide in a region where the value is smaller than the average value μ, and μ + 2σ in a region where the value is larger than the average value μ, as shown by the curve with the symbol D in FIG. The distribution is concentrated in the range of the degree.

For this reason, the weight determination unit 34c also obtains a normalization coefficient s _i for normalizing each feature amount using the following equation (5).

Note that when the feature amount without distribution bias is used, or when the distribution of the feature amount is accurately known when the number of samples of the reference block is large, the normalization coefficient s _i does not necessarily have to be obtained. Can be used.

The extraction unit 34d extracts a pavement of an unpaved road included in the image using the feature amount weighted by the weight determination unit 34c. Specifically, assuming that the value of the i-th feature amount in the block to be processed is f _i and the value of the i-th feature amount in the central block is f _ci , the extraction unit 34d represents the following equation (6): When the value of the evaluation function E is larger than the predetermined value T, it is evaluated that a wrinkle image is included in the processing target block, and the block is extracted. Note that the predetermined value T is appropriately set according to the values of the variables a, b, and c in the above-described equation (3) and the properties of the eyelid image.

  The RAM 40 is a rewritable volatile memory used to temporarily store data necessary for the image processing apparatus 30 to execute various image processes and the input image data as necessary.

  Next, the operation of the passage detection apparatus 1 having the above configuration will be described. Below, the case where the channel | path detection apparatus 1 performs the detection operation | movement which extracts a bag from the image obtained from the vehicle-mounted camera 100, and detects a channel | path area | region is demonstrated. In the detection operation, it is assumed that a vehicle (not shown) on which the in-vehicle camera 100 and the path detection device 1 are mounted is traveling on an unpaved road, and an input image in which soot is reflected is obtained from the in-vehicle camera 100.

  FIG. 12 is a flowchart showing the overall flow of the detection operation. Note that the flowchart shown in FIG. 12 is repeatedly executed at predetermined time intervals while the vehicle is running. When the detection operation is started, the input image is input from the in-vehicle camera 100 to the A / D converter 10 and converted into digital data, and is output to the image processing apparatus 30 as input image data (step S11: image input step). . And the process which extracts the center block which shows the approximate position of the eyelid as a channel | path area | region is performed first (step S12: extraction step). FIG. 13 is a flowchart showing details of the process of extracting the center block, and FIG. 14 is a diagram showing an example of an input image obtained from the in-vehicle camera 100. Here, as shown in FIG. 14A, it is assumed that an input image in which wrinkles are reflected is obtained.

  In the following, as shown in FIG. 14B, the horizontal direction of the input image is the X-axis direction, the vertical direction is the Y-axis direction, and the coordinates (X, Y) = (0, 0) to (639) of the input image. , 479) are represented as B (0, 0) to B (639, 479). That is, the input image data is a set of these luminance data B (0, 0) to B (639, 479). Such input image data is temporarily stored in the RAM 40 by the image processing apparatus 30.

  When the processing shown in FIG. 13 is started, first, the small area forming unit 31a provided in the central block extracting unit 31 of the image processing apparatus 30 reads input image data corresponding to the path detection target area in the input image from the RAM 40. At the same time, the Gabor filters F1 to F30 are read from the storage device 20, and the input image data is filtered using the Gabor filters F1 to F30, thereby generating intensity distribution data in each direction (step S21: filtering process).

  In the present embodiment, a case where the entire area of the input image is set as the path detection target area will be described as an example. That is, input image data corresponding to the entire area of the input image is read from the RAM 40. When a horizontal line can be extracted from the input image, an area below the horizontal line (that is, a ground area) may be set as a path detection target area. In this way, by setting only a region where a passage is surely present as a passage detection target region, it is possible to reduce the load of image processing described below, and to shorten the processing time.

  FIG. 15 is an explanatory diagram regarding the filtering process of the small region forming unit 31a. As illustrated in FIG. 15A, the small region forming unit 31 a performs the filtering process using the Gabor filters F <b> 1 to F <b> 30 as the luminance data of each pixel included in the convolution region having a certain pixel as the central pixel in the input image. Then, a convolution calculation with a Gabor filter, that is, a two-dimensional Gabor function GB (x, y) is performed, and the calculation result is acquired as the intensity of the central pixel corresponding to the direction parameter φ of the Gabor filter.

  That is, first, the small region forming unit 31a performs a convolution calculation between the luminance data of each pixel included in the convolution region having a certain pixel as a central pixel and the Gabor filter F1, and the calculation result is used as the direction parameter φ1 of the Gabor filter F1. Is obtained as the intensity of the center pixel corresponding to. Here, by creating each Gabor filter F1 to F30 separately for the real part filter and the imaginary part filter, the intensity obtained using the real part filter and the imaginary part filter are used. The final intensity can be obtained by squaring and adding the obtained intensities and obtaining the square root of the added value.

  Subsequently, the small region forming unit 31a switches to the Gabor filter F2 and similarly performs a convolution operation, and acquires the calculation result as the intensity of the central pixel corresponding to the direction parameter φ2 of the Gabor filter F2. Hereinafter, similarly, the small region forming unit 31a performs the convolution calculation while sequentially switching the Gabor filters, and acquires the calculation result as the intensity of the central pixel corresponding to the direction parameters φ1 to φn of the Gabor filters F1 to F30. To do.

  The small region forming unit 31a performs the filtering process as described above for each pixel of the coordinates (0, 0) to (639, 479) of the input image, whereby each small region forming unit 31a as illustrated in FIG. Direction intensity distribution data is generated and stored in the RAM 40. In FIG. 15B, intensity data obtained with each pixel at coordinates (0, 0) to (639, 479) as the central pixel is represented by I (0, 0) to I (639, 479). It is written.

  As can be seen from FIG. 15A, when the pixel close to the end of the input image is used as the central pixel, the convolution region protrudes from the input image. In this case, preset brightness data (for example, white data) Etc.) may be used as the luminance data of the convolutional region protruding from the input image. In addition, the size of the convolution region may be changed as appropriate according to the size of the passage to be detected. For example, when a heel corresponding to a wheel mark having a width of 25 cm to 50 cm is set as a detection target, the width is set to the width. It is desirable that the size is equal to or larger than the corresponding number of pixels (32 pixels), for example, 50 × 50 pixels.

  When the processing in step S21 described above is completed, the small region forming unit 31a reads the intensity distribution data in each direction from the RAM 40, and specifies the texture direction of each pixel in the input image based on the intensity distribution data (step S22). : Texture direction specifying process). Specifically, for example, when focusing on a pixel at coordinates (0, 0), a search is made for the largest intensity data I (0, 0) from the intensity distribution data in each direction, and the maximum intensity data I is obtained. The direction corresponding to the intensity distribution data including (0, 0) is specified as the texture direction of the pixel at coordinates (0, 0). For example, when the intensity distribution data corresponding to the direction φ3 includes the maximum intensity data I (0,0), the direction φ3 is the texture direction of the pixel at the coordinates (0,0).

  The small region forming unit 31a performs the texture direction specifying process as described above for each pixel at coordinates (0, 0) to (639, 479), thereby specifying the texture direction of each pixel, and the specifying result. Is stored in the RAM 40. FIG. 16 is an explanatory diagram regarding the texture direction specifying process of the small region forming unit 31a, where (a) shows the result of specifying the texture direction of each pixel, and (b) shows the input image and the result of specifying the texture direction. It is displayed in a superimposed manner. In FIG. 16, for the sake of convenience of description, the texture direction specification result is represented by a vector line indicating the texture direction. However, in actuality, the texture direction specification result is a value indicating the direction (for example, the texture direction is If φ1, the value of φ1).

  A passage such as a kite that exists on uneven ground is left as a result of a vehicle traveling around the ground several times, and the ground and the grass are stomped or solidified. As shown, the trace is observed as a discontinuous and unclear band-like or streak-like sparse region on the input image. Therefore, by performing the processing of step S21 and step S22 for extracting the wave component of the input image described above, the interval between the patterns forming the passage is indefinite and gentle, but the wave parallel to the traveling direction of the passage is gentle. Can be captured as an ingredient.

  When the process of step S22 described above is completed, the small area forming unit 31a performs a predetermined edge detection process based on the input image data, thereby extracting a strong edge area existing in the input image, and obtaining the extraction result. It memorize | stores in RAM40 (step S23: strong edge area | region extraction process). Here, a well-known technique can be adopted as a technique for extracting a strong edge region. For example, by applying a Sobel filter in the X-axis direction and a Sobel filter in the Y-axis direction, a pixel whose sum of squares of the intensity exceeds a certain threshold is searched, and a pixel within a certain distance from the pixel exceeding the threshold is searched. Extract the region as a strong edge region.

  When the process of step S22 is completed, the small region forming unit 31a sequentially selects each pixel in the input image as a voting source pixel, and exists on a straight line that starts from the voting source pixel and is parallel to the texture direction. Voting is performed on the pixels (step S24: voting process). 17-19 is explanatory drawing regarding the voting process of the small area formation part 31a. First, as shown in FIG. 17, the small area forming unit 31 a sets the vote count area for counting the number of votes in the memory space of the RAM 40 to the same size as the input image (that is, the number of votes corresponding to the number of 640 × 480 pixels). To ensure the total size).

  Then, as illustrated in FIG. 18A, when the small region forming unit 31 a selects a certain pixel as the voting source pixel, the small region forming unit 31 a grasps the texture direction of the voting source pixel from the result of specifying the texture direction of each pixel, and Voting is performed on pixels (voting target pixels) that exist on a straight line parallel to the texture direction from the pixel as a starting point, and the voting result is reflected in the vote counting area on the RAM 40 as shown in FIG. That is, as can be seen from FIG. 18B, the vote value “1” is added to the storage area corresponding to the vote target pixel in the vote count area on the RAM 40.

  The small area forming unit 31a performs the voting process as described above for each pixel, and sequentially reflects the voting result in the vote counting area on the RAM 40. Here, when performing the voting process as described above, the small area forming unit 31a determines pixels included in the periphery (for example, within two pixels) of the strong edge area extracted by the strong edge area extraction process as voting source pixels. Exclude from The reason for this will be described later.

  In the voting process described above, a case has been described in which voting is performed on a voting target pixel that exists on a straight line parallel to the texture direction from the voting source pixel. However, the present invention is not limited to this. Alternatively, voting may be performed on pixels (voting target pixels) included in the voting target area set along the texture direction. For example, as shown in FIG. 19 (a), the voting target area may be a belt-like (rectangular) shape parallel to the texture direction, or an arc shape or a triangular shape as shown in FIG. 19 (b). May be good.

  Here, when setting a belt-like voting target area as shown in FIG. 19A, as in the case of voting on the voting target pixel existing on a straight line parallel to the texture direction, What is necessary is just to vote equally with respect to each of the voting target pixels included in the voting target area. That is, the vote value “1” is equally added to the storage area corresponding to each vote target pixel in the vote total area on the RAM 40.

  On the other hand, when setting an arc-shaped or triangular voting target area as shown in FIG. 19B, the voting value is changed according to the distance between the voting source pixel and the voting target pixel. Is desirable. Specifically, the voting value of the voting target pixel located at a distance far from the voting source pixel is reduced (for example, voting value = 1 / distance from the voting source pixel). That is, the voting value of the voting target pixel at a position close to the voting source pixel is “1” or a value close to “1”, but the voting value of the voting target pixel at a position far from the voting source pixel is “0.5” or the like. The value after the decimal point. Furthermore, it is desirable to determine the voting value of each voting target pixel so that the total voting value of the voting target pixels located at the same distance in the voting target region is “1”. That is, for example, assuming that there are four voting target pixels located at the same distance, the voting value for each of the four voting target pixels is set to “0.25”.

  When the voting target area having an arc shape or a triangular shape is set in this way, the reason for changing the voting value according to the distance between the voting source pixel and the voting target pixel is that the voting is located far from the voting source pixel. This is because the target pixel has a high probability of overlapping with other voting target areas and tends to be voted more easily than the voting target pixel located at a close position, and it is difficult to accurately specify a vanishing point described later. Further, when the voting target area having an arc shape or a triangular shape is set, it is desirable that the center angle (the angle starting from the voting source pixel) is the same as the change in the direction parameter φ of the Gabor filters F1 to F30. .

  Subsequently, as shown in FIG. 20, the small area forming unit 31 a selects the pixel having the largest vote number based on the vote result by the voting process (that is, the vote count counted in the vote count area on the RAM 40). The vanishing point is identified (step S25: vanishing point identifying process). 20 and 21 are explanatory diagrams regarding the vanishing point specifying process of the small region forming unit 31a. The wave pattern of the corridor and the like is not completely parallel when viewed from a vehicle or the like, but is presumed to share the same vanishing point. For this reason, the vanishing point is identified by voting to the position (pixel) that is estimated to be the vanishing point by the voting process as described above, and utilizing the detection result that has statistically many wave patterns parallel to the passage. can do.

  Here, in the input image, if a high contrast object such as a tree, a power pole, or a signboard is reflected in addition to the passage, these patterns have a great influence on the total number of votes. Therefore, as described above, when performing the voting process, the pixels included in the strong edge area extracted by the strong edge area extraction process are excluded from the voting source pixels, so that the contrast of trees, utility poles, signs, etc. It is possible to prevent the extension line of an object having a high height from being identified as a vanishing point.

  By the way, as shown in FIG. 20, in the case where a path such as a wrinkle present in the input image extends in a straight line toward the horizontal line, the vanishing point is specified at one place. When the path existing in the input image is curved, the specific location of the vanishing point changes along the depth direction of the input image. In such a case, if the entire area of the input image (the entire area of the path detection target area) is processed at once, an accurate vanishing point cannot be obtained.

  Therefore, the above-described voting process and vanishing point specifying process are obtained by dividing the passage area detection target area (here, the entire area of the input image) into a plurality (M) along the depth direction as shown in FIG. It is desirable to carry out every rectangular divided area. 22 and 23 are explanatory diagrams regarding the small area dividing process of the small area forming unit 31a. Hereinafter, this divided area is referred to as a slice, and the symbols of the slices from the lowest level to the highest level of the input image are SL0 to SLM. That is, the vanishing point is obtained for each of the slices SL0 to SLM by performing the voting process and the vanishing point specifying process for each of the slices SL0 to SLM. In FIG. 22, the vanishing point P0 of the slice SL0 and the vanishing point P1 of the slice SL1 are representatively illustrated.

  Subsequently, as illustrated in FIG. 22, the small region forming unit 31 a surrounded each of the slices SL <b> 0 to SLM with a plurality of straight lines drawn radially from the respective vanishing points and the boundary lines of the slices. Each of the small areas is divided into a plurality (N) of small areas, and the path candidate parameters determined based on the ratio of the voting source pixels of the vanishing points included in the small areas are given (step S26: small) Region division processing). Since all the paths such as wrinkles that exist in each of the slices SL0 to SLM are represented by straight lines toward the vanishing point, it is estimated that a path exists in one of the small areas.

The path candidate parameter is one of path estimation information necessary for estimating the path area included in the input image, and is a parameter indicating a high possibility of being a part of the path area. As the path estimation information set in each of the small areas, there are the following five examples.
(1) Y direction (vertical direction) index Idy of the small area
(2) X direction (lateral direction) index Idx of the small area
(3) Vanishing point coordinates of the small region (4) Passage detection intensity of the small region (passage candidate parameter: any one of 0, 1, 2)
(5) Upper and lower end coordinates of the small area

  Here, the Y-direction index Idy of the small area is a parameter indicating the position of the small area in the Y direction. For example, as shown in FIG. 22, the Y-direction index Idy of all the small areas in the slice SL0 is “0”. The X direction index Idx of the small area is a parameter indicating the position of the small area in the X direction. For example, as shown in FIG. 22, the X direction index Idx of the leftmost small area existing in the slice SL0 is “0”.

  The vanishing point coordinates of the small area are the coordinates of the vanishing point of the slice where the small area exists. For example, as shown in FIG. 22, the vanishing point coordinates of all the small areas existing in the slice SL0 are the coordinates of the slice SL0. It becomes the coordinates of the vanishing point P0. The passage detection intensity of the small area corresponds to the passage candidate parameter, and “0” is the least likely that the small area is a part of the passage area, and “2” is the small area of the passage area. It is the most likely part.

  Specifically, the small area forming unit 31a determines the passage detection intensity of the small area as follows. In other words, the small area forming unit 31a first determines, for each small area, the voting source pixel of the vanishing point (the pixel having the texture direction facing the vanishing point) from the pixels included in the small area. Extract as (common item). Then, the small area forming unit 31a calculates the ratio r of effective voting pixels in the small area by dividing the number of effective voting pixels included in the small area by the total number of pixels included in the small area. Then, the small area forming unit 31a compares the effective vote pixel ratio r calculated as described above with the first threshold value TH1 and the second threshold value TH2 (where TH1 <TH2), and when r <TH1, The passage detection intensity of the small area is set to “0”. When TH1 ≦ r <TH2, the passage detection intensity of the small area is set to “1”. When r ≧ TH2, the passage detection intensity of the small area is set to “1”. “2”. Here, the pixels having the texture direction facing the vanishing point are set as effective voting pixels (common items). However, the pixels matching the texture direction having the most orientation around the small area are set as effective voting pixels. Various commonality may be used.

  As shown in FIG. 23, in the following, when the passage detection intensity of the small area is “2”, that is, when the possibility that the small area is a part of the passage area is the highest, the passage is displayed in a dark color. When the path detection intensity of the small area is “1”, that is, when the small area may be a part of the path area, the path is thin. It is assumed that the color is expressed by a single straight line toward the vanishing point. When the passage detection intensity of the small area is “0”, that is, when the possibility that the small area is a part of the passage area is the lowest, a straight line indicating the passage is not drawn.

  When the processing of step S26 described above is completed, the small area selection unit 31b provided in the central block extraction unit 31 performs the small area based on the path estimation information (particularly the path detection intensity) given to each small area. The areas are classified into path candidate areas (small areas with path detection intensity “1” or higher) and other non-path candidate areas (small areas with path detection intensity “0”) (step S27: noise filtering process). . 24-26 is explanatory drawing regarding the noise filtering process of the small area selection part 31b.

  As illustrated in FIG. 24, the small area selection unit 31b refers to the passage detection strengths set in the six small areas a, b, c, d, e, and f adjacent to the small area W to be processed. . The six small regions a to f are the closest small regions located at the upper left, upper right, left, right, lower left, and lower right of the small region W to be processed with reference to the position of each center line. select. Then, the small area selection unit 31b changes the passage detection intensity of the small area W to be processed to “1” when the following passage candidate additional conditions are satisfied for the above six small areas a to f.

That is, the small area W to be processed at this time is selected as a path candidate area and added as one of the path candidate areas.
Path candidate additional conditions:
(A or b) + c + d + (e or f) ≧ 3
and
The passage detection intensity of the small area W to be processed is “0”.
Here, a to f are variables that are “1” when the passage detection intensity of the six small regions a to f is “1” or more, and “0” when the passage detection intensity is “0”. (A or b) is a logical expression that is “1” when at least one of a and b is “1” or more, and “0” when both are “0”.

On the other hand, the small area selection unit 31b changes the passage detection intensity of the small area W to be processed to “0” when the following candidate passage deletion conditions are satisfied for the above six small areas a to f. That is, the small area W to be processed at this time is selected as a non-passage candidate area (noise component) and deleted from the path candidate area.
Path candidate deletion conditions:
a = 0 and b = 0 and e = 0 and f = 0
Or
c = 0 and d = 0

  The small areas to be processed are sorted into a path candidate area and a non-path candidate area as shown in FIG. 25 according to the path candidate addition condition and path candidate deletion condition as described above. FIG. 25A shows a case where the passage detection intensity of the small region W to be processed is changed from “0” to “1” (when selected as a passage candidate region), and FIGS. ) Shows a case where the passage detection intensity of the small area W to be processed is changed from “1” to “0” (when selected as a non-passage candidate area), and FIG. 25D shows the small area to be processed. The case where the passage detection intensity of W remains “0” is shown. The numbers in the figure indicate the passage detection intensity of the small area.

  The small area selection unit 31b sequentially performs the noise filtering process as described above for all the small areas in the input image, and repeats this process until the passage detection intensity of all the small areas does not change. Even if the passage detection intensity is a small region of “1” or more, if the small region exists in a position isolated from a group of small regions that are other passage candidate regions, noise due to erroneous detection of grass, trees, etc. Since it is considered to be a component, a small region that is a noise component can be removed from the path candidate region by performing the noise filtering process as described above.

  On the other hand, even if the passage detection intensity is a small area of “0”, it is highly likely that it is a part of the passage area if it is surrounded by a small area of which the passage detection intensity is “1” or more. . For this reason, a small region that is a part of the passage region is accurately obtained by changing the small region having the passage detection intensity “0” to the small region having the passage detection intensity “1” and adding the small region to the candidate passage region. (Passage candidate area) can be extracted.

  FIG. 26A shows a straight line representing a path existing in each small area based on the path detection intensity set in each small area before the above noise filtering process, and the noise filtering process was performed. FIG. 26B shows a straight line representing a path existing in each small area based on the path detection intensity of each subsequent small area. As shown in these drawings, it can be seen that by performing the noise filtering process, the small area existing at a position isolated from the group of small areas which are other path candidate areas is removed as a noise component.

  As shown in FIG. 26B, the path candidate area after the noise filtering process (the small area whose path detection intensity is “1” or more by the noise filtering process) forms several continuous groups. There are many. Accordingly, the passage area estimation unit 31c provided in the central block extraction unit 31 creates a group of continuous passage candidate areas (hereinafter referred to as a cluster) by grouping adjacent path candidate areas. (Step S28: Clustering process).

  Specifically, the passage area estimation unit 31c performs a process of assigning a cluster-specific ID to each passage candidate area using a labeling method in order to handle each cluster separately. 27 and 28 are explanatory diagrams relating to the clustering processing of the passage area estimation unit 31c. As shown in FIG. 27 (a), in the middle of the clustering process, the passage area estimation unit 31c confirms the passage detection intensity in order from the left end small area in the uppermost slice SLM in the X direction. If the path candidate area (path detection intensity is “1” or higher), a cluster ID is given. If the small area is a non-path candidate area (path detection intensity is “0”), no cluster ID is given.

It should be noted that when a cluster ID is given to a path candidate area, the following three rules are followed.
(1) If there is no small area having a cluster ID around, a new cluster ID is given (see FIG. 27A).
(2) If there is one small area having a cluster ID around it, the same cluster ID is given (see FIG. 27).
(3) When there are two or more areas having cluster IDs in the surrounding area, the cluster ID with the lowest number is given, and it is recorded that these cluster IDs are the same (see FIG. 27B). ).

  The passage area estimation unit 31c gives the cluster IDs to all the passage candidate areas in the input image according to the above three rules, and then scans the entire input image once more, so that the cluster ID recorded as the same is obtained. Replace with one cluster ID (the cluster ID with the lowest number). By the clustering process as described above, the same cluster ID is given to the adjacent path candidate areas, and a cluster which is an area group of continuous path candidate areas is created. As shown in FIG. 28, it can be seen that the path candidate regions existing in the input image are grouped into three clusters CL1, CL2, and CL3 by the clustering process.

Subsequently, the passage area estimation unit 31c selects an optimum cluster (a cluster that seems to be the most like a passage) from the clusters created by the clustering process as a passage area included in the input image (step S29: optimum cluster). Selection process). Specifically, the passage area estimation unit 31c selects an optimum cluster (hereinafter referred to as an optimum cluster) using the evaluation function f (F, H, C, A) represented by the above-described equation (2). . In the present embodiment, the first weighting factor k ₁ = 10000, the second weighting factor k ₂ = 1000, the third weighting factor k ₃ = 50, and the fourth weighting factor k ₄ = 1.

  29 and 30 are explanatory diagrams regarding the optimum cluster selection processing of the passage area estimation unit 30c. As shown in FIG. 29, in the evaluation function f (F, H, C, A), F is a first variable indicating continuity with the current location, and the evaluation target cluster is the foremost (bottommost) slice. “1” when SL0 is included, and “0” when SL0 is not included. H is a second variable indicating the length in the depth direction and is represented by the number of slices included in the cluster to be evaluated. C is a third variable indicating the number of effective path candidate areas, and is represented by the number of path candidate areas of path detection intensity “2” included in the cluster to be evaluated. A is a fourth variable indicating the total area, and is represented by the number of path candidate areas having path detection intensities “1” and “2” included in the cluster to be evaluated.

As described above, the evaluation function f (F, H, C, A) the largest weighting coefficient k ₁ to F is a first variable indicating the continuity of the current position is given in. This is because if the cluster to be evaluated has continuity with the current location (the area closest to the input image), the cluster to be evaluated is very likely to be a corridor or the like. It is. Note that the values of the weighting coefficients k _{1 to} k ₄ described above are examples, and may be determined as appropriate through experiments.

  For example, as shown in FIG. 28, the passage area estimation unit 31c assumes that the three clusters CL1, CL2, and CL3 have been created by the clustering process, and the above evaluation function for each of the three clusters CL1, CL2, and CL3. A value of f (F, H, C, A) is obtained, and a cluster having the largest value is selected as an optimum cluster (passage area included in the input image). Among the three clusters CL1, CL2, CL3, the cluster CL3 includes the foremost (lowermost) slice SL0, and thus the value of the evaluation function f (F, H, C, A) becomes the largest. Therefore, as shown in FIG. 30, the cluster CL3 is selected as the optimum cluster (passage area included in the input image).

  Next, the center line estimation unit 31d provided in the center block extraction unit 31 estimates the optimum cluster selected by the above optimum cluster selection process, that is, the center line of the passage area included in the input image (step S30: Centerline estimation process). FIG. 31 is a flowchart showing the center line estimation process of the center line estimation unit 30d in the center block extraction unit 31. As shown in FIG. 31, the centerline estimation unit 31d first calculates the center of gravity of each of the slices SL0 to SLM (step S30a).

  Specifically, the position of each center of gravity is obtained by using a small area existing in the slice as a trapezoid, and the position obtained by weighting and averaging these positions with the path detection intensity is set as the center of gravity of the entire slice. . 32 to 34 are explanatory diagrams regarding the centerline estimation processing of the centerline estimation unit 31d. By performing the above processing, the centroids of the slices SL0 to SLM are obtained as shown in FIG.

  Subsequently, the centerline estimation unit 31d creates a temporary centerline (step S30b). Specifically, when creating the first temporary center line, the center line estimation unit 31d uses the center of gravity of the lowermost slice SL0 as a starting point, as shown in FIG. A straight line is drawn in the direction toward the vanishing point P0. Then, as shown in FIG. 33B, the center line estimation unit 31d, when the above straight line reaches the boundary with the next slice SL1, the direction from the intersection of the straight line and the boundary toward the vanishing point P1 of the slice SL1. Draw a straight line. As shown in FIG. 33C, the center line estimation unit 31d sets a polygonal line obtained by repeating the above processing as the first temporary center line.

  Then, the centerline estimation unit 31d calculates and records the sum of squares of the distance (difference in the X direction) between the temporary centerline created as described above and the centroids of the slices SL0 to SLM ( Step S30c). Here, the centerline estimation unit 31d creates a plurality of temporary centerlines by looping the above-described steps S30b and S30c, and between each temporary centerline and the center of gravity of each slice SL0-SLM. Calculate and record the sum of squared distances.

  Specifically, when the center line estimation unit 31d creates a temporary center line for the second and subsequent times, the center line estimation unit 31d starts from the point shifted by 1 ° to the right from the starting point of the previous temporary center line, and is the same as the first time. A temporary center line is created by a simple method and the sum of squares is recorded. Then, the centerline estimation unit 31d aborts the search in the right direction when the created temporary centerline passes the right side of the centroid of all slices, and thereafter, from the starting point of the first temporary centerline. A temporary center line is created using the same method as the first time, starting from a point shifted by 1 ° to the left, and the sum of squares is recorded. Furthermore, if the created temporary center line passes through the left side of the center of gravity of all slices, the center line estimation unit 31d terminates the search in the left direction, ends the loop, and proceeds to the next step S30d.

  When the centerline estimation unit 31d creates a plurality of temporary centerlines and records the sum of squares thereof by the loop of steps S30b and S30c described above, the centerline having the smallest value of the sum of squares is optimized. The center line is selected (step S30d). An image of the process of calculating the sum of squares of the distance between the temporary center line and the centroids of the slices SL0 to SLM is shown in FIG. 32 (b), and an image showing the optimum center line selection result. Is shown in FIG. FIG. 34 shows an image of the optimum center line obtained by performing the center line estimation process on the cluster CL3 selected as the optimum cluster in FIG.

  When the above processing is completed, the input image data is divided into square blocks each having about 20 pixels. Then, the block through which the center line obtained by the above processing passes is extracted as the center block (step S31). FIG. 35 is a diagram illustrating a state in which the center block is extracted from the estimated center line. Now, it is assumed that the above-described processing is performed on the image shown in FIG. 35A to obtain the center line CL in the figure.

  The centerline estimation unit 31d divides the input image data into a plurality of blocks B as shown in FIG. In the example shown in FIG. 35B, the block B is divided into about 30 blocks B in the horizontal direction and about 23 blocks in the vertical direction. Then, the centerline estimation unit 31d extracts a block through which the centerline CL shown in FIG. 35A passes as a center block CB. Here, the center line CL shown in FIG. 35A extends from the bottom of the image to about 2/3 of the height of the image, but the six center blocks shown in FIG. CB is limited to the area from the bottom of the image to about ¼ of the height of the image. This is because the image becomes unclear as it goes to the upper part of the image, and the possibility that the center block is detected at an incorrect position is increased.

  When the extraction of the center block CB is completed (or in parallel with the process of step S12 of FIG. 12 or prior to the process of step S12), the vegetation detection unit 32 performs the vegetation detection process (step S13 of FIG. 12: Vegetation detection step). Specifically, as described above, the vegetation detection unit 32, based on the image data of the infrared image, for each pixel in the image, the reflectance calculated based on the image data related to the SWIR wavelength band, and the NIR By calculating the ratio (reflectance ratio) with the reflectance calculated based on the image data relating to the wavelength band (step S41 in FIG. 5), (2) by comparing the reflectance ratio with a predetermined threshold value, It is determined whether or not the observation object is a plant (step S42 in FIG. 5).

  When the detection process of whether or not each pixel of the input image data is vegetation is completed in step S13, the process of correcting the central block CB extracted in step S12 is performed in the central block correction unit 33 (step S14). : Center block correction step). Specifically, as described above, the vegetation ratio calculation unit 33a of the center block correction unit 33 calculates the ratio of plants in each of the blocks around the center block (for example, blocks in a horizontal row). Next, based on the calculation result of the plant ratio calculated by the vegetation ratio calculating unit 33a, the center block changing unit 33b sets the block having the smallest plant ratio as the “corrected center block”.

  FIG. 36 is a diagram illustrating an example of the corrected center block. In the example shown in FIG. 35 (b), six central blocks CB are arranged in the vicinity of the boundary between the fence and the grass. On the other hand, in the example shown in FIG. 36, it can be seen that a block located not near the boundary between the cocoon and the grass but near the approximate center of the cocoon is extracted. With this correction, a center block that is more accurate than the center block extracted by the center block extraction unit 31 can be obtained.

  When the correction of the center block CB is completed, a process for detecting the passage area is performed by the passage area detection unit 34 (step S15: detection step). When processing is started, processing for specifying a reference block is first performed. FIG. 37 is a diagram for explaining the reference block. Specifically, the center block CB corrected in step S12 and the block LB adjacent to the center block in the left-right direction are specified as the reference block. In the example shown in FIG. 37, a total of 18 blocks including 6 central blocks CB and 12 blocks LB are specified as reference blocks.

  When the above processing ends, first, the first calculation unit 34a performs processing for obtaining the average value and the standard deviation of the feature amount for each type of feature amount for the identified reference block (step S151). Next, a process for obtaining the average value of the feature values for all the blocks obtained by the feature value calculation unit 32 for each type of feature value is performed by the second calculation unit 34b (step S152). Here, an image above the horizon is reflected in a region R at the top of the image shown in FIG. 37, and the image in the region R is an image that does not participate in the detection of wrinkles as a passage region. For this reason, when performing the process of step S152, it is desirable to exclude the block in this area | region R and to obtain | require the average value of the feature-value about the remaining blocks.

When the above processing ends, the weight determining unit 34c determines the weight for each feature amount (step S153). Specifically, the feature value average value μ ^ _i and standard deviation σ ^ _i for the reference block obtained in step S151 and the average value μ _i for all the blocks obtained in step S152 are characterized. A process for determining the weight w ′ _i for each type of feature quantity is performed by substituting the formula (3) for each quantity type. When the obtained weight w ′ _i is a negative value, a process for determining the value to be zero is performed. When the weight determination unit 34c determines the weight w ′ _i described above, the weight determination unit 34c obtains the normalized weight w _i using the above-described equation (4) and normalizes each feature amount using the above-described equation (5). A process for obtaining a normalization coefficient s _i is also performed.

When the weighting for the feature amount is completed, the extraction unit 34d performs a process of extracting the wrinkles included in the image using the feature amount weighted by the passage area detection unit 34 (step S154). Specifically, a block to be processed is first identified, and the i-th feature value f _i of the block and the i-th feature value in the central block where the position of the block and the image in the vertical direction are the same. f _ci , and the weight w _i and the normalization coefficient s _i obtained in the process of step S153 are substituted into the evaluation function E shown in the above-described equation (6). Then, when the value of the evaluation function E is larger than the predetermined value T, it is evaluated that the image of the eyelid is included in the processing target block, and the block is extracted.

  When there is no central block whose vertical position is the same as that of the block to be processed, the central block closest to that block whose vertical position is lower than that block. The above evaluation is performed using. By performing the above processing for all the blocks (excluding blocks in the region R in FIG. 37), blocks containing wrinkles are extracted, and thereby the passage region is detected.

  FIG. 38 is a diagram illustrating an example of the detected passage area. By performing the above processing, the shaded block BE in FIG. 38 is detected as a passage area from the image shown in FIG. Referring to FIG. 38, the shaded blocks BE are scattered in portions slightly different from the wrinkles, but are mostly present in the portions where the wrinkles are reflected, and the wrinkles that are the passage areas are increased. It can be seen that it can be detected with accuracy.

  As described above, according to the present embodiment, the central block CB indicating the approximate position of the passage area included in the input image is extracted, and a plurality of types of filter processing are performed on the input image to evaluate the input image. A feature amount to be used is obtained for each type of filter processing, and a passage region is detected by extracting a region having a feature amount similar to the feature amount in the extracted central block CB from the input image. At that time, correction processing is performed so as to avoid the extracted center block from becoming a vegetation region as much as possible. For this reason, for example, even when grass grows in the center of the road, the passage area can be accurately detected. Further, the correction process of the central block is performed by (1) a process of integrating data indicating whether each pixel is vegetation for each pixel, or (2) a process of comparing the integration result between blocks. I am doing so. Therefore, the correction process can be performed by a simple addition / subtraction process and the processing time can be easily shortened. Therefore, for example, it is possible to detect the passage area with high accuracy in real time.

As mentioned above, although embodiment of this invention was described, this invention is not restrict | limited to the said embodiment, It can change freely within the scope of the present invention. For example, in the above embodiment, the example in which the weight w ′ _i is obtained using the above-described equation (3) when detecting the passage area has been described. However, the weight determined by the weight determination unit 34c is determined for the reference block. As long as the feature value has a large difference between the average value and the average value of all the plurality of blocks, and the feature value has a small standard deviation with respect to the reference block, it may be heavy. Accordingly, the weight w ′ _i may be obtained using the following equation (7) instead of the above equation (3).

  In the above embodiment, as a method for extracting the center block CB, the method using the center line of the passage area presumed to be a passage including ridges has been described as an example, but other methods are used. Is also possible. For example, a laser radar may be provided in the vehicle, and the center block may be specified based on the measurement result of the laser radar. Specifically, a specific part in the field of view of the in-vehicle camera (for example, the lowermost part of the field of view) is scanned with a laser radar, and the flattest portion of the obtained signal is estimated as 轍 and the central block CB is extracted. Good.

  Further, the vegetation detection unit 32 shown in FIG. 1 determines whether or not the imaging target is a plant based on the ratio (reflectance ratio) between the reflectance in the water absorption wavelength band and the reflectance in the near-infrared band. Although detected, for example, based on the ratio between the reflectance of visible light and the reflectance in the absorption wavelength band of water or the reflectance in the near-infrared band, it is detected whether the object to be photographed is a plant. It is good also as a simple structure. In this case, for example, the configuration of either the CCD light receiving element 118 or the InSb semiconductor light receiving element 115 can be omitted from the configuration of the in-vehicle camera 100 in FIG.

DESCRIPTION OF SYMBOLS 1 Passage detection apparatus 31 Central block extraction part 32 Vegetation detection part 33 Central block correction | amendment part 34 Passage area | region detection part 34a 1st calculation part 34b 2nd calculation part 34c Weight determination part 34d Extraction part B block

Claims (6)

A passage detection device that detects a passage region included in an input image that is an image obtained by photographing a predetermined photographing region,
Extraction means for extracting a central area of the passage area by performing predetermined image processing on the input image;
Vegetation detection means for detecting whether or not the photographing target is a plant, using an infrared image obtained by photographing the photographing region;
Correction means for correcting the position of the central region extracted by the extraction means based on the detection result by the vegetation detection means;
Passage area detection means for detecting the passage area with reference to a center area whose position is corrected by the correction means. The central region is one or more central blocks located in a central portion of the passage region;
The extracting means extracts one or a plurality of central blocks located in the center of the passage area as the central area;
The correction means corrects the position of the central block extracted by the extraction means based on the detection result by the vegetation detection means,
The passage detection apparatus according to claim 1, wherein the passage region detection unit detects the passage region on the basis of a central block whose position is corrected by the correction unit. Based on the detection result by the vegetation detection unit that is the detection target for detecting whether the correction unit detects the central region extracted by the extraction unit and the predetermined region around the central region as a plant. The path detection device according to claim 1, wherein the position of the central region extracted by the extraction unit is corrected. Based on the detection result by the vegetation detection unit that is the detection target for detecting whether the correction unit detects the central region extracted by the extraction unit and the predetermined region around the central region as a plant. The correction for changing the position of the central region extracted by the extracting means is performed at a position where the proportion of plants is relatively low in the central region and the surrounding predetermined region. 4. The passage detection device according to any one of 3 above. A path detection method for detecting a path area included in an input image that is an image obtained by shooting a predetermined shooting area,
An extraction process for extracting a central area of the passage area by performing predetermined image processing on the input image;
Using an infrared image obtained by imaging the imaging region, a vegetation detection process for detecting whether the imaging target is a plant,
Based on the detection result in the vegetation detection process, a correction process for correcting the position of the central region extracted in the extraction process,
A passage area detection process for detecting the passage area with reference to a center area whose position has been corrected in the correction process. A passage detection program for detecting a passage region included in an input image that is an image obtained by photographing a predetermined photographing region,
An extraction process for extracting a central area of the passage area by performing predetermined image processing on the input image;
Using an infrared image obtained by imaging the imaging region, a vegetation detection process for detecting whether the imaging target is a plant,
Based on the detection result in the vegetation detection process, a correction process for correcting the position of the central region extracted in the extraction process,
A passage detection program for executing, using a computer, a passage region detection step of detecting the passage region with reference to a center region whose position has been corrected in the correction step.