JP2016170163A - Imaging system, image processing system, mobile object control system, mobile object device, projection device, object detection method and object detection program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging system capable of obtaining an image effective for detecting object information while suppressing deterioration of visibility.SOLUTION: An imaging system includes: a projection device including a headlight device 10, installed in a vehicle body and configured to project uniform pattern light and random pattern light having different luminance distribution in different time zones; and a stereo camera 20 including two imaging parts on the right and the left, and configured to image a projection range of the projection device. With this, an image effective for detecting object information can be obtained while suppressing deterioration of visibility.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an imaging system, an image processing system, a moving body control system, a moving body device, a light projecting device, an object detection method, and an object detection program, and more specifically, an image capturing system including the light projecting device and the image capturing device, An image processing system including an imaging system, a mobile control system including the image processing system, a mobile device including the mobile control system, a light projecting device that projects pattern light, and an object detection method including a projecting step, The present invention relates to an object detection program including a procedure for projecting light.

  In recent years, techniques for detecting object information such as the presence / absence of an object and the distance to the object have been actively developed.

  For example, Patent Document 1 discloses a technique of projecting pattern light from a pattern illumination device, imaging an object irradiated with the pattern light with a compound eye imaging device, and obtaining an image effective for detecting object information. ing.

  However, the technique disclosed in Patent Document 1 cannot obtain an image effective for detecting object information while suppressing a decrease in visibility when mounted on a moving body.

  The present invention is an imaging system mounted on a moving body, and projects a first and second pattern lights having different luminance distributions in different time zones, and a light projecting range of the light projecting apparatus. An imaging system comprising: an imaging device including a plurality of imaging units that performs imaging.

  According to the present invention, it is possible to obtain an image effective for detecting object information while suppressing a decrease in visibility.

It is a top view of a vehicle provided with the mobile control system of one embodiment of the present invention. FIG. 2A is a side view of a vehicle including a moving body control system, and FIG. 2B is a front view of the vehicle including a moving body control system. It is a perspective view of the stereo camera of a moving body control system. It is a figure which shows an example of random pattern light. It is a block diagram which shows the hardware constitutions of a mobile body control system. It is a figure for demonstrating the parallax image generation means of an image process part. It is a figure for demonstrating the case of 8 propagation directions. It is a figure for demonstrating the minority parallax by subpixel estimation. It is explanatory drawing of the principle which derives | leads-out the distance from a stereo camera to an object. 10A is a diagram showing a reference image, FIG. 10B is a diagram showing a high-density parallax image with respect to FIG. 10A, and FIG. 10C is an edge parallax with respect to FIG. 10A. It is a conceptual diagram which shows an image. FIG. 11A is a conceptual diagram showing reference pixels in the reference image, and FIG. 11B shows the shift amount while sequentially shifting the corresponding pixel candidates in the comparison image with respect to the reference pixels in FIG. It is a conceptual diagram at the time of calculating. It is a graph which shows the cost value for every shift amount. It is a conceptual diagram for deriving a synthesis cost value. It is a graph which shows the synthetic | combination cost value for every parallax value. It is a flowchart for demonstrating the light projection control process 1 in the object detection apparatus of one Embodiment. It is a flowchart for demonstrating a parallax image generation process. It is a flowchart for demonstrating the parallax image generation process 1 at the time of lighting. It is a timing chart for demonstrating the parallax image generation process 1 at the time of lighting. It is a figure which shows the hardware constitutions of the mobile body control system of the modification 1. It is a flowchart for demonstrating the light projection control process 2 in the object detection apparatus of the modification 1. It is a flowchart for demonstrating the parallax image generation process 2 at the time of lighting. It is a timing chart for demonstrating the parallax image generation process 2 at the time of lighting. FIG. 23A and FIG. 23B are timing charts for explaining another example of the lighting-time parallax image generation processing. 24A and 24B are timing charts for explaining another example of the lighting-time parallax image generation processing.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[Overview]
As shown in FIGS. 1 to 2B, a mobile control system 100 according to an embodiment is mounted on a vehicle body (mobile) of a vehicle 1 as a mobile device, and includes a headlight device 10 and an imaging device. A stereo camera 20 (see FIG. 3), a processing device 30 (see FIG. 5), an ECU 50 (see FIG. 5) as an electronic control device, and the like. Here, the mobile control system 100 is supplied with electric power from an electric battery of the vehicle 1.

  Examples of the vehicle body that is a moving body include an automobile, a train, a motorcycle, a bicycle, a wheelchair, and an agricultural cultivator. Moreover, as a mobile body apparatus provided with the mobile body control system 100 and the mobile body by which this mobile body control system 100 is mounted, not only a vehicle but an aircraft, a ship, etc. may be sufficient, for example.

  The headlight device 10 includes a pair of left and right headlights 10a and 10b provided at the front portion of the vehicle body. The headlights 10a and 10b are turned on or off at the same time. Here, the headlights 10a and 10b are provided as standard equipment on the vehicle 1. Hereinafter, when it is not necessary to distinguish the headlights 10a and 10b, they are collectively referred to as “headlights”.

  Each headlight includes an LED array as a light source, in which a plurality of LEDs are arranged in an array, and a drive circuit that drives the LED array. The headlight can also be turned on / off by manually operating a headlight switch provided near the handle of the vehicle 1. The headlight may include, for example, a white light source instead of the LED as a light source.

  The light projected forward of the vehicle from the headlight is irradiated to the object when the object is within the light projection range of the headlight device 10. Here, the “light projection range” means a range (see FIG. 1) that includes a range in which light can be projected by each headlight of the headlight device 10.

  As an example, as shown in FIGS. 2A and 2B, the stereo camera 20 is attached in the vicinity of the rear-view mirror above the driver's seat inside the vehicle, and both eyes (the left and right imaging units 20a, 20b) so that the front of the vehicle can be imaged. That is, the stereo camera 20 can image the light projection range of the headlight device 10. The left and right images (luminance images) captured by both eyes of the stereo camera 20 are sent to the processing device 30 for each frame. The processing device 30 performs parallax calculation using the left and right images from the stereo camera 20.

  While the headlight device 10 is projecting light, pattern light is emitted to an object (for example, a person, another vehicle, a structure, a road, a tree, etc.) within the light projecting range, and thus the image is picked up with both eyes of the stereo camera 20. The pattern of the pattern light is superimposed (textured) on the objects in the left and right images.

  As shown in FIG. 5, the processing device 30 includes a CPU 31, a memory 29, a stereo camera control CPU I / F 32, an image processing unit 33, an image recognition processing unit 34, brightness information CPU I / F 35, and an image recognition result CPU I / F 37. , A start / end signal transfer I / F 38 and an image recognition result transfer I / F 40.

  The stereo camera control CPU I / F 32 is a transmission / reception interface between the CPU 31 and the stereo camera 20. The CPU 31 performs imaging control of the stereo camera 20 via the stereo camera control CPU I / F 32.

  The CPU 31 and the ECU 50 can transmit and receive via the start / end signal transfer I / F 38.

  The image processing unit 33 calculates brightness information from the left and right images (luminance images) from the stereo camera 20 and transmits the brightness information to the CPU 31 via the brightness information CPU I / F 35. The CPU 31 transmits the received brightness information to the ECU 50 via the brightness data transfer I / F 38.

  In addition, the image processing unit 33 performs gamma correction, distortion correction, and the like on the left and right images captured by the stereo camera 20, and then performs parallax calculation using the left and right images (luminance images). The image and the luminance image are sent to the image recognition processing unit 34.

  The image recognition processing unit 34 recognizes (detects) object information such as the presence / absence of an object, the type (type) of the object, and the distance to the object based on the parallax image and the luminance image from the image processing unit 33. Then, the image recognition result as the recognition result is transmitted to the CPU 31 via the image recognition result CPU I / F 37. The CPU 31 transmits the received image recognition result to the ECU 50 via the image recognition result transfer I / F 40.

  That is, the processing results in the image processing unit 33 and the image recognition processing unit 34 are sent to the CPU 31 of the processing device 30, corrected to a data format that can be received by the ECU 50, and then transferred to the ECU 50. The data format transferred to the ECU 50 mainly includes CAN I / F, LIN I / F, and the like.

  The ECU 50 controls the headlight according to the operation (ON / OFF) of the headlight switch by the driver or the like, and based on the processing information from the processing device 30, the braking device for the vehicle 1 is added to the headlight device 10. Control the steering device.

  The ECU 50 includes a CPU 51, a memory 52, a CPU I / F 53, a CPU I / F 55, a light emission control I / F 56, a braking control I / F 57, and a steering control I / F 58.

  The CPU 51 includes a light projection control unit 51a that controls the headlight device 10, a braking control unit 51b that controls the braking device, and a steering control unit 51c that controls the steering device.

  The CPU 51 acquires the brightness information from the CPU 31 via the start / end signal transfer I / F 38 and the CPU I / F 53.

  Further, the CPU 51 acquires the image recognition result from the CPU 31 via the image recognition result transfer I / F 40 and the CPU I / F 55.

  The braking control unit 51b and the steering control unit 51c acquire the image recognition result from the CPU 31 via the image recognition result transfer I / F 40 and the CPU I / F 55.

  The braking control unit 51b controls the braking device (for example, autobrake for avoiding danger) based on the acquired image recognition result.

  The steering control unit 51c performs control of the steering device (for example, auto steering for avoiding danger) based on the acquired image recognition result.

  Next, a parallax calculation algorithm using an image captured by the stereo camera 20 will be described with reference to FIGS.

  The image data captured by the left and right imaging units of the stereo camera 20 is subjected to processing such as gamma correction and distortion correction by the preprocessing unit of the image processing unit 33, respectively. For the left and right image data that has been subjected to the preprocessing, a parallax calculation algorithm is executed by a parallax image generating means (see FIG. 6) that is a post-processing unit of the image processing unit 33, and a parallax value is calculated for each pixel.

  By the way, the calculation method for generating the parallax image may be a method for calculating the parallax only for a portion having a strong texture (a portion having a strong edge) such as EBM (edge base). Alternatively, as in SGM (semi-global matching), the dissimilarity of matching is propagated from the periphery of each pixel by a recurrence formula, and the decimal disparity is calculated from the integer disparity that gives the minimum dissimilarity and the dissimilarity of the disparity adjacent thereto. It is also possible to calculate the parallax of each pixel of the whole image by using a subpixel estimation method (fitting by an equiangular straight line, a parabola, a high-order polynomial, etc.).

  SGM is more preferable than EBM because it can suppress missing pixels in the parallax data.

  Therefore, the parallax image generation unit of the present embodiment adopts the SGM method, and as an example, as shown in FIG. 7, the high frequency enhancement filter unit for the reference image, the high frequency enhancement filter unit for the comparison image, and the matching A degree calculator, a propagation similarity calculator, an integer parallax calculator, and a minority parallax calculator. The SGM method is disclosed in non-patent literature (Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information).

  Below, the parallax image generation means of this embodiment is demonstrated. In the SGM parallax calculation, as shown in FIG. 7, the dissimilarity is propagated from each direction of the target pixel for which the parallax is to be calculated (see formula (1) or (2)), and the energy S that is the sum of them is calculated. (Refer to equation (3)) is calculated, and the integer parallax D [pixel] that gives the minimum value of the energy S is derived as the integer parallax of the target pixel. Here, C expressed by the expressions (1) and (2) represents the degree of coincidence by the parallax search of the comparison image of the target pixel of the reference image. As an example of C, other stereo matching indices such as SAD, SSD, and NCC may be used. After calculating C, Lr is calculated from equation (1) or (2), and finally S is calculated from equation (3). Increasing the number of propagation directions improves the parallax calculation accuracy, but increases the size of the calculation processing circuit (the relationship between the parallax calculation accuracy and the processing circuit size is a trade-off).

  Further, when pursuing parallax accuracy, sub-parallax is calculated using a sub-pixel estimation method as shown in FIG. For example, in the equiangular straight line method, the decimal parallax D ′ is calculated using three S corresponding to the parallaxes D−1, D, and D + 1. Alternatively, in the parabola fit method, the decimal parallax D ′ is calculated using five S corresponding to the parallaxes D−2, D−1, D, D + 1, and D + 2.

  Note that whether or not the calculated parallax value is reliable is determined by reliability determination. If it is determined that the parallax value is not reliable, the parallax of the pixel is invalid parallax.

<Principles of ranging>
With reference to FIG. 9, the principle of deriving a parallax with respect to an object from a stereo camera by the stereo image method and measuring the distance from the stereo camera to the object with a parallax value indicating the parallax will be described. FIG. 9 is an explanatory diagram of the principle of deriving the distance from the stereo camera 20 to the object. Further, in the following, in order to simplify the description, the description will be made on a pixel-by-pixel basis, not on a predetermined region composed of a plurality of pixels.
When processing is performed in units of a predetermined area including a plurality of pixels instead of a single pixel, the predetermined area including the reference pixel is indicated as the reference area, and the predetermined area including the corresponding pixel is indicated as the corresponding area. Further, this reference area includes the case of only the reference pixel, and the corresponding area includes the case of only the corresponding pixel.

(Parallax value calculation)
First, the images captured by the image capturing unit 20a and the image capturing unit 20b illustrated in FIG. 9 are referred to as a reference image Ia and a comparative image Ib, respectively. In FIG. 9, it is assumed that the imaging unit 20a and the imaging unit 20b are installed in parallel equiposition. In FIG. 9, the point S on the object E in the three-dimensional space is mapped to a position on the same horizontal line of the imaging unit 20a and the imaging unit 20b. That is, the S point in each image is imaged at a point Sa (x, y) in the reference image Ia and a point Sb (X, y) in the comparative image Ib. At this time, the parallax value Δ is expressed by the following equation (4) using Sa (x, y) at the coordinates on the imaging unit 20a and Sb (X, y) at the coordinates on the imaging unit 20b. Is done.
Δ = X−x (4)

  Here, in the case as shown in FIG. 9, the distance between the point Sa (x, y) in the reference image Ia and the intersection of the perpendicular drawn from the imaging lens 11a on the imaging surface is set to Δa, and the comparison image Ib When the distance between the point Sb (X, y) and the intersection of the perpendicular line taken from the imaging lens 11b on the imaging surface is Δb, the parallax value Δ = Δa + Δb.

(Distance calculation)
Moreover, the distance Z between the imaging units 20a and 20b and the object E can be derived by using the parallax value Δ. Specifically, the distance Z is a distance from a plane including the focal position of the imaging lens 11a and the focal position of the imaging lens 11b to the specific point S on the object E. As shown in FIG. 9, using the focal length f of the imaging lens 11a and the imaging lens 11b, the baseline length B that is the length between the imaging lens 11a and the imaging lens 11b, and the parallax value Δ, the following ( The distance Z can be calculated by the equation 5).
Z = (B × f) / Δ (5)
According to the equation (5), the larger the parallax value Δ, the smaller the distance Z, and the smaller the parallax value Δ, the larger the distance Z.

<SGM method>
Subsequently, a distance measuring method using the SGM method will be described with reference to FIGS. 10A is a reference image, FIG. 10B is a conceptual diagram illustrating a high-density parallax image with respect to FIG. 10A, and FIG. 10C is a conceptual diagram illustrating an edge parallax image with respect to FIG.

  Here, the reference image is an image in which an object is indicated by luminance. The high-density parallax image is an image derived from the reference image by the SGM method, and is an image showing the parallax value at each coordinate of the reference image. The edge parallax image is an image derived by a conventionally used block matching method, and is an image showing only a parallax value of a portion having a relatively strong texture such as an edge portion of a reference image.

  The SGM method is a method for appropriately deriving the parallax value even for an object with a weak texture. Based on the reference image shown in FIG. This is a method for deriving a density parallax image. When the block matching method is used, the edge parallax image shown in FIG. 10C is derived based on the reference image shown in FIG. As can be seen by comparing the inside of the dashed ellipse in FIGS. 10B and 10C, the high-density parallax image can represent detailed information such as a road having a weak texture compared to the edge parallax image. More detailed distance measurement can be performed.

This SGM method does not calculate the cost value that is dissimilarity and immediately derives the parallax value, but after calculating the cost value, it further calculates the synthesis cost value that is the synthetic dissimilarity. This is a method of deriving a parallax value and finally deriving a parallax image (here, a high-density parallax image) indicating the parallax value in almost all pixels.
In the case of the block matching method, the cost value is calculated in the same way as the SGM method. However, as in the SGM method, a comparatively strong texture such as an edge portion is calculated without calculating a synthesis cost value. Only the parallax value is derived.

(Calculation of cost value)
First, a method for calculating the cost value C (p, d) will be described with reference to FIGS. FIG. 11A is a conceptual diagram showing reference pixels in the reference image, and FIG. 11B is a diagram showing the corresponding pixel candidates in the comparison image while sequentially shifting (shifting) the reference pixels in FIG. 11A. It is a conceptual diagram at the time of calculating quantity (deviation amount). FIG. 12 is a graph showing the cost value for each shift amount. Here, the corresponding pixel is a pixel in the comparison image that is most similar to the reference pixel in the reference image. In the following description, C (p, d) represents C (x, y, d).

  As shown in FIGS. 11A and 11B, a predetermined reference pixel p (x, y) in the reference image and an epipolar in the comparison image with respect to the reference pixel p (x, y). The cost of each corresponding pixel candidate q (x + d, y) with respect to the reference pixel p (x, y) based on each luminance value with a plurality of corresponding pixel candidates q (x + d, y) on the line (Epipolar Line) A value C (p, d) is calculated. d is the shift amount (shift amount) between the reference pixel p and the corresponding pixel candidate q, and in this embodiment, the shift amount in pixel units is represented. That is, in FIG. 11A and FIG. 11B, the corresponding pixel candidate q (x + d, y) is sequentially shifted by one pixel within a predetermined range (for example, 0 <d <25). A cost value C (p, d), which is a dissimilarity between the luminance values of the pixel candidate q (x + d, y) and the reference pixel p (x, y), is calculated. As a calculation method of the cost value C, when the cost value C indicates dissimilarity, a known method such as SAD (Sum of Absolute Difference) is applied.

The cost value C (p, d) calculated in this way can be represented by a graph of a cost curve that is a collection of cost values C for each shift amount d, as shown in FIG. In FIG. 12, the cost value C is 0 (zero) in the case of the shift amount d = 5, 12, and 19, and therefore the minimum value cannot be obtained. Thus, in the case of an object having a weak texture, it is difficult to obtain the minimum value of the cost value C.
(Computation cost value calculation)
Next, a method for calculating the combined cost value Ls (p, d) will be described with reference to FIGS. 13 and 14. FIG. 13 is a conceptual diagram for deriving a synthesis cost value. FIG. 14 is a graph of a synthesis cost curve showing a synthesis cost value for each parallax value. The calculation method of the synthesis cost value in the present embodiment is not only the calculation of the cost value C (p, d), but also the cost value when pixels around the predetermined reference pixel p (x, y) are used as the reference pixels. The combined cost value Ls (p, d) is calculated by consolidating the cost value C (p, d) at the reference pixel p (x, y).

Next, a method for calculating the synthesis cost value will be described in more detail. In order to calculate the combined cost value Ls (p, d), it is first necessary to calculate the route cost value Lr (p, d). The following expression (6) is an expression for calculating the route cost value Lr (p, d) and is substantially the same as the expression (1), and the expression (7) is the combined cost value Ls. This is an expression for calculating the above expression, and is substantially the same as the expression (3).
Lr (p, d) = C (p, d) + min {(Lr (pr, d), Lr (pr, d-1) + P1, Lr (pr, d + 1) + P1, Lrmin (p) -R) + p2} (6)

  Here, in Expression (6), r represents a direction vector in the aggregation direction, and has two components in the x direction and the y direction. min {} is a function for obtaining the minimum value. Lrmin (p−r) indicates the minimum value of Lr (p−r, d) when the shift amount d is changed in the coordinates obtained by shifting p by one pixel in the r direction. Note that Lr is applied recursively as shown in equation (6). P1 and P2 are fixed parameters determined in advance by experiments, and are parameters that make it easy for the parallax values Δ of adjacent reference pixels on the path to be continuous. For example, P1 = 48 and P2 = 96.

Further, as shown in the equation (6), Lr (p, d) is the cost value C in the reference pixel p (x, y), and the respective values in each pixel in the r direction shown in FIG. It is obtained by adding the minimum value of the path cost value Lr of the pixel. Thus, in order to obtain Lr at each pixel in the r direction, first, Lr is obtained from the endmost pixel in the r direction of the reference image p (x, y), and Lr is obtained along the r direction. .
Then, as shown in FIG. 13, Lr ₀ , Lr ₄₅ , Lr ₉₀ , Lr ₁₃₅ , Lr ₁₈₀ , Lr ₂₂₅ , Lr ₂₇₀ , and Lr ₃₁₅ in eight directions are obtained, and finally based on the equation (7). Thus, the synthesis cost value Ls is obtained.

The composite cost value Ls (p, d) calculated in this way is a graph of the composite cost curve in which the composite cost value Ls (p, d) is shown for each shift amount d as shown in FIG. Can be represented by In FIG. 14, the composite cost value Ls is calculated as a parallax value Δ = 3 because the minimum value is obtained when the shift amount d = 3.
In the above description, the number of r is assumed to be 8. However, the present invention is not limited to this. For example, the eight directions may be further divided into two to be divided into sixteen directions and three into 24 directions.
Further, although the cost value C is shown as “dissimilarity”, it may be expressed as “similarity” as a reciprocal of dissimilarity. In this case, as a method for calculating the cost value C, a known method such as NCC (Normalized Cross Correlation) is applied. Further, in this case, the parallax value Δ where the combined cost value Ls is not “minimum” but “maximum” is derived. In addition, you may represent as a "matching degree" including both dissimilarity and similarity.

  Note that a part of the processing of the processing device may be executed by the ECU, or a part of the processing of the ECU may be executed by the processing device. Moreover, you may comprise one control system which has both functions of a processing apparatus and ECU. In this case, the object detection program and the moving body control program may be stored in one memory.

  Further, instead of the memory, other storage media such as a hard disk may be used.

  Moreover, although the controlled object of a mobile body control system is a vehicle braking device and a steering device, it is not restricted to this. For example, only one of the braking device and the steering device may be set as the control target, or the drive source (for example, engine or motor) of the vehicle may be set as at least one of the control targets.

  In addition, the moving body control system includes a braking control unit and a steering control unit that control the vehicle body, but instead of or in addition to this, based on detection information of an object detection device described later, a driver or the like You may have the alarm device which outputs the alarm sound and warning display for calling attention.

  In addition, the LED array is turned on by a light emission pattern with an irregular light amount distribution and a random pattern light is projected. Instead, for example, a “pattern” represented by a projection mapping image is displayed in front of the host vehicle. It may be projected. In other words, the light projecting device may include an image forming unit that forms an image with light and emits the light that has formed the image as random pattern light. Further, random pattern light may be generated and projected by irradiating light to a mask pattern having an irregular pattern distribution.

  Further, although a headlight device is used as a light source unit of the light projecting device, a dedicated light source unit different from the headlight device may be provided separately. In addition, the headlight device is not limited to the standard equipment in the vehicle body, and may be a custom-made product or an optional product.

  Moreover, although the stereo camera 20 which has two imaging parts (two eyes) is employ | adopted as an imaging device, you may employ | adopt the imaging device which has three or more imaging parts.

[Details]
When the headlight switch is turned from OFF to ON, the light projection control unit 51a alternately transmits the uniform light emission pattern and the random light emission pattern stored in the memory 52 to the headlight, and the uniform light and the uniform light from the headlight. Random pattern light is projected alternately.

  On the other hand, when the headlight switch is turned from ON to OFF, the light projection control unit 51a transmits a light extinction trigger signal to the headlight, and terminates the light projection from the headlight.

  Further, when transmitting a random light emission pattern to the headlight, the light projection control unit 51a transmits a random light emission start signal to the CPU 31 via the CPU I / F 53 and the start / end signal transfer I / F 38.

  When receiving the random light emission start signal, the CPU 31 transfers the random light emission start signal to the stereo camera 20 via the stereo camera control I / F 32.

  When the stereo camera 20 receives the random light emission start signal, the stereo camera 20 captures a predetermined number of frames with both eyes and outputs the captured left and right images to the image processing unit 33 for each frame. When the stereo camera 20 captures a predetermined number of frames, the stereo camera 20 transmits an imaging end signal to the CPU 31 via the stereo camera control CPU I / F 32.

  When the CPU 31 receives the imaging end signal, it transfers it to the light projection control unit 51a via the start / end signal transfer I / F 38 and the CPU I / F 53.

  When receiving the imaging end signal, the light projection control unit 51a transmits a lighting trigger signal and a uniform light emission pattern to the headlight.

  In addition, when the uniform light emission pattern is transmitted to the headlight, the light projection control unit 51a transmits a uniform light emission start signal to the CPU 31 via the CPU I / F 53 and the start / end signal transfer I / F 38.

  Therefore, a “light projecting device” that includes the headlight device 10 and the light projecting control unit 51a and projects uniform pattern light and random pattern light alternately is configured. Furthermore, an “imaging system” that obtains an image of the light projection range of the light projection device is configured including the light projection device and the stereo camera 20. In addition, an object detection apparatus that detects object information from an image obtained by the imaging system is configured including the image processing system including the imaging system and the image processing unit 33.

  By the way, in general, when driving a vehicle, in a dark part such as in a daytime tunnel or at night, the dark part is brightened by projecting light forward from the headlight, but the contrast of the object originally in the dark part is Since it is low, it is difficult to obtain image contrast simply by turning on the headlight and irradiating it with uniform light. For this reason, in a dark part, when acquiring a parallax image ahead of a vehicle with a stereo camera, matching (correlation) of a right-and-left image becomes difficult at the time of parallax calculation. As a result, the number of invalid parallax pixels of a subject in front of the vehicle (person, other vehicle, obstacle, road surface, background, etc.) increases, and the accuracy of output parallax decreases.

  In view of this, in the present embodiment, as described in detail below, the “light projection from the headlight” is devised.

  For example, the headlight may be turned on / off by manually operating a headlight switch provided near the steering wheel of the vehicle 1 (mode 1). You may make it do (mode 2). Note that the moving body control system 100 corresponds to mode 1 as an example.

  In mode 1, a vehicle driver or a passenger (hereinafter referred to as “driver etc.”) turns on the headlight switch when the outside of the vehicle feels “dark” while the headlight is turned off ( Switch from OFF to ON) and turn on the headlight. More specifically, when the ECU 50 receives a lighting request signal by turning on the headlight switch via the USER I / F 80 (see FIG. 5), the light quantity distribution (current value distribution) stored in the memory 52 (see FIG. 5). First and second light emission pattern data different from each other are alternately transmitted to the headlight.

  In mode 2, the ECU 50 turns on the headlight when it is determined that the outside of the vehicle is “dark” while the headlight is turned off. Specifically, when the ECU 50 determines that the outside of the vehicle is “dark” based on the detection result from the brightness information detecting means for detecting the brightness information outside the vehicle, the first and second light emission pattern data Are alternately sent to the headlight.

  The “brightness information detection means” can be configured to include a stereo camera 20 and an image processing unit 33 (described later) that processes an image captured by the stereo camera 20. A combination of processing units, an illuminance sensor, or the like may be used. That is, when the brightness information detection unit is configured by the stereo camera 20 and the image processing unit 33, the image processing unit 33 determines that the image is “dark” when the total luminance value of all the pixels in the image is smaller than the threshold value. To do. Further, when the brightness information detecting means is an illuminance sensor, it is determined to be “dark” when the acquired value of the sensor is smaller than the threshold value. Of course, the brightness detection by the brightness information detecting means can take various forms as long as the brightness outside the vehicle can be acquired. The detection result of the brightness information detection means is transmitted to the ECU 50.

  In any of the modes 1 and 2, while the first light emission pattern data is transmitted to the headlight, the first pattern light corresponding to the first light emission pattern data is projected forward from the headlight to the vehicle. Then, while the second light emission pattern data is being transmitted to the headlight, the second pattern light corresponding to the second light emission pattern data is projected forward from the headlight to the vehicle. That is, when the headlight is turned on, the first and second pattern lights having different luminance distributions are alternately projected forward from the headlight. When the headlight drive circuit receives each light emission pattern data, the headlight drive circuit drives each LED of the LED array with a current value set for each LED in the light emission pattern data.

  In addition, when the first and second pattern lights are alternately projected, any of the first pattern lights may be projected first (the head), but whichever is the first, corresponds to the first pattern light. When the light emission pattern data to be transmitted is transmitted to the headlight, a lighting trigger signal is also transmitted.

  In mode 1, when the driver feels that the outside of the vehicle is “bright” while the headlight is on, the driver turns off the headlight switch (switching from ON to OFF) to turn off the headlight. . More specifically, when the ECU 50 receives a turn-off request signal by turning off the headlight switch via the USER I / F 80, the ECU 50 sends a turn-off trigger signal to the headlight to turn off the headlight.

  In mode 2, when the headlight is turned on, the ECU 50 transmits a turn-off trigger signal to the headlight when the outside of the vehicle is determined to be “bright” based on the detection result from the brightness information detecting means. Turn off the light.

  Here, the “first light emission pattern data” is data having a uniform light amount distribution (current value distribution), and the corresponding “first pattern light” has a uniform luminance distribution. Therefore, hereinafter, “first light emission pattern data” is also referred to as “uniform light emission pattern”, and “first pattern light” is also referred to as “uniform light pattern”. Here, “uniform luminance distribution” can be rephrased as “no contrast”.

  The “second light emission pattern data” is data having an irregular light amount distribution (current value distribution), and the corresponding “second pattern light” has an irregular luminance distribution (non-uniform luminance distribution). Have. Therefore, hereinafter, “second light emission pattern data” is also referred to as “random light emission pattern”, and “second pattern light” is also referred to as “random pattern light”.

  In the random pattern light, the binary or multi-value (three or more values) luminance portions are irregularly distributed (see FIG. 4).

  For example, when the light quantity distribution of the random light emission pattern is an irregular distribution of the binary light quantity part, the random pattern light also has an irregular distribution of the binary luminance part. Become.

  For example, if the light emission distribution of the random light emission pattern is an irregular distribution of multi-value light quantity portions, the random pattern light also has an irregular distribution of multi-value luminance portions. It becomes.

  Here, the time zone in which the random light emission pattern is transmitted to the headlight, that is, the light projection time Tr of the random pattern light is the time zone Tc in which the uniform light emission pattern is transmitted to the headlight, that is, the light projection time Tc of the uniform pattern light. (See FIG. 18). This is because, from the viewpoint of driver visibility, it is considered that the random pattern light projection time is preferably shorter than the uniform pattern light projection time. Tr and Tc are fixed. Tr: Tc is preferably about 1: (2 to 10). Tr: Tc may be an integer ratio or may not be an integer ratio. Note that Tc is preferably set according to the time required for generating a parallax image by the image processing unit 33 described later.

  Therefore, when the headlight is turned on, the uniform pattern light of the light projection time Tc and the random pattern light of the light projection time Tr alternately in the light projection range (more specifically, each time period is different from each other periodically. To) (see FIG. 18).

  For example, the brightness distribution (for example, contrast and pattern) of the random pattern light is manually adjusted by a driver or the like according to the brightness outside the vehicle, or automatically controlled (for example, the detection result from the brightness information detection means by the ECU 50). (Control based on the above). Specifically, the darker the vehicle exterior is, the lower the contrast of the object is. Therefore, the darker the vehicle exterior is, the higher the contrast of the random pattern light may be. For example, if a plurality of random light emission patterns having different light intensity distributions are stored in the memory 52, one random pattern light among a plurality of random pattern lights having different luminance distributions may be selectively projected as necessary. Can do.

  For example, the plurality of random pattern lights having different luminance distributions may be a plurality of random pattern lights having different contrasts and patterns, or may be a plurality of random pattern lights having different patterns and the same contrast, A plurality of random pattern lights having the same pattern and different contrasts may be used.

  When the uniform pattern light and the ranram pattern light are alternately projected on the projection range, an object (for example, a person, another vehicle, a structure, a road, a tree, etc.) within the projection range is displayed with the stereo camera 20. An image can be taken. In this case, the pattern of the uniform pattern light is superimposed on the object of the left and right images captured by the both eyes of the stereo camera 20 during the projection time Tc of the uniform pattern light, and the stereo camera is added to the projection time Tr of the random pattern light. A pattern of random pattern light is superimposed on the left and right image objects picked up by 20 eyes.

  Here, when an object is irradiated with random pattern light, the amount of texture increases and the contrast increases. On the other hand, even when the object is irradiated with uniform pattern light, the amount of texture hardly changes and the contrast hardly changes.

  Therefore, if the parallax calculation is performed using the left and right images captured by both eyes of the stereo camera 20 during the projection time Tr of the random pattern light, the left and right images can be easily matched and the parallax calculation accuracy can be improved. Can do.

  Hereinafter, the light projection control process 1 in the object detection apparatus of the present embodiment will be described with reference to the flowchart of FIG. Of the object detection program used for controlling the object detection device, a subprogram for operating the light projecting device including the headlight device 10 is stored in the memory 52 of the ECU 50. The light projection control process 1 is executed by the light projection control unit 51a according to this subprogram.

  The control here is performed when the safety mode (danger avoidance mode) is selected (ON) in the operation unit of the vehicle 1 by the driver or the like.

  When the safety mode is OFF, the light projection control unit 51a lights the headlight with a uniform light emission pattern according to the ON operation of the headlight switch by the driver or the like, and the headlight according to the OFF operation of the headlight switch. Turn off the light.

  In the first step S1, it is determined whether or not the headlight switch is ON. This determination is affirmed when the lighting control unit 51a receives a lighting request signal due to an ON operation of the headlight switch, and is negative when the lighting request signal is not received. If the determination in step S1 is affirmed, the process proceeds to step S2. If the determination in step S1 is negative, the process proceeds to step S5.

  In step S <b> 2, the lighting trigger signal and the uniform light emission pattern are transmitted to the headlight device 10, and the uniform light emission start signal is transmitted to the processing device 30. As a result, the uniform pattern light is projected from the headlight forward of the vehicle, and the CPU 31 receives the uniform light emission start signal.

  In the next step S <b> 3, a random light emission pattern is transmitted to the headlight device 10 and a random light emission start signal is transmitted to the processing device 30. As a result, random pattern light is projected from the headlight forward of the vehicle, and the CPU 31 receives a random light emission start signal.

  In the next step S4, it is determined whether an imaging end signal has been received. That is, it is determined whether an imaging end signal from the stereo camera 20 is received via the processing device 30. If the determination in step S4 is negative, the same determination is made again. That is, the state is “waiting for imaging end”. On the other hand, if the determination here is affirmed, the process proceeds to step S5.

  In step S5, it is determined whether the safety mode is ON. If the determination in step S5 is affirmative, the process returns to step S1. On the other hand, if the determination in step S5 is negative, the flow ends.

  Hereinafter, the parallax image generation processing in the object detection apparatus of the present embodiment will be described with reference to the flowchart of FIG. Of the object detection program used for controlling the object detection device, a subprogram for operating the stereo camera 20 and the processing device 30 is stored in the memory 29 of the processing device 30. The parallax image generation processing is executed by the processing device 30 according to this subprogram. Each determination in FIG. 16 is performed by the CPU 31.

  The control here is performed when the safety mode (danger avoidance mode) is selected (ON) in the operation unit of the vehicle 1 by the driver or the like.

  In the first step S11, it is determined whether or not a uniform light emission start signal has been received. If the determination in step S11 is negative, the process proceeds to step S12. On the other hand, if the determination in step S11 is affirmed, the process proceeds to step S14.

  In step S12, the stereo camera 20 captures a predetermined number of frames, and the captured left and right images are acquired for each frame. That is, here, the left and right images captured by the stereo camera 20 when the headlight is turned off are acquired for each frame.

  In the next step S13, a parallax image is generated from the acquired images for each of the left and right frames. Specifically, after the image processing unit 33 performs pre-processing such as gamma correction and distortion correction, parallax calculation is performed, and the pre-processed luminance image and the parallax image obtained by the parallax calculation are converted into the image recognition processing unit 34. Send to. When step S13 is executed, the process proceeds to step S15.

  In step S14, “lighting-time parallax image generation processing 1” described later is performed. When step S14 is executed, the process proceeds to step S15.

  In the next step S15, it is determined whether or not the safety mode is ON. If the determination in step S15 is negative, the flow ends. If the determination in step S15 is positive, the flow returns to step S11.

  The image recognition processing unit 34 performs image recognition using the parallax image generated by the parallax image generation processing. Specifically, when the uniform light emission start signal is not received, object detection is performed based on the parallax calculation result of the normal parallax image generation process, and when the uniform light emission start signal is received, the random pattern light described later is used. The object detection is performed using the result of the parallax calculation in the time zone when the light is projected. Then, the image recognition processing unit 34 recognizes (detects) information related to the object in the projection range, and transmits the result (image recognition result) to the ECU 50. The “information about the object” includes, for example, the presence / absence of the object, the distance to the object, the type of the object (for example, a person, a structure, a road, etc.). Note that the types of objects can be detected by storing parameters such as the sizes and shapes of a plurality of types of objects in the memory 29 in advance and detecting them based on the parameters.

  Subsequently, the ECU 50 executes various controls (controls for avoiding danger) based on the received image recognition result. Specifically, based on the image recognition result, the braking control unit 51b controls the braking device, and the steering control unit 51c controls the steering device.

  Hereinafter, “lighting-time parallax image generation processing 1” in step S14 in the “parallax image generation processing” will be described with reference to the flowchart of FIG. 17 and the timing chart of FIG.

  In the first step S21, it is determined whether or not a random light emission start signal has been received. If the determination here is affirmed, the process proceeds to step S22. On the other hand, if the determination here is denied, the same determination is made again. That is, the state is “waiting for reception”.

  In step S22, the stereo camera 20 captures a predetermined number of frames, and the captured left and right images are acquired for each frame. That is, here, when the random pattern light is projected from the headlight, the stereo camera 20 captures a predetermined number of frames, and the captured left and right images are acquired for each frame (see FIG. 18).

  In the next step S23, a parallax image is generated from the acquired images of the left and right frames. Specifically, after the image processing unit 33 performs pre-processing such as gamma correction and distortion correction, parallax calculation is performed, and the pre-processed luminance image and the parallax image obtained by the parallax calculation are converted into the image recognition processing unit 34. (See FIG. 18). When step S23 is executed, the process proceeds to step S24.

  In step S24, an imaging end signal is transferred to the ECU 50. Specifically, the imaging end signal from the stereo camera 20 is transferred to the light projection control unit 51a (see FIG. 18). When step S24 is executed, the flow ends.

  By the way, a random pattern light is projected to the front of the vehicle with respect to the dark part, and an image with a contrast that is advantageous for parallax calculation is obtained by imaging the projected range with a stereo camera. However, if only the random pattern light is continuously projected, the bright part and the dark part in the random pattern light are always mixed, so that the visibility is lowered, and there is a concern that the driver's stress and accidents are induced.

  The imaging system of the present embodiment described above is an imaging system mounted on a vehicle body, and projects the headlight device 10 and the projector that project first and second pattern lights having different luminance distributions in different time zones. A light projecting device including the light control unit 51a and a stereo camera 20 including two image capturing units that capture an image of the light projecting range of the light projecting device are provided.

  In this case, it is possible to obtain an image advantageous for parallax calculation capable of improving the matching accuracy of the left and right images while setting the ratio of the light projection time of the random pattern light in the total light projection time of the light projection device to less than 100% it can.

  As a result, in the imaging system of this embodiment, an image effective for detecting object information can be obtained while suppressing a decrease in visibility.

  In addition, since the light projector includes a headlight mounted on the vehicle body as a light source unit, the uniform pattern light and the random pattern light become illumination light that illuminates the dark part, and the random pattern light becomes effective light for parallax calculation. Become. That is, it is not necessary to provide a dedicated light source unit in addition to the headlight.

  Moreover, since the light projection time Tr of the random pattern light is shorter than the light projection time Tc of the uniform pattern light, it is possible to further suppress the decrease in visibility.

  Further, since the light projecting device alternately projects the uniform pattern light and the random pattern light, an image advantageous for parallax calculation can be obtained between the projections of the uniform pattern light.

  In addition, since the light projecting device periodically projects each of the uniform pattern light and the random pattern light, an image advantageous for parallax calculation can be obtained regularly and stably.

  In addition, since the imaging system of the present embodiment further includes an operation unit for switching on / off of light projection by the light projecting device, when the driver feels “dark”, the uniform pattern light and the random pattern Light can be projected alternately.

  In addition, the object detection device of the present embodiment is an object detection device mounted on a vehicle main body, and includes an imaging system and a processing device 30 that processes an image captured by the imaging system.

  In this case, it is possible to detect the object information with high accuracy from an image effective for detecting the object information obtained by the imaging system while suppressing a decrease in visibility.

  In addition, the moving body control system 100 of this embodiment is mounted on an object detection device and a vehicle main body (moving body), and an ECU 50 (control device) that controls the vehicle main body based on the processing result of the processing device 30 of the object detection device. ), It is possible to prevent the traveling vehicle body from colliding with an object.

  Moreover, since the vehicle 1 (moving body apparatus) is provided with the moving body control system 100 and the vehicle main body (moving body) in which the moving body control system 100 is mounted, it is excellent in safety.

  In addition, the light projecting device of the present embodiment is a light projecting device mounted on the vehicle main body, and is an imaging range (left eye imaging range and right eye) of the stereo camera 20 including two imaging units mounted on the vehicle main body. The uniform pattern light and the random pattern light are projected in different time zones within the range including the eye imaging range (see FIG. 1).

  In this case, it is possible to obtain an image advantageous for parallax calculation capable of improving the matching accuracy of the left and right images while setting the ratio of the projection time of the random pattern light in the total projection time to less than 100%.

  As a result, in the light projecting device of the present embodiment, it is possible to improve the detection accuracy of the object information while suppressing a decrease in visibility.

  As can be seen from the above description, the light projecting device, the imaging system, and the object detection device of the present invention are particularly effective when used in a dark environment, and are not necessarily mounted on a moving body such as a vehicle body. It is not necessary, and it is expected to be used in various fields. In this case, a light source unit instead of the headlight device 10 is used.

  Further, an object detection method using the object detection device of the present embodiment includes a step of projecting uniform pattern light in front of the vehicle body, a step of projecting random pattern light in front of the vehicle body, A step of imaging a range in which the random pattern light is projected from a plurality of positions, and a step of performing parallax calculation using a plurality of images individually captured from the plurality of positions in the imaging step.

  In this case, it is possible to generate a parallax image using an image advantageous for parallax calculation that can improve the matching accuracy of the left and right images while setting the ratio of the projection time of the random pattern light in the total projection time to less than 100%. it can.

  As a result, it is possible to improve the detection accuracy of object information while suppressing a decrease in visibility.

  Further, an object control program used for controlling the object detection apparatus of the present embodiment is an object detection program for causing a computer to execute the object detection method, and a procedure for projecting uniform pattern light in front of the vehicle body, It includes a procedure for projecting random pattern light in front of the vehicle body and a procedure for imaging a range in which the random pattern light is projected from a plurality of positions on the vehicle body.

  In this case, it is possible to generate a parallax image using an image advantageous for parallax calculation that can improve the matching accuracy of the left and right images while setting the ratio of the projection time of the random pattern light in the total projection time to less than 100%. it can.

  As a result, it is possible to improve the detection accuracy of object information while suppressing a decrease in visibility.

  Moreover, since the object detection program of this embodiment is divided and stored in the two memories 29 and 31, the object detection apparatus can be operated according to the object detection program as appropriate.

  Below, the modification of the said embodiment is demonstrated.

  The mobile control system 200 of the modification shown by FIG. 19 is a little different from the mobile control system 200 of the said embodiment in the structure of a processing apparatus and ECU. The moving body control system 200 corresponds to mode 2.

  Specifically, in the processing device 60 of the moving body control system 200, brightness information CPU I / F 35 and brightness information transfer I / F 38 are added to the processing device 30 of the above embodiment. Further, in the ECU 70 of the moving body control system 200, a CPU I / F 53 is added to the ECU 50 of the above embodiment.

  The image processing unit 33 of the processing device 60 calculates brightness information from an image (luminance image) captured by the stereo camera 20 and transmits the brightness information to the CPU 31 via the brightness information CPU I / F 35. The CPU 31 transmits the received brightness information to the ECU 70 via the brightness information transfer I / F 38.

  The CPU 51 of the ECU 70 acquires the brightness information from the CPU 31 via the brightness information transfer I / F 38 and the CPU I / F 53.

  In the mobile control system 200 according to the modified example, “lighting parallax image generation processing” in “light projection control processing” and “parallax image generation processing” is different from the above embodiment, and other processes and controls are the same as those in the above embodiment. The same is done.

  Below, the light projection control process 2 in the object detection apparatus of a modification is demonstrated with reference to the flowchart of FIG. Of the object detection program used for controlling the object detection device, a subprogram for operating the light projecting device including the headlight device 10 is stored in the memory 52 of the ECU 70. The light projection control process 2 is executed by the light projection control unit 51a according to this subprogram.

  The control here is performed when the safety mode (danger avoidance mode) is selected (ON) in the operation unit of the vehicle 1 by the driver or the like. When the safety mode is ON, the CPU 31 performs imaging in real time with the stereo camera 20 (see FIG. 22). Then, brightness information is calculated by the image processing unit 33 from an image (luminance image) ahead of the vehicle imaged by the stereo camera 20, and the brightness information is sent to the light projection control unit 51a via the CPU 31 in real time.

  When the safety mode is OFF, the light projection control unit 51a lights the headlight with a uniform light emission pattern according to the ON operation of the headlight switch by the driver or the like, and the headlight according to the OFF operation of the headlight switch. Turn off the light.

  In first step S31, it is determined whether the brightness information is less than a preset reference value. Here, the reference value is a level that generally feels “dim” or “dark” if the brightness information is less than the reference value, and generally feels “bright” if the brightness information is greater than or equal to the reference value. It is. If the determination in step S31 is negative, the process proceeds to step S35. On the other hand, if the determination in step S31 is affirmed, the process proceeds to step S32.

  In step S <b> 32, the lighting trigger signal and the uniform light emission pattern are transmitted to the headlight device 10, and the uniform light emission start signal is transmitted to the processing device 60. As a result, the uniform pattern light is projected from the headlight forward of the vehicle, and the CPU 31 receives the uniform light emission start signal.

  In the next step S33, a random light emission pattern is transmitted to the headlight device 10 and a random light emission start signal is transmitted to the processing device 60. As a result, random pattern light is projected from the headlight forward of the vehicle, and the CPU 31 receives a random light emission start signal.

  In the next step S34, it is determined whether or not an image output end signal has been received. That is, it is determined whether an image output end signal from the stereo camera 20 is received via the processing device 60. If the determination in step S34 is negative, the same determination is made again. That is, the state is “waiting for image output completion”. On the other hand, if the determination here is affirmed, the process proceeds to step S35.

  In step S35, it is determined whether the safety mode is ON. If the determination in step S35 is affirmed, the process returns to step S31. On the other hand, if the determination in step S35 is negative, the flow ends.

  Hereinafter, “lighting-time parallax image generation processing 2” of Modification 1 will be described with reference to the flowchart of FIG. 21 and the timing chart of FIG. The lighting parallax image generation process 2 is also executed by the processing device 60 according to the procedure of the subprogram stored in the memory 29. Each determination in FIG. 21 is performed by the CPU 31.

  In the first step S41, it is determined whether or not a random light emission start signal has been received. If the determination here is affirmed, the process proceeds to step S42. On the other hand, if the determination here is denied, the same determination is made again. That is, the state is “waiting for reception”.

  In step S42, the left and right images captured by the stereo camera 20 are acquired for each frame. That is, here, the left and right images of a predetermined number of frames captured by the stereo camera 20 when random pattern light is projected from the headlight are acquired for each frame (see FIG. 22).

  In the next step S43, a parallax image is generated from the acquired image for each of the left and right frames. Specifically, after the image processing unit 33 performs pre-processing such as gamma correction and distortion correction, parallax calculation is performed, and the pre-processed luminance image and the parallax image obtained by the parallax calculation are converted into the image recognition processing unit 34. (See FIG. 22). When step S43 is executed, the process proceeds to step S44.

  In step S44, an image output end signal is transferred to the ECU 50. Specifically, the image output end signal from the stereo camera 20 is transferred to the light projection control unit 51a. When step S44 is executed, the flow ends. When the stereo camera 20 receives the random light emission start signal, the stereo camera 20 outputs left and right images captured by both eyes to the image processing unit 33 for each frame. When the stereo camera 20 outputs a predetermined number of left and right images to the image processing unit 33, the stereo camera 20 transmits an image output end signal to the CPU 31 via the stereo camera control CPU I / F 32.

  The object detection device according to the modified example described above projects a uniform pattern light and a random pattern light alternately when the brightness information outside the vehicle body is less than a preset reference value (predetermined value). Since the parallax calculation is performed using the image captured when the light is projected, the parallax image can be generated stably and accurately while suppressing a decrease in visibility. In addition, since the contrast of the object originally in the bright part is already high, the necessity of projecting the random pattern light is low when the outside of the vehicle is equal to or higher than the reference value (when bright).

  Moreover, since the light projector acquires brightness information from the luminance information of the image captured by the stereo camera 20, the number of parts can be increased compared to the case where a dedicated detection unit for detecting brightness information is separately provided. Increase and cost increase can be suppressed.

  Examples 1 and 2 (used parallax calculation method: SGM algorithm) of the present invention are shown below (see Table 1).

<Example 1> (corresponding to the above embodiment)
While driving, turn on the headlights manually, judging that the driver is dark, etc., and pattern light with uniform pattern and non-uniform contrast from the headlights in time-division irradiation (alternate irradiation) did. The driver was not bothered and stressed. In addition, the number of invalid parallax pixels in front of the vehicle decreased.

<Example 2> (corresponding to the above modification)
Brightness information detection means was used. The headlight was turned on automatically after judging that it was dark automatically, and a pattern with a uniform pattern light and a non-uniform contrast was irradiated from the headlight to the front of the vehicle in a time-sharing manner. The driver was not bothered and stressed. In addition, the number of invalid parallax pixels in front of the vehicle decreased.

  In the second embodiment, a stereo camera is used as brightness information detection means, but the present invention is not limited to this. For example, a monocular camera, a polarization camera, a laser radar, or the like can be used.

  The present invention is not limited to the above-described embodiments and modifications, and can be changed as appropriate. For example, in the above-described embodiment and modification, a random light emission start signal is transmitted from the ECU to the processing device. Instead, as shown in FIGS. 23 (A) and 23 (B), the ECU It is also possible to transmit a random light emission signal having a pulse width as the light emission time Tr of the random pattern light to the processing device, and the stereo camera 20 may capture or output an image only while the random light emission signal is at a high level. Then, in synchronization with the falling edge of the random light emission signal, the uniform light emission start signal may be transmitted from the ECU to the processing device, and the lighting trigger signal and the uniform light emission pattern may be transmitted to the headlight device. In this case, there is no need to output an imaging end signal or an image output end signal from the stereo camera 20.

  In the above modification, a parallax image is generated from an image captured by the stereo camera 20 (hereinafter referred to as the former image) when random pattern light is projected. As shown in FIGS. 24A and 24B, a parallax image is also generated from an image captured by a stereo camera (hereinafter referred to as the latter image) when uniform pattern light is projected. You may do it. In this case, the image recognition processing unit 34 may perform image recognition using the parallax image generated from the former image. In this case, there is no need to output an imaging end signal or an image output end signal from the stereo camera 20. Note that the parallax image generated from the former image is a parallax image generated in the time period between the random light emission start signal or the rise of the random light emission signal and the next uniform light emission start signal.

  Moreover, in the said embodiment and modification, 1st pattern light is made into uniform pattern light, and 2nd pattern light is made into random pattern light, but it is not restricted to this, In short, the 1st and 2nd It is sufficient that the brightness distribution of the pattern light is different.

  Moreover, in the said embodiment and modification, although random pattern light with irregular luminance distribution is used as 2nd pattern light, it is not restricted to this, The point is pattern light with nonuniform luminance distribution. It is preferable.

  Moreover, in the said embodiment and the modification, although uniform pattern light with uniform luminance distribution is used as 1st pattern light, you may use somewhat nonuniform pattern light. Even in this case, it is preferable that the luminance distribution of the first pattern light has higher uniformity than the second pattern light.

  In the embodiment and the modification, Tr <Tc, but Tr ≧ Tc may be used. Further, each of Tr and Tc may be variable.

  DESCRIPTION OF SYMBOLS 10 ... Headlight apparatus (a part of light projection apparatus), 20 ... Stereo camera (imaging apparatus), 30 ... Processing apparatus, 50 ... ECU (control apparatus), 51a ... Light projection control part (a part of light projection apparatus) , 80 ... USERI / F80 (part of the operation unit), 100, 200 ... mobile control system.

JP 2013-190394 A

Claims (18)

An imaging system mounted on a moving body,
A light projecting device that projects the first and second pattern lights having different luminance distributions in different time zones;
An imaging system comprising: an imaging device including a plurality of imaging units that images a light projection range of the light projecting device.   The imaging system according to claim 1, wherein the light projecting device includes a headlight mounted on the moving body as a light source unit.   The imaging system according to claim 1 or 2, wherein the first pattern light has a uniform luminance distribution, and the second pattern light has a non-uniform luminance distribution.   The imaging system according to claim 1, wherein the second pattern light has an irregular luminance distribution.   The imaging system according to claim 1, wherein a projection time of the second pattern light is shorter than a projection time of the first pattern light.   The imaging system according to any one of claims 1 to 5, wherein the light projecting device projects the first and second pattern lights alternately.   The imaging system according to claim 6, wherein the light projecting device periodically projects each of the first and second pattern lights.   The imaging system according to claim 1, further comprising an operation unit for switching ON / OFF of light projection by the light projecting device.   9. The light projecting device according to claim 1, wherein the light projecting device projects light when a detection result of brightness information detecting means for detecting brightness information outside the moving body is less than a predetermined value. The imaging system according to one item. The imaging system according to any one of claims 1 to 9,
An image processing system comprising: a processing device that processes an image captured by the imaging device of the imaging system.   11. The parallax calculation according to claim 10, wherein the processing device performs a parallax calculation based on a plurality of images captured by each of the plurality of imaging units in a time zone in which the second pattern light is projected. Image processing system.   The image processing system according to claim 11, wherein the processing device performs object detection using a result of the parallax calculation. An image processing system according to any one of claims 10 to 12,
And a control device that is mounted on the mobile body and controls the mobile body based on a processing result of the processing device of the object detection device. The moving body control system according to claim 13,
A moving body apparatus comprising: the moving body on which the moving body control system is mounted. In the light projection device mounted on the moving body,
A light projecting device that projects first and second pattern lights having different luminance distributions in different time zones in an image capturing range of an image capturing device including a plurality of image capturing units mounted on the moving body.   16. The light projecting device according to claim 15, wherein the first pattern light has a uniform luminance distribution, and the second pattern light has a non-uniform luminance distribution. Projecting a first pattern light having a uniform luminance distribution in front of the moving body;
Projecting a second pattern light having a non-uniform luminance distribution in front of the moving body;
Imaging a range in which the second pattern light is projected from a plurality of positions on the moving body;
An object detection method comprising: performing parallax calculation using a plurality of images individually captured from the plurality of positions in the imaging step. An object detection program for causing a computer to execute the object detection method according to claim 17,
A procedure of projecting a first pattern light having a uniform luminance distribution in front of the moving body;
A procedure of projecting a second pattern light having a non-uniform luminance distribution in front of the moving body;
A procedure for imaging a range in which the second pattern light is projected from a plurality of positions on the moving body;
An object detection program including a procedure for performing parallax calculation using a plurality of images individually captured from the plurality of positions in the imaging procedure.