JP6642641B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging device.

Based on the image acquired by the camera mounted on the vehicle, the traveling environment of the vehicle is detected, and based on the detected traveling environment data, automatic traveling control such as constant speed traveling, following traveling, alarming, braking, steering assist, etc. A technology for performing driving assistance has been developed (see Patent Document 1).
In the related art, a camera is used to detect a line on a road. However, in a traveling environment such as in a tunnel or during rainfall, it may be difficult to detect the line.

JP 2010-79424 A

An imaging device according to an embodiment of the present invention includes an imaging unit that is mounted on a car and has a plurality of imaging regions with different imaging conditions, an input unit that inputs information on a direction of irradiation of a light of the car, has an imaging control unit for controlling the condition, and the imaging control unit in response to said information, to change the imaging conditions of the first imaging area that is part of the imaging region of the imaging unit, said imaging The imaging condition of a second imaging region, which is a part of the imaging region of the unit and is different from the first imaging region, is made different from that of the first imaging region .

1 is a schematic configuration diagram of a driving support device for a vehicle. FIG. 3 is a block diagram illustrating a configuration of a control device. FIG. 3 is a cross-sectional view of a stacked image sensor. FIG. 3 is a diagram illustrating a pixel array and a unit area of an imaging chip. FIG. 3 is a diagram illustrating a circuit in a unit area. FIG. 2 is a block diagram illustrating a functional configuration of an imaging device. FIG. 3 is a diagram illustrating positions of focus detection pixels on an imaging surface. It is the figure which expanded the area | region containing a part of focus detection pixel line. FIG. 2 is a block diagram illustrating the configuration of a camera having an image sensor. It is a figure which shows typically the image on the imaging surface of an imaging chip. 5 is a flowchart illustrating an overall flow of camera control processing executed by a control unit. 9 is a flowchart illustrating details of an imaging condition setting process. 9 is a flowchart illustrating a process in the case of a first traveling environment change. It is a figure which shows typically the image on the imaging surface of an imaging chip. It is a flow chart which illustrates processing in the case of the change of the 2nd running environment. FIG. 16A is a diagram schematically illustrating an image on the imaging surface of the imaging chip, FIG. 16A is a diagram at the time of a high beam, and FIG. 16B is a diagram at the time of a low beam. It is a flow chart which illustrates processing in the case of the change of the 3rd run environment. It is a figure which shows typically the image on the imaging surface of an imaging chip. It is a flow chart which illustrates processing in the case of the change of the 4th running environment. It is a figure which shows typically the image on the imaging surface of an imaging chip. It is a flowchart which illustrates the process in the case of the change of the 5th driving environment. FIG. 22A is a diagram schematically illustrating an image on an imaging surface of an imaging chip, FIG. 22A is a diagram before a lane change, and FIG. 22B is a diagram during a lane change.

Hereinafter, embodiments for implementing the present invention will be described with reference to the drawings.
<Scene where the camera is used>
FIG. 1 is a schematic configuration diagram of a driving support device 2 of a vehicle 1 equipped with a camera 3 according to an embodiment of the present invention. In FIG. 1, a driving support device 2 is mounted on a vehicle 1 such as an automobile. The driving support device 2 includes a camera 3, a control device 4, a first travel control unit 5, a second travel control unit 6, and the like. In the present description, an example in which an internal combustion engine is used as a driving source will be described. However, a motor may be used as a driving source, or a so-called hybrid using an internal combustion engine and a motor as driving sources may be used.

  The camera 3 includes an image pickup optical system having a plurality of lenses and an image pickup device (in this embodiment, a stacked image pickup device (see FIG. 3)), and is mounted, for example, in front of a ceiling in a vehicle cabin. The camera 3 is directed forward of the vehicle 1. The camera 3 acquires an image in the traveling direction of the vehicle 1, and performs distance measurement (distance measurement) to each subject (object) at a plurality of positions in the shooting screen based on the acquired image. The distance measurement is calculated by a distance measurement operation using an image signal from a focus detection pixel provided in the stacked imaging device. The focus detection pixels and the distance measurement will be described later. The image data and the distance measurement data acquired by the camera 3 are sent to the control device 4. Note that the cameras 3 may be provided outside the vehicle, the cameras 3 inside and outside the vehicle may cooperate, and the number of cameras 3 may be set as appropriate. For example, the white line detection described later may use the camera 3 outside the vehicle, and the recognition of an object or an obstacle may cooperate with the cameras 3 inside and outside the vehicle.

  The control device 4 includes a CPU 4a and a storage unit 4b as illustrated in FIG. The CPU 4a performs various calculations based on various programs stored in the storage unit 4b, using control parameters stored in the storage unit 4b, detection signals from sensors described below, and the like.

  The first travel control unit 5 performs constant-speed travel control and follow-up travel control based on an instruction from the control device 4. The constant speed traveling control is control for causing the vehicle 1 to travel at a constant speed based on a predetermined control program. When the speed of the preceding vehicle recognized by the control device 4 is equal to or lower than the target speed set in the vehicle 1 during the constant speed traveling control, the following traveling control is performed with respect to the preceding vehicle. This is control for running the vehicle while maintaining the inter-vehicle distance.

  The second travel control unit 6 performs driving support control based on an instruction from the control device 4. The driving support control outputs a steering control signal to the steering control device 9 so that the vehicle 1 travels along the road based on a predetermined control program, or avoids the vehicle 1 from colliding with an object. Is a control for outputting a brake control signal to the brake control device 8.

  1 further shows a throttle control device 7, a brake control device 8, a steering control device 9, a steering wheel 10, a turn signal switch 11, a vehicle speed sensor 12, a yaw rate sensor 13, a display device 14, A GPS device 15, a shift lever position detecting device 16, a microphone 17, a beam switch 18, and a rain sensor 19 are illustrated.

  The throttle control device 7 controls the opening of a throttle valve (not shown) according to the amount of depression of an accelerator pedal 7a. The throttle control device 7 also controls the opening of the throttle valve in accordance with a throttle control signal sent from the first travel control unit 5. The throttle control device 7 further sends a signal indicating the depression amount of the accelerator pedal 7a to the control device 4.

  The brake control device 8 controls the opening of a brake valve (not shown) according to the amount of depression of the brake pedal 8a. The brake control device 8 also controls the opening of the brake valve according to a brake control signal from the second travel control unit 6. The brake control device 8 further sends a signal indicating the depression amount of the brake pedal 8a to the control device 4.

  The steering control device 9 controls the steering angle of a steering device (not shown) according to the rotation angle of the steering wheel 10. The steering control device 9 also controls the steering angle of the steering device according to a steering control signal from the second travel control unit 6. The steering control device 9 further sends a signal indicating the rotation angle of the steering wheel 10 to the first travel control unit 5 and the control device 4, respectively.

  The turn signal switch 11 is an operation member for operating a turn signal (turn signal) device (not shown). The turn signal device is a blinking light emitting device that indicates a change in the course of the vehicle 1. When the occupant of the vehicle 1 operates the turn signal switch 11, an operation signal from the turn signal switch 11 is sent to the turn signal device, the second travel control unit 6, and the control device 4, respectively. The vehicle speed sensor 12 detects the vehicle speed V of the vehicle 1 and sends out a detection signal to the first travel control unit 5, the second travel control unit 6, and the control device 4, respectively.

  The yaw rate sensor 13 detects a yaw rate of the vehicle 1 and sends a detection signal to the second travel control unit 6 and the control device 4, respectively. The yaw rate is a change speed of the rotation angle of the vehicle 1 in the turning direction. The display device 14 displays information indicating a control state of the first travel control unit 5 and the second travel control unit 6, and the like. The display device 14 is configured by, for example, a HUD (Head Up Display) that projects information on a windshield. Note that a display unit of a navigation device (not shown) may be used as the display device 14.

  The GPS device 15 receives a radio wave from a GPS satellite and performs a predetermined calculation using information on the radio wave to calculate the position (latitude, longitude, and the like) of the vehicle 1. The position information calculated by the GPS device 15 is sent to a navigation device and the control device 4 (not shown). The shift lever position detecting device 16 detects a position of a shift lever (not shown) operated by an occupant of the vehicle 1 (for example, parking (P), reverse (R), drive (D), etc.). The shift lever position information detected by the shift lever position detection device 16 is sent to the control device 4.

  The microphone 17 includes, for example, a front microphone, a right microphone, and a left microphone. The front microphone has a directivity of collecting sound in front of the vehicle 1 exclusively. The right microphone has a directivity of collecting sound on the right side of the vehicle 1 exclusively. The left microphone has a directivity for collecting the sound on the left side of the vehicle 1 exclusively. Each sound information (front, right side, left side) collected by the microphone 17 is sent to the control device 4.

  The beam switching switch 18 is an operation member for switching the irradiation angle in the vertical direction of the lighting device (headlight) in at least two stages. For example, a “high beam” that irradiates the illumination light in a substantially horizontal direction and a “low beam” that irradiates the illumination light downward from the horizontal direction are switched. An operation signal from the beam switch 18 is sent to the illumination device and the control device 4 (not shown). The rainfall sensor 19 is a detector that detects raindrops by an optical type or a capacitance type, and is mounted inside or outside the vehicle. A detection signal (rain information) from the rain sensor 19 is sent to the control device 4.

<Detection of object>
The control device 4 performs image processing on an image from the camera 3 as follows in order to detect a traveling path of the vehicle 1 and an object. First, the control device 4 generates a distance image (depth distribution image) based on the distance measurement data at a plurality of positions in the shooting screen. The control device 4 performs a well-known grouping process based on the data of the distance image, and stores frames (windows) of three-dimensional road shape data, side wall data, object data, and the like stored in the storage unit 4b in advance. In addition to extracting white line data (including white line data along the road and white line (stop line: intersection information) data crossing the road), side wall data such as guardrails and curbs along the road, Objects and obstacles are classified and extracted as motorcycles, ordinary vehicles, large vehicles, pedestrians, telephone poles and other objects.
In the present description, the white or yellow line drawn on the traveling path is called a white line. In addition, the solid line and the broken line are called white lines.

  Further, the control device 4 performs shape detection, color detection, and light / dark detection of an object included in the image from the image data acquired by the camera 3. Specifically, data of a line such as a white line or a guardrail provided along the road is detected (extracted). Further, the control device 4 detects data of a tail lamp (tail lamp) in a lighting state of the preceding vehicle based on a light / dark state of each unit area 131 described later.

<Driving support>
The control device 4 recognizes a traveling path or an obstacle or an object / obstacle based on each of the information extracted as described above, that is, the white line data, the guardrail side wall data, and the object data. The second driving control unit 6 performs the driving support control on the basis of the above. That is, the vehicle 1 is made to travel along the road, and the collision of the vehicle 1 with the target is avoided.

<Driving control>
The control device 4 estimates the own vehicle traveling route in the following four ways, for example.
(1) Estimation of own vehicle traveling path based on white line White line data on both the left and right sides or one of the right and left sides of the traveling path is obtained from the image acquired by camera 3, and vehicle 1 travels from these white line data. When the shape of the lane in which the vehicle 1 is located can be estimated, the control device 4 estimates that the own vehicle traveling path is parallel to the white line in consideration of the width of the vehicle 1 and the position of the vehicle 1 in the current lane.

(2) Estimation of own vehicle traveling path based on side wall data such as guardrails, curbs, etc. Side wall data on both left and right sides or one side on the left and right sides of the running path are obtained from images acquired by camera 3, and from these side wall data When the shape of the lane in which the vehicle 1 is traveling can be estimated, the control device 4 considers the width of the vehicle 1 and the position of the vehicle 1 in the current lane, and the traveling path of the own vehicle is parallel to the side wall. It is estimated.

(3) Estimation of own vehicle traveling route based on preceding vehicle trajectory The control device 4 estimates the own vehicle traveling route based on the past traveling trajectory of the preceding vehicle stored in the storage unit 4b. The preceding vehicle refers to a forward vehicle closest to the vehicle 1 among objects traveling in the same direction as the vehicle 1.

(4) Estimation of Own Car Running Path Based on Running State of Vehicle 1 The control device 4 estimates the own vehicle traveling path based on the operating state of the vehicle 1. For example, the own vehicle traveling path is estimated using the turning curvature based on the detection signal from the yaw rate sensor 13 and the detection signal from the vehicle speed sensor 12. The turning curvature Cua is calculated by Cua = dψ / dt / V. dψ / dt is the yaw rate (the speed of change of the rotation angle in the turning direction), and V is the vehicle speed of the vehicle 1.

  The control device 4 estimates the travel area of the vehicle 1 at the position where the target object exists based on the own vehicle traveling path for each target object according to a predetermined travel control program stored in the storage unit 4b. The traveling area and the position of the object are compared to determine whether or not each object is within the traveling area. The control device 4 further recognizes the preceding vehicle based on a result of imaging by the camera 3. That is, the control device 4 sets the vehicle closest to the vehicle 1 to the preceding vehicle among the objects existing in the traveling area and traveling in the forward direction (the same direction as the vehicle 1).

  The control device 4 outputs inter-vehicle distance information between the preceding vehicle and the vehicle 1 and vehicle speed information of the preceding vehicle to the first travel control unit 5 as outside-vehicle information. Here, the vehicle speed information of the preceding vehicle was measured based on the vehicle speed V of the vehicle 1 acquired at predetermined time intervals and the image acquired by the camera 3 at the above-mentioned predetermined time intervals in synchronization with the acquisition timing of the vehicle speed V. It is calculated based on the change in the distance to the preceding vehicle (inter-vehicle distance) in the shooting screen.

  The first travel control unit 5 sends a throttle control signal to the throttle control device 7 so that the vehicle speed V detected by the vehicle speed sensor 12 converges to a predetermined vehicle speed (target speed) set in advance. As a result, the throttle control device 7 feedback-controls the opening of the throttle valve (not shown), and the vehicle 1 automatically runs at a constant speed.

  Further, the first travel control unit 5 is configured to control the vehicle speed information of the preceding vehicle input from the control device 4 during the travel control in the constant speed state when the vehicle speed information is equal to or lower than the target speed set in the vehicle 1. And sends a throttle control signal to the throttle control device 7 based on the inter-vehicle distance information input from the control device 4. Specifically, an appropriate inter-vehicle distance target value is set based on the inter-vehicle distance from the vehicle 1 to the preceding vehicle, the vehicle speed of the preceding vehicle, and the vehicle speed V of the vehicle 1, and the target value of the A throttle control signal is sent to the throttle control device 7 so that the inter-vehicle distance measured based on the distance converges to the target value of the inter-vehicle distance. As a result, the throttle control device 7 performs feedback control of the opening of the throttle valve (not shown), and causes the vehicle 1 to follow the preceding vehicle.

<Explanation of the stacked image sensor>
The stacked image sensor 100 provided in the camera 3 will be described. Note that this stacked-type image sensor 100 is described in Japanese Patent Application No. 2012-139026 filed earlier by the present applicant. FIG. 3 is a cross-sectional view of the stacked image sensor 100. The imaging device 100 includes a backside illumination type imaging chip 113 that outputs pixel signals corresponding to incident light, a signal processing chip 111 that processes pixel signals, and a memory chip 112 that stores pixel signals. The imaging chip 113, the signal processing chip 111, and the memory chip 112 are stacked, and are electrically connected to each other by a conductive bump 109 such as Cu.

  As shown in the figure, the incident light mainly enters in the Z-axis plus direction indicated by the white arrow. In the present embodiment, the surface of the imaging chip 113 on which incident light is incident is referred to as a back surface (imaging surface). As shown by the coordinate axes, the left direction perpendicular to the Z axis is defined as the plus direction of the X axis, and the near direction perpendicular to the Z axis and the X axis is defined as the plus direction of the Y axis. In the following figures, coordinate axes are displayed so that the directions of the respective figures can be understood with reference to the coordinate axes in FIG.

  One example of the imaging chip 113 is a back-illuminated MOS image sensor. The PD layer 106 is provided on the back side of the wiring layer 108. The PD layer 106 includes a plurality of PDs (photodiodes) 104 that are two-dimensionally arranged and accumulates charges corresponding to incident light, and a transistor 105 provided corresponding to the PD 104.

  The color filter 102 is provided on the incident side of the incident light in the PD layer 106 via the passivation film 103. The color filters 102 have a plurality of types that transmit different wavelength regions, and have a specific arrangement corresponding to each of the PDs 104. The arrangement of the color filters 102 will be described later. A set of the color filter 102, the PD 104, and the transistor 105 forms one pixel.

  On the incident side of the incident light in the color filter 102, a micro lens 101 is provided corresponding to each pixel. The micro lens 101 collects incident light toward the corresponding PD 104.

  The wiring layer 108 has a wiring 107 for transmitting the pixel signal from the PD layer 106 to the signal processing chip 111. The wiring 107 may be a multilayer, and may be provided with a passive element and an active element.

  A plurality of bumps 109 are arranged on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the opposite surface of the signal processing chip 111, and the imaging chip 113 and the signal processing chip 111 are aligned by pressing or the like. The bumps 109 are joined and electrically connected.

  Similarly, a plurality of bumps 109 are arranged on surfaces of the signal processing chip 111 and the memory chip 112 that face each other. These bumps 109 are aligned with each other, and the signal processing chip 111 and the memory chip 112 are pressurized or the like, whereby the aligned bumps 109 are joined and electrically connected.

  Note that the bonding between the bumps 109 is not limited to Cu bump bonding by solid-phase diffusion, and micro-bump bonding by solder melting may be employed. Further, for example, about one bump 109 may be provided for one block described later. Therefore, the size of the bump 109 may be larger than the pitch of the PD 104. In a peripheral region other than the pixel region where the pixels are arranged, a bump larger than the bump 109 corresponding to the pixel region may be additionally provided.

  The signal processing chip 111 has a TSV (through-silicon via) 110 that connects circuits provided on the front and back surfaces to each other. The TSV 110 is preferably provided in a peripheral area. Further, the TSV 110 may be provided in a peripheral area of the imaging chip 113 and the memory chip 112.

  FIG. 4 is a diagram illustrating the pixel array of the imaging chip 113 and the unit area 131. Particularly, a state in which the imaging chip 113 is observed from the back surface (imaging surface) side is shown. In the pixel area, for example, more than 20 million pixels are arranged in a matrix. In the example of FIG. 4, 16 pixels of 4 × 4 adjacent pixels form one unit region 131. The grid lines in the figure show the concept of forming the unit area 131 by grouping adjacent pixels. The number of pixels forming the unit area 131 is not limited to this, and may be about 1000, for example, 32 pixels × 64 pixels, or more or less.

  As shown in the partial enlarged view of the pixel area, the unit area 131 in FIG. 4 includes four so-called Bayer arrangements composed of four pixels of green pixels Gb, Gr, blue pixels B, and red pixels R in up, down, left, and right directions. The green pixels Gb and Gr are pixels having a green filter as the color filter 102, and receive light in a green wavelength band of incident light. Similarly, the blue pixel B is a pixel having a blue filter as the color filter 102 and receives light in the blue wavelength band, and the red pixel R is a pixel having a red filter as the color filter 102 and has a red wavelength band. Receives light.

  In the present embodiment, a plurality of blocks are defined so as to include at least one unit area 131 per block, and each block can control pixels included in each block with different control parameters. That is, image pickup signals having different image pickup conditions can be acquired between a pixel group included in a certain block and a pixel group included in another block. Examples of the control parameters include a frame rate, a gain, a thinning rate, the number of rows or columns to be added for adding pixel signals, the charge accumulation time or the number of accumulations, and the number of digitization bits (word length). The imaging element 100 can perform not only thinning in the row direction (X-axis direction of the imaging chip 113) but also thinning in the column direction (Y-axis direction of the imaging chip 113). Further, the control parameter may be a parameter in image processing after acquiring an image signal from a pixel.

  FIG. 5 is a diagram illustrating a circuit in the unit area 131. In the example of FIG. 5, one unit area 131 is formed by nine pixels of 3 × 3 adjacent pixels. Note that, as described above, the number of pixels included in the unit region 131 is not limited to this, and may be smaller or larger. The two-dimensional positions of the unit area 131 are indicated by reference signs A to I.

  The reset transistors of the pixels included in the unit region 131 can be individually turned on and off for each pixel. In FIG. 5, a reset line 300 for turning on / off the reset transistor of the pixel A is provided, and a reset line 310 for turning on / off the reset transistor of the pixel B is provided separately from the reset line 300. Similarly, a reset line 320 for turning on / off the reset transistor of the pixel C is provided separately from the reset lines 300 and 310. A dedicated reset line for turning on / off each reset transistor is also provided for the other pixels D to I.

  The transfer transistors of the pixels included in the unit region 131 can also be individually turned on and off for each pixel. In FIG. 5, a transfer line 302 for turning on and off the transfer transistor of the pixel A, a transfer line 312 for turning on and off the transfer transistor of the pixel B, and a transfer line 322 for turning on and off the transfer transistor of the pixel C are separately provided. Dedicated transfer lines for turning on and off the respective transfer transistors are also provided for the other pixels D to I.

  Further, the selection transistors of the pixels included in the unit region 131 can also be individually turned on / off for each pixel. In FIG. 5, a selection line 306 for turning on / off the selection transistor of the pixel A, a selection line 316 for turning on / off the selection transistor of the pixel B, and a selection line 326 for turning on / off the selection transistor of the pixel C are separately provided. Dedicated selection lines for turning on / off the respective selection transistors are also provided for the other pixels D to I.

  Note that the power supply wiring 304 is commonly connected to the pixels A to I included in the unit region 131. Similarly, the output wiring 308 is commonly connected to the pixels A to I included in the unit area 131. Further, the power supply wiring 304 is commonly connected between a plurality of unit areas, but the output wiring 308 is provided separately for each unit area 131. The load current source 309 supplies a current to the output wiring 308. The load current source 309 may be provided on the imaging chip 113 side or may be provided on the signal processing chip 111 side.

  By individually turning on and off the reset transistor and the transfer transistor in the unit region 131, the charge including the charge accumulation start time, the charge end time, and the transfer timing is independent of the pixels A to I included in the unit region 131. The accumulation can be controlled. In addition, by individually turning on and off the selection transistors in the unit region 131, the pixel signals of the pixels I can be output from the pixels A via the common output wiring 308.

  Here, a so-called rolling shutter system is known in which charge accumulation is controlled in a regular order for rows and columns for pixels A to I included in the unit region 131. When a pixel is selected for each row by the rolling shutter method and then a column is specified, in the example of FIG. 5, pixel signals are output in the order of “ABCDEFGHI”.

  By configuring the circuit based on the unit regions 131 as described above, the charge accumulation time can be controlled for each unit region 131. In other words, it is possible to output pixel signals at different frame rates between the unit areas 131. In addition, while the unit area 131 included in another area is rested while the charge accumulation (imaging) is performed in the unit area 131 included in a part area in the imaging chip 113, the predetermined area of the imaging chip 113 is reduced. Only the imaging can be performed, and the pixel signal can be output. Further, by switching an area where charge accumulation (imaging) is performed (accumulation control target area) between frames, it is possible to sequentially perform imaging in different areas of the imaging chip 113 and output a pixel signal.

  FIG. 6 is a block diagram illustrating a functional configuration of the image sensor 100 corresponding to the circuit illustrated in FIG. The analog multiplexer 411 sequentially selects the nine PDs 104 forming the unit area 131 and outputs each pixel signal to the output wiring 308 provided corresponding to the unit area 131. The multiplexer 411 is formed on the imaging chip 113 together with the PD 104.

  The pixel signals output via the multiplexer 411 are converted into CDS and A / D signals by a signal processing circuit 412 formed on the signal processing chip 111 and performing correlated double sampling (CDS) / analog / digital (A / D) conversion. D conversion is performed. The A / D-converted pixel signal is delivered to a demultiplexer 413 and stored in a pixel memory 414 corresponding to each pixel. The demultiplexer 413 and the pixel memory 414 are formed on the memory chip 112.

  The arithmetic circuit 415 processes the pixel signal stored in the pixel memory 414 and delivers it to the subsequent image processing unit. The arithmetic circuit 415 may be provided in the signal processing chip 111 or may be provided in the memory chip 112. In FIG. 6, connections for one unit area 131 are shown, but actually, these exist for each unit area 131 and operate in parallel. However, the arithmetic circuit 415 may not be provided for each unit area 131. For example, one arithmetic circuit 415 may perform sequential processing while sequentially referring to the value of the pixel memory 414 corresponding to each unit area 131. Good.

  As described above, the output wiring 308 is provided for each of the unit regions 131. Since the image pickup device 100 has the image pickup chip 113, the signal processing chip 111, and the memory chip 112 stacked, by using the electrical connection between the chips using the bump 109 for the output wiring 308, each chip can be moved in the plane direction. The wiring can be routed without increasing the size.

<Explanation of distance measurement>
FIG. 7 is a diagram illustrating the positions of the focus detection pixels on the imaging surface of the imaging device 100. In the present embodiment, the focus detection pixels are arranged discretely along the X-axis direction (horizontal direction) of the imaging chip 113. In the example of FIG. 7, 15 focus detection pixel lines 60 are provided at predetermined intervals. The focus detection pixels constituting the focus detection pixel line 60 output image signals for distance measurement. Normal imaging pixels are provided at pixel positions other than the focus detection pixel line 60 in the imaging chip 113. The imaging pixel outputs an image signal for monitoring outside the vehicle.

  FIG. 8 is an enlarged view of a region including a part of one of the focus detection pixel lines 60. FIG. 8 illustrates a red pixel R, a green pixel G (Gb, Gr), and a blue pixel B, a focus detection pixel S1, and a focus detection pixel S2. The red pixel R, the green pixel G (Gb, Gr), and the blue pixel B are arranged according to the above-described Bayer arrangement rule.

  The square area illustrated for the red pixel R, the green pixel G (Gb, Gr), and the blue pixel B indicates a light receiving area of the imaging pixel. Each imaging pixel receives a light beam passing through the exit pupil of the imaging optical system 31 (FIG. 9). That is, each of the red pixel R, the green pixel G (Gb, Gr), and the blue pixel B has a square mask opening, and light passing through these mask openings reaches the light receiving section of the imaging pixel. .

  Note that the shape of the light receiving regions (mask openings) of the red pixel R, the green pixel G (Gb, Gr), and the blue pixel B is not limited to a square, and may be, for example, a circle.

  The semicircular region illustrated for the focus detection pixel S1 and the focus detection pixel S2 indicates a light receiving region of the focus detection pixel. That is, the focus detection pixel S1 has a semicircular mask opening on the left side of the pixel position in FIG. 8, and light passing through the mask opening reaches the light receiving portion of the focus detection pixel S1. On the other hand, the focus detection pixel S2 has a semicircular mask opening on the right side of the pixel position in FIG. 8, and light passing through the mask opening reaches the light receiving portion of the focus detection pixel S2. As described above, the focus detection pixel S1 and the focus detection pixel S2 respectively receive a pair of light beams passing through different regions of the exit pupil of the imaging optical system 31 (FIG. 9).

  Note that the position of the focus detection pixel line on the imaging chip 113 is not limited to the position illustrated in FIG. Also, the number of focus detection pixel lines is not limited to the example of FIG. Further, the shape of the mask opening in the focus detection pixel S1 and the focus detection pixel S2 is not limited to a semicircle, but may be, for example, a square light receiving region (a mask aperture) in the imaging pixel R, the imaging pixel G, and the imaging pixel B. ) May be formed in a rectangular shape divided in the horizontal direction.

Further, the focus detection pixel line in the imaging chip 113 may be a line in which focus detection pixels are arranged along the Y-axis direction (vertical direction) of the imaging chip 113. An image sensor in which imaging pixels and focus detection pixels are two-dimensionally arranged as shown in FIG. 8 is known, and detailed illustration and description of these pixels are omitted.
In the example of FIG. 8, the configuration in which the focus detection pixels S1 and S2 each receive one of a pair of light beams for focus detection, that is, a so-called 1PD structure has been described. Instead, for example, as disclosed in Japanese Patent Application Laid-Open No. 2007-282107, a configuration in which each focus detection pixel receives both of a pair of light beams for focus detection, that is, a so-called 2PD structure may be employed. With the 2PD structure, image data can be read from the focus detection pixels, and the focus detection pixels do not become defective pixels.

  In the present embodiment, a pair of images formed by a pair of light beams passing through different regions of the imaging optical system 31 (FIG. 9) based on the ranging image signals output from the focus detection pixels S1 and S2. By detecting the image shift amount (phase difference), the focus adjustment state (defocus amount) of the imaging optical system 31 is calculated.

  Generally, the pair of images approach each other in a so-called front focus state in which the imaging optical system 31 forms a sharp image of an object (for example, a preceding vehicle) before the planned focal plane, and conversely, the object is positioned behind the planned focal plane. In a so-called back focus state that forms a sharp image of In the in-focus state where a sharp image of the object is formed on the predetermined focal plane, the pair of images relatively match. Therefore, the relative positional deviation amount of the pair of images corresponds to the distance (depth information) to the object.

  The calculation of the defocus amount based on the phase difference is known in the field of cameras, and a detailed description thereof will be omitted. Here, since the defocus amount and the distance to the object correspond one-to-one, the distance from the camera 3 to each object is obtained by obtaining the defocus amount for each data of the imaged object. Can be. That is, distance measurement (distance measurement) to the object can be performed at a plurality of positions on the shooting screen. The relationship between the defocus amount and the distance to the object is prepared in advance as a mathematical expression or a lookup table and stored in the nonvolatile memory 35b (FIG. 9).

<Explanation of camera>
FIG. 9 is a block diagram illustrating the configuration of the camera 3 having the above-described image sensor 100. 9, the camera 3 includes an imaging optical system 31, an imaging unit 32, an image processing unit 33, a work memory 34, a control unit 35, and a recording unit 36.

  The imaging optical system 31 guides a light beam from the object scene to the imaging unit 32. The imaging unit 32 includes the imaging element 100 and the driving unit 32a, and photoelectrically converts the image of the object formed on the imaging chip 113 by the imaging optical system 31. The drive unit 32a generates a drive signal required to cause the image sensor 100 (the image pickup chip 113) to perform independent accumulation control in the above-described block units. The instructions such as the position and shape of the block, its range, and the accumulation time are transmitted from the control device 4 to the drive unit 32a.

  The image processing unit 33 performs image processing on the image data captured by the imaging unit 32 in cooperation with the work memory 34. The image processing unit 33 performs image processing such as contour enhancement processing and gamma correction.

  The work memory 34 temporarily stores image data before and after image processing. The recording unit 36 records image data and the like on a storage medium including a nonvolatile memory. The control unit 35 is configured by, for example, a CPU, and controls the entire operation of the camera 3 according to a control signal from the control device 4. For example, a predetermined exposure calculation is performed based on an image signal captured by the imaging unit 32, and the accumulation time of the imaging chip 113 required for proper exposure is instructed to the driving unit 32a.

  The control unit 35 includes a distance measurement calculation unit 35a and a nonvolatile memory 35b. The distance measurement calculation unit 35a measures the distance (distance measurement) to the object at a plurality of positions on the shooting screen as described above. The image data obtained by the camera 3 and the distance measurement data calculated by the camera 3 are sent to the control device 4 (FIG. 1). The non-volatile memory 35b stores a program executed by the control unit 35 and information necessary for distance measurement.

<Block control of image sensor>
The control device 4 causes the image pickup device 100 (the image pickup chip 113) of the camera 3 to perform the independent accumulation control for each block described above. In the present embodiment, appropriate imaging conditions are set for different regions on the imaging surface of the imaging chip 113 in order to make it easier to find the line of the road when the traveling environment changes, for example, when traveling in a tunnel or when raining. .

  FIG. 10 is a diagram schematically illustrating an image of a subject (object) formed on the imaging chip 113. Although an inverted inverted image is actually formed, it is illustrated as an erect upright image for easy understanding. In FIG. 10, the vehicle 1 is traveling behind the preceding vehicle 84. The imaging surface (imaging area) 70 of the imaging chip 113 includes images of white lines 82a, 82b, 82c provided on the road, a preceding vehicle 84, and an oncoming vehicle 85. Among these objects, the white line 82a and the white line 82b indicating the boundary of the vehicle lane (traveling lane) in which the vehicle 1 travels are particularly important objects for the control device 4 to detect the traveling path. Therefore, in this description, the imaging region including the white lines 82a and 82b is referred to as a region of interest 71.

  The control device 4 sets different conditions between the attention area 71 on the imaging surface 70 and the other area 72 (non-interest area) to perform charge accumulation (imaging). Here, the size and position of the attention area 71 and the non-interest area 72 on the imaging surface 70 of the imaging chip 113 are also one of the imaging conditions.

The control device 4 controls the imaging by setting the first condition for the unit area 131 (FIG. 4) included in the attention area 71, and controls the unit area 131 included in the non-interest area 72 to the second condition. Is set to control imaging.
Note that a plurality of regions of interest 71 and non-regions of interest 72 may be provided, or a plurality of regions having different charge accumulation controls (imaging conditions) may be provided in the region of interest 71 or the non-region of interest 72. Furthermore, a rest area in which charge accumulation (imaging) is not performed in the row direction and the column direction of the imaging surface 70 may be provided.

<Explanation of flowchart>
Hereinafter, control processing of the camera 3 and setting of imaging conditions will be mainly described with reference to flowcharts (FIGS. 11 and 12). FIG. 11 is a flowchart illustrating an overall flow of control processing of the camera 3 executed by the control device 4. A program for executing the processing according to the flowchart in FIG. 11 is stored in the storage unit 4b of the control device 4. For example, when power supply is started from the vehicle 1 (system is turned on) or the engine is started, the control device 4 starts a program for performing the processing in FIG. 11.

  In step S10 of FIG. 11, the control device 4 determines whether or not the flag a = 0. The flag a is a flag that is set to 1 when the initialization has been completed and set to 0 when the initialization has not been completed. When the flag a = 0, the control device 4 makes an affirmative decision in step S10 and proceeds to step S20, and when the flag a ≠ 0, makes a negative decision in step S10 and proceeds to step S30.

  In step S20, the control device 4 performs an initial setting for the camera 3, and proceeds to step S30. The initial setting is to perform a predetermined setting for causing the camera 3 to perform a predetermined operation. Thereby, the camera 3 sets the same imaging conditions over the entire area of the imaging surface of the imaging element 100, and starts imaging at a frame rate of, for example, 60 frames per second (60 fps).

In step S30, the control device 4 performs an imaging condition setting process, and proceeds to step S40. The imaging condition setting process is a process of setting a region of interest 71 (FIG. 10) and the other non-region of interest 72 (FIG. 10) for the image sensor 100 of the camera 3 and determining the respective imaging conditions. Details of the imaging condition setting process will be described later. In the present embodiment, the frame rate of the attention area 71 is set higher than that of the non-interest area 72, the gain is increased, the thinning rate is reduced, and the accumulation time is set shorter. The camera 3 performs imaging based on this setting, and performs the above-described distance measurement (distance measurement).
It is not necessary to change all of the frame rate, the gain, the thinning rate, the accumulation time, and the like between the attention area 71 and the non-interest area 72, and at least one of them may be different.

  In step S40 of FIG. 11, the control device 4 acquires the image data, the distance measurement data, and the information from each unit in the vehicle 1 acquired by the camera 3 after the imaging condition setting process, and proceeds to step S50. In step S50, the control device 4 determines whether or not a setting for displaying information has been made. When the display setting has been performed, the control device 4 makes an affirmative decision in step 50 and proceeds to step S60. When the display setting is not performed, the control device 4 makes a negative determination in step 50 and proceeds to step S70.

In step S60, the control device 4 sends out display information to the display device 14 (FIG. 1), and proceeds to step S70. The display information is information according to the state of the vehicle 1 determined in the imaging condition setting process (S30), and for example, a message such as "entering a tunnel" or "headlight turned on" is displayed on the display device 14. Display.
Instead of transmitting the display information or together with the transmission of the display information, an audio signal for reproducing the message may be transmitted to an audio reproducing device (not shown). Also in this case, a voice device of a navigation device (not shown) may be used as a voice reproducing device (not shown).

  In step S70, the control device 4 determines whether or not an off operation has been performed. For example, when receiving an off signal (for example, a system off signal or an engine off signal) from the vehicle 1, the control device 4 makes an affirmative determination in step S70, performs a predetermined off process, and ends the process in FIG. For example, when not receiving the off signal from the vehicle 1, the control device 4 makes a negative determination in step S70 and returns to step S30. When returning to step S30, the above processing is repeated.

<Imaging condition setting process>
The details of the imaging condition setting processing (S30) will be described with reference to the flowchart of FIG. In the present embodiment, the imaging conditions for the attention area and the non-interest area are determined by exemplifying five changes in the traveling environment (road conditions) of the vehicle 1.

  In step S31 of FIG. 12, the control device 4 performs the above-described image processing on the image data acquired by the camera 3, and proceeds to step S33. In step S33, the control device 4 detects the white line data from the image data, and proceeds to step S35. The control device 4 sets the region including the white line 82a and the white line 82b in the photographing region 70 as the attention region 71 as described above. Referring to FIG. 10, a trapezoidal region (indicated by a broken line) including three of the white line 82a, the white line 82b, and the region between the white line 82a and the white line 82b is set as the attention region 71.

  In step S35, the control device 4 sets the noted area 71 as the first imaging area 71 and sets an area other than the first imaging area 71 as the second imaging area 72, and proceeds to step S37.

  In step S37, the control device 4 sends an instruction to the camera 3 to set the frame rate of the first imaging area 71 higher than the frame rate of the second imaging area 72. For example, the frame rate of the first imaging area 71 is 120 frames per second (120 fps), and the frame rate of the second imaging area 72 is 60 fps. This is to increase the frequency of acquiring information on a white line, which is a target to be noted during traveling. The processing up to step S37 described above is an example of processing under a normal driving environment (for example, driving in fine daylight).

  The control device 4 changes the imaging conditions for the attention area and the non-interest area when the traveling environment is different from the normal environment. FIG. 13 is a flowchart illustrating a process in the case of a change in the first driving environment (road condition). FIG. 14 is a diagram schematically illustrating an image of a subject (object) formed on the imaging chip 113 when the vehicle 1 approaches the entrance of the tunnel 83.

  In step S110 of FIG. 12, the control device 4 determines whether there is a tunnel. When the image acquired by the camera 3 includes the entrance of the tunnel, the control device 4 makes an affirmative decision in step S110 and proceeds to step S111 in FIG. If the image acquired by the camera 3 does not include the entrance of the tunnel, the control device 4 makes a negative determination in step S110 and proceeds to step S120. Note that, when a positive determination is made in step S110 at the time of the previous determination, but the control now proceeds to a negative determination in step S110, the control device 4 cancels the setting of the frame rate based on the flowchart of FIG.

  In step S111 of FIG. 13, the control device 4 detects a bright portion outside the tunnel 83 and a dark portion inside the tunnel 83 from the image acquired by the camera 3, and proceeds to step S112. For example, a bright portion and a dark portion (oblique lines) are detected from the subject image illustrated in FIG.

  In step S112 in FIG. 13, the control device 4 sets the dark area of the first imaging area 71 as the third imaging area 71a and sets the third imaging area 71a in the first imaging area 71 as shown in FIG. The region excluding is referred to as a remaining region 71b. That is, the first imaging region 71 is divided into a third imaging region 71a and a remaining region 71b.

  In step S113, the control device 4 sends an instruction to the camera 3 to set the frame rate of the third imaging area 71a lower than the frame rate of the remaining area 71b. The control device 4 reduces, for example, the frame rate of the third imaging region 71a to 60 fps. This is to obtain clear image information from the dark third imaging region 71a inside the tunnel 83. On the other hand, the frame rate of the remaining area 71b is kept at 120 fps.

  In step S114, the control device 4 further sets the dark area of the second imaging area 72 as the fourth imaging area 72a and excludes the fourth imaging area 72a of the second imaging area 72, as shown in FIG. The area is referred to as a remaining area 72b. That is, the second imaging region 72 is divided into a fourth imaging region 72a and a remaining region 72b.

In step S115, the control device 4 sends an instruction to the camera 3 to set the frame rate of the fourth imaging area 72a lower than the frame rate of the remaining area 72b. The control device 4 reduces, for example, the frame rate of the fourth imaging region 72a to 30 fps. This is to obtain clear image information from the dark fourth imaging region 72a inside the tunnel 83. Then, control device 4 proceeds to step S40 (FIG. 11).
In the above description, the case of the tunnel entrance has been described, but the same applies to the case of the tunnel exit. However, in the case of the tunnel entrance and the case of the tunnel exit, the relationship between the bright part and the dark part in the image is reversed, and the image outside the tunnel viewed from inside the tunnel becomes the bright part.

  FIG. 15 is a flowchart illustrating a process in the case of a change in the second traveling environment (road condition). FIG. 16 is a diagram schematically illustrating an image of a subject (object) formed on the imaging chip 113 when the vehicle 1 turns on the headlights (headlights). FIG. FIG. 16B shows a high beam state, and FIG. 16B shows a low beam state.

  In step 120 of FIG. 12, the control device 4 determines whether or not the headlight is turned on. The control device 4 makes an affirmative determination in step S120 when the image acquired by the camera 3 includes a bright portion by the headlight, and proceeds to step S121 in FIG. When the image acquired by the camera 3 does not include the bright part of the headlight, the control device 4 makes a negative determination in step S120 and proceeds to step S130. The control device 4 makes an affirmative determination in step S120 at the time of the previous determination, but when the control now turns to a negative determination in step S120, the controller 4 cancels the setting of the frame rate based on the flowchart of FIG.

In addition, instead of determining whether the headlight is on based on the detection of a bright portion in the image, the presence of lighting may be determined based on a lighting operation by the driver. In this case, the control device 4 treats the position of the irradiation area 86a in FIG. 16A as a bright portion of the image acquired by the camera 3 when the beam switch 18 is switched to the high beam side. When the beam switch 18 is switched to the low beam side, the controller 4 treats the position of the irradiation area 86b in FIG. 16B as a bright portion of the image acquired by the camera 3.
In step S121 in FIG. 15, the control device 4 detects an irradiation area (corresponding to the bright portion) by the headlight from the image acquired by the camera 3, and proceeds to step S122.

  In step S122, when the beam changeover switch 18 has been switched to the high beam side, the control device 4 performs fifth imaging on the area where the first imaging area 73 and the irradiation area 86a overlap as shown in FIG. A region 73a is defined, and a region of the first imaging region 73 excluding the fifth imaging region 73a is defined as a remaining region 73b. That is, the first imaging region 73 is divided into a fifth imaging region 73a and a remaining region 73b.

  In step S123, the control device 4 sends an instruction to the camera 3 to set the frame rate of the fifth imaging area 73a to be higher than the frame rate of the remaining area 73b. For example, when the frame rate of the remaining area 73b is 60 fps, the control device 4 increases the frame rate of the fifth imaging area 73a to 120 fps. This is to increase the frequency of acquiring image information from the fifth imaging region 73a that has been illuminated and brightened during the high beam.

  In step S124, the control device 4 further sets a region where the second imaging region 74 and the irradiation region 86a overlap with each other as a sixth imaging region 74a as shown in FIG. An area excluding the imaging area 74a is referred to as a remaining area 74b. That is, the second imaging region 74 is divided into a sixth imaging region 74a and a remaining region 74b.

  In step S125, the control device 4 sends an instruction to the camera 3 to set the frame rate of the sixth imaging region 74a higher than the frame rate of the remaining region 74b. For example, when the frame rate of the remaining area 74b is 60 fps, the frame rate of the sixth imaging area 74a is increased to 120 fps. This is to increase the frequency of acquiring image information from the sixth imaging region 74a that has been illuminated and brightened during the high beam. Then, control device 4 proceeds to step S40 (FIG. 11).

  Although the case of the high beam has been described with reference to FIG. 16A, the same can be applied to the case of the low beam. When the beam changeover switch 18 is set to the low beam side, the irradiation area 86b in FIG. 16B corresponds to a bright portion of the subject image. Then, in the comparison between FIG. 16A and FIG. 16B, the fifth imaging region 73a corresponds to the fifth imaging region 73c, the remaining region 73b corresponds to the remaining region 73d, and the sixth imaging region 74a corresponds to The remaining area 74b corresponds to the sixth imaging area 74c, and the remaining area 74b corresponds to the remaining area 74d.

  FIG. 17 is a flowchart illustrating a process in the case of a change in the third driving environment (road condition). FIG. 18 is a diagram schematically illustrating an image of a subject (target object) formed on the imaging chip 113 when the preceding vehicle 84 turns on the tail lamp (tail lamp).

  In step 130 of FIG. 12, the control device 4 determines whether or not the tail lamp of the preceding vehicle 84 has been recognized. When the control device 4 recognizes the lit tail lamp from the image acquired by the camera 3, the control device 4 makes an affirmative determination in step S130 and proceeds to step S131 in FIG. When the control device 4 does not recognize the lit tail lamp from the image acquired by the camera 3, the control device 4 makes a negative determination in step S130 and proceeds to step S140. When the determination in step S130 is affirmative at the time of the previous determination, but the control now proceeds to the negative determination in step S130, the control device 4 cancels the setting of the frame rate based on the flowchart of FIG. 17 described later.

  In step S131 of FIG. 17, the control device 4 detects the tail lamp of the preceding vehicle 84 from the image acquired by the camera 3, and proceeds to step S132. In FIG. 18, the image of the tail lamp 84a is included in the second imaging region 76 instead of the first imaging region 75 of FIG.

  In step S132 of FIG. 17, the control device 4 sets a predetermined area including the image of the tail lamp 84a of the preceding vehicle 84, for example, a rectangular area including the tail lamps 84a on both sides as the seventh imaging area 87. In step S133, the control device 4 separates the seventh imaging region 87 from the second imaging region 76 to perform the charge accumulation control under the same condition as that of the first imaging region 75. It is incorporated into the imaging area 75. The shape of the seventh imaging region 87 is not limited to a rectangle, but may be an ellipse or a trapezoid including tail lamps 84a on both sides.

  The control device 4 further sends an instruction to the camera 3 to set the frame rate of the seventh imaging area 87 to the same frame rate as that of the first imaging area 75. For example, when the frame rate of the first imaging area 75 is 120 fps, the frame rate of the seventh imaging area 87 is also increased to 120 fps. This is to increase the frequency of acquiring image information for the seventh imaging region 87 corresponding to the preceding vehicle 84. Then, control device 4 proceeds to step S40 (FIG. 11).

  FIG. 19 is a flowchart illustrating a process in the case of a change in the fourth traveling environment (road condition). FIG. 20 is a diagram schematically illustrating an image of a subject (object) formed on the imaging chip 113 when rainfall is detected.

  In step S140 of FIG. 12, the control device 4 determines whether there is rainfall information from the rainfall sensor 19 (FIG. 1). When rainfall information has been input, the control device 4 makes an affirmative determination in step S140, and proceeds to step S141 in FIG. When rainfall information is not input, control device 4 makes a negative determination in step S140 and proceeds to step S150. Note that, when the determination in step S140 is affirmative at the time of the previous determination, but the process now turns to a negative determination at step S140, the control device 4 cancels the setting of the frame rate based on the flowchart of FIG.

  Generally, during rainfall, the road surface becomes wet, so that it is difficult to identify a white line drawn on the road. Specifically, the contrast between the white line portion and the portion other than the white line is reduced as compared with the dry road surface.

  In step S141 of FIG. 19, the control device 4 sets the attention area in the image of the subject (object) formed on the imaging chip 113 as the first imaging area 77, as in step S35 of FIG. An area other than the first imaging area 77 is defined as a second imaging area 78 to divide them, and the process proceeds to step S142. 20, the control device 4 sets a trapezoidal region including the white line 82a, the white line 82b, and the region between the white line 82a and the white line 82b as the first imaging region 77, An area other than 77 is defined as a second imaging area 78.

In step S142 of FIG. 19, the control device 4 sends an instruction to the camera 3 to set the frame rate of the first imaging region 77 to be lower than the frame rate of the first imaging region 71 (FIG. 10). For example, when the frame rate of the first imaging area 71 is 120 fps, the frame rate of the first imaging area 77 is changed to 60 fps. This is to obtain clear image information from the first imaging region 77 in which the brightness of the white portion is reduced due to the wet road surface. Then, control device 4 proceeds to step S150 in FIG.
Instead of lowering the frame rate, the contrast of the image may be increased by adjusting the gradation curve.
For the same reason, the frame rate of the second imaging region 78 may be set lower than the frame rate of the second imaging region 72 (FIG. 10), or the contrast of the image may be reduced by adjusting the gradation curve. May be increased.

  FIG. 21 is a flowchart illustrating a process in the case of a change in the fifth driving environment (road condition). FIG. 22 is a diagram schematically illustrating an image of a subject (object) formed on the imaging chip 113 when a change in the course is detected. FIG. 22A shows the subject image before the lane change, and FIG. 22B shows the subject image during the lane change.

  In step 150 of FIG. 12, the control device 4 determines whether there is information on lane change. The navigation device (not shown) performs route guidance on which road (which lane) the vehicle 1 should travel in, for example, by comparing the position information input from the GPS device 15 with the map information. . When a lane change of the vehicle 1 is necessary, a lane change instruction is input to the control device 4 from the navigation device. When a lane change instruction is input, control device 4 makes an affirmative determination in step S150 and proceeds to step S151 in FIG. If a lane change instruction is not input, control device 4 makes a negative determination in step S150 and proceeds to step S40 in FIG. When the determination in step S150 is affirmative at the time of the previous determination, but the control now proceeds to the negative determination in step S150, the control device 4 cancels the setting of the frame rate based on the flowchart of FIG.

  In step S151 in FIG. 21, the control device 4 changes the first imaging region 79 in the image of the subject (object) formed on the imaging chip 113 as follows.

  Referring to FIG. 22 (a), the control device 4 determines a trapezoidal region including a white line 87a, a white line 87b, and a region between the white line 87a and the white line 87b as in the case of FIG. The first imaging region 79 is defined as a region other than the first imaging region 79, and the second imaging region 80 is defined as a second imaging region 80.

When the position information input from the GPS device 15 corresponds to the lane change position, the control device 4 changes the first imaging region 79 to the first imaging region 79A as described below. Referring to FIG. 22 (b), the new regions of interest are the white line 87b and the white line 87c that define the lane change destination, so the first imaging region 79A includes the white lines 87b and 87c. It is trapezoidal. The control device 4 gradually shifts the imaging region from the first imaging region 79 shown in FIG. 22A to the first imaging region 79A shown in FIG. 22B. The process of shifting the imaging region in this manner is continuously performed according to the change in the scene due to the lane change.
Note that instead of the operation of shifting the imaging region, the first imaging region 79 may be enlarged to include the first imaging region 79A.

  In step S152 of FIG. 21, the control device 4 sends an instruction to the camera 3 to cause the newly set frame rate of the first imaging region 79A to be set to the same frame rate as the first imaging region 79. This makes it possible to keep the frame rate of the attention area, which is the lane change destination, when changing the course, the same as the frame rate of the attention area before the lane change. Then, control device 4 proceeds to step S40 (FIG. 11).

According to the above-described embodiment, the following operation and effect can be obtained.
(1) The control device 4 sets the first imaging region 71 including the white lines 82a and 82b and the second imaging region 72 other than the first imaging region 71 in the camera 3, and sets the imaging conditions of the first imaging region 71. Are set differently from the imaging conditions of the second imaging region 72. For example, since the frame rate of the first imaging region 71 is set higher than the frame rate of the second imaging region 72, the white lines 82a and 82b can be reliably recognized in the image acquired by the camera 3. In addition, since the frame rate of the second imaging region 72 that does not contribute to the recognition of the white lines 82a and 82b is reduced, the power consumption of the camera 3 can be reduced and heat generation can be suppressed.
When the frame rate is high, the accumulation time is short, and when the frame rate is low, the accumulation time is long.

(2) Even when the traveling environment of the vehicle 1 changes, the control device 4 divides the first imaging region 71 into two imaging regions according to the brightness in the first imaging region 71, and sets the frame rate in each of the imaging regions. , The white lines 82a and 82b can be reliably recognized in the image acquired by the camera 3.
For example, in the case of the entrance of the tunnel 83 shown in FIG. 14, the dark portion in the first imaging region 71 is set as the third imaging region 71a, the other region is set as the remaining region 71b, and the frame of the third imaging region 71a is set. Since the rate is set lower than the frame rate of the remaining area 71b, the white lines 82a and 82b in the third imaging area 71a can be clearly recognized in the image acquired by the camera 3. By performing the same control for the second imaging region 72 as for the first imaging region 71, the imaging region related to traveling can be clearly recognized in the image acquired by the camera 3.

  For example, in the case of the high beam lighting shown in FIG. 16A, an area where the first imaging area 73 and the irradiation area 86a overlap is defined as a fifth imaging area 73a, and other areas are defined as remaining areas 73b. Since the frame rate of the fifth imaging region 73a is set higher than the frame rate of the remaining region 73b, the white lines 82a and 82b in the fifth imaging region 73a can be reliably recognized in the image acquired by the camera 3. By performing the same control for the second imaging region 72 as for the first imaging region 71, it is possible to increase the information amount of the imaging region related to traveling in the image acquired by the camera 3.

(3) As an example of a change in the traveling environment of the vehicle 1, when the tail lamp of the preceding vehicle 84 is recognized (FIG. 18), the area including the tail lamps 84a on both sides is set as the seventh imaging area 87, and the frame of the seventh imaging area 87 is set. By making the rate equal to the frame rate of the first imaging region 75, the image of the tail lamp 84a of the preceding vehicle 84 can be reliably recognized in the image acquired by the camera 3.

(4) As an example of a change in the traveling environment of the vehicle 1, when rainfall is detected (FIG. 20), the frame rate of the first imaging region 77 is set lower than the frame rate before rainfall. In the image, the white lines 82a and 82b having reduced contrast on a wet road surface can be easily recognized.

(5) As an example of a change in the traveling environment of the vehicle 1, when changing lanes (FIG. 22), the frame rate of the first imaging area 79A including the white lines 87b and 87c at the lane change destination is set to the first imaging area before the lane change. Since the frame rate is set to be the same as the frame rate of 79, the frame rate of the first imaging region before and after the lane change is kept constant, and the white line before and after the lane change can be reliably recognized in the image acquired by the camera 3.

The following modifications are also within the scope of the present invention, and one or more of the modifications can be combined with the above-described embodiment.
(Modification 1)
In the above-described embodiment, an example in which the camera 3 is controlled by the control of the control device 4 has been described. However, a configuration in which part of the control of the camera 3 is performed by the control unit 35 of the camera 3 may be employed.
Further, in the above embodiment, the frame rate is mainly described among the shooting conditions. However, the shooting conditions other than the frame rate may be changed for each shooting (imaging) area.

(Modification 2)
In the above description, an example has been described in which a white line drawn on a road surface along a road is recognized as a target object. However, not only white lines but also guardrails and curbs provided along the road are also noted. It may be included in the line as the target object.

(Modification 3)
In the above-described embodiment, an example has been described in which the imaging surface 70 is divided into a trapezoidal first imaging region and a second imaging region other than the first imaging region, and the respective frame rates are different. Instead, the imaging surface 70 may be divided into three or more imaging regions of a first imaging region, a second imaging region, and a third imaging region, and the frame rates of the imaging regions may be different. .
In the above-described embodiment, an example in which the first imaging region and the second imaging region are further divided into two imaging regions has been described. Instead, each of the first imaging region and the second imaging region may be divided into three or more imaging regions, and the frame rates of the three or more subdivided imaging regions may be different. Good.

(Modification 4)
In the above-described embodiment, the control device 4 is triggered by input of a lane change instruction from a navigation device (not shown) to trigger the first imaging region 79 shown in FIG. 22 (a) to the first imaging region 79 shown in FIG. 22 (b). The imaging area was gradually shifted to one imaging area 79A. Instead, the timing at which the lane change is performed may be determined by the control device 4. The control device 4 receives supply of point information for changing lanes from the car navigation device of the vehicle 1 or stores point information in the storage unit 4b in advance. The control device 4 determines lane change when the position information input from the GPS device 15 matches the point information at which the lane change is performed. When detecting the lane change, the control device 4 detects the lane change destination lane in the image acquired by the camera 3 and then moves from the first imaging region 79 shown in FIG. 22 (a) to the first imaging region 79 shown in FIG. 22 (b). The imaging area is shifted to the imaging area 79A.

(Modification 5)
In the above description, as the distance measurement performed by the camera 3, a method of calculating by distance measurement using an image signal from a focus detection pixel provided in the image sensor 100 is used. A method of performing distance measurement using an image may be used. Alternatively, a method of performing distance measurement using a millimeter wave radar separately from the camera 3 may be used.

  Although various embodiments and modified examples have been described above, the present invention is not limited to these contents. Other embodiments that can be considered within the scope of the technical concept of the present invention are also included in the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Vehicle 2 ... Driving assistance device 3 ... Camera 4 ... Control device 4b ... Storage part 5 ... 1st traveling control unit 6 ... 2nd traveling control unit 12 ... Vehicle speed sensor 14 ... Display device 15 ... GPS device 18 ... Beam Changeover switch 19 rainfall sensor 31 imaging optical system 35 control unit 35a distance measurement calculation units 71, 73, 75, 77, 79, 79A first imaging regions 72, 74, 76, 78, 80 second imaging Area 71a Third imaging area 72a Fourth imaging area 73a, 73c Fifth imaging area 74a, 74c Sixth imaging area 87 Seventh imaging area 100 Image sensor 113 Imaging chip

Claims (4)

In an imaging device mounted on a car,
An imaging unit having a plurality of imaging regions with different imaging conditions,
An input unit for inputting information on an irradiation direction of the car light,
An imaging control unit that controls imaging conditions of the imaging unit,
The imaging control unit changes an imaging condition of a first imaging region that is a part of the imaging unit according to the information, and is a part of the imaging unit of the imaging unit; An imaging device that sets an imaging condition of a second imaging region different from the first imaging region to that of the first imaging region . The imaging device according to claim 1 ,
The imaging device, wherein the imaging control unit changes a position of the first imaging region according to the information. The imaging apparatus according to claim 1 or 2,
The imaging device, wherein the imaging control unit changes an imaging speed of the first imaging region according to the information. The imaging device according to any one of claims 1 to 3 ,
The imaging apparatus, wherein the imaging control unit changes a thinning rate of a pixel in the first imaging area according to the information.

Images