JP2015228707A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To properly set imaging conditions of an imaging apparatus.SOLUTION: The imaging apparatus includes: an input part, mounted to a vehicle, for inputting information concerning pedestrians around the vehicle; an imaging part for imaging an outside part of the vehicle; and an imaging control part for changing imaging conditions on the basis of the information concerning the pedestrians around the vehicle and a direction of traveling of the vehicle.

Description

  The present invention relates to an imaging apparatus.

Based on the image acquired by the camera mounted on the vehicle, the driving environment of the vehicle is detected, and based on the detected driving environment data, automatic driving control such as driving following the preceding vehicle, warning, braking, steering assistance, etc. A technology for supporting driving has been developed (see Patent Document 1).
In the prior art, a solid-state imaging device such as a CCD is used for an in-vehicle camera. In-vehicle cameras that continue to acquire images of roads play an important role in automatic driving control and driving support. However, there are not many proposals for cameras that are premised to be installed in vehicles, and the usability of cameras is not sufficient. It was.

JP 2010-79424 A

  An image pickup apparatus according to the present invention is mounted on a vehicle and inputs an information on a pedestrian around the vehicle, an image pickup unit that images the outside of the vehicle, information on a pedestrian around the vehicle, and the An imaging control unit that changes imaging conditions of the imaging unit based on a traveling direction of the vehicle.

It is a schematic block diagram of the driving assistance device of a vehicle. It is a block diagram which illustrates the composition of a control device. It is sectional drawing of a multilayer type image pick-up element. It is a figure explaining the pixel arrangement | sequence and unit area | region of an imaging chip. It is a figure explaining the circuit of a unit area. It is a block diagram which shows the functional structure of an image pick-up element. It is a block diagram which illustrates the composition of a camera. It is the figure which expanded the area | region containing a part of focus detection pixel line. It is a block diagram which illustrates the composition of the camera which has an image sensor. It is a figure which illustrates the image pick-up surface of an image pick-up chip, an image pick-up field, an attention field, and a rest field. It is a flowchart explaining the flow of the control process of the camera which a control part performs. It is a flowchart explaining the detail of an initial setting process. It is a figure which illustrates the table of an initial setting value. It is a figure which illustrates the image pick-up surface of an image pick-up chip, an image pick-up field, an attention field, and a rest field. It is a figure which illustrates the image pick-up surface of an image pick-up chip, an image pick-up field, an attention field, and a rest field. It is a figure which illustrates the image pick-up surface of an image pick-up chip, an image pick-up field, an attention field, and a rest field. It is a flowchart explaining the detail of a driving assistance setting process. It is a figure explaining flag Em. It is a figure explaining distance Z. FIG. 20A is a diagram illustrating the movement of the position of the region of interest and the change in size when turning right at an intersection of a general road. FIG. 20B is a diagram illustrating the movement and size change of the region of interest when changing lanes while accelerating on an expressway. It is a figure explaining the size change of a turn signal direction and an attention area. It is a flowchart explaining the process at the time of operation of the turn signal switch by the modification 1.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.
<When to use the camera>
FIG. 1 is a schematic configuration diagram of a driving support device 2 for a vehicle 1 equipped with a camera 3 according to an embodiment of the present invention. In FIG. 1, a driving support device 2 is mounted on a vehicle 1 such as an automobile. The driving support device 2 includes a camera 3, a control device 4, a first travel control unit 5, a second travel control unit 6, and the like. In this description, an example in which an internal combustion engine is used as a drive source will be described, but a motor may be used as a drive source.

  The camera 3 includes an imaging optical system having a plurality of lenses and an imaging element (in this embodiment, a laminated imaging element (see FIG. 3)), and is attached, for example, in front of the ceiling in the vehicle interior. The camera 3 is directed to the front of the vehicle 1, and its mounting height (distance from the ground to the camera 3) is adjusted to, for example, 1.4 (m). The camera 3 acquires an image in the traveling direction of the vehicle 1 and performs distance measurement (ranging) to each subject (target object) at a plurality of positions in the shooting screen based on the acquired image. The distance measurement is calculated by a distance measurement calculation using an image signal from a focus detection pixel provided in the multilayer image sensor. The focus detection pixels and distance measurement will be described later. Image data and distance measurement data acquired by the camera 3 are sent to the control device 4. The cameras 3 may be provided outside the vehicle, the cameras 3 inside and outside the vehicle may cooperate, and the number of cameras 3 may be set as appropriate. For example, the camera 3 outside the vehicle may be used for white line detection, which will be described later, and the camera 3 inside and outside the vehicle may cooperate with each other to recognize an object or an obstacle.

  As illustrated in FIG. 2, the control device 4 includes a CPU 4a and a storage unit 4b. Based on various programs stored in the storage unit 4b, the CPU 4a performs various calculations using control parameters stored in the storage unit 4b, detection signals from sensors described later, and the like.

  The first travel control unit 5 performs constant speed travel control and follow-up travel control based on an instruction from the control device 4. The constant speed traveling control is control for causing the vehicle 1 to travel at a constant speed based on a predetermined control program. The follow-up running control is constant with respect to the preceding vehicle when the speed of the preceding vehicle recognized by the control device 4 is equal to or lower than the target speed set in the vehicle 1 during the constant speed running control. It is the control which makes it drive | work in the state holding the distance between two cars.

  The second traveling control unit 6 performs driving support control based on an instruction from the control device 4. Based on a predetermined control program, the driving support control outputs a steering control signal to the steering control device 9 so that the vehicle 1 travels along the road, or avoids the vehicle 1 from colliding with an object. In this control, a brake control signal is output to the brake control device 8.

  1 further includes a throttle control device 7, a brake control device 8, a steering control device 9, a steering wheel 10, a turn signal switch 11, a vehicle speed sensor 12, a yaw rate sensor 13, a display device 14, A GPS device 15, a shift lever position detection device 16, and a microphone 17 are shown.

  The throttle control device 7 controls the opening degree of a throttle valve (not shown) according to the depression amount of the accelerator pedal 7a. The throttle control device 7 also controls the opening degree of the throttle valve in accordance with a throttle control signal sent from the first travel control unit 5. The throttle control device 7 further sends a signal indicating the depression amount of the accelerator pedal 7 a to the control device 4.

  The brake control device 8 controls the opening degree of a brake valve (not shown) according to the depression amount of the brake pedal 8a. The brake control device 8 also controls the opening degree of the brake valve in accordance with a brake control signal from the second travel control unit 6. The brake control device 8 further sends a signal indicating the depression amount of the brake pedal 8 a to the control device 4.

  The steering control device 9 controls the steering angle of a steering device (not shown) according to the rotation angle of the steering wheel 10. The steering control device 9 also controls the steering angle of the steering device according to a steering control signal from the second travel control unit 6. The steering control device 9 further sends a signal indicating the rotation angle of the steering wheel 10 to the first traveling control unit 5 and the control device 4, respectively.

  The turn signal switch 11 is a switch for operating a turn signal (winker) device (not shown). The turn signal device is a blinking light emitting device that indicates a change in the course of the vehicle 1. When the turn signal switch 11 is operated by an occupant of the vehicle 1, an operation signal from the turn signal switch 11 is sent to the turn signal device, the second traveling control unit 6, and the control device 4. The vehicle speed sensor 12 detects the vehicle speed V of the vehicle 1 and sends detection signals to the first travel control unit 5, the second travel control unit 6, and the control device 4.

  The yaw rate sensor 13 detects the yaw rate of the vehicle 1 and sends detection signals to the second traveling control unit 6 and the control device 4, respectively. The yaw rate is the rate of change of the rotation angle of the vehicle 1 in the turning direction. The display device 14 displays information indicating a control state by the first traveling control unit 5 and the second traveling control unit 6. The display device 14 is configured by, for example, a HUD (Head Up Display) that projects information on a windshield. Note that a display unit of a navigation device (not shown) may be used as the display device 14.

  The GPS device 15 receives a radio wave from a GPS satellite and calculates a position (latitude, longitude, etc.) of the vehicle 1 by performing a predetermined calculation using information carried on the radio wave. The position information calculated by the GPS device 15 is sent to a navigation device (not shown) and the control device 4. The shift lever position detection device 16 detects the position (for example, parking (P), reverse (R), drive (D), etc.) of a shift lever (not shown) operated by an occupant of the vehicle 1. The shift lever position information detected by the shift lever position detection device 16 is sent to the control device 4.

  The microphone 17 is composed of, for example, a front microphone, a right side microphone, and a left side microphone. The front microphone has directivity for collecting sound in front of the vehicle 1 exclusively. The right side microphone has directivity for collecting sound on the right side of the vehicle 1 exclusively. The left side microphone has directivity for collecting sound on the left side of the vehicle 1 exclusively. Each piece of sound information (front, right side, left side) collected by the microphone 17 is sent to the control device 4.

<Detection of object>
The control device 4 performs image processing on the image from the camera 3 as follows in order to detect the travel path and the object of the vehicle 1. First, the control device 4 generates a distance image (depth distribution image) based on distance measurement data at a plurality of positions in the shooting screen. The control device 4 performs a well-known grouping process based on the distance image data, and frames (windows) such as three-dimensional road shape data, side wall data, and object data stored in the storage unit 4b in advance. Compared to, white line data (including white line data along the road and white line crossing the road (stop line: intersection information) data), side walls data such as guardrails, curbs, etc. existing along the road are extracted. Objects and obstacles are extracted by classifying them into other objects such as motorcycles, ordinary vehicles, large vehicles, pedestrians, and utility poles.
In the present description, the white or yellow line drawn on the travel path is called a white line. In addition, a white line including a solid line and a broken line is called.

<Driving support>
The control device 4 recognizes the travel path and the obstacle / obstruction that becomes an obstacle based on the information extracted as described above, that is, the data of the white line data, the guardrail side wall data, and the object data. Then, the second driving control unit 6 performs the driving support control. That is, the vehicle 1 is caused to travel along the road, and the vehicle 1 is prevented from colliding with an object.

<Running control>
For example, the control device 4 estimates the own vehicle traveling path in the following four ways.
(1) Self-vehicle travel path estimation based on white line White line data on both the left and right sides of the road or one of the left and right sides is obtained from the image acquired by the camera 3, and the vehicle 1 travels from these white line data. When the shape of the existing lane can be estimated, the control device 4 estimates that the own vehicle traveling path is parallel to the white line in consideration of the width of the vehicle 1 and the position of the vehicle 1 in the current lane.

(2) Self-vehicle traveling path estimation based on side wall data of guardrail, curbstone, etc. From the image acquired by the camera 3, side wall data on either the left or right side of the road or on either side is obtained. When the shape of the lane in which the vehicle 1 is traveling can be estimated, the control device 4 considers the width of the vehicle 1 and the position in the current lane of the vehicle 1 and the own vehicle traveling path is parallel to the side wall. Estimated.

(3) Self-vehicle traveling path estimation based on the preceding vehicle trajectory The control device 4 estimates the own vehicle traveling path based on the past traveling trajectory of the preceding vehicle stored in the storage unit 4b. The preceding vehicle refers to a vehicle closest to the vehicle 1 among the objects traveling in the same direction as the vehicle 1.

(4) Own vehicle travel path estimation based on travel trajectory of vehicle 1 The control device 4 estimates the own vehicle travel path based on the driving state of the vehicle 1. For example, the own vehicle traveling path is estimated using the turning curvature based on the detection signal from the yaw rate sensor 13 and the detection signal from the vehicle speed sensor 12. The turning curvature Cua is calculated by Cua = dψ / dt / V. dψ / dt is the yaw rate (speed of change of the rotation angle in the turning direction), and V is the vehicle speed of the vehicle 1.

  The control device 4 estimates the traveling region of the vehicle 1 at the position where the object exists for each object according to a predetermined traveling control program stored in the storage unit 4b based on the own vehicle traveling path. The traveling area and the object position are compared to determine whether or not each object is in the traveling area. The control device 4 further recognizes the preceding vehicle based on the imaging result of the camera 3. That is, the control device 4 sets the vehicle closest to the vehicle 1 as the preceding vehicle among the objects that exist in the travel region and travel in the forward direction (the same direction as the vehicle 1).

  The control device 4 outputs the inter-vehicle distance information between the preceding vehicle and the vehicle 1 and the vehicle speed information of the preceding vehicle to the first traveling control unit 5 as out-of-vehicle information. Here, the vehicle speed information of the preceding vehicle is measured based on the vehicle speed V of the vehicle 1 acquired every predetermined time and the image acquired by the camera 3 every predetermined time in synchronization with the acquisition timing of the vehicle speed V. It is calculated based on the change in the distance to the preceding vehicle (inter-vehicle distance) in the shooting screen.

  The first travel control unit 5 sends a throttle control signal to the throttle control device 7 so that the vehicle speed V detected by the vehicle speed sensor 12 converges to a predetermined vehicle speed (target speed) set in advance. Thereby, the throttle control device 7 feedback-controls the opening of a throttle valve (not shown), and the vehicle 1 is automatically driven at a constant speed.

  In addition, the first traveling control unit 5 performs the traveling control in the constant speed state when the vehicle speed information of the preceding vehicle input from the control device 4 is equal to or lower than the target speed set in the vehicle 1. Then, a throttle control signal is sent to the throttle control device 7 based on the inter-vehicle distance information input from the control device 4. Specifically, a target value of an appropriate inter-vehicle distance is set based on the inter-vehicle distance from the vehicle 1 to the preceding vehicle, the vehicle speed of the preceding vehicle, and the vehicle speed V of the vehicle 1, and an image acquired by the camera 3 is displayed. A throttle control signal is sent to the throttle control device 7 so that the inter-vehicle distance measured based on the distance converges to the target value of the inter-vehicle distance. As a result, the throttle control device 7 feedback-controls the opening of a throttle valve (not shown), and causes the vehicle 1 to travel following the preceding vehicle.

<Description of Laminated Image Sensor>
The laminated image sensor 100 provided in the camera 3 described above will be described. The multilayer image sensor 100 is described in Japanese Patent Application No. 2012-139026 filed earlier by the applicant of the present application. FIG. 3 is a cross-sectional view of the multilayer image sensor 100. The imaging device 100 includes a backside illumination type imaging chip 113 that outputs a pixel signal corresponding to incident light, a signal processing chip 111 that processes the pixel signal, and a memory chip 112 that stores the pixel signal. The imaging chip 113, the signal processing chip 111, and the memory chip 112 are stacked, and are electrically connected to each other by a conductive bump 109 such as Cu.

  As shown in the figure, incident light is incident mainly in the positive direction of the Z-axis indicated by a white arrow. In the present embodiment, in the imaging chip 113, the surface on the side where incident light enters is referred to as a back surface (imaging surface). Further, as shown in the coordinate axes, the left direction of the paper orthogonal to the Z axis is the X axis plus direction, and the front side of the paper orthogonal to the Z axis and the X axis is the Y axis plus direction. In the following several figures, the coordinate axes are displayed so that the orientation of each figure can be understood with reference to the coordinate axes in FIG.

  An example of the imaging chip 113 is a back-illuminated MOS image sensor. The PD layer 106 is disposed on the back side of the wiring layer 108. The PD layer 106 includes a plurality of PDs (photodiodes) 104 that are two-dimensionally arranged and store charges corresponding to incident light, and transistors 105 that are provided corresponding to the PDs 104.

  A color filter 102 is provided on the incident side of incident light in the PD layer 106 via a passivation film 103. The color filter 102 has a plurality of types that transmit different wavelength regions, and has a specific arrangement corresponding to each of the PDs 104. The arrangement of the color filter 102 will be described later. A set of the color filter 102, the PD 104, and the transistor 105 forms one pixel.

  On the incident light incident side of the color filter 102, a microlens 101 is provided corresponding to each pixel. The microlens 101 condenses incident light toward the corresponding PD 104.

  The wiring layer 108 includes a wiring 107 that transmits the pixel signal from the PD layer 106 to the signal processing chip 111. The wiring 107 may be multilayer, and a passive element and an active element may be provided.

  A plurality of bumps 109 are disposed on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the opposing surfaces of the signal processing chip 111, and the imaging chip 113 and the signal processing chip 111 are pressed and aligned. The bumps 109 are joined and electrically connected.

  Similarly, a plurality of bumps 109 are disposed on the mutually facing surfaces of the signal processing chip 111 and the memory chip 112. The bumps 109 are aligned with each other, and the signal processing chip 111 and the memory chip 112 are pressurized, so that the aligned bumps 109 are joined and electrically connected.

  The bonding between the bumps 109 is not limited to Cu bump bonding by solid phase diffusion, and micro bump bonding by solder melting may be employed. Further, for example, about one bump 109 may be provided for one block described later. Therefore, the size of the bump 109 may be larger than the pitch of the PD 104. Further, a bump larger than the bump 109 corresponding to the pixel region may be provided in a peripheral region other than the pixel region where the pixels are arranged.

  The signal processing chip 111 has a TSV (silicon through electrode) 110 that connects circuits provided on the front and back surfaces to each other. The TSV 110 is preferably provided in the peripheral area. The TSV 110 may also be provided in the peripheral area of the imaging chip 113 and the memory chip 112.

  FIG. 4 is a diagram for explaining the pixel array and the unit region 131 of the imaging chip 113. In particular, a state where the imaging chip 113 is observed from the back surface (imaging surface) side is shown. For example, 20 million or more pixels are arranged in a matrix in the pixel region. In the example of FIG. 4, adjacent 16 pixels of 4 × 4 pixels form one unit region 131. The grid lines in the figure indicate the concept that adjacent pixels are grouped to form a unit region 131. The number of pixels forming the unit region 131 is not limited to this, and may be about 1000, for example, 32 pixels × 64 pixels, or more or less.

  As shown in the partially enlarged view of the pixel region, the unit region 131 in FIG. 4 includes four so-called Bayer arrays, which are composed of four pixels of green pixels Gb, Gr, blue pixels B, and red pixels R, in the vertical and horizontal directions. The green pixels Gb and Gr are pixels having a green filter as the color filter 102, and receive light in the green wavelength band of incident light. Similarly, the blue pixel B is a pixel having a blue filter as the color filter 102 and receives light in the blue wavelength band, and the red pixel R is a pixel having a red filter as the color filter 102 and having a red wavelength band. Receives light.

  In the present embodiment, a plurality of blocks are defined so as to include at least one unit region 131 per block, and each block can control pixels included in each block with different control parameters. That is, it is possible to acquire imaging signals having different imaging conditions between a pixel group included in a certain block and a pixel group included in another block. Examples of the control parameters include a frame rate, a gain, a thinning rate, the number of addition rows or addition columns to which pixel signals are added, a charge accumulation time or accumulation count, a digitization bit number (word length), and the like. The imaging element 100 can freely perform not only thinning in the row direction (X-axis direction of the imaging chip 113) but also thinning in the column direction (Y-axis direction of the imaging chip 113). Furthermore, the control parameter may be a parameter in image processing after obtaining an image signal from a pixel.

  FIG. 5 is a diagram for explaining a circuit in the unit region 131. In the example of FIG. 5, one unit region 131 is formed by 9 pixels of 3 pixels × 3 pixels adjacent to each other. As described above, the number of pixels included in the unit region 131 is not limited to this, and may be less than this or more. The two-dimensional position of the unit region 131 is indicated by symbols A to I.

  The reset transistors of the pixels included in the unit region 131 are configured to be turned on and off individually for each pixel. In FIG. 5, a reset wiring 300 for turning on / off the reset transistor of the pixel A is provided, and a reset wiring 310 for turning on / off the reset transistor of the pixel B is provided separately from the reset wiring 300. Similarly, a reset line 320 for turning on and off the reset transistor of the pixel C is provided separately from the reset lines 300 and 310. Also for the other pixels D to I, dedicated reset wirings for turning on and off the respective reset transistors are provided.

  The transfer transistors of the pixels included in the unit region 131 are also configured to be turned on and off individually for each pixel. In FIG. 5, a transfer wiring 302 for turning on / off the transfer transistor of the pixel A, a transfer wiring 312 for turning on / off the transfer transistor of the pixel B, and a transfer wiring 322 for turning on / off the transfer transistor of the pixel C are separately provided. Also for the other pixels D to I, dedicated transfer wirings for turning on / off the respective transfer transistors are provided.

  Furthermore, the selection transistors of the pixels included in the unit region 131 are also configured to be turned on and off individually for each pixel. In FIG. 5, a selection wiring 306 for turning on / off the selection transistor of the pixel A, a selection wiring 316 for turning on / off the selection transistor of the pixel B, and a selection wiring 326 for turning on / off the selection transistor of the pixel C are separately provided. Also for the other pixels D to I, a dedicated selection wiring for turning on and off each selection transistor is provided.

  Note that the power supply wiring 304 is connected in common from the pixel A to the pixel I included in the unit region 131. Similarly, the output wiring 308 is commonly connected from the pixel A to the pixel I included in the unit region 131. Further, the power supply wiring 304 is commonly connected between a plurality of unit regions, but the output wiring 308 is provided for each unit region 131 individually. The load current source 309 supplies current to the output wiring 308. The load current source 309 may be provided on the imaging chip 113 side or may be provided on the signal processing chip 111 side.

  By individually turning on and off the reset transistor and the transfer transistor in the unit region 131, the charge including the charge accumulation start time, the accumulation end time, and the transfer timing independently from the pixel A to the pixel I included in the unit region 131 Accumulation can be controlled. In addition, by individually turning on and off the selection transistors in the unit region 131, the pixel signals of the pixels I from each pixel A can be output via the common output wiring 308.

  Here, a so-called rolling shutter system is known in which charge accumulation is controlled in a regular order with respect to rows and columns for pixels A to I included in the unit region 131. When a column is designated after selecting a pixel for each row by the rolling shutter method, pixel signals are output in the order of “ABCDEFGHI” in the example of FIG.

  Thus, by configuring the circuit with the unit region 131 as a reference, the charge accumulation time can be controlled for each unit region 131. In other words, it is possible to output pixel signals with different frame rates between the unit areas 131. Further, in the imaging chip 113, the unit area 131 included in another area is rested while the unit area 131 included in a part of the area is subjected to charge accumulation (imaging). Only the image can be taken and the pixel signal can be output. Furthermore, it is also possible to switch an area (accumulation control target area) where charge accumulation (imaging) is performed between frames and sequentially perform imaging in different areas of the imaging chip 113 to output pixel signals.

  FIG. 6 is a block diagram illustrating a functional configuration of the image sensor 100 corresponding to the circuit illustrated in FIG. 5. The analog multiplexer 411 sequentially selects the nine PDs 104 forming the unit region 131 and outputs each pixel signal to the output wiring 308 provided corresponding to the unit region 131. The multiplexer 411 is formed on the imaging chip 113 together with the PD 104.

  The pixel signal output via the multiplexer 411 is supplied to the signal processing chip 111 by a signal processing circuit 412 that performs correlated double sampling (CDS) / analog / digital (A / D) conversion. D conversion is performed. The A / D converted pixel signal is transferred to the demultiplexer 413 and stored in the pixel memory 414 corresponding to each pixel. The demultiplexer 413 and the pixel memory 414 are formed in the memory chip 112.

  The arithmetic circuit 415 processes the pixel signal stored in the pixel memory 414 and passes it to the subsequent image processing unit. The arithmetic circuit 415 may be provided in the signal processing chip 111 or may be provided in the memory chip 112. In FIG. 6, connection for one unit region 131 is shown, but actually these exist for each unit region 131 and operate in parallel. However, the arithmetic circuit 415 does not have to exist for each unit region 131. For example, one arithmetic circuit 415 may perform sequential processing while sequentially referring to the values of the pixel memory 414 corresponding to each unit region 131. Good.

  As described above, the output wiring 308 is provided corresponding to each of the unit regions 131. Since the image pickup device 100 includes the image pickup chip 113, the signal processing chip 111, and the memory chip 112, each chip is arranged in the plane direction by using an electrical connection between the chips using the bump 109 for the output wiring 308. Wiring can be routed without increasing the size.

<Explanation of distance measurement>
FIG. 7 is a diagram illustrating the position of the focus detection pixel on the imaging surface of the imaging device 100. In the present embodiment, focus detection pixels are discretely arranged along the X-axis direction (horizontal direction) of the imaging chip 113. In the example of FIG. 7, 15 focus detection pixel lines 60 are provided at a predetermined interval. The focus detection pixels constituting the focus detection pixel line 60 output an image signal for distance measurement. In the imaging chip 113, normal imaging pixels are provided at pixel positions other than the focus detection pixel line 60. The imaging pixel outputs an image signal for monitoring outside the vehicle.

  FIG. 8 is an enlarged view of a region including a part of one of the focus detection pixel lines 60. In FIG. 8, a red pixel R, a green pixel G (Gb, Gr), and a blue pixel B, a focus detection pixel S1, and a focus detection pixel S2 are illustrated. The red pixel R, the green pixel G (Gb, Gr), and the blue pixel B are arranged according to the rules of the Bayer arrangement described above.

  The square area illustrated for the red pixel R, the green pixel G (Gb, Gr), and the blue pixel B indicates a light receiving area of the imaging pixel. Each imaging pixel receives a light beam passing through the exit pupil of the imaging optical system 31 (FIG. 9). That is, the red pixel R, the green pixel G (Gb, Gr), and the blue pixel B each have a square-shaped mask opening, and light passing through these mask openings reaches the light-receiving portion of the imaging pixel. .

  In addition, the shape of the light receiving region (mask opening) of the red pixel R, the green pixel G (Gb, Gr), and the blue pixel B is not limited to a quadrangle, and may be, for example, a circle.

  The semicircular region exemplified for the focus detection pixel S1 and the focus detection pixel S2 indicates a light receiving region of the focus detection pixel. That is, the focus detection pixel S1 has a semicircular mask opening on the left side of the pixel position in FIG. 8, and the light passing through the mask opening reaches the light receiving portion of the focus detection pixel S1. On the other hand, the focus detection pixel S2 has a semicircular mask opening on the right side of the pixel position in FIG. 8, and light passing through the mask opening reaches the light receiving portion of the focus detection pixel S2. In this way, the focus detection pixel S1 and the focus detection pixel S2 respectively receive a pair of light beams that pass through different areas of the exit pupil of the imaging optical system 31 (FIG. 9).

  Note that the position of the focus detection pixel line in the imaging chip 113 is not limited to the position illustrated in FIG. Further, the number of focus detection pixel lines is not limited to the example of FIG. Further, the shape of the mask opening in the focus detection pixel S1 and the focus detection pixel S2 is not limited to a semicircular shape. For example, a rectangular light receiving region (mask opening) in the imaging pixel R, the imaging pixel G, and the imaging pixel B is used. Part) may be a rectangular shape divided in the horizontal direction.

The focus detection pixel line in the imaging chip 113 may be a line in which focus detection pixels are arranged along the Y-axis direction (vertical direction) of the imaging chip 113. An imaging device in which imaging pixels and focus detection pixels are two-dimensionally arranged as shown in FIG. 8 is known, and detailed illustration and description of these pixels are omitted.
In the example of FIG. 8, the configuration in which the focus detection pixels S <b> 1 and S <b> 2 each receive one of a pair of focus detection light beams, the so-called 1PD structure, has been described. Instead, as disclosed in, for example, Japanese Patent Application Laid-Open No. 2007-282107, a configuration in which the focus detection pixels receive both of a pair of light beams for focus detection, a so-called 2PD structure, may be used. By adopting the 2PD structure in this manner, it is possible to read image data from the focus detection pixels, and the focus detection pixels do not become defective pixels.

  In the present embodiment, a pair of images by a pair of light fluxes that pass through different regions of the imaging optical system 31 (FIG. 9) based on distance measurement image signals output from the focus detection pixel S1 and the focus detection pixel S2. By detecting the image shift amount (phase difference), the focus adjustment state (defocus amount) of the imaging optical system 31 is calculated.

  In general, the pair of images are close to each other in a so-called front pin state in which the imaging optical system 31 forms a sharp image of an object (for example, a preceding vehicle) before the planned focal plane, and conversely, the target object is behind the planned focal plane. In a so-called rear pin state that connects the sharp images of the two, they move away from each other. The pair of images relatively coincide with each other in a focused state that connects a sharp image of the object on the planned focal plane. Therefore, the relative positional deviation amount of the pair of images corresponds to the distance (depth information) to the object.

  Since the defocus amount calculation based on the phase difference is known in the field of cameras, detailed description thereof is omitted. Here, since the defocus amount and the distance to the object correspond one-to-one, the distance from the camera 3 to each object can be obtained by obtaining the defocus amount for each object. That is, distance measurement (ranging) to the object can be performed at a plurality of positions on the shooting screen. The relationship between the defocus amount and the distance to the object is prepared in advance as a mathematical expression or a lookup table and stored in the nonvolatile memory 35b (FIG. 9).

<Explanation of camera>
FIG. 9 is a block diagram illustrating the configuration of the camera 3 having the image sensor 100 described above. In FIG. 9, the camera 3 includes an imaging optical system 31, an imaging unit 32, an image processing unit 33, a work memory 34, a control unit 35, and a recording unit 36.

  The imaging optical system 31 guides the light beam from the object scene to the imaging unit 32. The imaging unit 32 includes the imaging device 100 and the driving unit 32a, and photoelectrically converts an image of an object formed on the imaging chip 113 by the imaging optical system 31. The drive unit 32a generates a drive signal necessary for causing the image sensor 100 (the image pickup chip 113) to perform independent accumulation control in units of blocks as described above. Instructions such as the position and shape of the block, its range, and accumulation time are transmitted from the control unit 35 to the drive unit 32a.

  The image processing unit 33 performs image processing on the image data captured by the imaging unit 32 in cooperation with the work memory 34. The image processing unit 33 performs color detection of an object included in the image in addition to image processing such as contour enhancement processing and gamma correction.

  The work memory 34 temporarily stores image data before and after image processing. The recording unit 36 records image data and the like on a storage medium constituted by a nonvolatile memory or the like. The control unit 35 is configured by a CPU, for example, and controls the overall operation of the camera 3 in accordance with a control signal from the control device 4. For example, a predetermined exposure calculation is performed based on the image signal imaged by the imaging unit 32, and the accumulation time of the imaging chip 113 necessary for proper exposure is instructed to the drive unit 32a.

  The control unit 35 includes a ranging calculation unit 35a and a nonvolatile memory 35b. The distance measurement calculation unit 35a performs distance measurement (distance measurement) to each of the objects at a plurality of positions on the shooting screen as described above. The image data acquired by the camera 3 and the distance measurement data calculated by the camera 3 are sent to the control device 4 (FIG. 1). The nonvolatile memory 35b stores a program executed by the control unit 35a and information necessary for distance measurement.

<Block control of image sensor>
The control device 4 causes the image pickup device 100 (image pickup chip 113) of the camera 3 to perform independent accumulation control in units of blocks as described above. Therefore, the following signals are input to the control device 4 from each part of the vehicle 1 (FIG. 2).
(1) Depression amount of the accelerator pedal 7a A signal indicating the depression amount of the accelerator pedal 7a is input from the throttle control device 7 to the control device 4.
(2) Depression amount of the brake pedal 8a A signal indicating the depression amount of the brake pedal 8a is input from the brake control device 8 to the control device 4.

(3) Rotation angle of steering wheel 10 A signal indicating the rotation angle of the steering wheel 10 is input from the steering control device 9 to the control device 4. The ratio between the rotation angle of the steering wheel 10 and the steering angle of the steering device depends on the gear ratio of the steering.
(4) Vehicle speed V of vehicle 1
A detection signal from the vehicle speed sensor 12 is input to the control device 4.

(5) Operation signal of the turn signal switch 11 An operation signal of the turn signal switch 11 is input to the control device 4.
(6) Shift lever operation position A signal indicating the shift lever operation position detected by the shift lever position detection device 16 is input to the control device 4.

(7) Position Information of Vehicle 1 Position information measured by the GPS device 15 is input from the GPS device 15 to the control device 4.
(8) Sound information around the vehicle 1 Sound information from the front, right side, and left side of the vehicle 1 collected by the microphone 17 is input to the control device 4, respectively.

  FIG. 10 shows the imaging surface of the imaging chip 113, the area where the charge accumulation (imaging) is performed in the imaging chip 113 (imaging area 81 and the attention area 82), and the charge accumulation (imaging) in the row direction and the column direction. It is a figure which illustrates the area | region (rest area | region 83) which does not exist. The attention area 82 is an area where charge accumulation (imaging) is performed under conditions different from those of the imaging area 81. The size and position of the imaging area 81 and the attention area 82 in the imaging chip 113 are also imaging conditions.

  The control device 4 controls the unit region 131 included in the imaging region 81 to perform imaging by setting the first condition, respectively, and controls the unit region 131 included in the attention region 82 to the second region. Control is performed so that imaging is performed with conditions set. The control device 4 pauses the unit area 131 included in the pause area 83 so as not to capture an image. Note that a plurality of attention regions 82 may be provided, or imaging conditions may be different among a plurality of attention regions. Further, the rest area 83 may not be provided.

<Description of flowchart>
Hereinafter, how to determine the imaging region 81 and the attention region 82 will be described with reference to flowcharts (FIGS. 11, 12, and 17). FIG. 11 is a flowchart illustrating the flow of control processing of the camera 3 executed by the control device 4. A program for executing the processing according to the flowchart of FIG. 11 is stored in the storage unit 4 b of the control device 4. For example, when the power supply is started from the vehicle 1 or the engine is started, the control device 4 starts a program for performing the processing shown in FIG.

  In step S10 of FIG. 11, the control device 4 determines whether or not the flag a = 0. The flag a is a flag that is set to 1 when the initial setting is completed and set to 0 when the initial setting is not completed. When the flag a = 0, the control device 4 makes a positive determination in step S10 and proceeds to step S20. When the flag a ≠ 0, the control device 4 makes a negative determination and proceeds to step S30.

  In step S20, the control device 4 performs an initial setting process and proceeds to step S30. Details of the initial setting process will be described later. In step S30, the control device 4 performs a travel assist setting process and proceeds to step S40. In the travel assist setting process, the imaging area 81 and the attention area 82 are determined for the imaging element 100. Details of the travel assist setting process will be described later.

In step S <b> 40, the control device 4 sends an instruction to the camera 3 to drive the imaging area 81 and the attention area 82 in the imaging element 100 under predetermined conditions to acquire an image. In the present embodiment, for example, when the vehicle speed V increases from 0, the control device 4 increases the frame rate of the attention area 82 compared to the imaging area 81, increases the gain, decreases the thinning rate, and shortens the accumulation time. Set. Thereby, imaging by the camera 3 is performed, and distance measurement (ranging) is performed at a plurality of positions on the shooting screen as described above.
Note that it is not necessary to change all of the frame rate, gain, thinning rate, accumulation time, and the like between the imaging region 81 and the attention region 82, and at least one of them may be different. Note that the control device 4 may set the attention area 82 not to be thinned out.

  In step S45, the control device 4 acquires image data and distance measurement data from the camera 3, and proceeds to step S50. In step S50, the control device 4 determines whether or not the setting for displaying information is performed. When the display setting is performed, the control device 4 makes a positive determination in step 50 and proceeds to step S60. When the display setting is not performed, the control device 4 makes a negative determination in step 50 and proceeds to step S70.

In step S60, the control device 4 sends display information for the display device 14 (FIG. 1), and proceeds to step S70. The display information is information according to the state of the vehicle 1 determined in the driving assist setting process (S30). For example, “stopped”, “emergency stop”, “turn right”, “turn left” The message “Masu” is displayed on the display device 14.
Instead of sending display information, or together with sending display information, an audio signal for reproducing the message may be sent to an audio playback device (not shown). Also in this case, a sound device of a navigation device (not shown) may be used as a sound reproducing device (not shown).

  In step S70, the control device 4 determines whether or not an off operation has been performed. For example, when the control device 4 receives an off signal (for example, an engine off signal) from the vehicle 1, the control device 4 makes a positive determination in step S <b> 70, performs a predetermined off process, and ends the process of FIG. 11. For example, if the control device 4 does not receive an off signal from the vehicle 1, the control device 4 makes a negative determination in step S70 and proceeds to step S80. In step S80, the control device 4 waits for a predetermined time (for example, 0.1 second) and returns to step S30. When returning to step S30, the above-described processing is repeated.

<Initial setting process>
FIG. 12 is a flowchart for explaining the details of the step S20 initial setting process of the flowchart of FIG. In step S21 of FIG. 12, the control device 4 inputs the position information of the vehicle 1 from the GPS device 15 (FIG. 1), and proceeds to step S22. In step S22, the control device 4 sets a flag indicating whether the traffic zone in which the vehicle 1 travels is left or right of the road, that is, left-side traffic or right-side traffic, based on the latitude and longitude included in the position information. Specifically, the country name where the vehicle 1 is used is determined based on the latitude and longitude. Then, a flag indicating whether the road in the country is left-hand traffic or right-hand traffic is set with reference to a database (not shown). A database indicating the relationship between the country name and the right and left of the traffic zone is stored in the storage unit 4b in advance.

  In step S23, the control device 4 sets a flag indicating the attachment position (right or left) of the steering wheel (steering wheel 10) in the vehicle 1 and proceeds to step S24. Information indicating whether the vehicle is a right steering wheel or a left steering wheel is stored in advance in the storage unit 4b as specification information of the vehicle 1. In step S24, the control device 4 determines an initial setting value based on the table illustrated in FIG. 13, and proceeds to step S25. Note that the order of step S21 and step S23 may be interchanged.

  According to FIG. 13, there are four ways of “1” to “4” depending on the combination of the mounting position (right or left) of the steering wheel 10 in the vehicle 1 and the position (right or left) of the traffic zone on the road. Initial setting values are prepared. In the case of left-hand traffic with the right handle, the initial setting value is “4”.

  In step S25, the control device 4 sets the initial position of the attention area 82. The initial position of the attention area 82 is a position corresponding to the initial setting value. Specifically, when the initial setting value is “1”, the initial position of the attention area 82 is (Xq1, Yq), and when the initial setting value is “2”, the initial position of the attention area 82 is (Xq2, Yq). When the initial setting value is “3”, the initial position of the attention area 82 is (Xq3, Yq), and when the initial setting value is “4”, the initial position of the attention area 82 is (Xq4, Yq).

  In this description, in the coordinate system representing the imaging area 81, the position of the attention area 82 is represented by the coordinates (Xq, Yq) at the center of the attention area 82. FIG. 10 exemplifies the attention area 82 when the initial setting value is “4”. Since the left steering is performed by the right handle, the attention area 82 is located on the driver's seat side (to the right) in the left lane. Is set so that the initial position (Xq4, Yq) is set.

  FIG. 14 exemplifies the attention area 82 when the initial set value is “1”. Since the left steering wheel is right-hand traffic, the attention area 82 is on the driver's seat side (left side) in the right-hand traffic band. The initial position (Xq1, Yq) is determined to set.

  FIG. 15 illustrates the attention area 82 in the case where the initial setting value is “3”. Since the left steering wheel is left-hand traffic, the attention area 82 is on the driver's seat side (to the left) in the left-hand traffic band. Is set so that the initial position (Xq3, Yq) is set.

  FIG. 16 illustrates the attention area 82 in the case where the initial setting value is “2”. Since the right steering is performed on the right handle, the attention area 82 is located on the driver's seat side (rightward) in the right traffic band. Is set to the initial position (Xq2, Yq).

  In step S <b> 26 of FIG. 12, the control device 4 sets the initial size of the attention area 82. In the present embodiment, the initial size (Px (X-axis direction) × Py (Y-axis direction)) of the attention area 82 is determined based on the size (dimensions) of the object (for example, the preceding vehicle). When the preceding vehicle is included in the image acquired by the camera 3, the control device 4 obtains the image height of the preceding vehicle captured by the imaging chip 113, the known focal length of the imaging optical system 31, and the distance measurement. The size (dimension) of the preceding vehicle is estimated based on the distance L from the vehicle 1 to the preceding vehicle. The number of pixels constituting an image obtained on the imaging chip 113 when a preceding vehicle having an estimated size (for example, width 3 (m) × height 1.4 (m)) is imaged from the rear of 1 (m). Let (Px (X-axis direction) × Py (Y-axis direction)) be the initial size.

Px and Py are calculated by the following expressions (1) and (2).
Px = ox × L (1)
Py = oy × L (2)
Here, ox is the number of pixels in the X-axis direction constituting the image of the preceding vehicle imaged by the imaging chip 113 at a distance of L (m). oy is the number of pixels in the Y-axis direction constituting the image of the preceding vehicle imaged by the imaging chip 113 at a distance of L (m). L is the inter-vehicle distance from the vehicle 1 to the preceding vehicle.
In the coordinate system representing the imaging region 81, the Yq representing the initial position is the center of the height of the image obtained on the imaging chip 113 when the preceding vehicle separated by 1 (m) is imaged (in this example, the preceding position). This corresponds to a vehicle height of 0.7 (m).

  In step S27, the control device 4 sends display information to the display device 14 (FIG. 1), sets 1 to the flag a, and ends the processing in FIG. The display information is information indicating that the initial setting process has been completed. For example, the display device 14 displays a message “initial setting is completed”.

<Driving assist setting process>
FIG. 17 is a flowchart illustrating details of the travel assist setting process. In step S310 of FIG. 17, the control device 4 makes a positive determination in step S310 when the position information of the shift lever input from the shift lever position detection device 16 (FIG. 1) is “P” (parking). Proceed to S320. If the shift lever position information input from the shift lever position detection device 16 (FIG. 1) is not “P”, the control device 4 makes a negative determination in step S310 and proceeds to step S420. The determination in step S310 may also be applied when the shift lever is “N” (neutral).

  In step S320, the control device 4 inputs the vehicle speed V from the vehicle speed sensor 12, and proceeds to step S330. The control device 4 changes the frame rate of the attention area 82 according to the vehicle speed V, for example. As described above, when the frame rate of the attention area 82 is set higher than the frame rate of the imaging area 81, the control device 4 sets the frame rate of the attention area 82 higher as the vehicle speed V increases, and the vehicle speed V The frame rate of the region of interest 82 is set lower as the value decreases. In this case, the control device 4 may apply such control that the frame rate of the imaging area 81 other than the attention area 82 is also proportional to the vehicle speed V. In step S330, the control device 4 inputs the depression amount of the brake pedal 8a from the brake control device 8 (FIG. 1), and proceeds to step S340.

In step S340, the control device 4 determines whether or not the flag Em = 0 based on the vehicle speed V and the depression amount (depression angle) of the brake pedal 8a. The flag Em is a flag that is set as illustrated in FIG. 18 based on the vehicle speed V and the amount of change in the depression amount (depression angle) of the brake pedal 8a. In the present embodiment, the case where Em = 1 is determined as emergency braking (rapid braking), and the case where Em = 0 is determined as normal braking. When Em = 0, the control device 4 makes a positive determination in step S340 and proceeds to step S350. When Em = 1, the control device 4 makes a negative determination in step S340 and proceeds to step S430.
Note that Em = 1 may be determined based on the amount of change in the opening of a brake valve (not shown) instead of the amount of change in the amount of depression (depression angle) of the brake pedal 8a. Further, Em = 1 may be determined based on the amount of change in the vehicle speed V, or Em = 1 may be determined based on the amount of change in the reduction ratio of a transmission (not shown).

  In step S350, the control device 4 inputs the depression amount of the accelerator pedal 7a from the throttle control device 7 (FIG. 1), and proceeds to step S360. In step S360, the control device 4 inputs the rotation angle θ of the steering wheel 10 from the steering control device 9, and proceeds to step S370. In step S370, the control device 4 determines whether or not a steering operation has been performed. If the rotation angle θ is larger than the predetermined value, the control device 4 makes a positive determination in step S370 and proceeds to step S380. If the rotation angle θ is equal to or smaller than the predetermined value, the control device 4 makes a negative determination in step S370 and proceeds to step S440.

In step S380, the control device 4 calculates the movement amount Xdist of the region of interest 82 in the X-axis direction based on the rotation angle θ of the steering wheel 10 and the vehicle speed V according to the following equation (3).
Xdist = θ × (V × 0.2) (3)
According to the above equation (3), the movement amount Xdist increases as the steering angle of the steering device (that is, the rotation angle θ of the steering wheel 10) increases and as the vehicle speed V increases.

In step S390, the control device 4 calculates, based on the initial position (XqN, Yq) of the region of interest 82 set in the initial setting process, the position (X coordinate) of the region of interest 82 during traveling by the following equation (4). To do.
Xq = XqN + Xdist (4)
However, N is one of the initial setting values 1 to 4 determined in the initial setting process. Xdist is the amount of movement of the region of interest 82 in the X-axis direction calculated in step S380, and corresponds to the number of pixels in the X-axis direction. By the processing in step S390, the position of the attention area 82 changes according to the steering operation. Further, the position of the attention area 82 also changes depending on the magnitude of the vehicle speed V.

In step S400, the control device 4 calculates the position (Y coordinate) of the attention area 82 during the traveling by the following equation (5) based on the initial position (XqN, Yq) of the attention area 82 set in the initial setting process. To do.
Yq = Yq + P (Z) (5)
However, P (Z) is the amount of movement of the region of interest 82 in the Y-axis direction, and is the number of pixels in the Y-axis direction corresponding to the depth Z (m). For example, it represents how many pixels the image of a road having a depth of 20 (m) corresponds to in the Y-axis direction. The relationship P (Z) between the depth Z and the number of pixels is stored in advance in the storage unit 4b (FIG. 2).

  In general, when the traveling direction is imaged on a flat straight road, the number of pixels in the Y-axis direction corresponding to the road image on the imaging chip 113 increases as the depth Z (m) from the vehicle 1 increases. Therefore, the value of Yq corresponding to the height center of the image when the preceding vehicle separated by 1 (m) is imaged is increased as the preceding vehicle to be noted becomes farther (that is, the depth Z becomes deeper).

The control device 4 determines the depth Z of the preceding vehicle to be noted by the following equation (6).
Z = Za + Zb (6)
However, Za is a braking distance (m) on a dry road, and Zb is a braking distance (m) on a wet road surface. Za and Zb are based on the values illustrated in FIG. In the present embodiment, the position of the attention area 82 is determined so that a preceding vehicle having a depth Z in front of the vehicle 1 (that is, a position away from the vehicle 1 by Z (m)) is included in the attention area 82. This is based on the idea of paying more attention to the distance than is necessary for stopping when the emergency brake is applied. The value (Za + Zb) of the depth Z corresponding to the vehicle speed V is stored in advance in the storage unit 4b (FIG. 2). According to the process of step S400, the position of the attention area 82 changes according to the change in the vehicle speed V.
With regard to the attention area 82 whose position has been changed in this way, at least one of the frame rate, the gain, the thinning rate, the accumulation time, and the like is made different between the imaging area 81 and the attention area 82.

In step S410, the control device 4 uses the initial size (Px × Py) of the attention area 82 set in the initial setting process as a base, and calculates the size (X_wid, Y_wid) of the attention area 82 during traveling by the following formula (7) and Calculation is performed according to (8), and the process of FIG. 17 is terminated.
X_wid = Px / Z (7)
Y_wid = Py / Z (8)
However, Px is the number of pixels in the X-axis direction set in step S26, and Py is the number of pixels in the Y-axis direction set in step S26. According to the above formulas (7) and (8), the size (X_wid, Y_wid) of the attention area 82 during traveling is the initial value of the attention area 82 as the preceding vehicle to be noticed becomes farther away (depth Z becomes deeper). It becomes smaller than the size (Px × Py). According to the process of step S410, the size of the attention area 82 changes according to the change in the vehicle speed V.
As for the attention area 82 whose size has been changed in this way, at least one of the frame rate, the gain, the thinning rate, the accumulation time, and the like is made different between the imaging area 81 and the attention area 82.

  In step S420, which proceeds after making a negative determination in step S310 described above, the control device 4 performs a setting process that is stopped and ends the process of FIG. In the stop setting process, for example, the position of the attention area 82 is determined so that the preceding vehicle 1 (m) away is included in the attention area 82. In addition, the size of the attention area 82 in the X-axis direction is maximized so that the object located near the side of the vehicle 1 is included in the attention area 82 as much as possible.

  In step S430, which proceeds after making a negative determination in step S340 described above, the control device 4 performs a setting process at the time of sudden brake determination, and ends the process of FIG. For example, the setting process at the time of sudden brake determination is to stop the thinning out in the attention area 82, increase the frame rate to the maximum, shorten the accumulation time, and set the gain high. Note that the control device 4 may increase the frame rate of the imaging area 81 other than the attention area 82. Moreover, the control apparatus 4 records with respect to the camera 3 so that the image acquired with the camera 3 may be preserve | saved in the recording part 36 for the predetermined time (for example, 5 seconds-15 seconds) after negative determination of step S340. The instructions are given.

  The control device 4 after the sudden stop further moves the attention area 82 to the initial position of the attention area 82 set in the initial setting process (step S25 in FIG. 12) and is set in the initial setting process (step S26 in FIG. 12). The size of the attention area 82 is changed to the initial size (Px × Py) of the attention area 82. As a result, the position and size of the attention area 82 changed according to the traveling vehicle speed V returns to the position and size suitable for stopping.

  In step S440, which proceeds after making a negative determination in step S370 described above, the control device 4 performs setting so as not to move the position (X coordinate) of the region of interest 82 during traveling. That is, when the rotation angle θ of the steering wheel 10 is equal to or smaller than a predetermined value, θ ← 0 is set, and the value of Xdist is also set to 0. That is, when the operation angle of the steering wheel 10 is less than the predetermined value, the position (X coordinate) of the attention area 82 is maintained. For this reason, it is useful for reducing the processing load at the time of a minute operation that is not a turning operation.

  FIG. 20A is a diagram illustrating the movement of the position of the attention area 82 and the change in the size of the attention area 82 when making a right turn at an intersection of general roads. According to the travel assist setting process, when the vehicle 1 is waiting for a right turn behind the preceding vehicle, the position of the attention area 82A is at the initial position, and the size of the attention area 82A is the initial size (Px × Py). Is almost the same. When the driver starts the steering operation in the right direction while the vehicle 1 moves forward, the position of the attention area 82B moves diagonally upward to the right. Since the vehicle speed V is low, the size of the attention area 82B is substantially the same as the initial size (Px × Py).

  FIG. 20B is a diagram illustrating the movement of the position of the attention area 82 and the change in the size of the attention area 82 when the lane is changed while accelerating to the right overtaking lane on the highway. According to the travel assist setting process, when the vehicle 1 travels at a high speed, the position of the attention area 82A is above the initial position, and the size of the attention area 82A is smaller than the initial size (Px × Py). When the driver performs a steering operation in the right direction while the vehicle 1 is accelerating, the position of the attention area 82B moves obliquely upward to the right. Since the vehicle speed V is high, the size of the attention area 82B is further reduced. FIG. 20 shows an example in the case of left-hand traffic, and it can be used as appropriate for left-hand traffic on right-hand traffic and lane changes for right-hand traffic. In addition, a driver's line of sight is detected by a line-of-sight detection device (not shown) (for example, a line-of-sight detection device is provided on the steering wheel), and an area where the driver is not gazing or a blind spot is set as the attention area 82. You may make it do. For eye gaze detection, the cornea reflection method that detects the direction of the user's gaze by reflecting infrared light on the driver's cornea, the Limbus tracking method that uses the difference in reflectance with respect to light between the cornea and the sclera, There is an image analysis method in which a video is captured by a camera and a line of sight is detected by image processing, and any line of sight detection method may be used.

According to the embodiment described above, the following operational effects can be obtained.
(1) The vehicle 1 includes a control device 4 that recognizes at least one of specifications of the mounted vehicle 1 and an operation on an operation unit of the vehicle 1, at least a region of interest 82, and an imaging region 81. Since the image pickup unit 32 that picks up the outside of the image pickup device 32 and the control device 4 that sets the image pickup condition of the region of interest 82 and the image pickup condition of the image pickup region 81 differently based on the recognition result by the control device 4 are provided. The imaging conditions of the camera 3 can be set appropriately.

(2) Since the control device 4 recognizes the mounting position (right or left) of the steering wheel (steering wheel 10) in the vehicle 1, it is possible to appropriately set the imaging condition of the camera 3 according to the driver's boarding position. it can.

(3) Since the setting unit sets the frame rate of the attention area 82 differently from the frame rate of the imaging area 81 according to the position of the steering wheel 10, for example, the attention area 82 on the driver's seat side (right). Thus, it is possible to appropriately set the imaging conditions of the camera 3 such as increasing the frame rate.

(4) The control device 4 that detects information related to the vehicle speed V of the vehicle 1 is provided. The control device 4 determines the imaging condition of the attention area 82 and the imaging condition of the imaging area 81 according to the detection result of the information related to the vehicle speed V. Therefore, the imaging conditions of the camera 3 can be appropriately set according to the vehicle speed V.

(5) The control device 4 changes at least one of the imaging condition of the attention area 82 and the imaging condition of the imaging area 81 when the information about the vehicle speed V increases and decreases. The imaging conditions of the camera 3 can be set appropriately, such as increasing the frame rate as the vehicle speed V increases.

(6) When the rotation angle θ of the steering wheel (steering wheel 10) exceeds a predetermined value, the control device 4 has at least one of the imaging frame rate of the attention area 82 and the imaging frame rate of the imaging area 81. Since the frame rate is changed high, the imaging condition of the camera 3 can be changed in the case of a turning operation.

(7) Since the control device 4 that transmits the display information for the display device 14 of the vehicle 1 based on the imaging result of the imaging unit 32 is provided, necessary information can be provided to the passenger of the vehicle 1.

(8) When the rotation angle θ of the steering wheel (steering wheel 10) does not reach a predetermined value, the control device 4 at least one frame of the imaging frame rate of the attention area 82 and the imaging frame rate of the imaging area 81 Since the setting of the rate is maintained, it is possible to avoid changing the imaging conditions during a minute operation that is not a turning operation. Thereby, for example, it is possible to prevent the frame rate of the attention area 82 from being changed more finely than necessary, which helps to reduce the processing load.

(9) The control device 4 sets the imaging frame rate, gain, thinning, pixel signal addition, accumulation, bit length to the imaging conditions that are different between the imaging conditions of the attention area 82 and the imaging conditions of the imaging area 81. Since at least one of the size of the imaging region and the position of the imaging region is included, the imaging condition of the camera 3 can be set appropriately.

(10) The control device 4 changes at least one center position between the center position of the attention area 82 and the center position of the imaging area 81 based on the detection result of the information related to the vehicle speed V. Accordingly, the imaging condition of the camera 3 can be appropriately set, such as changing the position of the attention area 82.

(11) Since the control device 4 changes at least one of the size of the attention area 82 and the size of the imaging area 81 based on the detection result of the information related to the vehicle speed V, attention is paid as the vehicle speed V changes. The imaging conditions of the camera 3 can be set appropriately, such as changing the size of the area 82.

(12) Since the control device 4 sets the imaging area 81 surrounding the attention area 82, the imaging condition of the camera 3 can be set appropriately.

(13) A steering wheel (steering wheel 10) is provided as an operation unit of the vehicle 1, and the control device 4 at least one of the center position of the attention area 82 and the center position of the imaging area 81 based on the operation of the steering wheel. Since the two center positions are changed, the imaging condition of the camera 3 can be appropriately set such that the position of the attention area 82 is changed in accordance with the change in the course of the vehicle 1. In the above-described embodiment, the camera 3 is controlled by the control device 4, but part of the control of the camera 3 may be performed by the control unit 35 of the camera 3.

The following modifications are also within the scope of the present invention, and one or a plurality of modifications can be combined with the above-described embodiment.
(Modification 1)
In the travel assist setting process, the control device 4 may be configured to change the position of the attention area 82 and the size of the attention area 82 in accordance with an operation signal from the turn signal switch 11. As illustrated in FIG. 21, the control device 4 changes the size of the attention area 82 based on the initial setting value determined in step S <b> 24 and the direction of the turn signal by the operation of the turn signal switch 11. The imaging condition of the area 82 is set.

  For example, referring to FIG. 10, when the right steering wheel and left-hand traffic and the initial setting value is “4”, the control device 4 includes the left end of the road in the attention area 82 if the turn signal is in the left direction. Perform such control. Specifically, the attention area 82 in FIG. 10 is expanded to the left. The reason why the attention area 82 is expanded to the left side is to prevent an accident involving a left turn. On the contrary, if the turn signal is in the right direction, the control device 4 performs control to include the oncoming lane in the attention area 82. Specifically, the attention area 82 in FIG. 10 is expanded to the right.

  Referring to FIG. 14, when the left steering wheel and the right-hand traffic and the initial setting value is “1”, the control device 4 includes the oncoming lane in the attention area 82 if the turn signal is in the left direction. Take control. Specifically, the attention area 82 in FIG. 14 is expanded to the left. On the contrary, the control device 4 performs control to include the right end of the road in the attention area 82 if the turn signal is in the right direction. Specifically, the attention area 82 in FIG. 14 is expanded to the right. The reason for expanding to the right is to prevent accidents involving right turns.

  Referring to FIG. 16, when the right steering wheel and right-hand traffic and the initial setting value is “2”, the control device 4 includes the oncoming lane in the attention area 82 if the turn signal is in the left direction. Take control. Specifically, the attention area 82 in FIG. 16 is greatly expanded to the left. On the contrary, the control device 4 performs control to include the right end of the road in the attention area 82 if the turn signal is in the right direction. Specifically, the attention area 82 in FIG. 16 is slightly expanded to the right. The reason for expanding to the right is to prevent accidents involving right turns.

  Referring to FIG. 15, when the left steering wheel and left-hand traffic and the initial setting value is “3”, the control device 4 includes the left end of the road in the attention area 82 if the turn signal is in the left direction. Take control. Specifically, the attention area 82 in FIG. 15 is slightly expanded to the left. The reason for expanding to the left is to prevent accidents involving left turns. On the contrary, if the turn signal is in the right direction, the control device 4 performs control to include the oncoming lane in the attention area 82. Specifically, the attention area 82 in FIG. 15 is greatly expanded to the right.

  FIG. 22 is a flowchart illustrating a process when the turn signal switch 11 is operated according to the first modification. When an operation signal is input from the turn signal switch 11 during the travel assist setting process, the control device 4 starts the process of FIG. 22 as a subroutine. In step S510 of FIG. 22, the control device 4 determines whether or not the turn signal direction is leftward. If the turn signal direction is leftward, the control device 4 makes a positive determination in step S510 and proceeds to step S520. If the turn signal direction is rightward, the control device 4 makes a negative determination in step S510 and proceeds to step S530.

  In step S520, the control device 4 determines whether or not the position of the traffic band is on the left. The control device 4 makes an affirmative determination in step S520 if the vehicle is on the left side and proceeds to step S550, and determines negative on step S520 if the vehicle is on the right side and proceeds to step S540.

  In step S540, the control device 4 controls the imaging unit 32 to include the oncoming lane in the attention area 82, and ends the process of FIG. In step S550, the control device 4 controls the imaging unit 32 so that the attention area 82 includes the left end of the road, and ends the process of FIG.

  In step S530, the control device 4 determines whether or not the position of the traffic zone is on the left. The control device 4 makes an affirmative decision in step S530 if the vehicle is on the left side and proceeds to step S560, and makes a negative decision on step S530 if the vehicle is on the right side and proceeds to step S570.

  In step S560, the control device 4 controls the imaging unit 32 to include the oncoming lane in the attention area 82, and ends the process of FIG. In step S570, the control device 4 controls the imaging unit 32 to include the right end of the road in the attention area 82, and ends the process of FIG.

  Note that, when the turn signal switch 11 is turned off after executing the processing of FIG. 22, the control device 4 cancels the size change of the attention area 82 of FIG. Further, when the turn signal switch 11 is turned on, even if the vehicle speed V is 0, the frame rate of the attention area 82 is made higher than the imaging area 81, the gain is increased, the thinning rate is lowered, and the accumulation time is increased. May be set short. However, when at least one of the frame rate, the gain, the thinning rate, the accumulation time, and the like are different between the imaging region 81 and the attention region 82, only the imaging conditions to be changed are changed.

  According to the modified example 1 described above, the imaging condition of the attention area 82 and the imaging condition of the imaging area 81 are set differently according to the operation of the turn signal switch 11. For example, when turning right or left at an intersection , The oncoming lane is included in the attention area 82 so that the oncoming vehicle can be reliably detected, or the road edge is included in the attention area 82 so as to prevent an entrainment accident, thereby appropriately setting the attention area 82. be able to. Furthermore, the imaging conditions can be appropriately set in the imaging area 81 and the attention area 82, such as setting the frame rate of the attention area 82 higher than that of the imaging area 81.

(Modification 2)
In the travel assist setting process, the control device 4 determines the position of the attention area 82 and the attention area 82 in accordance with a change in the distance between an object such as a two-wheeled vehicle, a normal vehicle, a large vehicle, and a pedestrian. You may comprise so that a size may be changed.

  In the second modification, for example, when the vehicle 1 approaches the preceding vehicle and the distance L (inter-vehicle distance) to the preceding vehicle becomes short, the control device 4 determines the position of the attention region 82 so that the preceding vehicle is included in the attention region 82. To do. Here, the change in the inter-vehicle distance L from the vehicle 1 to the preceding vehicle is captured by the distance measurement calculation unit 35a of the camera 3 based on images acquired by the camera 3 at predetermined intervals in synchronization with the acquisition timing of the vehicle speed V. It is obtained by sequentially measuring the distance L (inter-vehicle distance) to the preceding vehicle in the screen.

  The control device 4 calculates the position (Y coordinate) of the region of interest 82 during traveling using the inter-vehicle distance L instead of the depth Z in the above equation (5). Thereby, when imaging a preceding vehicle on a flat straight road, the value of Yq indicating the position of the region of interest 82 in the Y-axis direction on the imaging chip 113 increases as the inter-vehicle distance L becomes longer. Decreases with shortening.

  In addition, the control device 4 sets the size of the attention area 82 to be large because the preceding vehicle appears larger in the camera 3 when the inter-vehicle distance L changes and becomes shorter. On the contrary, the control device 4 sets the size of the attention area 82 to be small because the preceding vehicle appears small in the camera 3 when the inter-vehicle distance L changes and becomes long. The control device 4 calculates the size of the attention area 82 during traveling by substituting the inter-vehicle distance L in place of the depth Z in the above equations (7) and (8).

According to the processing in FIG. 17 described above, when the vehicle speed V increases, the size of the attention area 82 is set to be small. In the second modification, even when the vehicle speed L is high, the inter-vehicle distance L to the preceding vehicle is short. Since the size of the attention area 82 is set large, the preceding vehicle can be appropriately included in the attention area 82. For this reason, it is easier to detect the change in the traveling state of the preceding vehicle based on the image acquired by the camera 3 as compared to the case where the size of the attention area 82 is continuously set to be small.
As for the attention area 82 whose size and position are changed in this way, at least one of the frame rate, the gain, the thinning rate, the accumulation time, and the like may be different between the imaging area 81 and the attention area 82.

(Modification 3)
When the control device 4 detects an object around the vehicle 1 such as a two-wheeled vehicle, a normal vehicle, a large vehicle, or a pedestrian, in addition to the existing attention area 82, the control device 4 displays the attention area 82 including the object. It may be set anew. In the third modification, when the detected object moves, the control device 4 newly sets an attention area 82 including the object. For example, when the distance between the detected object and the vehicle 1 approaches within a predetermined distance, the control device 4 newly sets the attention area 82 including the object. And the control apparatus 4 cancels | releases the setting of the attention area 82 containing this target object, when the distance of the detected target object and the vehicle 1 leaves | separated more than predetermined distance.
Regarding the attention area 82 set in this way, at least one of the frame rate, the gain, the thinning rate, the accumulation time, and the like may be different between the imaging area 81 and the attention area 82.

  According to the third modification, the control device 4 that detects a moving object around the vehicle 1 based on information from the camera 3 is provided, and the control device 4 images the attention area 82 based on the detection result of the moving object. Since at least one imaging condition of the condition and the imaging condition of the imaging area 81 is changed, the imaging condition of the camera 3 can be appropriately set according to the presence or absence of a moving object.

  Further, when it is detected that the distance between the vehicle 1 and the moving object is approaching within a predetermined distance based on the information from the camera 3, the control device 4 captures the frame rate of the region of interest 82 and the imaging region 81. Since at least one frame rate of the imaging frame rate is changed to be high, detection of a change in the moving state of the moving object based on the image acquired by the camera 3 is facilitated.

(Modification 4)
The attention area 82 may be newly set based on the color of the image acquired by the camera 3. The control device 4 sets a region including a red object in the image as the attention region 82. By adding an area including a red object to the attention area 82, for example, a red signal, a railroad crossing alarm, a red light of an emergency vehicle, and the like can be included in the attention area 82.
Regarding the attention area 82 set in this way, at least one of the frame rate, the gain, the thinning rate, the accumulation time, and the like may be different between the imaging area 81 and the attention area 82.

(Modification 5)
The attention area 82 may be newly set based on sound information collected by the microphone 17 of the vehicle 1. For example, when the level of sound information on the right side of the vehicle 1 is input exceeding a predetermined value, the control device 4 expands the attention area 82 to the right side of the imaging area 81 or newly to the right side of the imaging area 81. An attention area 82 is set. The reason why the attention area 82 is provided on the right side is to collect information outside the vehicle on the right side of the vehicle 1.

Further, for example, when the level of sound information on the left side of the vehicle 1 exceeds a predetermined value, the control device 4 expands the attention area 82 to the left side of the imaging area 81 or to the left side of the imaging area 81. A new attention area 82 is set. The reason why the attention area 82 is provided on the left side is to collect information outside the vehicle on the left side of the vehicle 1.
Regarding the attention area 82 set in this way, at least one of the frame rate, the gain, the thinning rate, the accumulation time, and the like may be different between the imaging area 81 and the attention area 82.

(Modification 6)
Although the case where the attention area 82 including the preceding vehicle is set has been described in the above embodiment, the attention area 82 including the oncoming vehicle of the vehicle 1 may be set. In the modified example 6, the control device 4 recognizes the vehicle closest to the vehicle 1 as an oncoming vehicle from among the objects that exist in the above-described travel region and travel in the reverse direction (opposite the vehicle 1).

  The control device 4 sets an area corresponding to the position of the oncoming vehicle as the attention area 82 in the image acquired by the camera 3. The control device 4 sets the attention area 82 including the number plate of the oncoming vehicle and the face of the driver driving in the driver seat of the oncoming vehicle.

By setting the region including the number plate of the oncoming vehicle and the face of the driver of the oncoming vehicle as the attention region 82, the oncoming vehicle approaching the vehicle 1 can be appropriately included in the attention region 82.
Regarding the attention area 82 set in this way, at least one of the frame rate, the gain, the thinning rate, the accumulation time, and the like may be different between the imaging area 81 and the attention area 82.

(Modification 7)
In the above description, the example in which the imaging region 81 includes the attention region 82 (surrounding the attention region 82) has been described. However, the imaging region 81 and the attention region 82 may be set side by side. The size and position of the imaging area 81 and the attention area 82 can be changed by moving the boundary line between the imaging area 81 and the attention area 82 to the left and right. Regarding the attention area 82 set in this way, at least one of the frame rate, the gain, the thinning rate, the accumulation time, and the like may be different between the imaging area 81 and the attention area 82.

  In the above description, as a distance measurement performed by the camera 3, a method of calculating by distance measurement using an image signal from a focus detection pixel provided in the image sensor 100 is used. You may use the method of measuring distance using an image. Alternatively, a method of measuring distance using a millimeter wave radar separately from the camera 3 may be used.

  Although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other embodiments conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Vehicle 2 ... Driving assistance apparatus 3 ... Camera 4 ... Control apparatus 4b ... Memory | storage part 5 ... 1st traveling control unit 6 ... 2nd traveling control unit 7 ... Throttle control apparatus 7a ... Accelerator pedal 8 ... Brake control apparatus 8a Brake pedal 9 Steering control device 10 Steering wheel 11 Turn signal switch 12 Vehicle speed sensor 14 Display device 15 GPS device 16 Shift lever position detection device 17 Microphone 31 Imaging optical system 32 Imaging unit 32a Drive unit 33 ... image processing unit 34 ... work memory 35 ... control unit 35a ... distance measurement calculation unit 36 ... recording unit 60 ... focus detection pixel line 81 ... imaging areas 82, 82A, 82B ... attention area 83 ... rest area 100 ... imaging Element 113 ... Imaging chip

Claims (13)

In an imaging device mounted on a car,
An input unit for inputting information about pedestrians around the vehicle;
An imaging unit for imaging the outside of the vehicle;
An imaging apparatus comprising: an imaging control unit that changes imaging conditions of the imaging unit based on information about pedestrians around the vehicle and a traveling direction of the vehicle. The imaging device according to claim 1,
The imaging control unit is an imaging device that changes imaging conditions of the imaging unit when information about pedestrians around the vehicle changes. The imaging device according to claim 1 or 2,
The information relating to the pedestrians around the vehicle is an imaging device that is information relating to an interval between the vehicle and the pedestrians around the vehicle. In the imaging device according to any one of claims 1 to 3,
The imaging control unit is an imaging device that changes an imaging speed as an imaging condition of the imaging unit. In the imaging device according to any one of claims 1 to 4,
The imaging control unit is an imaging device that changes a pixel thinning rate of the imaging unit as an imaging condition of the imaging unit. In an imaging device mounted on a car,
An input unit for inputting information about pedestrians around the vehicle;
An imaging unit having a plurality of imaging regions with different imaging conditions;
An imaging apparatus comprising: an imaging control unit that changes imaging conditions of the imaging unit based on information about pedestrians around the vehicle and a traveling direction of the vehicle. The imaging device according to claim 6,
The information relating to the pedestrians around the vehicle is an imaging device that is information relating to an interval between the vehicle and the pedestrians around the vehicle. In the imaging device according to claim 6 or 7,
The imaging control unit is an imaging device that changes imaging conditions of the imaging unit when information about pedestrians around the vehicle changes. In the imaging device according to any one of claims 6 to 8,
The imaging control unit is an imaging device that changes an imaging condition of a first imaging region of the imaging unit as an imaging condition of the imaging unit. The imaging device according to claim 9,
The imaging control unit is an imaging device that changes an imaging speed of the first imaging region of the imaging unit as an imaging condition of the first imaging region of the imaging unit. In the imaging device according to any one of claims 9 or 10,
The imaging control unit is an imaging device that changes a pixel thinning rate of the first imaging region of the imaging unit as an imaging condition of the first imaging region of the imaging unit. In the imaging device according to any one of claims 9 to 11,
The imaging control unit is configured to make an imaging condition of a second imaging area different from the first imaging area different from that of the first imaging area. The imaging apparatus according to any one of claims 1 to 12,
The traveling direction of the vehicle is an imaging device that is a traveling direction by a right turn of the vehicle or a traveling direction by a left turn of the vehicle.

Images