JP6358214B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus.

Based on the image acquired by the camera mounted on the vehicle, the driving environment of the vehicle is detected, and based on the detected driving environment data, automatic driving control such as driving following the preceding vehicle, warning, braking, steering assistance, etc. A technology for supporting driving has been developed (see Patent Document 1).
However, in the future, it is expected that automobiles that are controlled automatically and automobiles that are driven manually by a driver will be mixed, but there have not been many proposals in this regard.

JP 2010-79424 A

An imaging apparatus mounted on a vehicle according to the present invention includes an imaging unit that captures an image of the outside of the vehicle, an input unit that inputs information about a moving body around the vehicle, and an input unit that inputs the information about the vehicle. An imaging control unit that changes imaging conditions of a partial area of the imaging unit based on information about a moving body around the vehicle .

It is a block diagram which shows the structure of the imaging system which concerns on one embodiment. It is a figure which illustrates arrangement | positioning of the traffic signal in an intersection. It is a figure which illustrates the signal device for motor vehicles. It is sectional drawing of a multilayer type image pick-up element. It is a figure explaining the pixel arrangement | sequence and unit area | region of an imaging chip. It is a figure explaining the circuit of a unit area. It is a block diagram which shows the functional structure of an image pick-up element. It is a figure which illustrates the position of the pixel for focus detection in an imaging surface. It is the figure which expanded the area | region containing a part of focus detection pixel line. It is a flowchart explaining the control by the control part of a motor vehicle. 11A is a diagram for explaining the positional relationship of the automobile, FIG. 11B is a diagram schematically showing a subject image of the front camera, and FIG. 11C is a schematic diagram of the subject image of the rear camera. FIG. It is a flowchart explaining control by the control part of a traffic light. It is a figure explaining control of the image pick-up part of the traffic signal for vehicles. It is a figure explaining control of the image pick-up part of the traffic signal for vehicles. It is a figure explaining control of the image pick-up part of the signal machine for pedestrians. FIG. 16A is a diagram illustrating an imaging scene by an imaging unit of a pedestrian traffic light, and FIG. 16B is a diagram illustrating setting of imaging conditions. It is a figure which illustrates the imaging scene by the imaging part of the signal device for motor vehicles.

FIG. 1 is a block diagram illustrating a configuration of an imaging system 1 including an imaging device according to an embodiment. The imaging system 1 uses an automobile 10, another car 20, a traffic signal generation device 30 and a traffic light 40.
Note that an information providing system or a VICS (registered trademark: Vehicle Information and Communication System) installed on the road may be used in place of or in combination with the traffic signal 40.

(Car 10)
The automobile 10 includes a vehicle operation unit 11, a GPS device 12, a navigation system (navigation system) 13, an optical system 14, a photoelectric conversion unit 15, a communication unit 16, a storage unit 17, a sensor 18, and a control unit. 19. In addition, although the description is omitted, the automobile 10 has a basic configuration as an automobile.

  The vehicle operation unit 11 includes various operation members related to the operation of the vehicle, such as a steering wheel (steering wheel), a turn signal switch, a shift lever, an accelerator, a brake, and a switch for switching between an automatic driving mode and a manual driving mode.

  The GPS device 12 calculates the position (longitude, latitude, etc.) of the automobile 10 based on a signal obtained by receiving radio waves from a GPS satellite. The position information calculated by the GPS device 12 is output to the navigation system 13 and the control unit 19.

  The navigation system 13 detects the current position of the automobile 10 from the GPS device 12, etc., acquires map data corresponding to the current position from a storage medium or a network, displays it on the liquid crystal monitor, and travels to the input destination It is a system to guide you. The navigation system 13 includes an operation unit that receives an operation from a user, the above-described liquid crystal monitor, a speaker that performs voice guidance, a reading unit that reads map data, and the like.

The optical system 14 includes a plurality of lenses, and forms a subject image on the photoelectric conversion unit 15. When the optical system 14 is directed in front of the automobile 10, an image in the traveling direction of the automobile 10 is acquired by the photoelectric conversion unit 15. When the optical system 14 is directed to the rear of the automobile 10, an image in the direction opposite to the traveling direction of the automobile 10 is acquired by the photoelectric conversion unit 15. The optical system 14 has an angle of view corresponding to a plurality of traveling lanes (two lanes, three lanes, etc.).
A plurality of optical systems 14 may be provided to form a stereo camera.

  The photoelectric conversion unit 15 is configured by stacking an imaging chip that outputs a pixel signal corresponding to light incident from the optical system 14, a signal processing chip that processes the pixel signal, and a memory chip that stores the pixel signal. The imaging device 100 is provided. As will be described in detail later, the imaging element 100 can individually set imaging conditions (including a case where imaging is not performed) for each unit region including each pixel or a plurality of pixels (for example, 16 pixels × 16 pixels).

In the present embodiment, the optical system 14 and the photoelectric conversion unit 15 constitute the imaging unit 5 of the camera to image an object (moving body, obstacle, etc.) around the automobile 10, or white lines (yellow, etc.) on the road (Including other color lines). The automobile 10 includes a front imaging unit 5 that acquires an image ahead of the automobile 10 and a rear imaging unit 5 that acquires an image behind the automobile 10.
In this description, a white line drawn on the road is called a white line. In addition, a white line including a solid line and a broken line is called.
Although not shown, a radar may be provided, and surrounding objects may be detected by the radar and the imaging unit 5 (the optical system 14 and the photoelectric conversion unit 15).

  The communication unit 16 performs wireless communication (including optical beacon, radio wave beacon, and visible light communication) with other devices such as other cars 20 and traffic lights 40. Any communication method may be used.

  The storage unit 17 is configured by, for example, a non-volatile semiconductor memory such as a flash memory, and stores various programs for driving the automobile 10 (including automatic driving) and control parameters.

  The sensor 18 includes various sensors such as one or a plurality of vehicle speed sensors and a yaw rate sensor. The vehicle speed sensor detects the vehicle speed V of the automobile 10 and sends a detection signal to the control unit 19 or the like. The yaw rate sensor detects the yaw rate of the automobile 10 and sends a detection signal to the control unit 19 or the like. The yaw rate is the rate of change of the rotation angle in the turning direction of the vehicle or the like.

  The control unit 19 controls the entire automobile 10 and includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like. In the present embodiment, the control unit 19 sets and controls the imaging conditions of each unit region of the imaging device 100 of the photoelectric conversion unit 15. Further, when the automatic operation mode is set by the vehicle operation unit 11, the control unit 19 detects a white line on the road by the imaging unit 5 (the optical system 14 and the photoelectric conversion unit 15) and uses the imaging unit 5. Then, a moving body, an obstacle, and the like around the automobile 10 are detected, and automatic driving is performed up to the destination input to the navigation system 13 in cooperation with the navigation system 13.

  In the present embodiment, the automatic driving mode means that the steering wheel, accelerator, brake, turn signal switch, shift lever, and the like are all automatically controlled by the control unit 19. In addition, manual operation means that a driver performs operations such as a steering wheel, an accelerator, a brake, a turn signal switch, and a shift lever, and there are cases where the transmission is an automatic transmission and a manual transmission. In addition, in the automatic operation mode, in addition to the fully automatic operation in which all the operations are performed by the control of the control unit 19, the control unit 19 performs the imaging unit 5, the GPS 12, and the communication unit 16 even when the user operates the vehicle operation unit 11. Also included is a semi-automatic operation in which the automobile 10 is stopped or decelerated so as to avoid a collision based on the output of the sensor 18 or the like. Thereby, safety can be ensured while enjoying the driving of the user's automobile 10. The semi-automatic driving includes a case where the control unit 19 controls a part of the steering wheel, accelerator, brake, turn signal switch, shift lever, and the like in place of the driver.

(Other car 20)
The other vehicle 20 includes a communication unit 21, a vehicle operation unit 22, a storage unit 23, an imaging unit 24, a control unit 25, and the like, and the function of each unit is the same as the function of the vehicle 10. Although the other vehicle 20 is also omitted in FIG. 1, it has a basic configuration as an automobile. However, the other vehicle 20 may be a vehicle that does not include the communication unit 21. In addition, in the other vehicle 20, an autonomous driving vehicle and a manual driving vehicle are mixed. Among these, vehicles having at least the automatic driving mode can communicate with each other by the communication unit 21, and can transmit / receive information regarding whether automatic driving or manual driving is performed and image data acquired by the imaging unit 24. It is configured.

(Traffic signal generator 30)
The traffic signal generation device 30 is a device that controls a signal lamp displayed on the display unit 42 of the traffic light 40, and includes a signal information generation unit 31, a storage unit 32, a communication unit 33, and a control unit 34. Yes. The traffic signal generation device 30 can be installed at each of a plurality of traffic lights 40 provided at an intersection or the like, but the traffic signal generation device 30 may control the plurality of traffic lights 40.

  The signal information generation unit 31 generates a traffic signal based on the types and installation positions of a plurality of traffic lights 40 installed at an intersection or the like, instructions from a traffic control center (not shown) regarding traffic, and the like.

  The storage unit 32 is configured by, for example, a nonvolatile semiconductor memory such as a flash memory, and stores various programs of the traffic signal generation device 30, control parameters, and the like.

  The communication unit 33 transmits the traffic signal generated by the signal information generation unit 31 to each of one or a plurality of traffic lights 40 by wired or wireless. The communication unit 33 transmits and receives information to and from the traffic control center.

  The control unit 34 controls the entire traffic signal generation device 30 and includes a CPU, a RAM, a ROM, and the like. Moreover, the control part 34 can analyze the traffic condition based on traffic volume etc., and can control the signal information generation part 31 based on it.

(Signal 40)
The traffic light 40 includes a communication unit 41, a display unit 42, an optical system 43, a photoelectric conversion unit 44, a storage unit 45, and a control unit 46. Although only one signal device 40 is shown in FIG. 1, a plurality of signal devices 40 are usually provided. For example, in the case of an intersection, as illustrated in FIG. 2, four traffic lights 40a for automobiles and eight traffic lights 40b for pedestrians are provided. The traffic light 40 receives a traffic signal corresponding to the installed position from the traffic signal generation device 30, and turns on and blinks the indicator lamp of the display unit 42.

  The communication unit 41 receives the traffic signal generated by the signal information generation unit 31 by wire or wireless. In addition, the communication unit 41 transmits and receives various information such as vehicle driving information and traffic information to and from the automobile 10, other cars 20, and other traffic signals 40.

  The display unit 42 includes a signal lamp, and displays the signal lamp according to the traffic signal received by the communication unit 41. Specifically, a signal lamp is turned on, blinked, or turned off in order to permit or limit movement such as travel and stop for vehicles traveling on the road and pedestrians crossing the road. The display unit 42 may turn on not only a red light, a yellow light, and a blue light, but also an arrow light indicating that the vehicle can go straight, can turn left, and can turn right at an intersection.

  In the present embodiment, the optical system 43 and the photoelectric conversion unit 44 constitute an imaging unit 50 of the camera. The optical system 43 includes a plurality of lenses, and forms a subject image on the photoelectric conversion unit 44. When the imaging unit 50 is provided in the traffic signal 40a for an automobile, the optical system 43 is mainly used for acquiring an image of the vehicle. When the imaging unit 50 is provided in the pedestrian traffic light 40b, the optical system 43 is mainly used for acquiring images of pedestrians (including bicycles). The imaging unit 50 is provided in the vicinity of the display unit 42 (signal lamp).

  The photoelectric conversion unit 44 is configured by stacking an imaging chip that outputs a pixel signal corresponding to light incident from the optical system 43, a signal processing chip that processes the pixel signal, and a memory chip that stores the pixel signal. The imaging device 100 is provided. The configuration is the same as that of the photoelectric conversion unit 15 in the automobile 10 described above. The imaging element 100 can set imaging conditions according to traffic signals for each unit region composed of each pixel or a plurality of pixels (for example, 16 pixels × 16 pixels).

  FIG. 3 is a diagram illustrating an automobile traffic light 40a arranged at an intersection. In FIG. 3, the imaging part 50 is installed in the lower vicinity of the display part 42a of the traffic light 40a. The imaging unit 50 includes, for example, a wide-angle lens as the optical system 43, and has an angle of view including four lanes on both sides of a plurality of driving lanes (such as two lanes on one side) illustrated in FIG.

  The storage unit 45 is configured by, for example, a nonvolatile semiconductor memory such as a flash memory, and stores the image data acquired by the photoelectric conversion unit 44.

  The control unit 46 controls the entire traffic signal 40 and includes a CPU, a RAM, a ROM, and the like. In the present embodiment, the control unit 46 performs display control of the signal lamp of the display unit 42 according to the traffic signal, and controls imaging using the imaging element 100 of the photoelectric conversion unit 44.

  In addition, when the imaging range by the imaging unit 50 of the pedestrian traffic light 40b is included in the imaging range by the imaging unit 50 of the traffic signal 40a for automobiles, the imaging unit 50 (optical system) of the traffic signal 40b for pedestrians 43, photoelectric conversion unit 44) may be omitted.

<Description of Laminated Image Sensor>
The laminated image sensor 100 provided in the imaging unit of the automobile 10, the other car 20, and the traffic signal 40 described above will be described. The multilayer image sensor 100 is described in WO13 / 164915, which was previously filed and filed by the applicant of the present application. FIG. 4 is a cross-sectional view of the multilayer image sensor 100. The imaging device 100 includes a backside illumination type imaging chip 113 that outputs a pixel signal corresponding to incident light, a signal processing chip 111 that processes the pixel signal, and a memory chip 112 that stores the pixel signal. The imaging chip 113, the signal processing chip 111, and the memory chip 112 are stacked, and are electrically connected to each other by a conductive bump 109 such as Cu.

  As shown in the figure, incident light is incident mainly in the positive direction of the Z-axis indicated by a white arrow. In the present embodiment, in the imaging chip 113, the surface on the side where incident light enters is referred to as a back surface (imaging surface). Further, as shown in the coordinate axes, the left direction of the paper orthogonal to the Z axis is the X axis plus direction, and the front side of the paper orthogonal to the Z axis and the X axis is the Y axis plus direction. In the following several figures, the coordinate axes are displayed so that the orientation of each figure can be understood with reference to the coordinate axes in FIG.

  An example of the imaging chip 113 is a back-illuminated MOS image sensor. The PD layer 106 is disposed on the back side of the wiring layer 108. The PD layer 106 includes a plurality of PDs (photodiodes) 104 that are two-dimensionally arranged and store charges corresponding to incident light, and transistors 105 that are provided corresponding to the PDs 104.

  A color filter 102 is provided on the incident side of incident light in the PD layer 106 via a passivation film 103. The color filter 102 has a plurality of types that transmit different wavelength regions, and has a specific arrangement corresponding to each of the PDs 104. The arrangement of the color filter 102 will be described later. A set of the color filter 102, the PD 104, and the transistor 105 forms one pixel.

  On the incident light incident side of the color filter 102, a microlens 101 is provided corresponding to each pixel. The microlens 101 condenses incident light toward the corresponding PD 104.

  The wiring layer 108 includes a wiring 107 that transmits the pixel signal from the PD layer 106 to the signal processing chip 111. The wiring 107 may be multilayer, and a passive element and an active element may be provided.

  A plurality of bumps 109 are disposed on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the opposing surfaces of the signal processing chip 111, and the imaging chip 113 and the signal processing chip 111 are pressed and aligned. The bumps 109 are joined and electrically connected.

  Similarly, a plurality of bumps 109 are disposed on the mutually facing surfaces of the signal processing chip 111 and the memory chip 112. The bumps 109 are aligned with each other, and the signal processing chip 111 and the memory chip 112 are pressurized, so that the aligned bumps 109 are joined and electrically connected.

  The bonding between the bumps 109 is not limited to Cu bump bonding by solid phase diffusion, and micro bump bonding by solder melting may be employed. Further, for example, about one bump 109 may be provided for one block described later. Therefore, the size of the bump 109 may be larger than the pitch of the PD 104. Further, a bump larger than the bump 109 corresponding to the pixel region may be provided in a peripheral region other than the pixel region where the pixels are arranged.

  The signal processing chip 111 has a TSV (silicon through electrode) 110 that connects circuits provided on the front and back surfaces to each other. The TSV 110 is preferably provided in the peripheral area. The TSV 110 may also be provided in the peripheral area of the imaging chip 113 and the memory chip 112.

  FIG. 5 is a diagram for explaining the pixel array and the unit region 131 of the imaging chip 113. In particular, a state where the imaging chip 113 is observed from the back surface (imaging surface) side is shown. For example, 20 million or more pixels are arranged in a matrix in the pixel region. In the example of FIG. 5, adjacent 16 pixels of 4 × 4 pixels form one unit region 131. The grid lines in the figure indicate the concept that adjacent pixels are grouped to form a unit region 131. The number of pixels forming the unit region 131 is not limited to this, and may be about 1000, for example, 32 pixels × 64 pixels, or more or less.

  As shown in the partially enlarged view of the pixel region, the unit region 131 in FIG. 5 includes four so-called Bayer arrays, which are composed of four pixels of green pixels Gb, Gr, blue pixels B, and red pixels R, vertically and horizontally. The green pixels Gb and Gr are pixels having a green filter as the color filter 102, and receive light in the green wavelength band of incident light. Similarly, the blue pixel B is a pixel having a blue filter as the color filter 102 and receives light in the blue wavelength band, and the red pixel R is a pixel having a red filter as the color filter 102 and having a red wavelength band. Receives light.

  In the present embodiment, a plurality of blocks are defined so as to include at least one unit region 131 per block, and each block can control pixels included in each block with different control parameters. That is, it is possible to acquire imaging signals having different imaging conditions between a pixel group included in a certain block and a pixel group included in another block. Examples of the control parameters include a frame rate, a gain, a thinning rate, the number of addition rows or addition columns to which pixel signals are added, a charge accumulation time or accumulation count, a digitization bit number (word length), and the like. The imaging element 100 can freely perform not only thinning in the row direction (X-axis direction of the imaging chip 113) but also thinning in the column direction (Y-axis direction of the imaging chip 113). Furthermore, the control parameter may be a parameter in image processing after obtaining an image signal from a pixel.

  FIG. 6 is a diagram for explaining a circuit in the unit region 131. In the example of FIG. 6, one unit region 131 is formed by 9 pixels of 3 pixels × 3 pixels adjacent to each other. As described above, the number of pixels included in the unit region 131 is not limited to this, and may be less than this or more. The two-dimensional position of the unit region 131 is indicated by symbols A to I.

  The reset transistors of the pixels included in the unit region 131 are configured to be turned on and off individually for each pixel. In FIG. 6, a reset wiring 300 for turning on / off the reset transistor of the pixel A is provided, and a reset wiring 310 for turning on / off the reset transistor of the pixel B is provided separately from the reset wiring 300. Similarly, a reset line 320 for turning on and off the reset transistor of the pixel C is provided separately from the reset lines 300 and 310. Also for the other pixels D to I, dedicated reset wirings for turning on and off the respective reset transistors are provided.

  The transfer transistors of the pixels included in the unit region 131 are also configured to be turned on and off individually for each pixel. In FIG. 6, a transfer wiring 302 for turning on / off the transfer transistor of the pixel A, a transfer wiring 312 for turning on / off the transfer transistor of the pixel B, and a transfer wiring 322 for turning on / off the transfer transistor of the pixel C are separately provided. Also for the other pixels D to I, dedicated transfer wirings for turning on / off the respective transfer transistors are provided.

  Furthermore, the selection transistors of the pixels included in the unit region 131 are also configured to be turned on and off individually for each pixel. In FIG. 6, a selection wiring 306 for turning on / off the selection transistor of the pixel A, a selection wiring 316 for turning on / off the selection transistor of the pixel B, and a selection wiring 326 for turning on / off the selection transistor of the pixel C are separately provided. Also for the other pixels D to I, a dedicated selection wiring for turning on and off each selection transistor is provided.

  Note that the power supply wiring 304 is connected in common from the pixel A to the pixel I included in the unit region 131. Similarly, the output wiring 308 is commonly connected from the pixel A to the pixel I included in the unit region 131. Further, the power supply wiring 304 is commonly connected between a plurality of unit regions, but the output wiring 308 is provided for each unit region 131 individually. The load current source 309 supplies current to the output wiring 308. The load current source 309 may be provided on the imaging chip 113 side or may be provided on the signal processing chip 111 side.

  By individually turning on and off the reset transistor and the transfer transistor in the unit region 131, the charge including the charge accumulation start time, the accumulation end time, and the transfer timing independently from the pixel A to the pixel I included in the unit region 131 Accumulation can be controlled. In addition, by individually turning on and off the selection transistors in the unit region 131, the pixel signals of the pixels I from each pixel A can be output via the common output wiring 308.

  Here, a so-called rolling shutter system is known in which charge accumulation is controlled in a regular order with respect to rows and columns for pixels A to I included in the unit region 131. When a column is designated after selecting a pixel for each row by the rolling shutter method, pixel signals are output in the order of “ABCDEFGHI” in the example of FIG.

  Thus, by configuring the circuit with the unit region 131 as a reference, the charge accumulation time can be controlled for each unit region 131. In other words, it is possible to output pixel signals with different frame rates between the unit areas 131. Further, in the imaging chip 113, the unit area 131 included in another area is rested while the unit area 131 included in a part of the area is subjected to charge accumulation (imaging). Only the image can be taken and the pixel signal can be output. Furthermore, it is also possible to switch an area (accumulation control target area) where charge accumulation (imaging) is performed between frames and sequentially perform imaging in different areas of the imaging chip 113 to output pixel signals.

  FIG. 7 is a block diagram illustrating a functional configuration of the image sensor 100 corresponding to the circuit illustrated in FIG. 6. The analog multiplexer 411 sequentially selects the nine PDs 104 forming the unit region 131 and outputs each pixel signal to the output wiring 308 provided corresponding to the unit region 131. The multiplexer 411 is formed on the imaging chip 113 together with the PD 104.

  The pixel signal output via the multiplexer 411 is supplied to the signal processing chip 111 by a signal processing circuit 412 that performs correlated double sampling (CDS) / analog / digital (A / D) conversion. D conversion is performed. The A / D converted pixel signal is transferred to the demultiplexer 413 and stored in the pixel memory 414 corresponding to each pixel. The demultiplexer 413 and the pixel memory 414 are formed in the memory chip 112.

  The arithmetic circuit 415 processes the pixel signal stored in the pixel memory 414 and passes it to the subsequent image processing unit. The arithmetic circuit 415 may be provided in the signal processing chip 111 or may be provided in the memory chip 112. Note that FIG. 7 shows connections for one unit region 131, but actually these exist for each unit region 131 and operate in parallel. However, the arithmetic circuit 415 does not have to exist for each unit region 131. For example, one arithmetic circuit 415 may perform sequential processing while sequentially referring to the values of the pixel memory 414 corresponding to each unit region 131. Good.

  As described above, the output wiring 308 is provided corresponding to each of the unit regions 131. Since the image pickup device 100 includes the image pickup chip 113, the signal processing chip 111, and the memory chip 112, each chip is arranged in the plane direction by using an electrical connection between the chips using the bump 109 for the output wiring 308. Wiring can be routed without increasing the size.

<Explanation of distance measurement>
FIG. 8 is a diagram illustrating the position of the focus detection pixel on the imaging surface of the imaging device 100. In the present embodiment, focus detection pixels are discretely arranged along the X-axis direction (horizontal direction) of the imaging chip 113. In the example of FIG. 8, 15 focus detection pixel lines 60 are provided at predetermined intervals. The focus detection pixels constituting the focus detection pixel line 60 output an image signal for distance measurement. In the imaging chip 113, normal imaging pixels are provided at pixel positions other than the focus detection pixel line 60. The imaging pixel outputs an image signal for monitoring a moving object, an obstacle, and the like.

  FIG. 9 is an enlarged view of a region including a part of one of the focus detection pixel lines 60. In FIG. 9, a red pixel R, a green pixel G (Gb, Gr), and a blue pixel B, a focus detection pixel P1, and a focus detection pixel P2 are illustrated. The red pixel R, the green pixel G (Gb, Gr), and the blue pixel B are arranged according to the rules of the Bayer arrangement described above.

  The square area illustrated for the red pixel R, the green pixel G (Gb, Gr), and the blue pixel B indicates a light receiving area of the imaging pixel. Each imaging pixel receives a light beam passing through the exit pupil of the imaging optical system. That is, the red pixel R, the green pixel G (Gb, Gr), and the blue pixel B each have a square-shaped mask opening, and light passing through these mask openings reaches the light-receiving portion of the imaging pixel. .

  In addition, the shape of the light receiving region (mask opening) of the red pixel R, the green pixel G (Gb, Gr), and the blue pixel B is not limited to a quadrangle, and may be, for example, a circle.

  The semicircular regions exemplified for the focus detection pixel P1 and the focus detection pixel P2 indicate the light receiving region of the focus detection pixel. That is, the focus detection pixel P1 has a semicircular mask opening on the left side of the pixel position in FIG. 9, and light passing through the mask opening reaches the light receiving portion of the focus detection pixel P1. On the other hand, the focus detection pixel P2 has a semicircular mask opening on the right side of the pixel position in FIG. 9, and light passing through the mask opening reaches the light receiving portion of the focus detection pixel P2. In this way, the focus detection pixel P1 and the focus detection pixel P2 respectively receive a pair of light beams that pass through different regions of the exit pupil of the imaging optical system.

  Note that the position of the focus detection pixel line in the imaging chip 113 is not limited to the position illustrated in FIG. Also, the number of focus detection pixel lines is not limited to the example of FIG. Further, the shape of the mask opening in the focus detection pixel P1 and the focus detection pixel P2 is not limited to a semicircular shape. For example, a rectangular light receiving region (mask opening) in the imaging pixel R, the imaging pixel G, and the imaging pixel B is used. Part) may be a rectangular shape divided in the horizontal direction.

  The focus detection pixel line in the imaging chip 113 may be a line in which focus detection pixels are arranged along the Y-axis direction (vertical direction) of the imaging chip 113. As shown in FIG. 9, an image pickup element in which image pickup pixels and focus detection pixels are two-dimensionally arranged is known, and detailed illustration and description of these pixels are omitted.

  In the example of FIG. 9, the configuration in which the focus detection pixels P <b> 1 and P <b> 2 each receive one of a pair of focus detection light beams, the so-called 1PD structure, has been described. Instead, as disclosed in, for example, Japanese Patent Application Laid-Open No. 2007-282107, a configuration in which the focus detection pixels receive both of a pair of light beams for focus detection, a so-called 2PD structure, may be used. By adopting the 2PD structure in this manner, it is possible to read image data from the focus detection pixels, and the focus detection pixels do not become defective pixels.

  In the present embodiment, based on the distance measurement image signals output from the focus detection pixel P1 and the focus detection pixel P2, the amount of image misalignment between a pair of images due to a pair of light beams passing through different regions of the imaging optical system ( By detecting the (phase difference), the focus adjustment state (defocus amount) of the imaging optical system is calculated.

  In general, the pair of images are close to each other in a so-called front pin state in which the imaging optical system forms a sharp image of an object (for example, a preceding vehicle) before the planned focal plane, and conversely, In a so-called rear pin state that connects sharp images, they move away from each other. The pair of images relatively coincide with each other in a focused state that connects a sharp image of the object on the planned focal plane. Therefore, the relative positional deviation amount of the pair of images corresponds to the distance (depth information) to the object.

  Since the defocus amount calculation based on the phase difference is known in the field of cameras, detailed description thereof is omitted. Here, since the defocus amount and the distance to the object correspond one-to-one, the distance from the camera to each object can be obtained by obtaining the defocus amount for each object. That is, distance measurement (ranging) to the object can be performed at a plurality of positions on the shooting screen. The relationship between the defocus amount and the distance to the object is prepared as a mathematical expression or a lookup table in advance and stored in a nonvolatile memory or the like.

<Automotive control>
Hereinafter, the control executed by the control unit 19 of the automobile 10 will be described with reference to the flowchart of FIG. Note that this flowchart is started by starting the automobile 10, for example, starting an engine or starting an operation system. A program for executing the processing according to the flowchart of FIG. 10 is stored in a storage medium such as a ROM in the control unit 19 or the storage unit 17 of the automobile 10.

  In step S <b> 1, the control unit 19 starts imaging by the imaging device 100 of the photoelectric conversion unit 15. As described above, the imaging unit 5 (the optical system 14 and the photoelectric conversion unit 15) of the automobile 10 acquires images before and after the automobile 10, respectively.

  In step S <b> 2, the control unit 19 communicates with another vehicle 20 (a vehicle traveling in a different traveling lane in the same traveling direction as the vehicle 10 in addition to a vehicle traveling in the same lane as the vehicle 10) via the communication unit 16. I do. In the present embodiment, as shown in FIG. 11A, the vehicle 73A before traveling on the same lane as the automobile 10 is an autonomous driving vehicle, and before traveling on the adjacent traveling lane (the same traveling direction). Assume that the vehicle 72A is a manually operated vehicle. Further, it is assumed that the rear vehicle 73B traveling in the same lane as the automobile 10 is an automatic driving vehicle, and the rear vehicle 72B traveling in the adjacent traveling lane (the same traveling direction) is a manual driving vehicle.

Whether the vehicle around the automobile 10 is an automatic driving vehicle or a manual driving vehicle is determined based on a communication result such as known inter-vehicle communication by the communication unit 16. Further, when an identification mark or the like is displayed on another vehicle 20, the determination may be made based on the imaging result by the imaging unit 5 (the optical system 14, the photoelectric conversion unit 15). The identification mark may display a predetermined mark or code on the body of the vehicle, or may display identification information on a display unit (not shown) provided on the vehicle roof or the like.
Note that the control unit 19 estimates that the vehicle is a manually-driven vehicle for other vehicles 20 that cannot determine whether the vehicle is an automatically-driven vehicle or a manually-driven vehicle based on a communication result (communication impossible) or an imaging result.

  In step S <b> 3, the control unit 19 individually sets imaging conditions for each unit region 131 (FIG. 5) of the imaging device 100 of the photoelectric conversion unit 15. FIG. 11B and FIG. 11C are diagrams schematically showing an image of a subject (target object) formed on the image sensor 100 that images the front and rear of the automobile 10, respectively. Although an inverted inverted image is actually formed, it is shown as an erect image for the sake of clarity. The white line 80a represents the lane marking on the left side of the road in the traveling direction, the white line 80b represents the boundary line of the driving lane (lane), and the white line 80c represents the lane marking on the right side of the road.

  As described above, since the vehicles 72A and 72B before and after traveling in the lane adjacent to the automobile 10 are manually operated vehicles, the control unit 19 sets the region including the vehicle 72A in FIG. 11B as the attention region 71A. . Then, the frame rate of the unit area of the image sensor 100 corresponding to the attention area 71A is set higher than the frame rate of the normal area (for example, 60 fps), and the thinning rate is 0 to 20% lower than that of the normal area.

Similarly, the control unit 19 sets a region including the vehicle 72B in FIG. 11C as a region of interest 71B. Then, the frame rate of the unit area of the image sensor 100 corresponding to the attention area 71B is set higher than the frame rate of the normal area (for example, 60 fps), and the thinning rate is 0 to 20% lower than that of the normal area.
The control unit 19 further changes the setting of the thinning rate according to the moving speed of the automobile 10 or the relative moving speed of the automobile 10 and another car 20. For example, as the relative movement speed increases, the thinning rate is changed to a lower value.

  FIG. 11B is an example in which all the regions except the attention region 71A are normal regions. However, the region surrounding the vehicle 73A that is an autonomous driving vehicle is defined as a quasi attention region 74A, and the attention region 71A and the quasi attention region 74A. The area except for may be a normal area. FIG. 11C shows an example in which all the regions except the attention region 71B are normal regions. The region surrounding the vehicle 73B that is an autonomous driving vehicle is a quasi attention region 74B, and the attention region 71B and the quasi attention region 74B. The area except for may be a normal area. The control unit 19 may set different imaging conditions for the semi-attention areas 74A and 74B between the fully automatic operation and the semiautomatic operation. In this case, the control part 19 should just set the frame rate of the image pick-up element 100 in the case of semi-automatic operation higher than the frame rate of the image pick-up element 100 in the case of fully automatic operation. Moreover, the control part 19 may set the imaging area | region as a normal area | region in the case of a fully automatic driving | operation.

  The control unit 19 sets the frame rate of the unit area of the image sensor 100 corresponding to the semi-attention areas 74A and 74B to be lower than the frame rate of the normal area for the image sensor 100 that images the front and rear of the automobile 10, respectively. (For example, 30 fps), the thinning rate is set to about 30 to 60%.

In addition to the attention areas 71A and 71B, the control unit 19 may set an area including a white line on the road as the attention area. The frame rate of the unit area of the image sensor 100 corresponding to the attention area is set to be higher than the frame rate of the normal area (for example, 60 fps) for the image sensor 100 that captures the front and rear of the automobile 10, respectively. 0 to 20%.
As described above, when imaging the manually-driven vehicle and the automatically-driven vehicle, the imaging element 100 can be used efficiently by changing the imaging condition for each unit region of the imaging element 100, and power consumption and heat generation can be achieved. Can be suppressed.

  In step S4 of FIG. 10, the control unit 19 determines whether or not communication with the traffic signal 40 is possible via the communication unit 16, that is, whether or not the traffic signal 40 (intersection) has been approached. When the communication with the traffic signal 40 is not possible (outside the communicable area), the control unit 19 makes a negative determination in step S4 and proceeds to step S7. On the other hand, if communication with the traffic light 40 is possible (within the communicable area), the control unit 19 makes an affirmative decision in step S4 and proceeds to step S5.

In step S5, the control unit 19 receives information based on the image acquired by the photoelectric conversion unit 44 of the traffic signal 40 (the traffic signal 40a for a car or the traffic signal 40b for a pedestrian). For example, when the automobile 10 makes a left turn (when a vehicle such as the United States makes a right turn in a right-hand traffic area), the control unit 19 receives information related to the person from the pedestrian traffic light 40b. In this case, the control unit 46 of the traffic light 40 determines the presence or absence of a pedestrian based on the image acquired by the photoelectric conversion unit 44, and the control unit 19 receives information about the pedestrian determined by the control unit 46.
Note that the control unit 19 of the automobile 10 may receive the image data acquired by the photoelectric conversion unit 44 of the traffic light 40 and determine the presence or absence of a pedestrian based on the received image data.

  When the automobile 10 makes a right turn (when the vehicle turns to the left in a right-hand traffic area), the control unit 19 receives information from the automobile traffic signal 40a such as whether the oncoming lane is a self-driving car or a manual driving car. To do. Furthermore, the control part 19 receives the information regarding traffic signal switching (for example, signal switching information such as a change from a green signal to a red signal after a few seconds) from the traffic light 40a for automobiles.

  In step S6, the control unit 19 sets the imaging condition of the imaging device 100 of the photoelectric conversion unit 15 based on the information obtained in step S5. For example, when the automobile 10 makes a left turn at an intersection, the control unit 19 sets the frame rate of the unit area of the image sensor 100 corresponding to the left side of the shooting screen higher than the frame rate of the unit area corresponding to the right side of the shooting screen. The frame rate is changed in accordance with the speed of the automobile 10 and the moving speed of the pedestrian (for example, 4 km / h).

  For example, when the car 10 moving at a speed of 50 km / h decelerates to about 10 km / h due to a left turn, the frame rate of the unit area corresponding to the left side of the shooting screen is lowered as compared with that before the deceleration. In addition, when the automobile 10 makes a left turn, the control unit 19 sets the imaging condition of the image sensor 100 depending on whether a pedestrian crossing the sidewalk approaches the automobile 10 or whether a pedestrian moves away from the automobile 10. To do. That is, the control unit 19 increases the frame rate of the unit area of the image sensor 100 corresponding to the person approaching the automobile 10 and lowers the thinning rate so that the pedestrian away from the automobile 10 (especially, the automobile 10 is scheduled to pass). The frame rate of the unit area of the image sensor 100 corresponding to the person who has already crossed the pedestrian crossing) is increased and the thinning rate is increased.

  When the automobile 10 makes a right turn at an intersection, the control unit 19 sets the frame rate of the unit area corresponding to the right side of the shooting screen when the other vehicle 20 that goes straight from the oncoming lane is a manually operated vehicle. Is relatively higher than the frame rate of the unit area corresponding to the left side of the shooting screen, and the thinning rate is lowered. Further, when a pedestrian crossing the right-hand pedestrian crossing is approaching the automobile 10, the frame rate of the unit area of the image sensor 100 corresponding to the approaching pedestrian is further increased to further increase the thinning rate. make low.

  When the vehicle 10 makes a right turn or a left turn, the control unit 19 predicts an imaging region to be noticed according to the operation state of the turn signal switch and the operation amount of the steering wheel (the pedestrian or the like may be captured). It is also possible to predict the imaging area and change the imaging conditions.

Moreover, the control part 19 receives the information regarding switching of a traffic signal from the traffic signal 40a for motor vehicles, and changes the imaging condition of the image pick-up element 100 based on the information. For example, the control unit 19 receives information on the traffic signal switching that changes from a green signal to a red signal after a few seconds from the traffic light 40a for the automobile, and when the automobile 10 decelerates, the frame of the image sensor 100 that captures the front. Control the rate to be lower than before deceleration or increase the thinning rate. On the other hand, the control unit 19 performs control so that the imaging condition of the imaging device 100 that captures the rear side is maintained as it is.
In addition, when the automobile 10 decelerates, it is expected that the following vehicle approaches the automobile 10, so that the frame rate of the image sensor 100 that captures the rear is made higher than before the deceleration or the thinning rate is lowered. May be performed.
When the automobile 10 changes the speed, the control unit 19 may change the imaging condition by predicting a speed change according to the operation amount of the brake or the accelerator (the pedal depression amount).

  In step S7, the control unit 19 determines whether the engine (or operating system) is on. If the engine (or the driving system) is on, the control unit 19 makes an affirmative decision in step S7 and repeats the processes in and after step S2. If the engine (or the driving system) is off, the control unit 19 makes a negative determination in step S7 and ends the process according to this flowchart.

<Control of traffic light>
Next, the control executed by the control unit 46 of the traffic light 40 will be described with reference to the flowchart of FIG. A program for executing the processing according to the flowchart of FIG. 12 is stored in a storage medium such as a ROM in the control unit 46 or the storage unit 45.

  In step S <b> 10, the control unit 46 determines whether a traffic signal has been received from the traffic signal generation device 30. When the control unit 46 receives the traffic signal from the traffic signal generation device 30, the control unit 46 makes a positive determination in step S10 and proceeds to step S11. When the traffic signal from the traffic signal generator 30 is not received, the control unit 46 makes a negative determination in step S10 and waits for reception.

  The control unit 46 performs display control of the display unit 42 in step S11. For example, the control unit 46 performs control to switch the signal lamp display of the display unit 42 from red to blue according to the traffic signal received from the traffic signal generation device 30.

In step S12, the control unit 46 communicates with one or a plurality of vehicles or other traffic signals 40. Vehicles to be communicated by the control unit 46 can include the automobile 10 and other vehicles 20 including the communication unit 21.
The communication target vehicle may be a vehicle within a predetermined range from the intersection or the traffic light 40, or may be a vehicle that can communicate via an information providing system (not shown) installed on the road.

  The control unit 46 acquires information on the movement method such as whether the vehicle or a vehicle around the vehicle is an automatic driving vehicle or a manual driving vehicle from the communication target vehicle. Furthermore, the control part 46 acquires the information regarding the driving | running state of this vehicle or the surrounding vehicle of this vehicle from a communication object vehicle. For example, the control unit 19 or the control unit 25 of the communication target vehicle predicts a course change (right turn or left turn) at an intersection from the state of a turn signal switch for operating a winker (direction indicator). The control unit 46 acquires the right turn prediction information or the left turn prediction information predicted by the communication target vehicle from the communication target vehicle as information related to the driving situation.

  Note that the control unit 46 may determine whether the vehicle is an automatic driving vehicle or a manual driving vehicle based on a communication result with the communication target vehicle. Further, when an identification mark or the like is displayed on the vehicle, the control unit 46 may determine based on the imaging result by the imaging unit (the optical system 43 and the photoelectric conversion unit 44). Furthermore, the change of the course of the vehicle (right turn or left turn) may be determined by the control unit 46 based on the imaging result of the imaging unit (optical system 43, photoelectric conversion unit 44) from the operating state of the winker of the vehicle. Alternatively, the determination may be made based on whether the vehicle is on the left turn lane or the right turn lane.

  The control unit 46 can also communicate with other traffic signals 40 including the traffic signal 40a for automobiles or the traffic signal 40b for pedestrians, and can acquire information on traffic conditions at intersections and the like. The control unit 46 further communicates with the traffic signal generation device 30 related to the traffic signal generation of each traffic signal as necessary, and acquires information about the display state of the traffic signal.

  In step S <b> 13, the control unit 46 sets the imaging condition of the image sensor 100 of the photoelectric conversion unit 44 based on the signal lamp display state of the display unit 42 and the information acquired in step S <b> 12. Details of setting of imaging conditions in the imaging device 100 of the photoelectric conversion unit 44 will be described later.

  In step S14, the control unit 46 causes the imaging unit (the optical system 43 and the photoelectric conversion unit 44) to perform imaging under the imaging conditions set in step S13.

  In step S <b> 15, the control unit 46 transmits the image data acquired in step S <b> 14 to the automobile 10, another car 20, another traffic signal 40, or the like via the communication unit 41. Further, the control unit 46 performs estimation based on information extracted by performing image processing or image analysis based on the image data, for example, data such as vehicle direction and speed, and identification of an operating state of a winker or the like. Information on the travel lane of the vehicle is also transmitted in the same manner.

  Further, the control unit 46 recognizes and recognizes the estimated travel lane of the communication target vehicle (the automobile 10 or the other vehicle 20) or the obstacle object (including the vehicle or the pedestrian) based on the analyzed information. A message may be generated based on the result, and the message may be transmitted from the communication unit 41 to the communication target vehicle. The message is, for example, “a motorcycle comes from behind”, “a pedestrian crosses”, “an oncoming vehicle goes straight”, and the like. The control unit 46 repeatedly executes the processing from step S10 to step S15.

<Setting imaging conditions in traffic light>
FIG. 13 shows control of imaging conditions in the imaging device 100 of the imaging unit 50-1 installed integrally with or in the vicinity of the traffic light 40a for the automobile when the traffic light 40a for the automobile displays a green light. It is a figure which shows the example of.

  In FIG. 13, a traffic light 40a for an automobile for a lane A (lane) A, an imaging unit 50-1, and a traffic signal generation device 30-1 are shown. Other traffic lights at the intersection are not shown. The control unit 46 of the automobile traffic signal 40a sets, as the attention area 71, the range of the traveling lane in which the vehicle is moving due to the green signal within the imaging area 70 of the imaging unit 50-1. ).

  The control unit 46 controls the unit area of the image sensor 100 corresponding to the attention area 71 by increasing the frame rate or lowering the thinning rate than the unit areas other than the attention area 71. When there is a vehicle 72 that is a manually operated vehicle that is not automatic driving in the attention area 71 of FIG. 13, the control unit 46 sets the area surrounding the vehicle 72 as the special attention area 75. Then, the frame rate of the unit area of the image sensor 100 corresponding to the special attention area 75 is made higher than that of the unit area corresponding to the attention area 71. The control unit 46 also controls the thinning rate of the unit area of the image sensor 100 corresponding to the special attention area 75 to be lower than the thinning rate of the unit area corresponding to the attention area 71.

  In addition, the control unit 46 also lowers the thinning rate for the region including the vehicle 76 and the vehicle 77 that are temporarily stopped for the right or left turn in the attention region 71 than the unit region corresponding to the attention region 71. By doing so, it is possible to take an image with high resolution.

  FIG. 14 shows an imaging condition in the imaging device 100 of the imaging unit 50-1 installed integrally with or in the vicinity of the automobile traffic signal 40a when the automobile traffic signal 40a displays a red signal. It is a figure which shows the example of control.

  In FIG. 14, a traffic light 40a for an automobile for a lane A (lane) A, an imaging unit 50-1, and a traffic signal generation device 30-1 are shown. Other traffic lights at the intersection are not shown. The control unit 46 sets a pedestrian crossing where the pedestrian 90 can enter and its vicinity as the attention region 71 within the range of the imaging region 70 of the imaging unit 50-1 (shaded range).

  The control unit 46 controls the unit area of the image sensor 100 corresponding to the attention area 71 by increasing the frame rate or lowering the thinning rate than the unit areas other than the attention area 71. Further, when recognizing the pedestrian 90, the control unit 46 sets a region in a predetermined range including the pedestrian 90 as the special attention region 75. Then, the frame rate of the unit area of the image sensor 100 corresponding to the special attention area 75 is made higher than that of the unit area corresponding to the attention area 71. The control unit 46 also controls the thinning rate of the unit area of the image sensor 100 corresponding to the special attention area 75 to be lower than the thinning rate of the unit area corresponding to the attention area 71.

  FIG. 15 is a diagram illustrating an example of control of imaging conditions of the imaging unit 50-2 installed as a unit with the pedestrian traffic light 40b or in the vicinity thereof. In FIG. 15, a traffic light 40b for a pedestrian, an imaging unit 50-2, and a traffic signal generation device 30-2 are shown. Other traffic lights at the intersection are not shown. The control unit 46 of the traffic light 40b for the pedestrian sets the pedestrian crossing where the pedestrian 90 can enter and the vicinity thereof as the attention region 71 within the range of the imaging region 70 of the imaging unit 50-2 (the shaded range). ).

  The control unit 46 controls the unit area of the image sensor 100 corresponding to the attention area 71 by increasing the frame rate or lowering the thinning rate than the unit areas other than the attention area 71. Further, when recognizing the pedestrian 90, the control unit 46 sets a region in a predetermined range including the pedestrian 90 as the special attention region 75. Then, the frame rate of the unit area of the image sensor 100 corresponding to the special attention area 75 is made higher than that of the unit area corresponding to the attention area 71. The control unit 46 also controls the thinning rate of the unit area of the image sensor 100 corresponding to the special attention area 75 to be lower than the thinning rate of the unit area corresponding to the attention area 71.

  FIG. 16A is a diagram illustrating a scene imaged by the imaging unit 50-2 installed in the pedestrian traffic light 40b. FIG. 16B is a diagram illustrating setting of imaging conditions based on a subject recognition result using image data acquired by the imaging unit 50-2. In FIG. 16A, an imaging unit 50-2 having the above-described imaging device 100 is installed in a pedestrian traffic light 40b. According to the image sensor 100 according to the present embodiment, it is possible to measure not only the vertical and horizontal directions but also the movement in the depth direction, and therefore it is possible to measure the speed Vo at which the pedestrian 90 as the subject moves. .

  In FIG. 16B, the control unit 46 sets, as the attention region 71, a range including a pedestrian 90 crossing a pedestrian crossing within the imaging region 70 of the imaging unit 50-2. Then, the control unit 46 changes the imaging conditions for the unit areas of the image sensor 100 corresponding to the attention area 71 with the unit areas other than the attention area 71. At this time, the control unit 46 varies the imaging conditions depending on the speed Vo of the pedestrian 90.

  For example, when the absolute value of the speed Vo of the pedestrian 90 is large, the control unit 46 sets the frame rate of the unit area of the imaging device 100 corresponding to the attention area 71 including the pedestrian 90 from the unit area other than the attention area 71. The control is performed by increasing the value or decreasing the thinning rate.

  Further, the control unit 46 may change the imaging condition for the attention area 71 based on the positional relationship between the nearby vehicle or building and the pedestrian 90. For example, when there is a vehicle operating the blinker for turning right or left at the intersection in the imaging region 70, the vehicle may enter a pedestrian crossing. The region closer to the vehicle is set as a special attention region. Then, the frame rate of the unit area of the image sensor 100 corresponding to the special attention area is made higher than that of the unit area corresponding to the attention area 71. The control unit 46 also controls the thinning rate of the unit area of the image sensor 100 corresponding to the special attention area to be lower than the thinning ratio of the unit area corresponding to the attention area 71.

  Further, also in the attention area 71, the control unit 46 may change the imaging conditions for a plurality of pixels or areas 78P indicated by oblique lines and a plurality of pixels or areas 78S not indicated by oblique lines. In the example of FIG. 16B, the imaging conditions are different between pixels or regions adjacent in the checkerboard pattern in the vertical and horizontal directions, but the present invention is not limited to this.

  In addition, when the control unit 46 recognizes an object such as another pedestrian or a bicycle within the range of the imaging region 70, the control unit 46 may add a region including each of the plurality of objects to the attention region 71. . Then, in the plurality of regions of interest 71, the imaging conditions may be different between the plurality of pixels or regions 78P indicated by oblique lines and the plurality of pixels or regions 78S not indicated by oblique lines.

  FIG. 17 is a diagram illustrating a scene imaged by the imaging unit 50-1 installed in the traffic signal 40a for an automobile at the intersection. In FIG. 17, a traffic light 40a for an automobile includes a display unit 42a having an indicator lamp that indicates whether the vehicle can go straight, can turn left, or can turn right. An example of setting the imaging condition based on the subject recognition result using the image data captured by the imaging unit 50-1 will be described.

FIG. 17 shows a state in which the display unit 42a of the traffic light 40a lights up an indicator lamp indicating whether the vehicle can go straight and can turn left, and waits for a right turn. The control unit 46 of the traffic light 40a for the automobile sets the frame rate of the unit area of the image sensor 100 corresponding to the traveling lane A through which the vehicle traveling straight or turns left, and the traveling lane B through which the vehicle traveling straight passes, to other traveling lanes. Control is performed by increasing the frame rate of the unit area corresponding to C or by reducing the thinning rate. In other words, the control unit 46 performs control by reducing the frame rate of the unit area corresponding to the travel lane C or setting the thinning rate high.
Note that lowering the frame rate includes setting not to perform imaging in the unit area.

  In the control of the imaging conditions of the image sensor 100 in the automobile 10 or the traffic signal 40 (the traffic signal 40a for the automobile, the traffic signal 40b for the pedestrian) according to the embodiment described above, imaging is performed almost simultaneously with the display switching timing of the display unit 42. The conditions may be changed, or the imaging conditions may be changed with a certain time interval from the display switching timing. Alternatively, for a certain period of time immediately after display switching, the imaging condition may be changed using both the attention area set before the display switching and the attention area to be set after the display switching as the attention area.

According to the embodiment described above, the following operational effects can be obtained.
(1) The imaging device of the automobile 10 (or the traffic light 40) is configured to drive the imaging unit 5 (or the imaging unit 50) including the imaging device 100 that can set imaging conditions for a plurality of regions and other vehicles 20 around it. A control unit 19 (or control unit 41) that sets imaging conditions for a plurality of unit areas based on the method is provided. Thereby, an imaging condition suitable for the driving method of the surrounding car 20 can be set for the imaging device 100.

(2) Since the control unit 19 (or the control unit 41) sets different imaging conditions for each region in which another vehicle 20 with a different driving method is imaged, the vehicle 20 with a different driving method is selected with respect to the image sensor 100. Different imaging conditions may be set between the areas to be imaged.

(3) The driving method of the other vehicle 20 is an automatic driving method and a manual driving method, and the control unit 19 (or the control unit 41) is configured to capture an area for imaging the vehicle 20 of the automatic driving method and a manual driving method. Since different imaging conditions are set for the area where the car 20 is imaged, the area for imaging the automatic driving type car 20 and the area for imaging the manual driving type car 20 are different for the image sensor 100. Imaging conditions can be set.

(4) The control unit 19 (or the control unit 41) sets the frame rate of the region for imaging the manually operated vehicle 20 higher than the frame rate of the region for imaging the automatically driven vehicle 20. Thereby, since the frequency which images the car 20 of a manual driving system increases more than the frequency which images the car 20 of an automatic driving system, the attention degree with respect to the car 20 of a manual driving system can be raised. That is, it is possible to obtain quick and accurate information about the unpredictable behavior of the manually operated vehicle 20.

(5) The control unit 19 (or the control unit 41) sets a pixel thinning rate in a region where the manual driving type vehicle 20 is imaged to be lower than a pixel thinning rate in a region where the automatic driving type vehicle 20 is imaged. To do. As a result, the amount of information about the manually operated vehicle 20 can be made larger than the amount of information about the automatically operated vehicle 20. That is, it is possible to obtain more accurate information about the unpredictable behavior of the manually operated vehicle 20.

(6) Since the control unit 19 (or the control unit 41) that acquires information on the movement method of the other vehicles 20 in the surroundings is provided, for example, even when the other vehicles 20 in the surroundings are replaced, the latest acquired Based on the information, an imaging condition can be set for the imaging device 100.

(7) Since the control unit 19 (or the control unit 41) acquires information through communication with another vehicle 20, the image capturing condition of the image sensor 100 is appropriately set based on new information obtained through communication. Can be set to

(8) Since the control unit 19 (or the control unit 41) acquires information by imaging the other vehicle 20 by the imaging unit 5 (or the imaging unit 50), it is based on new information even in a situation where communication is not possible. The imaging conditions of the image sensor 100 can be set appropriately.

(9) The control unit 19 acquires information from the traffic signal 40 different from that of the other cars 20. Thereby, even in a situation where communication with another vehicle 20 is not possible, it is possible to appropriately set the imaging condition of the imaging element 100 based on the new information.

(10) The control unit 19 (or the control unit 41) acquires information indicating whether the other vehicle 20 is an automatically driven vehicle or a manually driven vehicle, and captures an image of the automatically driven vehicle and an imaged region of the manually driven vehicle. Set different imaging conditions. Thereby, for example, the imaging condition can be appropriately set for each unit region on the imaging surface of the imaging device 100 so that the degree of attention about the manually driven vehicle is higher than that of the autonomous driving vehicle. In particular, by increasing the frame rate and decreasing the pixel thinning rate for a manually driven vehicle, it is possible to quickly and accurately obtain information about the unpredictable behavior of the manually driven vehicle.

(11) Since the automobile 10 includes the imaging devices (1) to (10) described above, the imaging conditions for the imaging device can be appropriately set according to the movement method of the other cars 20 around the automobile 10.

(12) The control unit 19 (or the control unit 41) of the automobile 10 changes the imaging condition of the imaging device 100 according to the operation of the steering wheel, the turn signal switch, the accelerator, the brake, or the like in the automobile 10. Thereby, according to the course change of the motor vehicle 10, speed change, etc., an imaging condition can be appropriately set for every unit area in the imaging surface of the image sensor 100.

(13) The traffic light 40 is based on information related to movement of the imaging unit 50 including the imaging element 100 that can independently set imaging conditions of the plurality of areas, and the movement of the automobile 10 and the other cars 20. And a control unit 46 for setting imaging conditions. Thereby, imaging conditions can be set for the imaging device 100 in accordance with the traffic conditions of the automobile 10 and other cars 20 moving at an intersection or the like.

(14) The information related to the movement of the automobile 10 and the other car 20 includes a signal that permits the movement of the automobile 10 and the other car 20, and a signal that does not allow the movement of the automobile 10 and the other car 20. The unit 46 sets the imaging condition based on each signal. Thereby, the imaging conditions of the image sensor 100 can be appropriately set for the case where the automobile 10 and the other car 20 move and the case where the automobile 10 and the other car 20 do not move, respectively.

(15) The signal for permitting movement of the automobile 10 and the other vehicle 20 includes a signal for permitting straight travel, a signal for permitting a left turn, and a signal for permitting a right turn. The imaging conditions are set based on the above. Thereby, when the car 10 and the other car 20 go straight, turn left, or turn right, the imaging conditions of the image sensor 100 can be set appropriately.

(16) Since the control unit 46 changes the imaging condition in accordance with the switching of the information related to the movement of the automobile 10 and the other cars 20, the imaging unit 100 appropriately changes the imaging condition at the timing when the signal is switched. Can do.

(17) Since the traffic signal 40 includes the control unit 46 that acquires information related to the movement of the surrounding automobile 10 and the other vehicle 20, for example, even when the surrounding other vehicle 20 is replaced, based on the acquired latest information. Thus, imaging conditions can be set for the imaging device 100.

(18) Since the control unit 46 acquires information through communication with the vehicle 10 and the other vehicle 20, the imaging condition of the image sensor 100 can be appropriately set based on new information obtained through communication. .

(19) Since the control unit 46 acquires information by imaging the automobile 10 and the other vehicle 20 by the imaging unit 50, the imaging unit 100 appropriately sets the imaging condition of the imaging element 100 based on new information even in a situation where communication is not possible. Can be set.

(20) The control unit 46 acquires information indicating whether the vehicle 10 or the other vehicle 20 is an automatic driving type vehicle or a manual driving type vehicle as information related to movement, and an area for imaging the automatic driving type vehicle; Different imaging conditions are set for the region where the manually operated vehicle is imaged. Thereby, for example, the imaging condition can be appropriately set for each unit region on the imaging surface of the imaging device 100 so as to increase the degree of attention about the manually driven vehicle compared to the automatically driven vehicle. In particular, by increasing the frame rate and decreasing the pixel thinning rate for a manually operated vehicle, information can be quickly and accurately acquired about the unpredictable behavior of the manually operated vehicle.

(21) The control unit 46 acquires information indicating the course change of the automobile 10 and the other vehicle 20, and changes the region for setting the imaging condition based on the course change of the automobile 10 and the other vehicle 20. Thereby, the imaging condition of the image sensor 100 can be appropriately changed at the timing of the course change.

(22) Since the vehicle 10 and the communication unit 41 that communicates with the other vehicle 10 and the other vehicle 20 different from the other vehicle 20 are provided, for example, the information of the other vehicle 20 around the traffic signal 40 is obtained from the vehicle 10. Can be sent to.

(23) The automobile 10 and the traffic light 40 measure the moving speed of the object to be imaged, and change the imaging condition of the imaging device 100 according to the magnitude, direction, and orientation of the speed. Therefore, it is possible to appropriately set the imaging condition for each region on the imaging surface of the imaging device 100 according to the movement state of the object.

(24) The automobile 10 acquires information such as how many seconds later the signal changes, and reflects it in the driving situation of the automobile 10. Therefore, smooth operation can be realized in consideration of signal switching in advance.

(25) The automobile 10 or the traffic light 40 includes both the attention area set in the immediately preceding signal and the attention area to be set in the signal after the switching as the attention area for a certain time immediately after the signal switching. The imaging conditions of the image sensor 100 are set. Thereby, it is possible to appropriately set the imaging condition of the imaging device 100 with respect to a transitional state in signal switching.

(26) With the imaging system 1 including the automobile 10 and the traffic light 40, it is possible to realize a more organized traffic system based on information acquisition and communication by accurate and efficient imaging.

  In the above-described embodiment, the imaging unit 5 and the imaging unit 50 are respectively controlled by the control performed by the control unit 19 of the automobile 10 or the control unit 46 of the traffic light 40. However, part of the control of the imaging unit 5 and the imaging unit 50 is performed. May be performed by a control circuit (CPU or the like) inside the imaging unit.

  Further, a part of the processing performed by the control unit 46 of the traffic light 40 may be performed by the control unit 34 of the signal information generating device 30. The imaging unit 50 such as a camera does not necessarily have to be associated with the traffic signal 40, and may be installed according to traffic signals such as intersections or traffic conditions.

  Furthermore, in the above-described embodiment, the display unit / audio reproduction unit of the navigation system 13 is used for displaying the message, but a separate display / reproduction device may be used. Furthermore, a display / reproduction device configured by a HUD (Head Up Display) that projects information on the windshield of the automobile 10 and a speaker that reproduces audio information may be used.

  The following modifications are also within the scope of the present invention, and one or a plurality of modifications can be combined with the above-described embodiment.

(Modification 1)
The control unit 46 of the traffic light 40 acquires information on the ratio of the autonomous driving vehicle on the road or at the intersection by imaging by the imaging unit 50 or communication by the communication unit 41, and changes the imaging condition according to the ratio. Also good. For example, the control unit 46 performs control so as to increase the thinning rate in a time zone in which the ratio of the autonomous driving vehicle is large compared to when the ratio of the autonomous driving vehicle is small. Thereby, power consumption can be saved, and more efficient imaging can be performed.

(Modification 2)
The control unit 19 of the automobile 10 or the control unit 46 of the traffic light 40 may identify a sign such as an initial driver sign or an elderly driver sign in addition to identifying whether the car is an automatic driving car or a manual driving car. . Then, different imaging conditions are set in the unit area of the image sensor 100 corresponding to the driver of the first driving and the elderly driving car. For example, the vehicle on which the initial driver sign is displayed may be imaged at a frame rate that is higher than the unit area of the image sensor 100 corresponding to the manually operated vehicle. Thereby, according to a target object, an imaging condition can be appropriately set for every unit region in the imaging surface of the image sensor 100.

(Modification 3)
In the above description, distance measurement and detection of surrounding moving objects and obstacles are performed by imaging using the image sensor 100, but a radar (not shown) may be used in combination. Thereby, it is possible to acquire traffic information with higher reliability by utilizing the characteristics of the image sensor 100 and the radar.

  While various embodiments and modifications have been described above, the present invention is not limited to these contents. The aspect using combining each structure shown by embodiment and the modification is also contained in the scope of the present invention. Other embodiments conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Imaging system 5, 50 (50-1, 50-2) ... Imaging part 10 ... Car 19, 46 ... Control part 20 ... Other car 30 ... Traffic signal generator 40 ... Traffic light 40a ... Traffic light 40b for motor vehicles ... Pedestrian traffic light 42 ... Display section 70 ... Imaging areas 71, 71A, 71B ... Attention areas 72, 72A, 72B, 73A, 73B, 76, 77 ... Vehicles 74A, 74B ... Semi-attention area 75 ... Special attention area 90 ... Pedestrian 100 ... Image sensor 113 ... Imaging chip

Claims (10)

In an imaging device mounted on a car,
An imaging unit for imaging the outside of the vehicle;
An input unit for inputting information on a moving body around the vehicle from the outside;
An imaging control unit which on the basis of the information related to the mobile around the vehicle which is input to the input unit, changes the imaging conditions of a partial region of the imaging unit,
An imaging apparatus having The imaging device according to claim 1,
The image pickup control unit is an image pickup apparatus that changes a region in which the image pickup condition of the image pickup unit is changed according to information about a moving body around the vehicle input to the input unit. The imaging device according to claim 1 or 2 ,
The input unit is an imaging apparatus in which information on a position of a moving body around the vehicle is input as information about the moving body around the vehicle . In the imaging device according to any one of claims 1 to 3,
The input unit is an imaging apparatus in which information about the speed of a moving body around the vehicle is input as information about the moving body around the car . In the imaging device according to any one of claims 1 to 4,
A communication unit that communicates with a moving body around the vehicle;
The input unit is an imaging device in which information about a moving body around the vehicle is input by communication with the moving body around the vehicle by the communication unit. In the imaging device according to any one of claims 1 to 4,
Having a communication unit that communicates with the traffic light,
The input unit is an imaging apparatus in which information about a moving body around the vehicle is input by communication with the traffic signal by the communication unit. In the imaging device according to any one of claims 1 to 6 ,
The input unit is, as information on the moving body around the car, whether the driving method of the car around the car, or whether the car traveling around the car is being driven by a primitive driver or an elderly driver An imaging device to which information on whether or not is input . In the imaging device according to any one of claims 1 to 7 ,
The input device is an imaging device in which information about a pedestrian around the vehicle is input as information about a moving body around the vehicle . In the imaging device according to any one of claims 1 to 8,
The imaging control unit is an imaging device that changes an imaging speed of a partial region of the imaging unit based on information about a moving body around the vehicle input to the input unit. In the imaging device according to any one of claims 1 to 9 ,
The image pickup control unit is an image pickup apparatus that changes a thinning rate of pixels in a partial region of the image pickup unit based on information about a moving body around the vehicle input to the input unit.

Images