JP4862129B2 - Driving support device using in-vehicle camera device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a driving support apparatus using an in board camera for supporting a driver for selecting a correct route just before the route selection spot based on the recoded images photographed circumference scenery, traffic-control sign of routing spot designated on a map from far spot in advance, by interlocking a map device with photographed image. <P>SOLUTION: A camera 110 zoom-photographs the designated spots such as a crossroads with a CMOS sensor 112 of global shutter type, records its image signal on the recording medium by the recording/reproducing circuit 122, wherein, by the CMOS sensor 112 of global shutter type, a photographed image of a moving object without deformation can be obtained without a mechanical shutter. Before the designated spot, on an upper frame 133a of a display screen 132, the photographed image from the signal recording/reproducing circuit 122 is displayed and at the same time the upper image information of the road map from a map navigation 140 and the image information of a running spot and designated spot are made to display, for drive supporting by allowing the comparison with real scenery of the designated point and the recognition of the positional information. <P>COPYRIGHT: (C)2007,JPO&amp;INPIT

Description

  The present invention relates to a driving support device using an in-vehicle camera device, and in particular, by taking a magnified image of a designated point passing through a few minutes later by using an in-vehicle camera device using a CMOS sensor, which is a solid-state imaging device, The present invention relates to a driving support device having a function of making the shooting information useful for driving operation just before passing a point.

  FIG. 9 shows a configuration diagram of an example of a conventional in-vehicle camera device. In the figure, the in-vehicle camera device is composed of a camera 210 and a processor 220 which are a group of image pickup devices, and these are mounted on an automobile. The camera 210 condenses the light from the subject with the lens 211, and an optical image is formed on the imaging surface of a rolling shutter type CMOS area sensor (hereinafter referred to as “rolling type CMOS sensor”) 213, which will be described later, via the mechanical shutter 212. And imaging signals obtained by photoelectric conversion by the rolling CMOS sensor 213 are output. The mechanical shutter 212 is used for controlling the exposure time.

  The processor 220 receives the imaging signal output from the camera 210 as an input by the signal processing circuit 221, performs predetermined signal processing by the signal processing circuit 221, and then outputs the image signal from the output terminal 222 to the outside. The video signal output from the processor 220 is supplied to, for example, a monitor device and displayed. Accordingly, the in-vehicle camera device is used for the purpose of accurately performing garage entry, for example, based on a captured image captured by the camera 210 when entering the garage.

  FIG. 10 shows a configuration diagram of an example of a conventional driving support apparatus. In the figure, the driving support device 230 is mounted on an automobile, and includes a map device 231 and a display unit (monitor) 232, an integrated type of the map device 231 and the display unit 232, the map device 231 and the display unit 232. There is a connected type. The map device 231 stores top view information on the map in advance, and provides the information to the display unit 232. The display unit 232 receives the top view information of the map from the map device 231 and displays the top view of the driving map on the screen frame 233 to support driving.

  Next, the in-vehicle camera device of FIG. 9 will be described in more detail. In the in-vehicle camera device shown in FIG. 9, a solid-state imaging device used for the camera 210 is easier to manufacture than a CCD (Charge Coupled Devise) and known as low power consumption. A CMOS sensor 213 whose image quality is improved by improving the technology is used. The CMOS sensor 213 is a conventionally known rolling shutter type CMOS sensor (see, for example, Patent Document 1).

  FIG. 11 shows an equivalent circuit diagram of an example of the conventional CMOS sensor. In the CMOS sensor shown in the figure, for simplicity, the unit pixel 1 has a 2 × 2 pixel arrangement in which two horizontal pixels and two vertical pixels are arranged. The unit pixel 1 includes a photodiode (PD) 2 for photoelectrically converting a subject image, a signal charge amplification MOS field effect transistor (hereinafter referred to as MOSFET) 3, a charge transfer MOSFET 4, a reset MOSFET 5, and a selection. The power supply line 6 is connected to the drains of the MOSFETs 3 and 5, and the source of the amplification MOSFET 3 is connected to the drain of the selection MOSFET 7.

  The gate electrode of the amplification MOSFET 3 is in a floating diffusion (FD), and the charge of the photodiode 2 is transferred to the gate electrode (FD) of the amplification MOSFET 3 through the drain-source of the charge transfer MOSFET 4. The potential of the gate electrode (FD) of the amplification MOSFET 3 is reset by the reset MOSFET 5.

  When the selection MOSFET 7 is turned on, the source of the amplification MOSFET 3 is conducted to the pixel output line 8 through the drain and source of the selection MOSFET 7. The pixel output line 8 is connected to the drain of the constant current supply MOSFET 9. The constant current supply MOSFET 9 acts as a load of the source follower circuit of the amplification MOSFET 3. The constant current supply MOSFET 9 is controlled by the gate potential of the gate potential supply line 13.

  The reset control line 10, the charge transfer control line 11, and the pixel selection control line 12 are connected to the gate electrodes of the reset MOSFET 5, the charge transfer MOSFET 4, and the selection MOSFET 7, respectively. It is supplied from the pulse supply terminals 15, 14, and 16 through the drains and sources of the MOSFETs 19, 20, and 21, respectively.

  The vertical shift register 17 is a circuit for selecting a 2 × 2 pixel row for row sequential scanning, and the vertical shift register output lines 18-1 and 18-2 are connected to the gate electrodes of the MOSFETs 19, 20, and 21 in each row. It is connected and determines which row of pixels is controlled by the pulse supplied to the terminals of the pulse supply terminals 15, 14, 16.

  The read block 22 is connected to a capacitor 23 for holding a reset signal output, a capacitor 24 for holding an optical signal output, switching MOSFETs 25 and 26 for selecting which one to hold, and horizontal output lines 27 and 28. Switch MOSFETs 29 and 30. The switching MOSFETs 25 and 26 are switching-controlled by pulses supplied from the terminals 37 and 38 to the gate electrodes.

  The horizontal shift register 34 is a horizontal shift register output line 35-connected to the gates of the MOSFETs 29 and 30 for switching which column of the 2 × 2 pixels is to be output to the horizontal output lines 27 and 28. 1 and the output potential to 35-2. In addition, a potential for resetting the horizontal output lines 27 and 28 is supplied from the terminal 33, and the reset timing is performed by switching the switching MOSFETs 31 and 32 with a pulse supplied from the terminal 36. The horizontal output lines 27 and 28 are connected to the input terminal of the differential amplifier 39. The differential amplifier 39 takes the difference between the reset signal output and the optical signal output, and outputs the difference signal from the amplifier output terminal 40 to the outside of the sensor.

  Next, the operation of the conventional CMOS sensor shown in FIG. 11 will be described with reference to the timing chart of FIG. Note that all MOSFETs in FIG. 11 are N-type. Therefore, the MOSFET is turned on when the gate potential is high (High) and turned off when the gate is low (Low).

  First, as shown in FIG. 12D, the potential of the vertical shift register output line 18-1 becomes High at time t1, thereby selecting the pixel 1 in the first row. Subsequently, as shown in FIG. 12C, the input pulse at the pulse supply terminal 16 becomes High at time t2, and the selection MOSFET 7 of the pixel 1 in the first row is turned on. The source of the amplification MOSFET 3 of the pixel 1 is connected to the constant current supply MOSFET 9 through the drain / source of the selection MOSFET 7 and the pixel output line 8 to form a source follower circuit.

  In this state, first, a high-level pulse is supplied to the pulse supply terminal 15 as shown in FIG. 12B, and the gate electrode of the amplification MOSFET 3 passes through the drain and source of the reset MOSFET 5 of the pixel 1 in the first row. (FD) is reset. Thereafter, at time t3, the input pulse of the pulse supply terminal 37 becomes High as shown in FIG. 12 (I), the switching MOSFET 25 is turned on, and the capacitor 23 outputs from the source follower circuit of the pixel 1 in the first row. The reset signal output is held.

  Next, when a high pulse is applied to the pulse supply terminal 14 at time t4 as shown in FIG. 12A, the charge transfer MOSFET 4 in the pixel 1 in the first row is turned on, and the pixel 1 in the first row. The charge accumulated in the photodiode 2 is transferred to the gate electrode (FD) of the amplification MOSFET 3 through the drain / source of the charge transfer MOSFET 4. At time t5, when a high pulse is applied to the pulse supply terminal 38 as shown in FIG. 12 (J), the optical signal output output from the source follower circuit of the pixel 1 in the first row is held in the capacitor 24. Is done. Subsequently, as shown in FIG. 12C, the input pulse at the pulse supply terminal 16 becomes Low at time t6, so that the selection MOSFET 7 in the pixel 1 in the first row is turned off, and the pixel in the first row. The output from 1 disappears.

  During this time, the input signal of the terminal 36 is High as shown in FIG. 12 (H), and the horizontal output lines 27 and 28 are in the reset state. However, when the input signal at the terminal 36 becomes Low as shown in FIG. 12 (H) at the time t6 and the High pulse shown in FIG. 12 (F) is applied to the horizontal shift register output line 35-1 in this state. Since the switching MOSFETs 29 and 30 in the first column are turned on, the signals of the capacitors 23 and 24 in the first column are output to the horizontal output lines 27 and 28 through the switching MOSFETs 29 and 30 in the first column, respectively. And supplied to the differential amplifier 39. The differential amplifier 39 calculates the difference between each signal of the capacitors 23 and 24 in the first column, that is, the reset signal output and the optical signal output, and removes the optical signal from which the noise caused by the threshold variation of the amplification MOSFET 3 is removed. Output from the output terminal 40.

  Next, when a high pulse is applied to the terminal 36 at time t7 shown in FIG. 12 (H), the horizontal output lines 27 and 28 are reset again, and then to the horizontal shift register output line 35-2, as shown in FIG. 12 (G). As shown, the high pulse is applied at time t8, and the switching MOSFETs 29 and 30 in the second column are turned on, so that the signals of the capacitors 23 and 24 in the second column are switched to the switching MOSFETs 29 and 30 in the second column. Are output to the horizontal output lines 27 and 28, supplied to the differential amplifier 39, and the second column signal is output from the differential amplifier 39 to the output terminal 40 in the same manner as the first column.

  After that, at time t9 shown in FIG. 12D, the potential of the vertical shift register output line 18-1 becomes Low, and the processing for the first row is completed. Next, at time t10, as shown in FIG. 12E, the potential of the vertical shift register output line 18-2 becomes High, the same processing as in the first row is performed, and the reading of all pixels is completed.

  Therefore, in the case of this CMOS sensor, the timing of photoelectric conversion by the photodiodes 2 in the first and second rows is different. Such an imaging method is called a rolling shutter or a focal plane.

JP 2003-17677 A

  When taking an image using such a rolling CMOS sensor 213, for simplicity, description will be made together with a conceptual diagram shown in FIG. 13A of pixels P11 to P24 in 2 rows and 4 columns, and pixels P11 to P14 in a certain row. Is read in order from left to right (solid arrow I), and when one line is read, it returns to the beginning of the next line (dashed arrow II), and then reads the next line again from left to right ( The operation of solid arrow III) is repeated. Therefore, since the exposure process is performed for the first time when each pixel is selected, the signal acquisition times do not coincide with each other when all the pixels are read. When all the pixels of the CMOS sensor have been read, one picture has been read.

  For example, when a rectangle 150 moving from the left to the right of the screen is photographed as shown in FIG. 13B, if the photograph is performed using the conventional rolling CMOS sensor 213, the photographed images have different photographing times in each row. If the rectangle to be photographed moves while reading the pixels as described above, the image is captured as a parallelogram image 151 distorted by the movement of the rectangle 150 as shown in FIG. .

  Accordingly, for example, as shown in FIG. 14A, when the vehicle mounted with the on-vehicle camera device zooms in on a distant view of the vicinity of the intersection 104 where the vehicle will pass after a few minutes, the intersection 104 The captured image of the vehicle 105 moving at a high speed from left to right is significantly distorted and out of focus, as indicated by 105 'in FIG. 5B, and in front of the intersection 104 shown in FIG. As shown by 106 ′ and 107 ′ in the same figure (B), the in-vehicle camera device itself is also moving in each of the captured images of the traffic light 106 fixed on the left side and the post office 107 on the right side of the intersection 104. Therefore, it will be distorted.

  9 uses a mechanical shutter 212 in front of the light incident side of the rolling CMOS sensor 213 to perform exposure for one frame period of all lines corresponding to the open period, By sequentially reading line by line, the exposure process and the signal reading process can be separated, and as a result, a captured image of a moving subject such as a vehicle moving at high speed can be prevented from being distorted. However, the control of the mechanical shutter 212 is complicated, the number of parts is large, the whole apparatus becomes large, and it is disadvantageous in terms of power consumption.

  It is conceivable to use the imaging signal from such an in-vehicle camera device for driving support. However, as a conventional on-vehicle multipurpose device, although there is a composite type of an audio device, a map device, and a display unit, the on-vehicle camera device of FIG. 9 and the driving support device of FIG. 10 are conventionally provided separately, The in-vehicle camera device has limited uses such as monitoring of garage storage, and there is no one that connects these and performs driving support using an image captured by the in-vehicle camera device.

  Further, in the conventional in-vehicle camera device, for example, the camera uses the rolling CMOS sensor 213 to recognize a traffic sign at night, bad weather, and a point before and after a tunnel with a large difference in brightness. There is a problem that receives a lot of restrictions from such points. Furthermore, in the conventional driving support device, the map information provided by the map device 231 is top view information, not solid information, and is insufficient as information for route selection. The information on the map provided by the device 231 is ready-made information, and it is difficult to immediately reflect the running information in the driving operation.

  The present invention has been made in view of the above points, and a clear captured image without distortion is obtained during high-speed traveling, and based on the captured image, it is immediately reflected in traffic sign recognition, route selection, driving operation, and the like. An object of the present invention is to provide a driving support device using an in-vehicle camera device capable of providing imaging information during traveling that can be performed.

  Further, another object of the present invention is based on an image obtained by photographing and recording in advance a landscape around a route selection point designated on a map and a traffic sign by linking a map device and a captured image. An object of the present invention is to provide a driving support device using an in-vehicle camera device for assisting the driver in selecting an accurate route immediately before the route selection point.

In order to achieve the above object, the present invention is mounted on an automobile, and an on-vehicle camera that supports driving using an image signal output from the solid-state image sensor by imaging a subject outside the automobile with the solid-state image sensor. a driving support device using the device, holes timing of the start and end of the exposure of an optical image of an object on the photoelectric conversion regions of a plurality of all pixels obtained by photoelectric conversion by exposure so that all pixel simultaneous It is equipped with a global shutter type CMOS sensor as a solid-state image sensor that sequentially outputs as a video signal from each pixel the hole charge accumulated during the exposure period after accumulating the charge in all pixels, and specified in advance while driving the car Both the in-vehicle camera device that captures the point and the video signal output from the global shutter type CMOS sensor perform predetermined signal processing and output the video signal Recording / reproducing means for recording and reproducing a video signal subjected to signal processing on a recording medium, and road map information between at least a traveling point where the automobile is currently traveling and a designated point, and each image of the traveling point and the designated point The map information generating means that is generated together with the information and the image of the video signal from the recording / reproducing means are displayed in the first display area at a position in the vicinity before reaching the designated point, and the map information generating means Display means for displaying road map information and image information of travel points and designated points in the second display area, and driving assistance is provided by images displayed in the first and second display areas of the display means. Here, the global shutter CMOS sensor described above is formed in a first conductivity type well formed on a semiconductor substrate and a second region different from the predetermined first region in the well, and is connected to the well. A photodiode having a buried portion of the second conductivity type, photoelectrically converting an optical image to accumulate hole charges , a ring-shaped gate electrode formed on the first region via a gate oxide film, and a ring-shaped A first source portion of the first conductivity type formed in a region in the well corresponding to the central opening of the gate electrode, and the gate so as not to reach the outer periphery of the ring-shaped gate electrode around the first source portion; a second conductivity type source region near portion for storing been hole charges transferred from and connected to the first source region is formed are embedded in in the well so as not to contact the oxide film photodiode, the wells The first source type and the first drain portion of the first conductivity type formed separately from the source vicinity region portion in a third region different from the first region of the first region, and are accumulated in the source vicinity region portion. A ring-shaped gate transistor for outputting the hole charge as an imaging signal, and a transfer gate electrode formed on the first region so as to cover a part of the ring-shaped gate electrode, and the transfer gate electrode and the ring-shaped gate Each pixel has a transfer gate transistor that controls the potential difference with the electrode and transfers the hole charges accumulated in the photodiode to the ring gate transistor all at once.
A well is continuously present in the region immediately below the gate oxide film from the transfer gate electrode to the ring-shaped gate electrode , and from the photodiode to the source vicinity region , and the buried portion The accumulated hole charge is controlled by controlling the potential of the ring gate electrode and the potential of the transfer gate electrode so that a potential difference as a barrier does not occur in the well between the ring gate electrode and the transfer gate electrode, respectively. After all the pixels are transferred all at once to the source vicinity region through only the well and the hole charge is output as an imaging signal, the potential of the ring-shaped gate electrode and the potential of the transfer gate electrode are the first drain. in parts of the potential below the potential is reset, until the accumulation of hole charge in the photo diode are transferred next hole charge starts imaging During the output period of No. be continuously performed, the imaging signal is characterized by continuously outputted for each frame.

  In the present invention, a video signal obtained by photographing a designated point with an in-vehicle camera device equipped with a global shutter type CMOS sensor during traveling is recorded on a recording medium by the recording / reproducing means, and then reaches the designated point. Since the photographed image of the designated point from the recording / reproducing unit is displayed in the first area of the display unit at a position near the front, the designated point can be selected from the automobile traveling at high speed without having a mechanical shutter. The captured image of each subject at the specified point obtained by shooting can be displayed without distortion, and the road map information and the image information of the traveling point and the specified point are displayed in the second area of the display means. Thus, the driver can grasp the positional relationship between the current traveling position and the designated point, and the stereoscopic image of the designated point displayed in the first area can be compared with the actual scenery at the designated point. It can be.

  In the present invention, the global shutter type CMOS sensor transfers the charges accumulated in the photoelectric conversion region by photoelectric conversion to the source vicinity region where the signal output storage capacity is large. Compared to a wider dynamic range.

  According to the present invention, by using a global shutter type CMOS sensor, a photographed image of each subject at a designated point obtained by photographing a designated point from an automobile traveling at high speed without having a mechanical shutter can be obtained. Since it can be displayed without distortion without causing image deformation depending on the readout method, complicated control when using a mechanical shutter is unnecessary, and the entire apparatus is not enlarged. Power consumption can also be reduced.

  Further, according to the present invention, the road map information and each image information of the travel point and the designated point are displayed in the second area of the display means, so that the driver can know the positional relationship between the current travel position and the designated point. As well as making it possible to compare the stereoscopic image of the designated point displayed in the first area with the actual scenery at the designated point, the driver can discriminate the traffic sign before the designated point and drive The user can drive and move while checking the actual scenery around the designated point, and when the designated point approaches the vicinity of an intersection where it is easy to make a mistake in traffic signs, etc. It is displayed as an image for the driver, so the driver can select the correct route, guide the car, and use it for driving operation, driving support, such as route selection, sign compliance, and prevent mistakes in route selection It is possible to perform a safe driving.

  In addition, according to the present invention, the global shutter type CMOS sensor transfers the charges accumulated in the photoelectric conversion region by photoelectric conversion to the source vicinity region where the signal output storage capacity is large. A wide dynamic range is realized compared to a CMOS sensor, so traffic signs can be recognized more clearly than before, for example, at night, in bad weather, and before and after tunnels with a large difference in brightness. Compared to this, driving assistance such as more accurate route selection and sign compliance can be performed.

  Next, embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows a block diagram of an embodiment of a driving support device using an in-vehicle camera device according to the present invention. As shown in the figure, the driving support device of the present embodiment includes a camera 110 that is an imaging device group, a recorder 120 that is a recording / reproducing device group, a display 130 that is a display device group, and a map navigation 140. It is installed in the car. The camera 110 includes a lens 111 and a global shutter type CMOS area sensor (hereinafter referred to as “global type CMOS sensor”) 112 which will be described later. An optical image from a subject is placed on the imaging surface of the global type CMOS sensor 112 by the lens 111. The imaging signal obtained by imaging and photoelectric conversion by the global CMOS sensor 112 is output to the recorder 120.

  The recorder 120 includes a signal processing circuit 121, a signal recording / reproducing circuit 122, and an output terminal 123. The signal processing circuit 121 performs predetermined signal processing such as gamma correction on the image pickup signal from the camera 110. Then, the video signal obtained by the signal processing is supplied to the signal recording / reproducing circuit 122, temporarily recorded in a recording medium in a predetermined signal format, and output to the display 130 via the output terminal 123.

  The display 130 includes a display processing unit 131 and a display screen 132 thereof. The video signal supplied from the recorder 120 is subjected to predetermined signal processing for display by the display processing unit 131, and a screen frame of the display screen 132 is displayed. An image is displayed on the upper part 133a. In addition, on the screen frame lower portion 133 b of the display screen 132, map top view information and the like from the map device 141 constituting the map navigation 140 are displayed. The display screen 132 is provided at a position that the driver can visually recognize while driving.

  The map navigation 140 generates top view information of a road map between the travel point where the vehicle on which the device is mounted is currently traveling by the map device 141 and the designated point to be imaged, and at least the travel point and Each image information of the designated point is generated and supplied to the display 130. Note that the map navigation 140 obtains the position information of each image information of the travel point and the designated point by a known method using, for example, a GPS (Global Positioning System) and the like. Although image information is generated, its method is well-known, and therefore detailed description thereof is omitted.

  FIG. 2 shows a block diagram of an embodiment of the display processing unit 131 in FIG. As shown in FIG. 2, the display processing unit 131 includes an input processing unit 1311, a memory 1312, and a buffer 1313, and an image displayed on the display screen 132 by the input processing unit 1311 for a video signal input from the recorder 120. The display processing is performed based on the operation signal related to the brightness, resolution, size, and signal compression control of the technical viewpoint, and the processed video signal is stored in the memory 1312.

  The memory 1312 reads the stored video signal at the designated read address, supplies it to the buffer 1313, temporarily holds it, and then outputs it to the display 130. The read address indicates, for example, an address for selecting a screen frame, an address where element data of the screen touch panel is stored, and the like.

Next, the configuration and operation of the global CMOS sensor 112 used in this embodiment will be described in detail. 3 (A) is plan view of an example of one pixel of the global type CMOS sensor 112 described above, FIG. (B) is a cross-sectional view taken along line X-X 'in FIG. (A). FIG. 3 (A), the as shown in (B), p on the p ⁺ -type substrate 41 ^- grown -type epitaxial layer 42, there is a n-well 43 to the surface of the epitaxial layer 42. On the n-well 43, a gate electrode 45 having a ring shape as a first gate electrode is formed with a gate oxide film 44 interposed therebetween.

An n ⁺ -type source region 46 is formed on the surface of the n-well 43 corresponding to the center portion of the ring-shaped gate electrode 45, a source vicinity p-type region 47 is formed adjacent to the source region 46, and An n ⁺ -type drain region 48 is formed at a position apart from the source region 46 and the p-type region 47 near the source. Further, a buried p ⁻ -type region 49 is present in the n-well 43 below the drain region 48. The buried p ⁻ -type region 49 and the n-well 43 constitute the buried photodiode 50 shown in FIG.

  Between the embedded photodiode 50 and the ring-shaped gate electrode 45, there is a transfer gate electrode 51 which is a second gate electrode. The drain region 48, the ring-shaped gate electrode 45, the source region 46, and the transfer gate electrode 51 include a drain electrode wiring 52, a ring-shaped gate electrode wiring 53, a source electrode wiring (output line) 54, and a transfer gate electrode, which are metal wirings, respectively. A wiring 55 is connected. Further, a light shielding film 56 is formed above each of the above components as shown in FIG. 3B, and an opening 57 is formed at a position corresponding to the embedded photodiode 50 in the light shielding film 56. Has been. The light shielding film 56 is formed of a metal or an organic film. The light reaches the embedded photodiode 50 through the opening 57 and is photoelectrically converted.

  Next, the pixel structure of the CMOS sensor and the entire structure of the image sensor will be described with reference to FIG. 4 expressed by an electric circuit. In the figure, first, pixels are arranged in a pixel spread area 61 in m rows and n columns. In FIG. 4, one pixel 62 of s rows and t columns among these m rows and n columns pixels is represented by an equivalent circuit. The pixel 62 includes a ring-shaped gate MOSFET 63, a photodiode 64, and a transfer gate MOSFET 65. The drain of the ring-shaped gate MOSFET 63 is the n-side terminal of the photodiode 64 and the drain electrode wiring 66 (corresponding to 52 in FIG. 3). , The source of the transfer gate MOSFET 65 is connected to the p-side terminal of the photodiode 64, and the drain is connected to the back gate of the ring-shaped gate MOSFET 63.

In FIG. 3B, the ring-shaped gate MOSFET 63 has a p-type region 47 near the source directly below the ring-shaped gate electrode 45 as a gate region, and an n ⁺ -type source region 46 and an n ⁺ -type drain region 48. An n-channel MOSFET. In FIG. 3B, the transfer gate MOSFET 65 has an n well 43 just below the transfer gate electrode 51 as a gate region, a p ⁻ type region 49 embedded with a photodiode 50 as a source region, and a p-type region 47 near the source. A p-channel MOSFET serving as a drain.

  In FIG. 4, in order to read a signal for one frame from each pixel of m rows and n columns, there is a circuit 67 for generating a frame start signal for giving a signal to start reading. The frame start signal may be given from outside the image sensor. This frame start signal is supplied to the vertical shift register 68. The vertical shift register 68 outputs a signal indicating which row of pixels is read out from each pixel of m rows and n columns.

  The pixels in each row are connected to a control circuit that controls the potentials of the ring-shaped gate electrode, transfer gate electrode, and drain electrode, and these control circuits are supplied with the output signal of the vertical register 68. For example, the ring-shaped gate electrode of each pixel in the s-th row is connected to the ring-shaped gate potential control circuit 70 via the ring-shaped gate electrode wiring 69 (corresponding to 53 in FIG. 3), and the transfer gate electrode of each pixel is Are connected to the transfer gate potential control circuit 72 via the transfer gate electrode wiring 71 (corresponding to 55 in FIG. 3), and the drain electrode of each pixel is drained via the drain electrode wiring 66 (corresponding to 52 in FIG. 3). It is connected to the potential control circuit 73. Each control circuit 70, 72, 73 is supplied with an output signal from the vertical shift register 68.

  Since the ring-shaped gate electrode is controlled for each row, wiring is performed in the horizontal direction. However, since the transfer gate electrode is controlled simultaneously for all pixels, the wiring direction is not limited and the vertical direction may be used. Here, it is expressed as wiring in the horizontal direction. The drain potential control circuit 73 controls all the pixels at the same time, but may be controlled for each row. Therefore, the drain potential control circuit 73 is represented by being connected to both the frame start signal and the vertical register 68.

  The source electrode of the ring-shaped gate MOSFET 63 of the pixel 62 is branched into two via a source electrode wiring 74 (corresponding to 54 in FIG. 3), one of which is supplied to a source potential control circuit 75 that controls the source electrode potential via a switch SW1. The other is connected to the signal readout circuit 76 via the switch SW2. When reading the signal, the switch SW1 is turned off and the switch SW2 is turned on. When the source potential is controlled, the switch SW1 is turned on and the switch SW2 is turned off. Since the signal is output in the vertical direction, the wiring direction of the source electrode is set to be vertical.

  The signal readout circuit 76 is configured as follows. The output of the pixel 62 is performed from the source of the ring-shaped gate MOSFET 63, and a load, for example, a current source 77 is connected to the output line 74. Therefore, it is a source follower circuit. One end of each of the capacitor C1 and the capacitor C2 is connected to the current source 77 via the switch sc1 and the switch sc2. One end of each of the capacitors C1 and C2 whose other ends are grounded is connected to the inverting input terminal and the non-inverting input terminal of the differential amplifier 78, and the potential difference between the capacitors C1 and C2 is output from the differential amplifier 78. It is like that.

  Such a signal readout circuit 76 is called a CDS circuit, and various circuits other than the method described here have been proposed, and the circuit is not limited to this circuit. The signal output from the signal readout circuit 76 is output via the output switch swt. The output switches swt in the same column are subjected to switching control by a signal output from the horizontal shift register 79.

Next, a method for driving the CMOS sensor shown in FIG. 4 will be described with reference to the timing chart of FIG. First, in the period shown in FIG. 5A, light is incident on the embedded photodiode (50 in FIG. 3A, 64 in FIG. 4, etc.), and an electron / hole pair is generated by the photoelectric conversion effect. Holes are accumulated in the buried p ⁻ -type region 49 of the diode. At this time, the potential of the transfer gate electrode 51 is the same as the drain potential Vdd, and the transfer gate MOSFET 65 is off. These accumulations are performed at the same time as the previous frame read operation is being performed.

  In the subsequent period shown in FIG. 5 (2), when the reading of the previous frame is completed, a new frame start signal is transmitted as shown in FIG. First, all the pixels are performed simultaneously from the photodiode (50 in FIG. 3A, 64 in FIG. 4) to the p-type region (47 in FIG. 3) near the source of the ring-shaped gate electrode (45 in FIG. 3). It is to transfer the hole. Therefore, as shown in FIG. 5B, the transfer gate control signal output from the transfer gate potential control circuit 72 falls from Vdd to Low2, the potential of the transfer gate electrode (41 in FIG. 3) becomes Low2, and the transfer gate MOSFET 65 Turns on.

  At this time, the potential of the ring-shaped gate electrode wiring 69 controlled by the ring-shaped gate potential control circuit 70 changes from Low to Low1 as shown in FIG. 5C, but Low2 is larger than Low1. Low1 may be the same as Low. Most simply, Low1 = Low = 0 (V) is set.

  On the other hand, the source potential of all the pixels including the source potential supplied from the source potential control circuit 75 to the source of the ring-shaped gate MOSFET 63 from the source electrode wiring 74 via the switch SW1 is as shown in FIG. The potential is set to S1. S1> Low1, which keeps the ring-shaped gate MOSFET 63 off and prevents current from flowing. As a result, charges (holes) accumulated in the photodiodes of all the pixels are transferred all at once under the ring-shaped gate electrodes of the corresponding pixels.

  In the region under the ring-shaped gate electrode 45 shown in FIG. 3B, the p-type region 47 near the source has the lowest potential, so the holes accumulated in the photodiode reach the p-type region 47 near the source. Accumulated in. As a result of the accumulation of holes, the potential of the p-type region 47 near the source rises.

Subsequently, in the period shown in FIG. 5 (3), as shown in FIG. 5 (B), the transfer gate electrode becomes Vdd again, and the transfer gate MOSFET 65 is turned off. As a result, in the photodiode (50 in FIG. 3A, 64 in FIG. 4 and the like), electron-hole pairs are generated again by the photoelectric conversion effect, and holes start to accumulate in the buried p ⁻ -type region 49 of the photodiode. This accumulation operation is continued until the next charge transfer.

  On the other hand, since the read operation is performed in units of rows, the potential of the ring-shaped gate electrode is low as shown in FIG. 5C in the period (3) of reading the first row to the (s−1) th row. In this state, a standby state is entered with holes accumulated in the p-type region 47 near the source. The source potential can take various values depending on the value of the signal from the pixel while the signal is read from another row. The ring-shaped gate electrode potential can take various values for each row, but is set to Low in the s-th row, and the ring-shaped gate MOSFET 63 is in an off state.

  In the subsequent period shown in FIGS. 5 (4) to (6), pixel signal readout is performed. This signal readout operation will be described representatively for the pixel 62 in the s-th row and the t-th column. First, in the state where holes are accumulated in the p-type region 47 near the source, the vertical shift register 68 shown in FIG. In the period (4) in which the output signal is at a low level as shown in FIG. 5H, the ring-shaped gate electrode 45 is controlled by the control signal output from the ring-shaped gate potential control circuit 70 to the ring-shaped gate electrode wiring 69. Is increased from Low to Vg1, as shown in FIG.

Here, the potential Vg1 is between the potentials Low, Low1, and Vdd described above.
Low ≦ Low1 ≦ Vg1 ≦ Vdd (where Low <Vdd)
Is an electric potential that holds the inequality. In the period (4), the switch SW1 is turned off as shown in FIG. 5I, the switch SW2 is turned on as shown in FIG. 5J, and the switch sc1 is turned on as shown in FIG. The switch sc2 is turned off as shown in FIG. As a result, the source follower circuit connected to the source of the ring-shaped gate MOSFET 63 works, and the source potential of the ring-shaped gate MOSFET 63 becomes S2 (= Vg1-Vth1) in the period (4) as shown in FIG. Become. Here, Vth1 is a threshold voltage of the ring-shaped gate MOSFET 63 in a state in which there is a hole in the back gate (p-type region 47 near the source). The source potential S2 is stored in the capacitor C1 through the switch sc1 that is turned on.

  In the subsequent period shown in FIG. 5 (5), the potential of the ring-shaped gate electrode 45 is set as shown in FIG. 5 (K) by the control signal output from the ring-shaped gate potential control circuit 70 to the ring-shaped gate electrode wiring 69. At the same time as raising to High1, the switch SW1 is turned on and the switch SW2 is turned off as shown in FIGS. 1I and 1J, and the source potential output from the source potential control circuit 75 is shown in FIG. Raise to Highs as shown. Here, High1 and Highs> Low1.

  The values of the potentials High1 and Highs may be the same or different, but High1 and Highs ≦ Vdd are desirable for simplicity of design. In a simple setting, High1 = Highs = Vdd. Further, it is desirable to set the potential so that the ring-shaped gate MOSFET 63 is turned on and no current flows. As a result, the potential of the p-type region 47 near the source rises, and holes are discharged to the epitaxial layer 42 beyond the barrier of the n-well 43 (reset).

  In the subsequent period shown in FIG. 5 (6), the same signal readout state as in the period (4) is set again. However, unlike the period (4), as shown in FIGS. 5M and 5N, the switch sc1 is turned off and the switch sc2 is turned on. The ring-shaped gate electrode has the same Vg1 as that in the period (4) as shown in FIG. However, in this period (6), holes are discharged to the substrate in the immediately preceding period (5), and no holes exist in the p-type region 47 near the source, so the source potential of the ring-shaped gate MOSFET 63 is as shown in FIG. L), the period (6) is S0 (= Vg1-Vth0). Here, Vth0 is the threshold voltage of the ring-shaped gate MOSFET 63 in a state where there is no hole in the back gate (p-type region 47 near the source).

  The source potential S0 is stored in the capacitor C2 through the switch sc2 that is turned on. The differential amplifier 78 outputs the potential difference between the capacitors C1 and C2. That is, the differential amplifier 78 outputs (Vth0−Vth1). This output value (Vth0-Vth1) is a change in threshold value due to hole charge. Thereafter, among the pulses shown in FIG. 5F output from the horizontal shift register 79, the output switch swt in FIG. 4 is turned on based on the output pulse in the t-th column shown in FIG. As schematically shown by hatching in FIG. 5 (P), the threshold change due to the Hall charge from the differential amplifier 78 is output outside the sensor as the output signal Vout of the pixel 62 during the ON period.

  Subsequently, in the period indicated by (7) in FIG. 5, the potential of the ring-shaped gate electrode 45 is set to low again as shown in FIG. 5B, and all the p-type regions 47 near the source have no holes. It waits until the signal processing of the next row is completed (until the readout of the pixels of the s + 1 row to the nth row is completed). During these readout periods, the photodiode 64 is accumulating holes due to the photoelectric conversion effect. Thereafter, the process returns to the period (1) and repeats from the hole transfer. Thereby, the output signal shown in FIG. 5G is read from each pixel.

  3A and 3B, in the solid-state imaging device, the ring-shaped gate MOSFET 63 having the ring-shaped gate electrode 45 is an amplification MOSFET. As shown in FIG. It is a kind of CMOS sensor in the sense that it has an amplifying MOSFET. In this CMOS sensor, the charge (hole) accumulated in the photodiode is transferred to the p-type region 47 in the vicinity of the source under the ring-shaped gate electrode of the corresponding pixel at the same time. Is realized.

  Note that the potential supply of the source electrode wiring 74 at the time of resetting in the period (5) of FIG. That is, in the period (5), both the switches SW1 and SW2 are turned off, and the source electrode wiring 74 is floated. Here, when the potential of the ring-shaped gate electrode wiring 69 is High1, the ring-shaped gate MOSFET 63 is turned on, current is supplied from the drain to the source electrode, and the source electrode potential rises. As a result, the potential of the p-type region 47 in the vicinity of the source is raised, and holes are discharged to the p-type epitaxial layer 42 beyond the barrier of the n-well 43 (reset). The source electrode potential when the holes are completely discharged becomes High1-Vth0. This method can reduce the number of transistors that supply Highs in the source potential control circuit 75, and as a result, the chip area can be reduced.

  In the global type CMOS sensor 112 in the present embodiment, the exposure from the subject is performed in the same one frame period without the timing being shifted for each line. This corresponds to the period (1) in FIG. After a certain period of exposure, the charges of all the pixels are simultaneously transferred to a predetermined region of each pixel (the back gate of the ring-shaped gate MOSFET 63 in FIG. 4) by the transfer gate (transfer gate MOSFET 65 in FIG. 4) in the global CMOS sensor 112. It is transferred to the source vicinity p-type region 47)) of FIG. This corresponds to the period (2) in FIG. Thereafter, signals from each pixel are sequentially read out by the readout circuit within the readout period. This corresponds to the periods (3) to (7) in FIG.

  As a result, as shown in FIG. 6A, even when a subject 150 moving from left to right is imaged on the screen, for example, the captured image is an image exposed at the same time. A captured image as indicated by 160 is obtained, and image distortion different from the image of the subject 150 does not occur.

  Next, the photographing method, the captured image, the driving support operation, and the like according to the present embodiment will be described in more detail. In the driving support apparatus of the embodiment shown in FIG. 1, as shown in FIG. 7, the camera 110 is fixed at a predetermined position such as the roof of the automobile 100, and images a subject at the front position in the running direction of the automobile 100. . Here, for example, a case will be described in which a landmark, a landscape, and the like of a designated point through which the automobile 100 passes after a few minutes are magnified and the photographing information is used for driving operation immediately before the automobile 100 passes through the designated point.

  As shown in FIG. 8, a confirmation point 1000 m before the intersection 104, which is a designated point (Z point) where the automobile 100 equipped with the driving support device of the present embodiment will pass, for example, after a few minutes, That is, when the vehicle has traveled to the check point (X point), the camera 110 zooms in on the actual scene 108 at the intersection 104, and the video signal obtained at that time is recorded and reproduced in the recorder 120 of FIG. In addition to recording on the recording medium by the circuit 122, an image is displayed on the screen frame upper part 133 a of the display screen 132 of the display 130.

  As a result, as shown in FIG. 8 (A), the screen frame upper part 133a has an image obtained by taking a magnified image of the intersection 104 from the X point to the Z point that is currently traveling, that is, the intersection 104 is rapidly moved from the left to the right. The image 105a of the moving vehicle 105, the image 106a of the traffic light 106 fixedly installed on the front left side of the intersection 104, and the image 107a of the post office 107 on the right side of the intersection 104 are displayed in three dimensions.

  Here, in this embodiment, since the global type CMOS sensor 112 is used, image distortion as shown in FIG. 14B when the image is taken using the conventional rolling type CMOS sensor does not occur. The image 105a of the vehicle 105 moving at high speed is also displayed stationary, and similarly, the signal image 106a and the image 107a of the post office 107 are also clearly displayed without distortion. Further, in this embodiment, since a mechanical shutter is not used, complicated control when a mechanical shutter is used is unnecessary, the entire apparatus is not enlarged, and power consumption can be further reduced. In particular, in the case of a one million pixel video camera, the power consumption is 350 mW for the CCD sensor of 831 mW, which is 350 mW, which can be reduced to about one third.

  At this time, the screen frame lower part 133b of the display screen 132 in FIG. 1 is an image of a top view of a road map from the point X to the point Z where the automobile 100 is currently traveling, as shown in FIG. 8A. Is displayed based on the map information from the map device 141 together with the image indicating the current position of the car. Thus, the driver can confirm the positional relationship between the X point and the Z point that are currently traveling. As a result, the driver can visually confirm the situation at the Z point based on the three-dimensional landscape image at the Z point of the screen frame upper part 133a and the positional relationship information of the screen frame lower part 133b. Therefore, the driver can drive with peace of mind and perform a judgment operation with a margin.

  When the automobile 100 further travels and approaches the intersection 104 and arrives at a preset position, for example, a point 100m before the intersection 104 (Y point), the signal of FIG. The video signal of the intersection 104 zoomed at the X point recorded on the recording medium by the recording / reproducing circuit 122 is reproduced and supplied to the display 130. As shown in FIG. The captured image of the intersection 104 that is zoomed at the same point X as the image displayed at the point is displayed again.

  At the same time, as shown in FIG. 8B, the lower part of the screen frame 133b shows a map of the road map from the X point to the Z point, which is the check point, based on the map information supplied from the map device 141. An image of the top view and an image 145 showing the Y point where the automobile 100 is currently traveling are displayed. The image 145 of the Y point is displayed with a flash “moment” on the map, for example, to alert the driver.

  As a result, the driver can compare the three-dimensional landscape image of the Z point visually recognized in advance displayed on the upper part 133a of the screen frame at the Y point close to the designated point (Z point) while comparing with the actual landscape 108. The vehicle 100 can enter the intersection 104, which is the Z point, by immediately reflecting driving information in the driving operation, and from the image displayed on the lower part 133b of the screen frame, the Y point and the Z point currently running By grasping the positional relationship, the driver can safely drive in compliance with the traffic rules and can make a judgment operation with a margin. In addition, it is possible to prevent mistakes in route selection, oversight of road signs, unexpected encounters, and the like.

Note that the global CMOS sensor 112 used in the camera 110 of the present embodiment, as described with reference to FIGS. 3 to 5, photoelectrically converts the p-type region 47 near the source of the ring-shaped gate MOSFET having a large storage capacity. Charges accumulated in the p ⁻ -type region 49 can be collectively transferred. Therefore, since a wider dynamic range can be realized compared with the conventional rolling type CMOS sensor, traffic signs can be recognized more clearly than before, for example, at night, in bad weather, and at points before and after a tunnel with a large difference in brightness. it can.

It is a block diagram of one embodiment of a driving support device using an in-vehicle camera device of the present invention. It is a block diagram of one Embodiment of the display process part in FIG. FIG. 2 is a plan view of one pixel of a global shutter type CMOS area sensor used in the present invention and a cross-sectional view taken along line X-X ′. 1 is an equivalent circuit diagram showing an overall configuration of a global shutter type CMOS area sensor used in the present invention. 5 is a timing chart for explaining the operation of the equivalent circuit of FIG. 4. It is a figure which shows an example of the to-be-photographed object image by the global shutter type | mold CMOS area sensor used by this invention, and its picked-up image. It is explanatory drawing of the driving assistance device of this invention mounted in the motor vehicle. It is explanatory drawing which shows the distance image | photographing and display image in the check point of the driving assistance apparatus of this invention, redisplay of the image | photographed image in the position before a designated point, and comparison with a real landscape. It is a block diagram of an example of the conventional vehicle-mounted camera apparatus. It is a block diagram of an example of the conventional driving assistance device. It is an equivalent circuit diagram of an example of a rolling shutter type CMOS sensor. 12 is a timing chart for explaining the operation of FIG. 11. It is a figure which shows an example of the reading method by a rolling shutter type | mold CMOS sensor, a to-be-photographed object image, and its picked-up image. It is a figure which shows an example of a to-be-photographed object and a picked-up image by a rolling shutter type CMOS sensor.

Explanation of symbols

43 n well 45 ring-shaped gate electrode 46 n ⁺ type source region 47 near source p type region 48 n ⁺ type drain region 49 buried p ⁻ type region 50, 64 photodiode 51 transfer gate electrode 52, 66 drain electrode wiring 53, 69 Ring-shaped gate electrode wiring 54, 74 Source electrode wiring (output line)
55, 71 Transfer gate electrode wiring 61 Pixel covering area 62 Pixel 63 Ring-shaped gate MOSFET
65 Transfer gate MOSFET
DESCRIPTION OF SYMBOLS 100 Car which mounts the driving assistance device of this invention 110 Camera 111 Lens 112 Global shutter type CMOS area sensor 120 Recorder 121 Signal processing circuit 122 Signal recording / reproducing circuit 130 Display 131 Display processing part 132 Display screen 133a Screen frame upper part 133b Screen frame lower part 140 Map Navigation 141 Map Device 1311 Input Processing Unit 1312 Memory 1313 Buffer

Claims (1)

A driving support device using an in-vehicle camera device that is mounted on an automobile, images a subject outside the automobile of the automobile with a solid-state imaging device, and performs driving assistance using a video signal output from the solid-state imaging device. ,
The hole charge obtained by performing photoelectric conversion by exposing the optical image of the subject to the photoelectric conversion regions of all the pixels so that the start and end timing of exposure is the same for all the pixels, A vehicle equipped with a global shutter type CMOS sensor as the solid-state imaging device that sequentially outputs the hall charge accumulated during the exposure period as the video signal from each pixel, and images a designated point specified in advance while the vehicle is running A camera device;
Recording / reproducing means for performing predetermined signal processing on the video signal output from the global shutter type CMOS sensor and outputting the video signal, and recording and reproducing the video signal subjected to the signal processing on a recording medium; ,
Map information generating means for generating road map information between at least a travel point where the vehicle is currently traveling and the designated point together with image information of the travel point and the designated point;
The image of the video signal from the recording / reproducing means is displayed in a first display area at a position in the vicinity before reaching the designated point, and the road map information generated by the map information generating means and the traveling Display means for displaying each image information of the point and the designated point in the second display area;
Providing driving assistance with images displayed in the first and second display areas of the display means,
The global shutter CMOS sensor is
A first conductivity type well formed on a semiconductor substrate; and a second conductivity type buried portion formed in a second region different from a predetermined first region in the well and connected to the well. A photodiode for photoelectrically converting the optical image to accumulate hole charges ;
A ring-shaped gate electrode formed on the first region via a gate oxide film, and a first conductivity type first formed in a region in the well corresponding to a central opening of the ring-shaped gate electrode The first source is formed so as to be embedded in the well so as not to reach the outer periphery of the ring-shaped gate electrode and around the first source part and so as not to contact the gate oxide film. A source vicinity region portion of a second conductivity type that accumulates the hole charge transferred from the photodiode connected to the portion, and a third region different from the first region in the well, the first source portion and and a first conductivity type first drain portion of which is formed spaced apart on the vicinity of the source region portion, a ring-shaped Getoto for outputting the accumulated holes charges in the vicinity of the source region portion as the image pickup signal And Njisuta,
A photodiode having a transfer gate electrode formed on the first region so as to cover a part of the ring-shaped gate electrode; and controlling a potential difference between the transfer gate electrode and the ring-shaped gate electrode to control the photodiode. A transfer gate transistor that transfers the hole charges accumulated in the pixel to the ring gate transistor all at once,
For each pixel,
The well is continuously present in a region immediately below the gate oxide film from the transfer gate electrode to the ring-shaped gate electrode and from the photodiode to the source vicinity region. And
The hole charges accumulated in the buried portion have the potential of the ring-shaped gate electrode and the potential of the transfer gate electrode so that a potential difference serving as a barrier does not occur in the well between the ring-shaped gate electrode and the transfer gate electrode. By controlling each potential, all pixels are transferred to the source vicinity region portion only through the well,
The region near the source after the hole charge is output as the imaging signal is reset with the potential of the ring-shaped gate electrode and the potential of the transfer gate electrode being equal to or lower than the potential of the first drain portion,
Before Symbol accumulation of hole charge in the photodiode is continuously performed even during the output period of the image signal until the transfer of the next hole charge is initiated, that the imaging signals are output consecutively for each frame A driving support device using an in-vehicle camera device.

Images