JP2014116741A - Imaging apparatus and imaging system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus that easily has high sensitivity for visible light and near infrared radiation without a process of eliminating visible light noise for a near infrared image.SOLUTION: The imaging apparatus comprises: plural first pixels (101) that convert visible rays of light into electric signals; plural second pixels (101) that convert near infrared radiation into electric signals; and a scanning circuit (102) that causes outputting, in a first frame, signals including the electric signals from the first pixels and causes outputting signals including only the electric signals from the second pixels.

Description

  The present invention relates to an imaging apparatus and an imaging system.

  It is known that near-infrared rays of 800 nm to 1000 nm are permeable to living bodies. A technique for visualizing a drug by injecting a drug that emits near-infrared fluorescence and emits near-infrared fluorescence into the body and observing the fluorescence from outside the body is attracting attention. A monochrome imaging device having sensitivity in the near-infrared region can capture fluorescence from within the body. In addition, it is required to simultaneously output color information and simultaneously monitor a visible light image and a near-infrared image. Although a method of photographing using each of the visible light imaging device and the near-infrared imaging device and superimposing the images can be considered, there is a difficulty in miniaturization and cost reduction. There is a demand for obtaining a visible light image and a near-infrared image with a single imaging device.

  Patent Document 1 discloses a solid-state imaging device in which a filter that transmits visible light and a filter that transmits near-infrared light are arranged on pixels of the imaging device, and the sensitivity of visible light and near-infrared light is obtained by a single imaging device. Has been. In Patent Document 2, a filter having transparency in a specific visible light region and near infrared region is arranged on a pixel of the imaging device, and visible light and near infrared light are received by the same pixel, and each frame is received. A technique for switching between visible light output and near infrared light output is disclosed. In addition, a technique has been disclosed in which near infrared rays are output by adding signals for four pixels.

JP 2008-76084 A JP 2010-35168 A

  Near-infrared rays have a lower conversion efficiency inside the silicon than visible rays, so that the near-infrared rays output is generally smaller than the visible ray output. Therefore, when higher sensitivity is required for near-infrared wavelengths, there is a possibility that sufficient sensitivity cannot be obtained in the arrangement of the filter disclosed in Patent Document 1.

  In order to solve the above problem, Patent Document 2 discloses a technique of adding signals for four pixels to near-infrared output. However, since visible light and near infrared light are received by the same pixel, visible light noise is mixed into the near infrared image, and in order to obtain a low noise near infrared image, visible light noise is separately removed. It is necessary to perform processing.

  An object of the present invention is to provide an imaging apparatus and an imaging system that can easily have high sensitivity to visible light and near infrared light without performing processing for removing visible light noise on the near infrared image.

  The imaging apparatus according to the present invention includes a plurality of first pixels that convert visible light into an electrical signal, a plurality of second pixels that convert near-infrared light into an electrical signal, and a plurality of the first pixels in a first frame. A scanning circuit that outputs a signal including an electrical signal of the pixel and outputs a signal including only the electrical signals of the plurality of second pixels in the second frame.

  An imaging apparatus and an imaging system having high sensitivity to visible light and near infrared light can be easily provided without performing processing for removing visible light noise on the near infrared image.

It is a figure which shows the structural example of a solid-state imaging device. It is a figure which shows the pixel arrangement pattern in 1st Embodiment. It is a figure which shows the equivalent circuit of the pixel part in 1st Embodiment. It is a figure which shows the read-out sequence in 1st Embodiment. It is a figure which shows the read-out timing in 1st Embodiment. It is a figure which shows the image of the output image in 1st Embodiment. It is a figure which shows the pixel arrangement pattern in 2nd Embodiment. It is a figure which shows the equivalent circuit of the pixel part in 2nd Embodiment. It is a figure which shows the read-out timing in 2nd Embodiment. It is a figure which shows the pixel arrangement pattern in 3rd Embodiment. It is a figure which shows the image of the output image in 3rd Embodiment. It is a figure which shows the pixel arrangement pattern in 3rd Embodiment. It is a figure which shows the image of the output image in 3rd Embodiment. It is a figure which shows the pixel arrangement pattern in 4th Embodiment. It is a figure which shows the image of the output image in 4th Embodiment. It is a block diagram which shows the structural example of an imaging system.

(First embodiment)
FIG. 1 is a block diagram illustrating a configuration example of a solid-state imaging apparatus according to the first embodiment of the present invention. The solid-state imaging device includes a pixel unit 101, a vertical scanning circuit 102, a reading circuit 103, an output amplifier 105, and a horizontal scanning circuit 104. The pixel unit 101 includes a plurality of pixels arranged in a two-dimensional matrix that accumulates charges generated according to incident light and outputs a signal. The vertical scanning circuit 102 is pixel control means for reading out pixel signals from the pixel unit 101 to the reading circuit 103. The readout circuit 103 processes signals from the pixels of the pixel unit 101. The horizontal scanning circuit 104 controls signal transfer from the reading circuit 103 to the output amplifier 105.

  FIG. 2 is a diagram illustrating an arrangement example of some pixels of the pixel unit 101. The pixel unit 101 has a plurality of pixels arranged in a two-dimensional matrix. In the n-th row, a pixel (hereinafter referred to as an R pixel) in which a color filter (visible light filter) that transmits red (visible light) wavelength is disposed, and a color filter that transmits green (visible light) wavelength ( Pixels (hereinafter referred to as G pixels) on which visible light filters are arranged are alternately arranged. In the (n + 2) th row, pixels (hereinafter referred to as B pixels) on which color filters (visible light filters) that transmit blue (visible light) wavelength are arranged and G pixels are alternately arranged. In the (n + 1) th row and the (n + 3) th row, pixels (hereinafter referred to as IR pixels) on which filters that transmit near-infrared wavelengths are arranged. The R pixel, the G pixel, and the B pixel are visible light pixels (first pixels), and convert visible light into an electrical signal. The IR pixel is a near-infrared pixel (second pixel), and converts the near-infrared ray into an electrical signal. Here, n is an odd number. In this case, the visible light transmission filter pixels are arranged in the odd rows, and the near infrared transmission filter pixels are arranged in the even rows. The arrangement order of the color filters in the odd lines is arbitrary, and here, the reading start line is defined as the first line.

  FIG. 3 is an equivalent circuit diagram for four pixels arranged in the same column of the pixel portion 101 of FIG. The pixel in the nth row includes a light receiving element 301a and a transfer transistor 302a. The pixel in the (n + 1) th row includes a light receiving element 301b and a transfer transistor 302b. The pixel in the (n + 2) th row includes a light receiving element 301c and a transfer transistor 302c. The pixels in the (n + 3) th row include a light receiving element 301d and a transfer transistor 302d. The light receiving elements 301a to 301d are, for example, photodiodes, and generate and accumulate charges by photoelectric conversion. The transfer transistors 302a and 302b transfer the charges accumulated in the light receiving elements 301a and 301b to a floating diffusion (hereinafter referred to as FD) 306a, respectively. Pixels in the n-th and n + 1-th rows adjacent in the vertical direction share one FD 306a. The transfer transistors 302c and 302d transfer the charges accumulated in the light receiving elements 301c and 301d to the FD 306b, respectively. Pixels in the n + 2 and n + 3 rows adjacent in the vertical direction share one FD 306b. The amplification transistors 304a and 304b amplify the voltages of the FDs 306a and 306b, respectively. The row selection transistors 305a and 305b select the output voltages of the amplification transistors 304a and 304b, respectively, and output them to the vertical output line 307. The reset transistors 303a and 303b are provided between the FDs 306a and 306b and the power supply voltage node, respectively, and reset the signals of the FDs 306a and 306b and the light receiving elements 301a to 301d. The transfer transistors 302a to 302d, the reset transistors 303a and 303b, and the row selection transistors 305a and 305b are controlled by the vertical scanning circuit 102 in FIG. The vertical scanning circuit 102 includes a scanning circuit for reading and a scanning circuit for resetting, and controls a reading operation and a resetting operation.

  FIG. 4 is a diagram showing a read sequence in the present embodiment. After time t2, electrical signals from a plurality of visible light pixels (first pixels) are read out and output in the first frame. Next, after time t3, the electrical signal of the IR pixel (second pixel) is read and output in the second frame. Further, resetting the light receiving elements 301a to 301d can be performed after time t4 after reading the signal of the IR pixel, thereby extending the charge accumulation time of the IR pixel with respect to the visible light pixel. Sensitivity can be increased.

  FIG. 5 is a diagram showing signal readout timings of pixels for four rows in the same column shown in FIG. The reset signal RES1 indicates a reset signal for the gates of the reset transistors 303a in the n-th row and the (n + 1) -th row. The transfer signal TX1 indicates a transfer signal of the gate of the transfer transistor 302a in the nth row. The transfer signal TX2 indicates a transfer signal of the gate of the transfer transistor 302b in the (n + 1) th row. The row selection signal SEL1 indicates a row selection signal of the gates of the row selection transistors 305a in the nth row and the (n + 1) th row. The reset signal RES2 indicates a reset signal for the gates of the reset transistors 303b in the (n + 2) th row and the (n + 3) th row. The transfer signal TX3 indicates a transfer signal of the gate of the transfer transistor 302c in the (n + 2) th row. The transfer signal TX4 indicates a transfer signal of the gate of the transfer transistor 302d in the (n + 3) th row. The row selection signal SEL2 indicates a row selection signal for the gates of the row selection transistors 305b in the (n + 2) th row and the (n + 3) th row.

  In FIG. 4, times t1 and t4 indicate the start of the reset operation of the pixels in the first row. Time t2 indicates the start of reading of the first row of visible light pixels. Time t3 indicates the start of reading of the first row of near-infrared pixels. 4 and 5, when the reset signal RES1 becomes a high level, the reset transistors 303a of the n-th and n + 1-th rows are turned on, and when the reset signal RES2 becomes a high level, the reset transistors of the n + 2-th and n + 3-th rows 303b turns on. At time t11, the transfer signals TX1 and TX2 become high level, the transfer transistors 302a and 302b in the n-th and n + 1-th rows are turned on, and the FD 306a and the light receiving elements 301a and 301b in the n-th and n + 1-th rows are reset. Is done. Subsequently, the transfer signals TX3 and TX4 become high level, the transfer transistors 302c and 302d in the n + 2 and n + 3 rows are turned on, and the FD 306b and the light receiving elements 301c and 301d in the n + 2 and n + 3 rows are reset.

  Next, at time t21, the reset signal RES1 becomes low level, and the reset transistors 303a in the n-th row and the (n + 1) -th row are turned off. At the same time, the row selection signal SEL1 goes high and the row selection transistors 305a in the nth and n + 1th rows are turned on, so that the pixels in the nth and n + 1th rows become active. Next, when the transfer signal TX1 becomes high level, the transfer transistor 302a in the n-th row is turned on, and the signal of the light-receiving element 301a in the n-th row is read out to the reading circuit 103 via the vertical output line 307. During this period, the transfer signal TX2 is fixed at the low level, the transfer transistor 302b in the (n + 1) th row is turned off, and the signal of the light receiving element 301b in the (n + 1) th row is not read out. Thereafter, the reset signal RES1 becomes a high level, and the reset transistors 303a in the n-th row and the (n + 1) -th row are turned on. Subsequently, the reset signal RES2 becomes low level, and the reset transistors 303b in the (n + 2) th row and the (n + 3) th row are turned off. At the same time, the row selection signal SEL2 becomes a high level and the row selection transistors 305b in the n + 2 and n + 3 rows are turned on, so that the pixels in the n + 2 row and the n + 3 row become active. Next, when the transfer signal TX3 becomes high level, the transfer transistor 302c in the (n + 2) th row is turned on, and the signal of the light receiving element 301c in the (n + 2) th row is read out to the reading circuit 103 via the vertical output line 307. During this period, the transfer signal TX4 is fixed at the low level, the transfer transistor 302d in the (n + 3) th row is turned off, and the signal of the light receiving element 301d in the (n + 3) th row is not read out. As described above, the signals of the visible light pixels in the nth and n + 2th rows of the odd-numbered rows are read out by the transfer signals TX1 and TX3.

  Next, at time t31, the reset signal RES1 becomes low level, and the reset transistors 303a in the n-th row and the (n + 1) -th row are turned off. At the same time, the row selection signal SEL1 goes high and the row selection transistors 305a in the nth and n + 1th rows are turned on, so that the pixels in the nth and n + 1th rows become active. Next, when the transfer signal TX2 becomes high level, the transfer transistor 302b in the (n + 1) th row is turned on, and the signal of the light receiving element 301b in the (n + 1) th row is read out to the reading circuit 103 via the vertical output line 307. During this period, the transfer signal TX1 is fixed at a low level, the transfer transistor 302a in the nth row is turned off, and the signal of the light receiving element 301a in the nth row is not read out. Subsequently, the reset signal RES2 becomes low level, and the reset transistors 303b in the (n + 2) th row and the (n + 3) th row are turned off. At the same time, the row selection signal SEL2 becomes a high level and the row selection transistors 305b in the n + 2 and n + 3 rows are turned on, so that the pixels in the n + 2 row and the n + 3 row become active. Next, when the transfer signal TX4 becomes high level, the transfer transistor 302d in the (n + 3) th row is turned on, and the signal of the light receiving element 301d in the (n + 3) th row is read out to the reading circuit 103 via the vertical output line 307. During this period, the transfer signal TX3 is fixed at the low level, the transfer transistor 302c in the (n + 2) th row is turned off, and the signal of the light receiving element 301c in the (n + 2) th row is not read out. As described above, the signals of the near-infrared pixels in the n + 1th and n + 3th rows of the even-numbered rows are read out by the transfer signals TX2 and TX4. After time t41, the reset operation of the pixels in all rows is performed again in the same manner as time t11 and after, and the above operation is repeated.

  FIGS. 6A and 6B are diagrams showing a visible light image frame and a near-infrared image frame read out by the above operation. The solid-state imaging device outputs a visible light pixel signal in the first frame of FIG. 6A and a near-infrared pixel signal in the second frame of FIG. With the arrangement pattern of visible light pixels and near-infrared pixels shown in FIG. 2, the visible light image of FIG. 6 (a) and the near-infrared image of FIG. 6 (b) can be output separately with an easy driving sequence. In the imaging apparatus, the visible light image in FIG. 6A and the near-infrared image in FIG. 6B are output to separate frames. Therefore, for example, by alternately outputting the first frame and the second frame, a visible light image and a near-infrared image are simultaneously displayed as a moving image by relatively simple image processing without requiring an external memory or the like. It becomes possible. In addition, in the case where it is desired to display an image with priority on sensitivity at a reduced resolution, the range of 4 pixels surrounded by the broken lines in FIGS. 6A and 6B is configured as one pixel. In that case, in the visible light pixel of FIG. 6A, a pixel output is constituted by the color information of each pixel in the broken line. In the near-infrared pixel of FIG. 6B, the sensitivity can be improved by adding signals for four broken lines. The addition here may be performed within the imaging apparatus or may be performed outside the imaging apparatus. That is, in the second frame in FIG. 6B, the vertical scanning circuit 102 and the horizontal scanning circuit 104 add and output the electrical signals of the plurality of IR pixels within the broken line.

  Further, when it is desired to improve the sensitivity of the near-infrared pixel with respect to the visible light pixel, the signal of the near-infrared pixel may be amplified by the reading circuit 103 with a gain and output. In FIG. 2, by arranging visible light pixels and near infrared pixels in units of rows, by setting the readout gain in the first frame or the second frame, respectively, with simple control, The output of the visible light pixel and the output of the near infrared pixel can be adjusted.

(Second Embodiment)
Since the configuration of the imaging apparatus according to the second embodiment of the present invention is the same as that of the first embodiment, differences from the first embodiment will be described. FIG. 7 is a diagram illustrating a pixel arrangement pattern according to the second embodiment of the present invention. In the nth row, R pixels and G pixels are alternately arranged. In the (n + 1) th row, G pixels and B pixels are alternately arranged. In addition, IR pixels are arranged in the (n + 2) th row and the (n + 3) th row. These are repeated in subsequent lines. FIG. 8 shows an equivalent circuit of some pixels of the pixel unit 101 according to the present embodiment. FIG. 8 shows an equivalent circuit diagram of four pixels arranged in the same column in the pixel arrangement of FIG. The pixel in the nth row includes a light receiving element 801a, a transfer transistor 802a, a reset transistor 803a, an amplification transistor 804a, a row selection transistor 805a, and an FD 806a. The pixel in the (n + 1) th row includes a light receiving element 801b, a transfer transistor 802b, a reset transistor 803b, an amplification transistor 804b, a row selection transistor 805b, and an FD 806b. The pixel in the (n + 2) th row includes a light receiving element 801c, a transfer transistor 802c, a reset transistor 803c, an amplification transistor 804c, a row selection transistor 805c, and an FD 806c. The pixel in the (n + 3) th row includes a light receiving element 801d, a transfer transistor 802d, a reset transistor 803d, an amplification transistor 804d, a row selection transistor 805d, and an FD 806d. Row select transistors 805a and 805c are connected to the vertical output line 807a. Row select transistors 805b and 805d are connected to the vertical output line 807b. This embodiment has two vertical output lines 807a and 807b for the same column. Accordingly, in the present embodiment, signals for two rows can be simultaneously read out to the vertical output lines 807a and 807b, so that the reading speed is improved as compared with the first embodiment.

  The transfer transistors 802a to 802d transfer the charges generated in the light receiving elements 801a to 801d to the FDs 806a to 806d, respectively. The amplification transistors 804a to 804d amplify the voltages of the FDs 806a to 806d, respectively. The row selection transistors 805a and 805c select and output the outputs of the amplification transistors 804a and 804c to the vertical output line 807a, respectively. The row selection transistors 805b and 805d select the outputs of the amplification transistors 804b and 804d and output them to the vertical output line 807b, respectively. The reset transistors 803a to 803d are connected between the FDs 806a to 806d and the power supply voltage node, respectively, and control reset of the signals of the FDs 806a to 806d and the light receiving elements 801a to 801d. The transfer transistors 802a to 802d, the reset transistors 803a to 803d, and the row selection transistors 805a to 805d are controlled by the vertical scanning circuit 102.

  The read sequence of this embodiment is the same as that of the first embodiment. FIG. 9 is a read timing chart of the present embodiment corresponding to the read sequence of FIG. 4, and shows the read timing of signals for four rows in the same column. The reset signal RES1 indicates a reset signal of the gates of the reset transistors 803a and 803b in the nth row and the (n + 1) th row. The transfer signal TX1 indicates a transfer signal of the gates of the transfer transistors 802a and 802b in the nth row and the (n + 1) th row. The row selection signal SEL1 indicates a row selection signal of the gates of the row selection transistors 805a and 805b in the nth row and the (n + 1) th row. The reset signal RES2 indicates a reset signal of the gates of the reset transistors 803c and 803d in the n + 2 row and the n + 3 row. The transfer signal TX2 indicates a transfer signal of the gates of the transfer transistors 802c and 802d in the (n + 2) th row and the (n + 3) th row. The row selection signal SEL2 indicates a selection signal for the gates of the row selection transistors 805c and 805d of the (n + 2) th row and the (n + 3) th row.

  When the reset signal RES1 becomes high level, the reset transistors 803a and 803b in the n-th and n + 1-th rows are turned on. When the reset signal RES2 becomes high level, the reset transistors 803c and 803d in the n + 2-th and n + 3-th rows are turned on. To do. At time t11, the transfer signal TX1 becomes high level, the transfer transistors 802a and 802b in the nth and n + 1th rows are turned on, and the FDs 806a and 806b and the light receiving elements 801a and 801b in the nth and n + 1th rows are reset. Is done. Subsequently, when the transfer signal TX2 becomes high level, the transfer transistors 802c and 802d in the n + 2 and n + 3 rows are turned on, and the FDs 806c and 806d and the light receiving elements 801c and 801d in the n + 2 and n + 3 rows are reset. Is done.

  Next, at time t21, the reset signal RES1 becomes low level, and the reset transistors 803a and 803b are turned off. At the same time, the row selection signal SEL1 becomes a high level and the row selection transistors 805a and 805b are turned on, so that the pixels in the n-th and n + 1-th rows become active. Next, when the transfer signal TX1 becomes high level, the transfer transistors 802a and 802b in the n-th and n + 1-th rows are turned on. Then, the signals of the light receiving elements 801a and 801b in the nth row and the (n + 1) th row are read out to the reading circuit 103 via the vertical output lines 807a and 807b, respectively.

  Subsequently, at time t31, the reset signal RES2 becomes low level, and the reset transistors 803c and 803d are turned off. At the same time, the row selection signal SEL2 becomes high level and the row selection transistors 805c and 805d are turned on, so that the pixels in the (n + 2) th row and the (n + 3) th row become active. Next, when the transfer signal TX2 becomes high level, the transfer transistors 802c and 802d in the (n + 2) th row and the (n + 3) th row are turned on. Then, the signals of the light receiving elements 801c and 801d in the (n + 2) th row and the (n + 3) th row are read out to the reading circuit 103 via the vertical signal lines 807a and 807b, respectively. After time t41, the reset operation for all rows is performed again, and the above operation is repeated, similarly to the time after time t11.

  With the above operation, the read visible light image frame and the near-infrared image frame are as shown in FIGS. 6A and 6B as in the first embodiment. The solid-state imaging device outputs a visible light pixel signal in the first frame of FIG. 6A and a near-infrared pixel signal in the second frame of FIG. With the arrangement pattern of visible light pixels and near-infrared pixels shown in FIG. 7, a visible light image and a near-infrared image can be output separately with an easy driving sequence. The imaging apparatus outputs the visible light image of FIG. 6A and the near-infrared image of FIG. 6B to separate frames. Therefore, for example, by outputting the first frame and the second frame alternately, a visible light image and a near-infrared image are simultaneously displayed as a moving image with relatively simple image processing without the need for an external memory or the like. It becomes possible to do. Also. In the case where it is desired to display an image with priority given to sensitivity with reduced resolution, one pixel is constituted by a range of four pixels surrounded by a wavy line in FIGS. 6 (a) and 6 (b). In that case, in the visible light pixel, the pixel output is constituted by the color information of each pixel in the broken line range, and in the near-infrared pixel, the signal for the four pixels in the broken line range is added to obtain the sensitivity. Can be improved.

  In the present embodiment, the pixel arrangement shown in FIG. 7 is operated with the driving pattern shown in FIG. 9, thereby improving the reading speed and increasing the frame rate compared to the first embodiment. On the other hand, it becomes a more suitable structure.

(Third embodiment)
In the third embodiment of the present invention, the same points as in the first embodiment are omitted, and different points will be described. FIG. 10 is a diagram illustrating a pixel arrangement pattern according to the third embodiment of the present invention. In FIG. 10, the G pixel in the row including the R pixel and the G pixel is replaced with an IR pixel. . That is, in the nth row, R pixels and IR pixels are alternately arranged. Each row of the two-dimensional matrix of pixels is either a row containing a plurality of visible light pixels or a row containing only a plurality of IR pixels. FIGS. 11A and 11B show images output in the first frame and the second frame according to the driving sequence and the driving pattern shown in FIGS. The solid-state imaging device outputs visible light pixel and IR pixel signals in the first frame of FIG. 11A, and outputs IR pixel signals in the second frame of FIG. 11B. The vertical scanning circuit 102 outputs an image including electrical signals of a plurality of visible light pixels in the first frame of FIG. 11A, and electrical signals of a plurality of IR pixels in the second frame of FIG. 11B. Output an image that contains only

  The signal of the IR pixel output to the first frame in FIG. 11A can be used as information for correcting the IR component included in the aperture pixel, and is compared with the first embodiment. A visible light image with higher image quality can be obtained. In addition, the IR pixel output in the second frame in FIG. 11B has the same accumulation time as the IR pixel output in the first frame for the purpose of obtaining the incident near-infrared information. There is no need. Further, since the second frame does not have to have the same pixel signal gain as the first frame, an IR image with higher sensitivity can be obtained by increasing the accumulation time or applying gain. be able to. As described above, the present embodiment can provide a lower noise image with respect to the image output in the first frame as compared with the first embodiment.

  Further, the present embodiment may have the pixel arrangement pattern shown in FIG. FIG. 12 shows an example in which replacement is performed so that G pixels remain in all columns when replacing G pixels with IR pixels. The nth to n + 3th rows in FIG. 12 have the same pixel arrangement pattern as the nth to n + 3th rows in FIG. In the (n + 4) th row, R pixels and G pixels are alternately arranged. In the n + 5th row and the n + 7th row, IR pixels are arranged. In the (n + 6) th row, IR pixels and B pixels are alternately arranged. In this case, the output image has the first frame in FIG. 13A and the second frame in FIG. In the first frame of FIG. 11A, since G pixels are arranged in all columns, this G pixel signal can be used as information for performing offset correction between columns, and a more preferable arrangement pattern It has become. Moreover, the arrangement | positioning is not limited to the pattern of FIG. 12 as long as the objective of including G pixel in all the columns is satisfy | filled.

(Fourth embodiment)
In the fourth embodiment of the present invention, the same points as those of the second embodiment are omitted, and different points will be described. FIG. 14 shows a pixel arrangement pattern according to an embodiment of the present invention. FIG. 14 is an arrangement in which the G pixel in the row including the R pixel and the G pixel is replaced with the IR pixel with respect to FIG. That is, in the nth row, R pixels and IR pixels are alternately arranged. According to the drive sequence and the drive pattern shown in FIGS. 4 and 9, the image output in the first frame and the second frame is the first frame in FIG. 11A as in the third embodiment. And the second frame of FIG. The signal of the IR pixel output to the first frame in FIG. 11A can be used as information for correcting the IR component included in the aperture pixel, and is compared with the first embodiment. A visible light image with higher image quality can be obtained. In addition, the IR pixel output in the second frame in FIG. 11B has the same accumulation time as the IR pixel output in the first frame for the purpose of obtaining the incident near-infrared information. There is no need. In addition, the second frame does not have to have the same gain with respect to the pixel signal as the first frame. Therefore, by increasing the accumulation time or applying the gain, the IR is more sensitive. An image can be obtained. As described above, this embodiment can provide a lower noise image than the image output in the first frame, as compared to the second embodiment.

  Further, the present embodiment may be the pixel arrangement pattern shown in FIG. FIG. 15 shows an example in which replacement is performed so that G pixels remain in all columns when replacing G pixels with IR pixels. The nth to n + 3th lines in FIG. 15 are the same as the nth to n + 3th lines in FIG. In the (n + 4) th row, R pixels and G pixels are alternately arranged. In the (n + 5) th row, IR pixels and B pixels are alternately arranged. The n + 6 line and the n + 7 line are the same as the n + 2 line and the n + 3 line. In this case, the output image has a first frame in FIG. 13A and a second frame in FIG. In the images shown in FIGS. 13A and 13B, G pixels are arranged in all columns in the first frame with respect to the images shown in FIGS. 11A and 11B. The pixel signal can be used as information for offset correction between columns, and the arrangement pattern is more suitable. Further, the present embodiment only needs to satisfy the purpose of including G pixels in all columns, and the arrangement is not limited to the pattern of FIG.

(Fifth embodiment)
FIG. 16 is a diagram illustrating an example of an imaging system to which an imaging apparatus according to the fifth embodiment of the present invention is applied. The imaging system 501 includes an optical system 502, the imaging device 100, a signal processing unit 503, and an external device 506. The signal processing unit 503 includes an imaging signal processing circuit 504 and an image signal processing unit 505. In addition, the imaging system 501 may include a memory unit, a timing generator, and the like. The imaging device 100 is the imaging device according to the first to fourth embodiments. The light incident on the optical system 502 forms an image of the subject on the imaging surface (pixel unit 101) of the imaging device 100. The imaging apparatus 100 converts an image of a subject formed on the imaging surface into an image signal, reads the image signal from the pixel unit 101, and outputs the image signal. The imaging signal processing circuit 504 is connected to the imaging device 100 and processes an image signal output from the imaging device 100. The image signal processing unit 505 performs arithmetic processing such as various corrections on the image signal processed by the imaging signal processing circuit 504 to generate image data. This image data is output to an external device (for example, a personal computer) 506. When the signal supplied from the imaging apparatus 100 to the imaging signal processing circuit 504 is an analog signal, the digital signal is output by an A / D converter or the like in the imaging signal processing circuit 504 before being supplied to the image signal processing unit 505. It is desirable to convert to

  The signal processing unit 503 forms a visible light image based on the first frame output from the imaging device 100, and forms a near-infrared image based on the second frame. Further, the signal processing unit 503 may alternately display the visible light image and the near infrared image on the external device (display device) 506 for each frame, or synthesize the visible light image and the near infrared image to the external device (display device). (Display device) 506 may be displayed.

  Since the imaging apparatus 100 outputs visible light image data and near-infrared image data for each frame as described in the first to fourth embodiments, the image signal processing unit 505 has, for example, a visible light component from a near-infrared image. This eliminates the need for a correction system that removes noise. Therefore, a high-sensitivity image can be output for visible light and near infrared light without requiring special correction.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

101 pixel section, 102 vertical scanning circuit, 103 readout circuit, 104 horizontal scanning circuit, 105 output amplifier

Claims (7)

A plurality of first pixels for converting visible light into electrical signals;
A plurality of second pixels that convert near-infrared rays into electrical signals;
A scanning circuit that outputs a signal including electrical signals of the plurality of first pixels in the first frame, and outputs a signal including only the electrical signals of the plurality of second pixels in the second frame. An imaging apparatus characterized by the above. The plurality of first pixels and the plurality of second pixels are arranged in a two-dimensional matrix,
The imaging apparatus according to claim 1, wherein each row of the two-dimensional matrix is either a row including a plurality of the first pixels or a row including only the plurality of the second pixels.   The imaging apparatus according to claim 1, wherein in the second frame, electric signals of the plurality of second pixels are added and output. The imaging device according to any one of claims 1 to 3,
An optical system for forming an image on the imaging device;
An imaging system comprising: a signal processing unit that processes an image output from the imaging device to generate image data.   The imaging system according to claim 4, wherein the signal processing unit forms a visible light image based on the first frame and forms a near-infrared image based on the second frame.   The imaging system according to claim 5, wherein the signal processing unit alternately displays the visible light image and the near-infrared image for each frame.   The imaging system according to claim 5, wherein the signal processing unit displays the visible light image and the near-infrared image by combining them.

Images