JP2017108223A - Imaging apparatus, control method therefor and imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To more easily and accurately correct smear of which the distribution is not uniform.SOLUTION: An imaging apparatus 100 comprises: an imaging device 102 including a pixel array in which a first pixel for receiving light from a subject and a second pixel for detecting smear are arrayed, and an output part for separately outputting a first image signal that is read out of the first pixel and a second image signal that is read out of the second pixel; and correction means 105 which uses a signal value of the second pixel included in the second image signal to correct a signal value of the first pixel corresponding to a position of the second pixel.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus, a control method thereof, and an imaging element.

  2. Description of the Related Art Conventionally, in an image sensor such as a CCD or CMOS image sensor, it is known that a phenomenon that affects image quality such as fixed pattern noise, shading, flicker, smear, etc. occurs in the image sensor. For this reason, some imaging apparatuses generate correction values for each phenomenon using image signals output from the image sensor, and correct the image signals using the generated correction values.

  In a CMOS image sensor, a band-like luminance variation may occur in the horizontal direction of the same line. Japanese Patent Application Laid-Open No. 2004-133620 proposes a technique for obtaining a smear amount using a signal in the HOB area of the image sensor and subtracting it from the signal value in the effective pixel area in order to correct the horizontal smear.

  Also, in Patent Document 2, a light-shielded dummy pixel is arranged in an effective image, a smear amount is obtained using a signal output from the dummy pixel, and the smear amount is subtracted from the signal value of the effective pixel. Correction techniques have been proposed.

JP 2009-5169 A JP 2011-199426 A

  However, in the technique proposed in Patent Document 1, when the smear component is not uniform in the horizontal direction, the smear amount in the HOB area is different from the smear amount in the effective image, so that the correction is performed accurately according to the occurrence of smear. It may not be possible.

  In the technique proposed in Patent Document 2, signals of all effective pixels and dummy pixels are mixedly output from the image sensor. For this reason, a circuit for processing the output signal needs to perform signal processing by excluding signals from the dummy pixels, and the processing may be complicated.

  The present invention has been made in view of the above-described problems of the prior art, and provides an imaging apparatus, a control method thereof, and an imaging element that can more easily correct smear with nonuniform distribution more easily. The purpose is to do.

  In order to solve this problem, for example, an imaging apparatus of the present invention has the following configuration. That is, the first image signal read out from the first pixel has a first pixel that receives light from the subject and a pixel array in which second pixels for detecting smear are arranged. And an output device that separates and outputs the second image signal read from the second pixel, and the signal value of the second pixel included in the second image signal is used. Correction means for correcting the signal value of the first pixel corresponding to the position of the second pixel.

  According to the present invention, it is possible to more easily correct smear with a non-uniform distribution more easily.

1 is a block diagram illustrating a functional configuration example of a digital camera 100 as an example of an imaging apparatus according to an embodiment of the present invention. FIG. 3 is a diagram illustrating a configuration example of the image sensor 102 according to the first embodiment. FIG. 6 is a diagram for explaining smear detection pixels and effective pixels of the image sensor 102 according to the first embodiment. The figure explaining the smear correction process which concerns on Embodiment 1. FIG. FIG. 3 is a block diagram showing an example functional configuration of a digital camera 100 according to a second embodiment. FIG. 10 is a diagram illustrating a configuration example of an image sensor 102 according to the second embodiment. FIG. 6 is a diagram for explaining an example of a stacked image sensor according to a third embodiment. FIG. 9 is a block diagram illustrating an example of a functional configuration of a smartphone 800 as an example of a mobile phone according to a fourth embodiment.

(Embodiment 1)
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. Hereinafter, an example in which the present invention is applied to an arbitrary digital camera capable of imaging will be described as an example of an imaging apparatus. However, the present invention can be applied not only to a digital camera but also to any device that can capture an image. These devices may include, for example, a game machine, a tablet terminal, a personal computer, a clock-type or glasses-type information terminal, a monitoring device, or a medical device.

(Configuration of digital camera 100)
FIG. 1 is a block diagram illustrating a functional configuration example of a digital camera 100 as an example of an imaging apparatus according to the present embodiment. One or more of the functional blocks shown in FIG. 1 may be realized by hardware such as an ASIC or a programmable logic array (PLA), or may be realized by a programmable processor such as a CPU or MPU executing software. May be. Further, it may be realized by a combination of software and hardware. Therefore, in the following description, even when different functional blocks are described as the operation subject, the same hardware can be realized as the subject.

  The photographing optical system 101 includes a zoom lens, a diaphragm, a focus lens, a shift lens, and the like, and forms an optical image from a subject on an image sensor. The imaging element 102 has a configuration in which a plurality of pixels each having a photoelectric conversion element are two-dimensionally arranged. As will be described later with reference to FIG. 2, the image sensor 102 has effective pixels that receive the subject optical image and smear detection pixels, respectively, and reads from the image signals read from the effective pixels and the smear detection pixels. It is configured such that it can be read out separately from the correction image signal that has been output.

  The smear detection unit 103, the smear correction value generation unit 104, and the smear correction unit 105 include a signal processing circuit or a signal processing module. The smear detection unit 103 detects smear generated using the correction image signal read from the smear detection pixel of the image sensor 102, and generates smear information. The smear information is information indicating the presence / absence and occurrence position of smear. The smear correction value generation unit 104 generates a smear correction value using the correction image signal and the smear information output from the smear detection unit 103. The smear correction unit 105 corrects the image signal by subtracting the smear correction value generated by the smear correction value generation unit 104 from the signal value of the image signal output from the effective pixel of the image sensor 102.

  The signal processing unit 106 includes a signal processing circuit or a signal processing module, and performs predetermined signal processing such as white balance on the image signal output from the smear correction unit 105. The display unit 107 includes a display panel including an LCD, an LED, and the like, and displays an image signal output from the signal processing unit 106, an operation image for operating the digital camera 100, and the like.

  The recording unit 108 includes, for example, a recording medium such as a nonvolatile semiconductor memory and a hard disk, or a recording medium such as an SD card, and records the image signal output from the signal processing unit 106.

  The control unit 109 includes, for example, a CPU or MPU, and controls the entire digital camera 100 by developing and executing a program stored in a non-volatile memory such as a ROM in a work area of a volatile memory such as a RAM. To do. The control unit 109 controls reading of each signal from the image sensor 102.

(Configuration of the image sensor 102)
Next, the detailed configuration of the image sensor 102 will be described with reference to FIG. The unit pixel 201 includes a pixel having the configuration shown in FIG. The floating diffusion amplifier (FD amplifier) 202 detects the accumulated charge amount of the unit pixel 201 as a voltage.

  The row readout control line 203a and the row readout control line 203b are respectively connected to the unit pixels 201 at a predetermined interval (for example, every column), and readout control signals are sent from the vertical scanning circuit 208a or the vertical scanning circuit 208b to each unit pixel. Is transmitted. FIG. 2A shows the row readout control lines 203a-20 to 203a-23 or 203b-20 connected to the unit pixels 201 of the 20th row to the 23rd row of the unit pixels 201 arranged two-dimensionally. ~ 203b-23.

  The row reset control line 204a and the row reset control line 204b are respectively connected to the unit pixels 201 at a predetermined interval, and transmit a reset control signal from the vertical scanning circuit 208a or the vertical scanning circuit 208b to each unit pixel. Row reset control lines connected to, for example, the unit pixels 201 in the 20th to 23rd rows of the unit pixels 201 arranged in a two-dimensional shape are shown as 204a-20 to 204a-23 or 204b-20 to 204b-23. Yes.

  Further, the row selection line 205a and the row selection line 205b are respectively connected to the unit pixels 201 with a predetermined interval, and transmit a row selection control signal to each unit pixel from the vertical scanning circuit 208a or the vertical scanning circuit 208b. Row selection lines connected to, for example, the unit pixels 201 in the 20th to 23rd rows of the unit pixels 201 arranged in a two-dimensional shape are shown as 205a-20 to 205a-23 or 205b-20 to 205b-23. .

  A column signal line 206 provided for each column transmits the output signal of the unit pixel 201 of each column to the column circuit 207. The column circuit 207 converts the pixel signal output from each unit pixel 201 under the control of the horizontal scanning circuit 209 from analog to digital, and outputs it. The digital signal output from the column circuit 207 is input to the pixel signal selection circuit 211 or the pixel signal selection circuit 212 via the horizontal signal line 210, and the signal output terminal 213 or the signal output terminal according to the operation of each pixel signal selection circuit. 214. For example, the pixel signal selection circuit 211, the pixel signal selection circuit 212, the signal output terminal 213, and the signal output terminal 214 constitute a digital signal output unit.

  A timing generator (TG) 215 generates a reset timing signal and a read timing signal according to the setting. The vertical synchronization signal input terminal 216 and the horizontal synchronization signal input terminal 217 are terminals for inputting a synchronization signal from the outside. The setting signal line 218 inputs a setting signal for setting the reset and read timings for the TG 215 from the outside. The read row timing signal 219 output from the TG 215 is input to the vertical scanning circuit 208a and the vertical scanning circuit 208b. Further, the read column timing signal 220 output from the TG 215 is input to the horizontal scanning circuit 209.

  FIG. 2B shows a configuration example of the unit pixel 201 shown in FIG. A photodiode (PD) 301 that is a photoelectric conversion element generates and accumulates charges corresponding to the amount of incident light by photoelectric conversion. The transfer transistor 303 performs an on / off operation in response to the read control signal 302 input from the row read control line 203a or 203b. The floating diffusion (FD) 304 is constituted by a capacitance having a predetermined capacity, and generates a voltage corresponding to the electric charge accumulated in the capacitance. The reset transistor 306 performs an on / off operation in response to a reset control signal 305 input from the row reset read control line 204a or 204b. In the FD 304, the accumulated charge is reset according to the reset voltage supplied from the reset level input unit 307 in accordance with the operation of the reset transistor 306. When the transfer transistor 303 is turned on, the charge generated in the PD 301 is transferred and accumulated. The row selection transistor 309 is turned on / off in response to a row selection control signal 308 input from the row selection line 205 a or the row selection line 205, and a voltage signal corresponding to the amount of charge transferred to the FD 304 and stored is output from the pixel output 310. Output. Here, the FD amplifier 202 in FIG. 2A includes the FD 304 and the row selection transistor 309 in FIG.

(Overview of smear correction processing)
FIG. 3 schematically shows a pixel array of the image sensor 102 including smear detection pixels and effective pixels. Of the plurality of pixels arranged in a two-dimensional manner, for example, effective pixels and smear detection pixels are alternately arranged in the horizontal direction (row direction). A column 4a indicates a column in which effective pixels are arranged, and a column 4b indicates a column in which smear detection pixels are arranged. A plurality of pixels arranged in the VOB area and the HOB area are shielded from light so that light from the subject does not enter.

  FIG. 3 schematically shows a state in which horizontal smear in the row direction occurs when a high-brightness subject is photographed, and a pixel with a brighter color (that is, white) outputs a signal with higher luminance. It shows the situation. In a row indicated as a smear region, a smear in the horizontal direction (row direction) occurs with respect to a high-brightness object, and a signal with a high luminance is output as compared with pixels in other regions in the vertical direction (column direction). Further, the luminance level is not uniform in the horizontal direction, and the luminance level is different for each column.

  Next, smear detection and smear correction processing will be described with reference to FIGS. FIG. 4 (a) shows an image signal generated by extracting a pixel column constituting the effective image 4a, and FIG. 4 (b) is generated by extracting a pixel column constituting the smear detection pixel 4b. Each correction image signal is shown. FIG. 4C shows an image after the smear correction is performed by applying the correction value obtained based on the smear detection pixel to the signal of the effective pixel.

  FIG. 4D is a graph showing the output signal value of the pixel in the smear region of FIG. 4A, and the pixel included in the range A outputs the highest signal value. Further, the output of the pixel changes (decreases) as it moves away from the range A, and the pixels in the HOB area output a signal value of a predetermined level associated with the dark current.

  FIG. 4E is a graph showing the output signal value of the smear detection pixel. Like the pixels in the HOB area, the smear detection pixels are shielded so as not to receive light from the subject, and smear components generated in each pixel can be detected. In the example shown in FIG. 4E, it is shown that a smear component that is high in the range A and decreases as it leaves this range is detected. That is, it indicates that a smear that is not uniform in the horizontal direction (row direction) occurs with respect to the high-luminance subject.

  FIG. 4 (f) shows the result of subtracting the smear component shown in FIG. 4 (e) from the output signal value of the effective pixel shown in FIG. 4 (d). More specifically, it is possible to appropriately correct the smear component generated in the effective pixel adjacent to the smear detection pixel by subtracting the output signal value of the adjacent smear detection pixel from the signal value of the effective pixel. Show.

(Specific operation related to smear correction processing)
Next, a specific operation related to the smear correction process of this embodiment will be described. For example, when there is a shooting instruction from the user via an input unit (not shown) of the digital camera 100, this processing is started together with the start of the shooting operation. This is realized by the control unit 109 developing and executing a program stored in a ROM (not shown) in a work area of a RAM (not shown).

  The control unit 109 sets a timing for resetting each pixel group and a timing for reading a signal from each pixel group for each field or frame for the TG 215 in the image sensor 102. When an optical image of a subject is formed on the light receiving surface of the image sensor 102 through the photographing optical system 101, each unit pixel is displayed on a PD (photodiode) 301 in each unit pixel 201 on the image sensor 102. Photoelectric charges corresponding to the amount of light incident on 201 are generated.

  In the FD 304 in each unit pixel 201 of the image sensor 102, a voltage corresponding to the electric charge accumulated in the capacitance having a predetermined capacity is generated. When the reset control signal 305 is input, the reset transistor 306 operates and the charge accumulated in the FD 304 is reset by the power supply voltage VDD input from the reset level input unit 307.

  On the other hand, the PD 301 stores photocharges according to the amount of light incident on each unit pixel 201. In this state, when the transfer transistor 303 is turned on due to the level change of the read control signal 302, the photocharge accumulated in the PD 301 moves to the FD 304. At this time, in the FD 304, the voltage level changes by the amount of photocharge that has moved with respect to the reset level. The change in the voltage level becomes the signal level of the pixel. When the row selection transistor 309 is turned on by the change in the level of the row selection control signal 308, a voltage signal corresponding to the charge amount of the FD 304 is output. The output analog pixel signal is transmitted to the column circuit 207 via the column signal line 206. The column circuit 207 amplifies the analog pixel signal and converts it into a digital signal.

  Digital signals converted by the column circuit 207 of each column are sequentially read out by the horizontal scanning circuit 209 and input to the pixel signal selection circuit 211 and the pixel signal selection circuit 212 through the horizontal signal line 210. The pixel signal selection circuit 211 selects an image signal output from the effective pixel 4 a according to an instruction from the control unit 109, while the pixel signal selection circuit 212 is output from the smear detection pixel 4 b according to an instruction from the control unit 109. Select the image signal for correction. Thereafter, the image signal of the effective pixel 4 a is output separately from the signal output terminal 213 and the correction image signal is output from the signal output terminal 214.

  Next, the smear detection unit 103 detects smear using the correction image signal output from the signal output terminal 214, and generates smear information indicating the presence / absence of the smear and the occurrence position. More specifically, since smear occurs in the horizontal direction (row direction) with respect to the high-luminance subject, the smear detection unit 103 changes the signal amount in the vertical direction (column direction) of the smear detection pixel (that is, the signal amount). Difference). The smear detection unit 103 detects smear when the calculated change amount is equal to or greater than a predetermined threshold. Note that the method of detecting smear in the horizontal direction (row direction) is not limited to a method that uses a change in the signal amount of effective pixels. In such a case, smear may be detected. The smear detection unit 103 outputs smear information indicating the presence / absence of the smear and the occurrence position (for example, coordinate position or line number) to the smear correction value generation unit 104 based on the result of detecting the smear.

  The smear correction value generation unit 104 generates a smear correction value for the image signal using the smear information generated by the smear detection unit 103 and the correction image signal. Specifically, the smear correction value generation unit 104 specifies a smear position from the smear information, and generates a correction value using the signal value of the correction image signal corresponding to the smear position. Note that the smear correction value generation unit 104 calculates the correction value for the effective pixel adjacent to the smear detection pixel, so that the correction image signal is taken into account in the horizontal direction (row direction) between the correction image signals. Interpolation may be performed. That is, when the smear detection pixel column and the effective pixel column are alternately arranged, the correction image signal is interpolated in consideration of the change in the horizontal direction between the correction image signals. A smear correction value suitable for the column can be generated. The smear correction unit 105 corrects smear by subtracting the smear correction value output from the smear correction value generation unit 104 from the image signal output from the signal output terminal 213 of the image sensor 102.

  The signal processing unit 106 acquires the image signal subjected to the smear correction processing from the smear correction unit 105, performs predetermined image processing, and displays the image signal on the display unit 107 according to an instruction from the control unit 109, or causes the recording unit 108 to The image signal is recorded, and a series of operations related to this processing is completed.

  In this embodiment, the example in which the smear detection pixels are alternately arranged with the effective pixels has been described. However, the arrangement of the smear detection pixels may be dynamically changeable. Since the smear detection pixel described in the present embodiment is composed of the unit pixel 201, the effective pixel and the smear detection pixel are moved by eliminating the charge accumulation time (to zero) in order to shield the pixel. Each image signal can be read out by switching between them. In this way, the unit pixel can be used effectively according to the situation, for example, by changing the smear detection density according to the subject.

  As described above, in the present embodiment, the image signal read from the effective pixel is read from the smear detection pixel by using the image sensor 102 in which the effective pixel and the smear detection pixel are alternately arranged for each column. Correction was performed using a correction value obtained from the correction image signal that was output. At this time, the image sensor separates and outputs the image signal read from the effective pixel and the correction image signal read from the smear detection pixel. By doing so, a circuit that receives a signal from the image sensor can easily detect smear, and when the smear spreads in the horizontal direction, correction processing according to the change in the horizontal direction of the smear becomes possible. That is, it is possible to more easily correct smear with a non-uniform distribution more easily.

(Embodiment 2)
Next, Embodiment 2 will be described. In the first embodiment, the smear detection unit 103 provided outside the image sensor 102 is configured to detect smear using the correction image signal output from the image sensor 102. In the present embodiment, the smear detection unit is disposed inside the image sensor and the smear information is output from the image sensor 102. Although the configuration related to the image sensor and the smear detection unit of the present embodiment is different from that of the first embodiment, other configurations are the same as those of the first embodiment. For this reason, the same reference numerals are assigned to the same components, and redundant descriptions are omitted, and differences will be mainly described.

  With reference to FIG. 5, a functional configuration example of the digital camera 100 in which the image sensor 501 includes a smear detection unit 502 will be described.

  The image sensor 501 outputs the image signal read from the effective pixel from the signal output terminal 213 and inputs the correction image signal read from the smear detection pixel to the smear detection unit 502. The image sensor 501 outputs smear information generated by the smear detection unit 502 from the signal output terminal 214. Further, the image sensor 501 cuts out a correction image signal related to the position where smear is detected, and outputs it from the signal output terminal 214 together with smear information.

  The smear detection unit 502 detects smear based on the correction image signal that is arranged in the image sensor 501 and read from the smear detection pixel. Note that the smear detection process in the smear detection unit 502 is the same as the process described in the first embodiment. The control unit 109 controls the entire digital camera 100 and also controls the reading method of the image sensor 102 and the smear detection unit 502.

  Furthermore, the smear correction process according to the present embodiment will be described. For example, when there is a shooting instruction from the user via an input unit (not shown) of the digital camera 100, this processing is started together with the start of the shooting operation. The operation of the unit pixel 201 of the image sensor 102 is the same as that of the first embodiment, and a signal output from each unit pixel 201 is converted into a digital signal by the column circuit 207 via the column signal line 206. The converted digital signal is input to the pixel signal selection circuit 211 and the pixel signal selection circuit 212 as shown in FIG.

  The smear detection unit 502 detects smear using the correction image signal of the smear detection pixel, generates smear information indicating the presence / absence of the smear and the occurrence position based on the detection result, and outputs the smear information from the signal output terminal 214 To do.

  Based on the smear information generated by the smear detection unit 502, the pixel selection circuit 212 acquires only the image signal at the position where smear is detected in the smear detection pixel 4b, and outputs it from the signal output terminal 214 as a correction signal. Let By doing so, the correction image signal in the area where smear does not occur is not constantly output, so that an increase in the amount of output signals accompanying an increase in the number of pixels and an increase in processing burden on the receiving side are prevented. be able to. In other words, the amount of signal can be reduced by limiting the correction image signal output from the image sensor 102 to the position or row where smear occurs.

  Thereafter, the smear correction value generation unit 104 acquires the smear information and the correction image signal output from the signal output terminal 214, and generates a smear correction value for the correction image signal as in the first embodiment. In the subsequent processing, as in the first embodiment, the smear correction unit 105 corrects smear by subtracting the smear correction value from the image signal output from the signal output terminal 213 of the image sensor 102. Then, the control unit 109 displays the corrected image signal on the display unit 107, for example, and ends a series of operations related to this processing.

  In the present embodiment, the image sensor 501 includes the smear detection unit 502 inside and outputs the smear information and the correction image signal in which the smear is generated. However, the smear correction value generation unit 104 and the smear correction unit are further provided. 105 may be provided in the image sensor 501. In this way, smear correction processing can be realized in the image sensor, and the image sensor can output only an effective image on which smear correction has been performed, and the amount of signal to be output can be reduced.

  As described above, in the present embodiment, the image sensor 501 includes the smear detection unit 502 inside, and the image sensor 501 detects smear inside and outputs the presence / absence of the smear and the occurrence position of the smear. Only the image signal for position correction is output. By doing so, it becomes possible to correct smear adaptively and accurately according to the occurrence of smear, and to reduce the amount of signal output from the image sensor.

(Embodiment 3)
Next, Embodiment 3 will be described. In the present embodiment, a case where the image sensor is configured as a stacked type will be described. In the present embodiment, the configuration is the same as that of the first embodiment except that the imaging element is a stacked type. For this reason, the same reference numerals are assigned to the same components, and redundant descriptions are omitted, and differences will be mainly described.

  FIG. 7 shows an image sensor 700 according to this embodiment. In the image sensor 700, an image sensor chip 710 including a photoelectric conversion element such as a unit pixel 201 and a high-speed logic process chip 720 including a configuration for performing digital processing such as a column circuit 207 are stacked at a chip level.

  7A is an oblique projection of the image sensor 700, and FIG. 7B is a top view of the chip 710 and the chip 720. The pixel portion 711 of the image sensor chip 710 includes a configuration for reading out pixel signals of the unit pixel 201, the FD amplifier 202, the TG 215, and the like. The high-speed logic process chip 720 includes a configuration capable of processing including digital data, such as the column circuit 207, the horizontal scanning circuit 209, the pixel signal selection circuit 211, and the pixel signal selection circuit 212.

  Further, the high-speed logic process chip 720 may include a smear detection unit 103. In the high-speed logic process chip 720, the smear detection unit 103 detects smear and outputs smear information and a correction image signal related to the generated smear.

  Further, the high-speed logic process chip 720 may include a smear correction value generation unit 104 and a smear correction unit 105 in addition to the smear detection unit 103, and the corrected image signal after the smear correction is supplied to the signal processing unit 106. You may make it output.

  As described above, in the present embodiment, since a large area can be allocated to the signal processing circuit by using the multilayer imaging element, at least a part of the signal processing circuit related to the smear correction process described above can be easily imaged. It can be mounted on the element.

(Embodiment 4)
Next, a fourth embodiment will be described. In the present embodiment, an example in which smear correction processing is performed in the mobile phone 800 will be described. FIG. 8 shows a functional configuration example of the mobile phone 800 as the present embodiment. The mobile phone 800 according to the present embodiment has an electronic mail function, an Internet connection function, an image capturing / playback function, and the like in addition to a voice call function.

  In FIG. 8, a communication unit 801 communicates audio data and image data with another mobile phone or a device having a communication function according to a communication carrier contracted by a user or by a communication method such as a wireless LAN. The voice processing unit 802 converts voice data from the microphone 803 into a format suitable for outgoing call and transmits the voice data to the communication unit 801 during a voice call. In addition, the voice processing unit 802 decodes the voice data from the communication partner transmitted from the communication unit 801 and transmits the decoded voice data to the speaker 804.

  The imaging unit 805 includes any of the imaging elements of Embodiments 1 to 3 described above, captures a subject, and outputs an image signal. The image processing unit 806 includes the smear detection unit 103, the smear correction value generation unit 104, the smear correction unit 105, and the signal processing unit 106 in the above-described embodiment. At the time of shooting an image, the image processing unit 806 processes the image signal shot by the imaging unit 805, converts it into a format suitable for recording, and outputs it. In addition, when the recorded image is reproduced, the image processing unit 806 processes the reproduced image and transmits the processed image to the display unit 807. The display unit 807 includes a liquid crystal display panel of about several inches, for example, and displays various screens according to instructions from the control unit 809. The nonvolatile memory 808 stores data such as address book information, e-mail data, or image data captured by the imaging unit 805.

  A control unit 809 includes a CPU, a memory, and the like, and controls each unit of the mobile phone 800 according to a control program stored in a memory (not shown). The control unit 809 includes the control unit 109 of the above-described embodiment. The operation unit 810 includes a power button, number keys, and other various operation keys for a user to input data. The card IF 811 records and reproduces various data with respect to the memory card 812. The external IF 813 transmits data stored in the nonvolatile memory 808 and the memory card 812 to the external device, and receives data transmitted from the external device. The external IF 813 performs communication by a known communication method such as a wired communication method such as USB or wireless communication.

  Next, a voice call function in the mobile phone 800 will be described. When making a call to the other party, the user operates the number key of the operation unit 810 to input the number of the other party, or displays the address book stored in the nonvolatile memory 808 on the display unit 807, Select and instruct to make a call. When the transmission is instructed, the control unit 809 transmits the communication unit 801 to the other party. When receiving a call to the other party, the communication unit 801 outputs the other party's voice data to the voice processing unit 802 and transmits the user's voice data to the other party.

  When transmitting an e-mail, the user uses the operation unit 810 to instruct mail creation. When mail creation is instructed, the control unit 809 displays a mail creation screen on the display unit 807. The user inputs a transmission destination address and a text using the operation unit 180 and instructs transmission. When the mail transmission is instructed, the control unit 809 transmits address information and mail body data to the communication unit 801. The communication unit 801 converts mail data into a format suitable for communication and transmits it to a transmission destination. When an email is received, the communication unit 801 converts the received mail data into a format suitable for display and displays a display unit 807. To display.

  Next, the photographing function in the mobile phone 800 will be described. When the user operates the operation unit 810 to set the shooting mode and then instructs to shoot a still image or a moving image, the imaging unit 805 shoots still image data or moving image data and sends it to the image processing unit 806. The image processing unit 806 performs the above smear correction processing on the captured still image data and moving image data, performs processing such as encoding, and stores it in the nonvolatile memory 808. Also, the image processing unit 806 transmits the captured still image data and moving image data to the card IF 811. The card IF 811 stores still images and moving image data in the memory card 812.

  In addition, the mobile phone 800 can transmit a file including a captured still image or moving image data as an attached file of an e-mail. Specifically, when an e-mail is transmitted, an image file stored in the nonvolatile memory 808 or the memory card 812 is selected, and transmission is instructed as an attached file.

  In addition, the mobile phone 800 can transmit a file including a captured still image or moving image data to an external device such as a PC or another telephone using the external IF 813. The user operates the operation unit 810 to select an image file stored in the nonvolatile memory 808 or the memory card 812 and instruct transmission. The control unit 809 controls the external IF 813 so that the selected image file is read from the nonvolatile memory 808 or the memory card 812 and transmitted to the external device.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 102 ... Image sensor, 103 ... Smear detection part, 104 ... Smear correction value generation part, 105 ... Smear correction part, 201 ... Unit pixel

Claims (23)

A first pixel that receives light from a subject, a pixel array in which second pixels for detecting smear are arranged, a first image signal read from the first pixel, and the first pixel An imaging device comprising: an output unit that separates and outputs the second image signal read from the two pixels;
Correction means for correcting the signal value of the first pixel corresponding to the position of the second pixel by using the signal value of the second pixel included in the second image signal;
An imaging apparatus characterized by that. The imaging device according to claim 1, wherein the second pixels are arranged at predetermined intervals in a direction in which smear occurs. The imaging device according to claim 1, wherein light from the subject is shielded from the second pixel. The correction means corrects the corresponding signal value of the first pixel using the signal value of the second pixel in which the signal value due to smear is accumulated,
The imaging apparatus according to any one of claims 1 to 3, wherein The imaging element can change the predetermined interval when the second pixels are arranged at predetermined intervals in a direction in which smear occurs.
The imaging apparatus according to any one of claims 1 to 4, wherein the imaging apparatus is characterized in that The imaging element controls the second pixel to block light from the subject by eliminating the charge accumulation time of the second pixel.
The imaging apparatus according to claim 5. The imaging device further includes detection means for detecting the occurrence of smear when the change in the signal value of the second pixel is greater than a predetermined threshold in a direction perpendicular to the direction in which smear occurs.
The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus. The image sensor outputs only the second pixel detected by the detection means as the second image signal.
The imaging apparatus according to claim 7. The image pickup device has a configuration in which a first chip on which the pixel array is arranged and a second chip on which the output unit is arranged are stacked.
The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus. The second chip further includes detection means for detecting the occurrence of smear when a change in the signal value of the second pixel is greater than a predetermined threshold in a direction perpendicular to the direction in which the smear occurs.
The imaging apparatus according to claim 9. The second chip outputs the second image signal of only the second pixel detected by the detection means;
The imaging apparatus according to claim 10. The correction means further includes detection means for detecting the occurrence of smear when the change in the signal value of the second pixel is greater than a predetermined threshold in a direction perpendicular to the direction in which smear occurs.
The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus. A control method for an imaging apparatus including an imaging element having a pixel array in which a first pixel that receives light from a subject and a second pixel for detecting smear is arranged,
An output step in which the image sensor separates and outputs the first image signal read from the first pixel and the second image signal read from the second pixel;
A correcting step in which the correcting means corrects the value of the first pixel corresponding to the position of the second pixel by using the signal value of the second pixel included in the input second image signal; Having
And a method of controlling the imaging apparatus. The method according to claim 13, wherein the second pixels are arranged at predetermined intervals in a direction in which smear occurs. The method according to claim 13 or 14, wherein the second pixel blocks light from the subject. A pixel array in which a first pixel that receives light from a subject, and a second pixel for detecting smear;
Separating means for separating the first image signal read from the first pixel and the second image signal read from the second pixel;
Correction means for correcting the signal value of the first pixel corresponding to the position of the second pixel by using the signal value of the second pixel included in the second image signal;
And an output unit that outputs a first image signal including the first pixel corrected by the correction unit. The image pickup device according to claim 16, wherein the second pixels are arranged at predetermined intervals in a direction in which smear occurs. The image sensor according to claim 16 or 17, wherein the second pixel is shielded from light from the subject. The imaging device includes a first chip on which the pixel array is arranged, a second chip on which the separation unit, the correction unit, and the output unit are arranged, and is stacked on the first chip.
The image pickup device according to any one of claims 16 to 18, wherein the image pickup device is provided. A pixel array in which a first pixel that receives light from a subject, and a second pixel for detecting smear;
An output unit that separates and outputs the first image signal read from the first pixel and the second image signal read from the second pixel;
An image pickup device comprising: The image pickup device according to claim 20, wherein the second pixels are arranged at predetermined intervals in a direction in which smear occurs. The image sensor according to claim 20 or 21, wherein the second pixel is shielded from light from the subject. The said image pick-up element has the structure which laminated | stacked the 1st chip | tip with which the said pixel arrangement | sequence is arrange | positioned, and the 2nd chip | tip with which the said output part is arrange | positioned. The imaging device according to item.