JP6703784B2 - Imaging device, control method thereof, and imaging device - Google Patents

Description

The present invention relates to an image pickup apparatus, a control method therefor, and an image pickup element.

Conventionally, in an image sensor such as a CCD or a CMOS image sensor, it is known that a phenomenon that affects image quality such as fixed pattern noise, shading, flicker, and smear occurs in the image sensor. Therefore, some image pickup apparatuses generate a correction value for each phenomenon using the image signal output from the image pickup element, and correct the image signal using the generated correction value.

In a CMOS image sensor, there is a case where a band-shaped luminance variation occurs in the horizontal direction of the same line. Patent Document 1 proposes a technique for calculating the amount of smear using a signal in the HOB region of the image sensor to correct the horizontal smear and subtracting the amount from the signal value in the effective pixel region.

Further, in Patent Document 2, a dummy pixel that is shielded from light is arranged in an effective image, a smear amount is obtained using a signal output from the dummy pixel, and the smear amount is subtracted from the signal value of the effective pixel to eliminate the smear. A correction technique has been proposed.

JP, 2009-5169, A JP, 2011-199426, A

However, in the technique proposed in Patent Document 1, when the smear component is not uniform in the horizontal direction, the smear amount in the HOB area and the smear amount in the effective image are different, so that the smear amount is accurately corrected according to the occurrence state of the smear. It may not be possible.

Further, in the technique proposed in Patent Document 2, signals of all effective pixels and dummy pixels are mixedly output from the image sensor. Therefore, the circuit that processes the output signal needs to perform signal processing by excluding the signal from the dummy pixel, which may complicate the processing.

The present invention has been made in view of the above-mentioned problems of the related art, and provides an image pickup apparatus, a control method therefor, and an image pickup element capable of correcting smear having a non-uniform distribution more easily. The purpose is to do.

In order to solve this problem, for example, the imaging device of the present invention has the following configuration. That is, the first image signal read from the first pixel has a first pixel that receives light from the subject and a pixel array in which second pixels for detecting smear are arranged. And an image sensor including an output unit that separates and outputs the second image signal read from the second pixel, and uses the signal value of the second pixel included in the second image signal. Te, a correction means for correcting a signal value of a first pixel corresponding to the position of the pixel second, was closed, the second pixels be controlled to charge storage time becomes 0 Is characterized by.

According to the present invention, it is possible to more easily correct smear with uneven distribution with high accuracy.

FIG. 3 is a block diagram showing a functional configuration example of a digital camera 100 as an example of an image pickup apparatus according to an embodiment of the present invention. FIG. 3 is a diagram showing a configuration example of the image sensor 102 according to the first embodiment. FIG. 3 is a diagram illustrating smear detection pixels and effective pixels of the image sensor 102 according to the first embodiment. FIG. 6 is a diagram illustrating smear correction processing according to the first embodiment. FIG. 3 is a block diagram showing a functional configuration example of the digital camera 100 according to the second embodiment. FIG. 3 is a diagram showing a configuration example of the image sensor 102 according to the second embodiment. FIG. 6 is a diagram illustrating an example of a stacked image sensor according to the third embodiment. A block diagram showing a functional configuration example of a smartphone 800 as an example of a mobile phone according to a fourth embodiment.

(Embodiment 1)
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. An example in which the present invention is applied to an arbitrary digital camera capable of capturing an image will be described below as an example of the image capturing apparatus. However, the present invention is not limited to a digital camera and can be applied to any device capable of capturing an image. These devices may include, for example, game machines, tablet terminals, personal computers, clock-type or eyeglass-type information terminals, monitoring equipment or medical equipment.

(Configuration of digital camera 100)
FIG. 1 is a block diagram showing a functional configuration example of a digital camera 100 as an example of the image pickup apparatus according to the present embodiment. Note that one or more of the functional blocks illustrated in FIG. 1 may be implemented by hardware such as an ASIC or a programmable logic array (PLA), or by a programmable processor such as a CPU or MPU executing software. May be. Also, it may be realized by a combination of software and hardware. Therefore, in the following description, even when different functional blocks are described as the main subject of operation, the same hardware can be mainly implemented.

The photographing optical system 101 includes a zoom lens, a diaphragm, a focus lens, a shift lens, and the like, and forms an optical image from a subject on an image sensor. The image sensor 102 has a configuration in which a plurality of pixels each having a photoelectric conversion element are arranged two-dimensionally. As will be described later with reference to FIG. 2, the image sensor 102 has an effective pixel that receives an optical image of a subject and a pixel for smear detection, respectively, and reads from the image signal read from the effective pixel and the smear detection pixel. It is configured such that it can be read out separately from the correction image signal that is output.

The smear detection unit 103, the smear correction value generation unit 104, and the smear correction unit 105 include a signal processing circuit or a signal processing module. The smear detection unit 103 detects smear generated using the correction image signal read from the smear detection pixel of the image sensor 102, and generates smear information. The smear information is information indicating the presence or absence of smear and the generation position. The smear correction value generation unit 104 generates a smear correction value using the correction image signal and the smear information output from the smear detection unit 103. The smear correction unit 105 corrects the image signal by subtracting the smear correction value generated by the smear correction value generation unit 104 from the signal value of the image signal output from the effective pixel of the image sensor 102.

The signal processing unit 106 includes a signal processing circuit or a signal processing module, and performs predetermined signal processing such as white balance on the image signal output from the smear correction unit 105. The display unit 107 includes a display panel including an LCD, an LED, and the like, and displays an image signal output from the signal processing unit 106, an operation image for operating the digital camera 100, and the like.

The recording unit 108 includes a recording medium such as a non-volatile semiconductor memory or a hard disk, or a recording medium such as an SD card, and records the image signal output from the signal processing unit 106.

The control unit 109 includes, for example, a CPU or MPU, and controls the entire digital camera 100 by expanding a program stored in a non-volatile memory such as a ROM into a work area of a volatile memory such as a RAM and executing the program. To do. The control unit 109 also controls the reading of each signal from the image sensor 102.

(Structure of the image sensor 102)
Next, a detailed configuration of the image sensor 102 will be described with reference to FIG. The unit pixel 201 includes a pixel having the configuration shown in FIG. 2B as one unit. The floating diffusion amplifier (FD amplifier) 202 detects the accumulated charge amount of the unit pixel 201 as a voltage.

The row read control line 203a and the row read control line 203b are connected to the unit pixels 201 at predetermined intervals (for example, for each column), and a read control signal is sent from the vertical scanning circuit 208a or the vertical scanning circuit 208b to each unit pixel. To transmit. FIG. 2A shows row read control lines 203a-20 to 203a-23 or 203b-20 connected to, for example, the unit pixels 201 on the 20th to 23rd rows of the unit pixels 201 arranged two-dimensionally. ~ 203b-23.

In addition, the row reset control line 204a and the row reset control line 204b are connected to the unit pixels 201 at predetermined intervals, and a reset control signal is transmitted from the vertical scanning circuit 208a or the vertical scanning circuit 208b to each unit pixel. For example, the row reset control lines connected to the unit pixels 201 on the 20th to 23rd rows of the unit pixels 201 arranged two-dimensionally are shown as 204a-20 to 204a-23 or 204b-20 to 204b-23. There is.

Further, the row selection line 205a and the row selection line 205b are respectively connected to the unit pixels 201 at predetermined intervals, and a row selection control signal is transmitted from the vertical scanning circuit 208a or the vertical scanning circuit 208b to each unit pixel. For example, the row selection lines connected to the unit pixels 201 on the 20th to 23rd rows of the unit pixels 201 arranged two-dimensionally are shown as 205a-20 to 205a-23 or 205b-20 to 205b-23. .

The column signal line 206 provided for each column transmits the output signal of the unit pixel 201 of each column to the column circuit 207. The column circuit 207 converts the pixel signal output from each unit pixel 201 under the control of the horizontal scanning circuit 209 into an analog-to-digital signal by an internal A/D conversion circuit and outputs the converted signal. The digital signal output from the column circuit 207 is input to the pixel signal selection circuit 211 or the pixel signal selection circuit 212 through the horizontal signal line 210, and the signal output terminal 213 or the signal output terminal is input in accordance with the operation of each pixel signal selection circuit. It is output from 214. For example, the pixel signal selection circuit 211, the pixel signal selection circuit 212, the signal output terminal 213, and the signal output terminal 214 form an output unit of a digital signal.

The timing generator (TG) 215 generates a reset timing signal and a read timing signal according to the setting. The vertical sync signal input terminal 216 and the horizontal sync signal input terminal 217 are terminals for inputting sync signals from the outside. Further, the setting signal line 218 inputs a setting signal for setting the reset and read timings for the TG 215 from the outside. The read row timing signal 219 output from the TG 215 is input to the vertical scanning circuit 208a and the vertical scanning circuit 208b. The read column timing signal 220 output from the TG 215 is input to the horizontal scanning circuit 209.

FIG. 2B shows a configuration example of the unit pixel 201 shown in FIG. A photodiode (PD) 301, which is a photoelectric conversion element, generates and accumulates charges according to the amount of incident light by photoelectric conversion. The transfer transistor 303 turns on/off according to the read control signal 302 input from the row read control line 203a or 203b. The floating diffusion (FD) 304 is composed of a capacitance having a predetermined capacitance, and generates a voltage according to the charges accumulated in the capacitance. The reset transistor 306 is turned on/off according to the reset control signal 305 input from the row reset read control line 204a or 204b. With the operation of the reset transistor 306, the FD 304 resets the charges accumulated according to the reset voltage supplied from the reset level input unit 307. Then, when the transfer transistor 303 is turned on, the charge generated in the PD 301 is transferred and accumulated. The row selection transistor 309 is turned on/off according to the row selection control signal 308 input from the row selection line 205 a or the row selection line 205, and a voltage signal according to the amount of charge transferred to and accumulated in the FD 304 is output from the pixel output 310. Output. Here, the FD amplifier 202 of FIG. 2A is composed of the FD 304 and the row selection transistor 309 of FIG. 2B.

(Outline of smear correction processing)
FIG. 3 schematically shows a pixel array of the image sensor 102 including smear detection pixels and effective pixels. Among a plurality of pixels arranged two-dimensionally, for example, effective pixels and smear detection pixels are arranged alternately in the horizontal direction (row direction). The column 4a shows a column in which effective pixels are arranged, and the column 4b shows a column in which smear detection pixels are arranged. Further, the plurality of pixels arranged in the VOB area and the HOB area are shielded from light so that light from the subject does not enter.

FIG. 3 schematically shows a state in which horizontal smear occurs in the row direction when a high-brightness subject is photographed, and pixels of a region indicated by a brighter color (that is, white) output a higher-brightness signal. It shows the situation. In a row shown as a smear area, smear occurs in the horizontal direction (row direction) with respect to a high-luminance object, and a high-luminance signal is output as compared with pixels in other areas in the vertical direction (column direction). Further, the brightness level is not uniform in the horizontal direction, and the brightness level is different for each column.

Next, the smear detection and smear correction processing will be described with reference to FIGS. FIG. 4A shows an image signal generated by extracting a pixel column forming the effective image 4a, and FIG. 4B shows an image signal generated by extracting a pixel column forming the smear detection pixel 4b. Each of the correction image signals is shown. And FIG.4(c) has shown the image after applying the correction value obtained based on the smear detection pixel to the signal of an effective pixel, and performing smear correction.

FIG. 4D is a graph showing the output signal value of the pixel in the smear area of FIG. 4A, and the pixel included in the range A outputs the highest signal value. Further, the output of the pixel changes (decreases) as the distance from the range A increases, and the pixel in the HOB region outputs the signal value of a predetermined level due to the dark current.

FIG. 4E is a graph showing the output signal value of the smear detection pixel. The smear detection pixel is shielded so as not to receive the light from the subject like the pixel in the HOB area, and the smear component generated in each pixel can be detected. The example shown in FIG. 4E indicates that the smear component that is high in the range A and decreases as the distance from the range is detected. That is, it means that non-uniform smear is generated in the horizontal direction (row direction) with respect to the high-luminance subject.

4F shows the result of subtracting the smear component shown in FIG. 4E from the output signal value of the effective pixel shown in FIG. 4D. More specifically, by subtracting the output signal value of the adjacent smear detection pixel from the signal value of the effective pixel, it is possible to appropriately correct the smear component generated in the effective pixel adjacent to the smear detection pixel. Showing.

(Specific operation related to smear correction processing)
Next, a specific operation related to the smear correction process of this embodiment will be described. For example, when there is a shooting instruction from the user via an input unit (not shown) of the digital camera 100, this processing is started together with the start of the shooting operation. This is realized by the control unit 109 expanding a program stored in a ROM (not shown) into a work area of a RAM (not shown) and executing the program.

The control unit 109 sets the timing for resetting each pixel group and the timing for reading a signal from each pixel group for each field or frame for the TG 215 in the image sensor 102. When an optical image of a subject is formed on the light receiving surface of the image sensor 102 through the photographic optical system 101, the PD (photodiode) 301 in each unit pixel 201 on the image sensor 102 has each unit pixel Photocharges corresponding to the amount of incident light on 201 are generated.

In the FD 304 in each unit pixel 201 of the image sensor 102, a voltage corresponding to the electric charge accumulated in the capacitance having a predetermined capacitance is generated. When the reset control signal 305 is input, the reset transistor 306 operates, and the power source voltage VDD input from the reset level input unit 307 resets the charges accumulated in the FD 304.

On the other hand, photoelectric charges are accumulated in the PD 301 according to the amount of light incident on each unit pixel 201. In this state, when the transfer transistor 303 is turned on by the level change of the read control signal 302, the photocharge accumulated in the PD 301 moves to the FD 304. At this time, in the FD 304, the voltage level changes by the amount of the photocharges moved with respect to the reset level. The change in the voltage level becomes the signal level of the pixel, and when the row select transistor 309 is turned on due to the change in the level of the row select control signal 308, a voltage signal according to the charge amount of the FD 304 is output. The output analog pixel signal is transmitted to the column circuit 207 via the column signal line 206. The column circuit 207 amplifies the analog pixel signal and converts it into a digital signal.

The digital signal converted by the column circuit 207 of each column is sequentially read by the horizontal scanning circuit 209 and input to the pixel signal selection circuit 211 and the pixel signal selection circuit 212 through the horizontal signal line 210. The pixel signal selection circuit 211 selects the image signal output from the effective pixel 4a according to the instruction from the control unit 109, while the pixel signal selection circuit 212 outputs from the smear detection pixel 4b according to the instruction from the control unit 109. Image signal for correction to be selected. After that, the image signal of the effective pixel 4a is separately output from the signal output terminal 213 and the correction image signal is output from the signal output terminal 214.

Next, the smear detection unit 103 detects smear using the correction image signal output from the signal output terminal 214, and generates smear information indicating the presence or absence of smear and the generation position. More specifically, since smear is generated in the horizontal direction (row direction) for a high-luminance subject, the smear detection unit 103 changes the signal amount of the smear detection pixel in the vertical direction (column direction) (that is, the signal amount). Difference) is calculated. The smear detection unit 103 detects smear when the calculated change amount is equal to or larger than a predetermined threshold. The method for detecting the smear in the horizontal direction (row direction) is not limited to the method using the change in the signal amount of the effective pixel, and for example, the change amount in the vertical direction (column direction) of the smear detection pixel in the HOB area is equal to or more than a predetermined threshold If it is, smear may be detected. The smear detection unit 103 outputs smear information indicating the presence or absence of smear and the generation position (for example, coordinate position or line number) to the smear correction value generation unit 104 based on the result of detection of smear.

The smear correction value generation unit 104 uses the smear information generated by the smear detection unit 103 and the correction image signal to generate a smear correction value for the image signal. Specifically, the smear correction value generation unit 104 specifies the smear position from the smear information and generates the correction value using the signal value of the correction image signal corresponding to the smear position. Since the smear correction value generation unit 104 calculates the correction value for the effective pixel adjacent to the smear detection pixel, the correction image signal is calculated in consideration of the change in the horizontal direction (row direction) between the correction image signals. You may interpolate. That is, when the rows of smear detection pixels and the rows of effective pixels are alternately arranged, the correction image signals are interpolated in consideration of the change in the horizontal direction between the correction image signals, thereby A smear correction value suitable for the column can be generated. The smear correction unit 105 corrects smear by subtracting the smear correction value output from the smear correction value generation unit 104 from the image signal output from the signal output terminal 213 of the image sensor 102.

The signal processing unit 106 acquires the image signal subjected to the smear correction process from the smear correction unit 105, performs predetermined image processing, and causes the display unit 107 to display the image signal according to an instruction from the control unit 109, or the recording unit 108. The image signal is recorded, and a series of operations related to this processing is ended.

In the present embodiment, the example in which the smear detection pixels are arranged alternately with the effective pixels has been described, but the arrangement of the smear detection pixels may be dynamically changeable. Since the smear detection pixel described in this embodiment is composed of the unit pixel 201, the effective pixel and the smear detection pixel are moved by eliminating the charge accumulation time (set to 0) in order to shield the pixel from light. Each image signal can be read out by selectively switching. By doing so, the unit pixel can be effectively utilized in accordance with the situation such as changing the smear detection density according to the subject.

As described above, in the present embodiment, the image sensor 102 in which the effective pixels and the pixels for smear detection are alternately arranged for each column is used to read the image signal read from the effective pixels from the smear detection pixels. The correction is performed based on the correction value obtained from the correction image signal that is output. At this time, the image pickup device separates and outputs the image signal read from the effective pixel and the correction image signal read from the smear detection pixel. By doing so, the circuit that receives the signal from the image pickup device can easily detect the smear, and when the smear spreads in the horizontal direction, the correction process according to the horizontal change of the smear becomes possible. That is, it becomes possible to more easily correct smear with uneven distribution with high accuracy.

(Embodiment 2)
Next, a second embodiment will be described. In the first embodiment, the smear detection unit 103 provided outside the image sensor 102 is configured to detect smear using the correction image signal output from the image sensor 102. In the present embodiment, the smear detector is arranged inside the image sensor to output the smear information from the image sensor 102. The configuration relating to the image sensor and the smear detection unit of this embodiment is different from that of the first embodiment, but the other configurations are the same as those of the first embodiment. For this reason, the same components are designated by the same reference numerals, duplicate description is omitted, and different points will be mainly described.

With reference to FIG. 5, a functional configuration example of the digital camera 100 in which the image sensor 501 includes the smear detection unit 502 will be described.

The image sensor 501 outputs the image signal read from the effective pixel from the signal output terminal 213 and inputs the correction image signal read from the smear detection pixel to the smear detection unit 502. The image sensor 501 outputs the smear information generated by the smear detector 502 from the signal output terminal 214. Further, the image sensor 501 cuts out a correction image signal related to the position where the smear is detected, and outputs it from the signal output terminal 214 together with the smear information.

The smear detection unit 502 is arranged in the image sensor 501 and detects smear based on the correction image signal read from the smear detection pixel. The smear detection process in the smear detection unit 502 is the same as the process described above in the first embodiment. The control unit 109 controls the entire digital camera 100, and also controls the reading method of the image sensor 102 and the smear detection unit 502.

Further, the operation of the smear correction process according to this embodiment will be described. For example, when there is a shooting instruction from the user via an input unit (not shown) of the digital camera 100, this processing is started together with the start of the shooting operation. The operation of the unit pixel 201 of the image sensor 102 is similar to that of the first embodiment, and the signal output from each unit pixel 201 is converted into a digital signal by the column circuit 207 via the column signal line 206. The converted digital signal is input to the pixel signal selection circuit 211 and the pixel signal selection circuit 212 as shown in FIG. 6, and at the same time, is input to the smear detection unit 502.

The smear detection unit 502 detects smear using the correction image signal of the smear detection pixel, generates smear information indicating the presence or absence of smear and the generation position based on the detection result, and outputs the smear information from the signal output terminal 214. To do.

The pixel selection circuit 212 acquires only the image signal at the position where smear is detected in the smear detection pixel 4b based on the smear information generated by the smear detection unit 502, and outputs it as a correction signal from the signal output terminal 214. Let By doing so, the correction image signal in the area where smear does not occur is not constantly output, so that an increase in the output signal amount and an increase in the processing load on the receiving side due to the increase in the number of pixels are prevented. be able to. In other words, the correction image signal output from the image sensor 102 can be limited to the position or row where smear is occurring, and the signal amount can be reduced.

After that, the smear correction value generation unit 104 acquires the smear information and the correction image signal output from the signal output terminal 214, and generates the smear correction value for the correction image signal as in the first embodiment described above. Also in the subsequent processing, the smear correction unit 105 corrects smear by subtracting the smear correction value from the image signal output from the signal output terminal 213 of the image sensor 102, as in the first embodiment. Then, the control unit 109 causes the display unit 107 to display the corrected image signal and ends the series of operations related to the present process.

In the present embodiment, the image sensor 501 includes the smear detection unit 502 therein to output the smear information and the image signal for correction in which the smear has occurred. You may make it equip 105 in the image pick-up element 501. With this configuration, the smear correction process can be realized in the image sensor, and the image sensor can output only the smear-corrected effective image, thereby reducing the output signal amount.

As described above, in the present embodiment, the image sensor 501 includes the smear detector 502 therein, the image sensor 501 internally detects smear, outputs the presence or absence of smear, and the generation position, and the smear is detected. Only the image signal for position correction is output. By doing so, it is possible to adaptively and accurately correct the smear according to the occurrence state of the smear, and it is possible to reduce the signal amount output from the image sensor.

(Embodiment 3)
Next, a third embodiment will be described. In the present embodiment, a case where the image pickup device is configured as a stacked type will be described. Note that the present embodiment has the same configuration as that of the first embodiment except that the image pickup element is a laminated type. For this reason, the same components are designated by the same reference numerals, duplicate description is omitted, and different points will be mainly described.

FIG. 7 shows an image sensor 700 according to this embodiment. In the image sensor 700, an image sensor chip 710 including a photoelectric conversion element such as the unit pixel 201 and a high-speed logic process chip 720 including a configuration for performing digital processing such as a column circuit 207 are stacked at a chip level.

FIG. 7A is a perspective view of the image sensor 700, and FIG. 7B is a top view of the chip 710 and the chip 720. The pixel portion 711 of the image sensor chip 710 includes a configuration for reading out pixel signals of the unit pixel 201, the FD amplifier 202, the TG 215, and the like. The high-speed logic process chip 720 includes a column circuit 207, a horizontal scanning circuit 209, a pixel signal selection circuit 211, a pixel signal selection circuit 212, and other components capable of processing including digital data.

Further, the high speed logic process chip 720 may include the smear detection unit 103. In the high-speed logic process chip 720, the smear detection unit 103 detects smear, and outputs smear information and a correction image signal related to the smear that has occurred.

Further, the high-speed logic process chip 720 may include a smear correction value generation unit 104 and a smear correction unit 105 in addition to the smear detection unit 103, and the corrected image signal after smear correction is sent to the signal processing unit 106. You may make it output.

As described above, in the present embodiment, since a large area can be allocated to the signal processing circuit by using the stacked image sensor, at least a part of the signal processing circuit related to the smear correction processing described above can be easily imaged. It can be mounted on the device.

(Embodiment 4)
Next, a fourth embodiment will be described. In this embodiment, an example in which the mobile phone 800 performs smear correction processing will be described. FIG. 8 shows an example of the functional configuration of the mobile phone 800 according to this embodiment. The mobile phone 800 of this embodiment has an e-mail function, an Internet connection function, an image capturing/reproducing function, and the like in addition to a voice call function.

In FIG. 8, a communication unit 801 communicates voice data and image data with another mobile phone or a device having a communication function according to a communication carrier contracted by a user or a communication method such as a wireless LAN. The voice processing unit 802 converts the voice data from the microphone 803 into a format suitable for transmission and transmits the voice data to the communication unit 801 during a voice call. Further, the voice processing unit 802 decodes voice data from the communication partner transmitted from the communication unit 801 and transmits the voice data to the speaker 804.

The image capturing unit 805 includes any of the image capturing elements according to the first to third embodiments described above, captures a subject, and outputs an image signal. The image processing unit 806 includes the smear detection unit 103, the smear correction value generation unit 104, the smear correction unit 105, and the signal processing unit 106 according to the above-described embodiment. The image processing unit 806 processes the image signal captured by the image capturing unit 805 during image capturing, converts the image signal into a format suitable for recording, and outputs the format. Further, the image processing unit 806 processes the reproduced image and transmits it to the display unit 807 when reproducing the recorded image. The display unit 807 includes, for example, a liquid crystal display panel of about several inches, and displays various screens in accordance with an instruction from the control unit 809. The non-volatile memory 808 stores data such as address book information, email data, or image data captured by the image capturing unit 805.

The control unit 809 includes a CPU and a memory, and controls each unit of the mobile phone 800 according to a control program stored in a memory (not shown). The control unit 809 includes the control unit 109 of the above-described embodiment. The operation unit 810 includes a power button, number keys, and other various operation keys for the user to input data. The card IF 811 records/reproduces various data on/from the memory card 812. The external IF 813 transmits the data stored in the non-volatile memory 808 or the memory card 812 to the external device, and also receives the data transmitted from the external device. The external IF 813 performs communication by a known communication method such as a wired communication method such as USB or wireless communication.

Next, the voice call function of the mobile phone 800 will be described. When making a call to the other party, the user operates the number key of the operation unit 810 to input the number of the other party, or the address book stored in the non-volatile memory 808 is displayed on the display unit 807, and the other party is called. Select and instruct to make a call. When an instruction to make a call is made, the control unit 809 makes a call to the communication partner via the communication unit 801. When the communication partner receives an incoming call, the communication unit 801 outputs the partner's voice data to the voice processing unit 802 and also transmits the user's voice data to the partner.

In addition, when sending an electronic mail, the user uses the operation unit 810 to instruct mail creation. When the mail creation is instructed, the control unit 809 displays a mail creation screen on the display unit 807. The user uses the operation unit 180 to input a destination address and a body text, and gives an instruction for transmission. When the mail transmission is instructed, the control unit 809 transmits the address information and the mail text data to the communication unit 801. The communication unit 801 converts the mail data into a format suitable for communication and transmits the data to the destination, and when receiving the electronic mail, converts the received mail data into a format suitable for display and displays the display unit 807. To display.

Next, the shooting function of the mobile phone 800 will be described. When the user operates the operation unit 810 to set the shooting mode and then instructs to shoot a still image or a moving image, the imaging unit 805 shoots the still image data or the moving image data and sends it to the image processing unit 806. The image processing unit 806 performs the above-described smear correction processing on the captured still image data or moving image data, performs processing such as encoding, and stores the nonvolatile memory 808. The image processing unit 806 also transmits the captured still image data and moving image data to the card IF 811. The card IF 811 stores still image and moving image data in the memory card 812.

In addition, the mobile phone 800 can transmit a file including a captured still image or moving image data as an attachment file of an email. Specifically, when sending an e-mail, an image file stored in the non-volatile memory 808 or the memory card 812 is selected and an instruction to send as an attached file is given.

Further, the mobile phone 800 can also transmit a file including a captured still image or moving image data to an external device such as a PC or another phone by the external IF 813. The user operates the operation unit 810 to select an image file stored in the non-volatile memory 808 or the memory card 812 and gives an instruction for transmission. The control unit 809 controls the external IF 813 so that the selected image file is read from the non-volatile memory 808 or the memory card 812 and transmitted to an external device.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. It can also be realized by the processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

102... Image sensor, 103... Smear detection part, 104... Smear correction value generation part, 105... Smear correction part, 201... Unit pixel

Claims (12)

A first pixel that receives light from a subject, a pixel array in which a second pixel for detecting smear is arranged, a first image signal read from the first pixel, and the first image signal. An image sensor including an output unit that separates and outputs a second image signal read from the second pixel;
Using said second signal value of the second pixels included in the image signal, have a, and correcting means for correcting the signal value of the first pixel corresponding to the position of the pixel second,
The image pickup device, wherein the second pixel is controlled so that the charge accumulation time becomes zero . The correction means corrects the signal value of the corresponding first pixel by using the signal value of the second pixel in which the signal value by smear is accumulated.
The image pickup apparatus according to claim 1 , wherein the image pickup apparatus is provided. The image pickup device is capable of changing the predetermined interval when the second pixels are arranged at predetermined intervals in the direction in which smear occurs.
The imaging apparatus according to claim 1 or 2, characterized in that. The image pickup device further includes detection means for detecting the occurrence of smear when the change in the signal value of the second pixel is larger than a predetermined threshold value in the direction perpendicular to the direction in which smear occurs.
The imaging apparatus according to any one of claims 1 3, characterized in that. The image pickup device outputs only the second pixel detected by the detection means as the second image signal.
The image pickup apparatus according to claim 4 , wherein The image pickup device has a structure in which a first chip in which the pixel array is arranged and a second chip in which the output unit is arranged are stacked.
The imaging apparatus according to any one of claims 1 to 5, characterized in that. The second chip further includes detection means for detecting the occurrence of smear when the change in the signal value of the second pixel is larger than a predetermined threshold value in the direction perpendicular to the direction in which the smear occurs.
The image pickup apparatus according to claim 6 , wherein The second chip outputs the second image signal of only the second pixel detected by the detection means,
The image pickup apparatus according to claim 7 , wherein The correction means further includes detection means for detecting the occurrence of smear when the change in the signal value of the second pixel is larger than a predetermined threshold value in the direction perpendicular to the direction in which smear occurs.
The imaging device according to claim 2 or 3 , wherein. A method for controlling an image pickup apparatus including an image pickup element having a pixel array in which a first pixel for receiving light from a subject and a second pixel for detecting smear are arranged,
An output step in which the image pickup device separates and outputs a first image signal read from the first pixel and a second image signal read from the second pixel;
A correcting step of correcting the value of the first pixel corresponding to the position of the second pixel using the signal value of the second pixel included in the input second image signal; , I have a,
The method of controlling an image pickup device, wherein the second pixel is controlled so that a charge accumulation time becomes zero . A pixel array in which a first pixel for receiving light from a subject and a second pixel for detecting smear are arrayed;
Separation means for separating the first image signal read from the first pixel and the second image signal read from the second pixel;
Correction means for correcting the signal value of the first pixel corresponding to the position of the second pixel using the signal value of the second pixel included in the second image signal;
Have a, and output means for outputting a first image signal comprised of corrected first pixel by the correction means,
The second pixel is controlled so that the charge accumulation time becomes zero . The image pickup device includes a first chip in which the pixel array is arranged, a second chip in which the separating unit, the correcting unit, and the output unit are arranged, and which is laminated with the first chip.
The image pickup device according to claim 11 , wherein: