JP2021158491A - Imaging device - Google Patents

Abstract

To prevent a plurality of lateral smears from overlapping and complicating in an imaging device having a configuration in which a single sensor has a plurality of optical systems.SOLUTION: An imaging device includes: a plurality of optical systems that can be shielded from light independently of each other; one imager on which images formed by the plurality of optical systems are projected; variation quantity detection means that has high-luminance subject detection means configured to detect a high-luminance subject in a photographic image and by which a quantity of variation of a signal from a black level is detected from a light-shielded image in an image on the basis of an image photographed by the imager; correction quantity detection means that calculates a quantity of correction from the detected quantity of variation; and correction means that corrects the photographic image to the calculated quantity of correction. When a high-luminance subject is present in the photographic image, some of the plurality of optical systems are imaged, and some other are shielded from light. The quantity of variation of a signal caused by the photographed image is detected from the shielded image of the imager and corrected.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image pickup apparatus, and more particularly to an image pickup apparatus having two optical systems and forming an image in parallel on one image pickup device.

Conventionally, in a solid-state image sensor having a configuration in which a differential amplifier in each row amplifies a signal output from a pixel in each row to a vertical output line in a pixel array and outputs the signal to the output amplifier, it is strong against some pixels. When light hits the image, a phenomenon occurs in which the left and right output levels of a high-brightness subject differ from other levels as if a band was drawn (hereinafter referred to as horizontal smear) according to the image signal. It has been known.

This means that when strong light hits some pixels, an optical signal with an excessive signal level is output from some of the pixels to the vertical output line. At this time, even if the potential of the vertical output line is clipped by the MOS transistor, the optical signal whose signal level becomes excessive flows into the differential amplifier until the MOS transistor is turned on. In this case, when the differential amplifier amplifies the optical signal, the constant current circuit included in the differential amplifier causes an excessive current to flow to the ground line, so that the differential amplifiers in the other rows are affected by this and the appropriate signal is output. Is generated without being able to output.

Patent Document 1 discloses, in order to correct this horizontal smear, a method of detecting the amount of horizontal smear from a horizontal OB region arranged in the horizontal direction of the effective image origin region and correcting it by performing a clamping operation.

Further, conventionally, an imaging device for stereoscopic photography is known.

For example, Patent Document 2 discloses a lens in which two sets of optical systems are arranged in parallel and two image circles are imaged in parallel on one image sensor. The two sets of optics are arranged horizontally with a predetermined distance apart. This distance is called the baseline length. When viewed from the imaging side, the image formed by the right optical system is recorded as a moving image or still image for the right eye, and the image formed by the left optical system is recorded as a moving image or still image for the left eye. .. At the time of reproduction, by viewing using a known 3D display or so-called VR goggles, the image for the right eye is displayed on the right eye of the viewer, and the image for the left eye is displayed on the left eye. At this time, since an image having a parallax is projected on the right eye and the left eye depending on the baseline length of the photographing lens, the viewer can obtain a stereoscopic effect.

Japanese Unexamined Patent Publication No. 2013-106186 Japanese Unexamined Patent Publication No. 2012-003022

However, even in the conventional image pickup apparatus for stereoscopic photography as disclosed in Patent Document 2 described above, horizontal smear as disclosed in Patent Document 1 occurs.

Two images are imaged on one image sensor by two sets of optical systems, but in the case of stereoscopic vision, almost the same subject is photographed although there is parallax, so when a high-brightness subject is photographed. Causes horizontal smearing due to the high-intensity subjects in both of the two images.

Further, since the amount of horizontal smear is not uniform in the horizontal direction but has a shading component and changes in the horizontal direction, there is also a problem that the detection in the horizontal OB portion is not sufficient.

Therefore, an object of the present invention is to accurately correct lateral smear in an image pickup apparatus having one image pickup element from two sets of optical systems.

In order to achieve the above object, the image pickup device of the present invention has a plurality of optical units, a light-shielding unit capable of independently shielding each optical unit, and an image formed by the plurality of optical systems. One image sensor on which the image is projected, a high-brightness subject detection means for detecting a high-brightness subject in a captured image, a storage unit for storing the captured image, and a horizontal smear component detected from the image to detect horizontal smear. In an image sensor having a horizontal smear correction amount generator for calculating a correction amount and a horizontal smear correction unit for correcting a captured image using the calculated horizontal smear correction amount, a high-brightness subject is present in the captured image. In this case, some of the plurality of optical systems are shielded from light, and the horizontal smear correction amount is calculated from the light-shielded image of the image sensor to perform horizontal smear correction.

According to the present invention, the horizontal smear can be corrected with high accuracy by detecting the horizontal smear component from the shaded effective image region and calculating the correction amount.

It is a block diagram which shows the structural example of the image pickup apparatus of this invention. It is a conceptual diagram explaining the horizontal smear detection and horizontal smear correction. It is a conceptual diagram explaining the horizontal smear detection and horizontal smear correction. It is a conceptual diagram explaining the horizontal smear detection and horizontal smear correction. It is a flowchart explaining the operation of 1st Example. It is a flowchart explaining the operation of 1st Example.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

In this embodiment, the configuration and operation for accurately correcting the smear will be described with reference to the drawings.

FIG. 1 is a block diagram showing a configuration example of an imaging device capable of capturing a stereoscopic image according to the present embodiment. Reference numerals 101 and 102 are optical units composed of a zoom lens, a focus lens, a shift lens, and the like, and in this case, they have two optical units for stereoscopic photography. Reference numerals 103 and 104 are light-shielding portions that block the optical images from the optical portions of 101 and 102, respectively.

The image sensor 105 is capable of forming an optical image of the optical unit 101 and an optical image of the optical unit 102 on the left and right sides of the effective image as an electrical image signal.

The high-brightness subject detection unit 106 detects whether or not a high-brightness subject that generates horizontal smear exists in the image signal output from the image sensor 105, and if so, the captured image signal has horizontal smear. The generation information is added, and the control unit 107 is notified that horizontal smear is generated.

The storage unit 108 stores a recording image and a horizontal smear amount detection image. The horizontal smear correction amount generation unit 109 detects the horizontal smear amount from the horizontal smear amount detection image of the storage unit 108 and generates the correction amount.

The horizontal smear correction unit 110 subtracts and corrects the horizontal smear correction amount generated by the horizontal smear correction amount generation unit 109 from the recording image read from the storage unit 108.

The camera signal processing circuit 111 performs camera signal processing such as white balance on the video signal output from the horizontal smear correction unit 110. The display unit 112 displays the video signal output from the camera signal processing unit 111 on a display device such as a liquid crystal panel. The recording unit 113 is a recording unit that records the video signal output from the camera signal processing unit 113 on a recording medium such as an SD card.

Further, the control unit 107 is for controlling the reading method and the light-shielding unit of each image sensor. 2, 3 and 4 are conceptual diagrams for explaining horizontal smear detection and horizontal smear correction.

FIG. 2a shows a state in which the light-shielding portions of 103 and 104 are not shaded and the images of the optical portions of 101 and 102 are imaged on the image sensor 105. In FIG. 2a, R is the effective image on the right side imaged from the optical unit 101, L is the effective image on the left side imaged from the optical unit 102, and O is the optical shading around the effective pixel portion by the image sensor. It is an OB part representing the black level.

AR and AL in FIG. 2a are high-luminance subjects. It is assumed that the high-intensity subject AR in the right image is in the position of the vertical position V horizontal position HR, and the high-intensity subject AL in the left image is in the position of the vertical position V horizontal position HL. In an imaging device for stereoscopic photography as in this embodiment, although there is left-right parallax, almost the same subject is photographed, so that the vertical positions V of the two high-brightness subjects AR and AL do not change, but only the horizontal position changes. Is imaged.

SR in FIG. 2a is a smear area in the right image R, SL is a horizontal smear area in the left image L, SO is a horizontal smear area in the OB part, and horizontal smear occurs in the horizontal direction of the high-brightness subjects AR and AL. It represents the region where the signal level has changed. Since the horizontal smear is generated in the horizontal direction of the high-brightness subject, the vertical positions of the horizontal smear regions SR, SL, and SO are also V.

FIG. 2b shows the signal level in the horizontal direction at the vertical position V in FIG. 2a.

The positions of HR and HL in which a high-brightness subject exists have high signal values, and in the horizontal smear areas SR, SL, and SO, the signal level increases as the position is closer to the high-brightness subject.

FIG. 2c shows the amount of variation in the signal level due to the horizontal smear generated by the high-luminance subject AL (hereinafter referred to as the horizontal smear component). 2d shows a horizontal smear component generated by the high-luminance subject AR.

FIG. 2e is obtained by obtaining the horizontal smear component when the high-luminance subject AR and AL are generated from the horizontal smear component of FIGS. 2c and 2d.

FIG. 2f shows that the horizontal smear is corrected by subtracting the horizontal smear component of FIG. 2e from the captured image signal with horizontal smear shown in FIG. 2b. The image is an image without horizontal smear as shown in FIG. 2g.

FIG. 3a shows a state in which the light-shielding portion of 104 is shielded from light and only the image of the optical portion of 101 is imaged on the image sensor 105.

In FIG. 3a, R is the effective image on the right side imaged from the optical portion of 101, L is the effective image portion on the left side shaded by the light-shielding portion of 104, and O is the optical shading around the effective pixel portion by the image sensor. It is an OB part representing the black level.

AR in FIG. 3a is a high-luminance subject. It is assumed that the high-intensity subject AR in the right image is at the same vertical position V and horizontal position HR as in FIG. 2a.

SR in FIG. 3a is a smear area in the right image R, SL is a horizontal smear area in the left image L, and SO is a horizontal smear area in the OB section. Represents the changed area.

FIG. 3b shows the signal level in the horizontal direction at the vertical position V in FIG. 3a. The position of HR in which a high-luminance subject exists has a high level signal value, and in the horizontal smear areas SR, SL, and SO, the signal level increases as the position is closer to the high-luminance subject.

Compared with FIG. 2b, since the high-luminance subject AL does not exist, the horizontal smear component in the horizontal smear region SR in the right image R is reduced. The horizontal smear region SL in the left image L and the horizontal smear region SO in the OB portion O are the values of the horizontal smear component with respect to the black level.

FIG. 3c shows the amount of the lateral smear component generated in the left direction of the high-luminance subject AR. From the changes in the horizontal smear components of SL and SO, the horizontal smear components in SR can also be obtained by calculation.

FIG. 4a shows a state in which the light-shielding portion of 103 is shielded from light and only the image of the optical portion of 102 is imaged on the image sensor 105.

In FIG. 4a, L is the effective image on the left side imaged from the optical portion of 102, R is the effective image portion on the right side shaded by the light-shielding portion of 103, and O is the optical shading around the effective pixel portion by the image sensor. It is an OB part representing the black level.

AL in FIG. 4a is a high-luminance subject. It is assumed that the high-intensity subject AL in the left image L is at the same vertical position V and horizontal position HL as in FIG. 2a.

SL in FIG. 4a is a smear area in the left image L, SR is a horizontal smear area in the right image R, and SO is a horizontal smear area in the OB section. Represents the changed area.

FIG. 4b shows the signal level in the horizontal direction at the vertical position V in FIG. 4a. The position of HL where a high-brightness subject exists has a high level signal value, and in the horizontal smear areas SL, SR, and SO, the signal level increases as the position is closer to the high-brightness subject.

Compared with FIG. 2b, since the high-luminance subject AR does not exist, the horizontal smear component in the horizontal smear region SL in the right image L is reduced. The horizontal smear region SL in the left image L and the horizontal smear region SO in the OB portion O are the values of the horizontal smear component with respect to the black level.

FIG. 4c shows the amount of the lateral smear component generated in the right direction of the high-luminance subject AL. From the change in the lateral smear component of SR, the lateral smear component in SL can also be obtained by calculation.

FIG. 4d shows the horizontal smear component in the left-right direction by combining the horizontal smear component on the left side of the high-luminance subject shown in FIG. 3c and the horizontal smear component on the right side of the high-luminance subject shown in FIG. 4c. ..

The horizontal smear components shown in FIGS. 2c and 2d described above represent the horizontal smear components shown in FIG. 4d as if the high-luminance subject is at the HL and HR positions.

Next, the operation of the first embodiment of the present invention will be described with reference to FIGS. 1, 2, 3, and 4 according to the flowchart of FIG.

FIG. 5a shows the imaging operation flow of the imaging device when the user acquires an image for correcting the horizontal smear at the time of shooting, and FIG. 5b shows the correction flow of the imaging device that corrects the horizontal smear with respect to the recorded image. ..

First, when the user gives an imaging instruction for image recording to the imaging device, the imaging device captures the image for recording (501). At this time, the light-shielding portions of 103 and 104 do not block light, pass through the optical units 101 and 102, and capture the image formed on the image sensor 105.

Next, the high-brightness subject detection unit 106 detects whether or not a high-brightness subject that causes lateral smear is present in the image signal output from the image sensor 105 (502).

Specifically, the signal level at which horizontal smear occurs is set in advance by the image sensor 105, and when the signal level of the captured image signal is exceeded, it is determined as a high-luminance subject.

When the high-luminance subject does not exist, the output image signal from the image sensor 105 is stored in the storage unit 108 (503).

When a high-brightness subject exists, the control unit 107 is notified, and the horizontal smear generation information indicating that the high-brightness subject exists and the position information of the high-brightness subject are transmitted to the output image signal from the image sensor 105. It is additionally stored in the storage unit 108 (504). At this time, an image corresponding to FIG. 2a having two high-intensity subjects on the left and right and a horizontal smear area is stored.

After that, the control unit 107 shields the light-shielding unit 104 from light, and captures an image for detecting lateral smear that has passed only through the optical unit 101 (505). Since the left image is shielded from light, an image corresponding to FIG. 3a is captured and stored in the storage unit 108. Further, the position of the high-brightness subject is detected by the high-brightness subject detection unit 106, and the position information is added.

After that, the control unit 107 shields the light-shielding unit 103 from light, and captures an image for detecting lateral smear that has passed only through the optical unit 102 (506). Since the right image is shielded from light, an image corresponding to FIG. 4a is captured and stored in the storage unit 108. Further, the position of the high-brightness subject is detected by the high-brightness subject detection unit 106, and the position information is added.

The above 501 to 506 are performed within a sufficiently short time in which the position of the subject does not change.

Next, the horizontal smear correction flow will be described with reference to FIG. 5b. The horizontal smear correction amount generation unit 109 checks whether or not the horizontal smear generation information is added to the recording image stored in the storage unit (507), and if it is added, generates the horizontal smear correction amount. Do (508).

Specifically, first, the horizontal smear component detection image (FIG. 3a) whose left side is shaded is read out from the storage unit 108, and the horizontal smear components of the horizontal smear regions SL and SO are acquired. Further, by calculating the horizontal smear component of the horizontal smear region SR from the changes in the horizontal smear components of SL and SO, the horizontal smear component (FIG. 3c) generated on the left side of the high-luminance subject AR is detected.

Next, the horizontal smear component detection image (FIG. 4a) whose right side is shaded is read out from the storage unit 108, and the horizontal smear region and the horizontal smear component of SR are acquired. Further, by calculating the horizontal smear component of the horizontal smear region SL from the change of the horizontal smear component of SR, the horizontal smear component (FIG. 4c) generated on the right side of the high-luminance subject AL is detected.

Then, both the left and right lateral smear components (FIG. 4d) of a single high-luminance subject are calculated from the left lateral smear component (FIG. 3c) and the right lateral smear component (FIG. 4c) of the high-luminance subject.

Since two high-brightness subjects exist at the HR and HL positions in the recording image (FIG. 2a), the horizontal smear component of the high-brightness subject at the HL position (FIG. 2c) and the horizontal smear of the high-brightness subject at the HR position. The horizontal smear component (FIG. 2e) when there are two high-luminance subjects is calculated from the component (FIG. 2d) and used as the horizontal smear correction amount.

After that, the horizontal smear correction unit 110 corrects the image by subtracting the horizontal smear correction amount calculated by the horizontal smear correction amount calculation unit 109 from the recording image (FIG. 2a), and the horizontal smear correction unit 110 corrects the image (FIG. 2g). ) Is output (509).

The imaging flow of 501 to 506 and the correction flow of 507 to 509 need not be continuously performed, and need not be performed in the same imaging device.

In this way, in an image pickup device that has two optical systems and forms an image in parallel on one image sensor, the horizontal smear correction is performed with high accuracy by shielding the optical systems one by one and obtaining the horizontal smear component. Will be possible.

Next, the operation of the first embodiment of the present invention will be described with reference to FIGS. 1, 2, 3, and 4 according to the flowchart of FIG.

The flow of this embodiment is to detect the presence or absence of a high-brightness subject in advance and enable lateral smear correction at the same time when the recording image is captured.

Since the configuration shown in FIG. 1 and the method of horizontal smear correction shown in FIGS. 2, 3, and 4 are the same as those in the first embodiment, the description thereof will be omitted.

FIG. 6 is an image pickup operation flow of the image pickup apparatus when the user acquires an image for correcting the lateral smear at the time of shooting.

First, the image pickup device performs an LV operation for imaging and displaying (601) before the user gives an image pickup instruction for image recording. Even during the LV operation, the image sensor outputs an image for LV. The high-brightness subject detection unit 106 detects whether or not a high-brightness subject that causes lateral smear is present in the image signal output from the image sensor 105 (602).

If the high-luminance subject does not exist, the LV operation is continued as it is. When a high-brightness subject exists, the horizontal smear generation information indicating that the high-brightness subject exists and the position information of the high-brightness subject are stored in the storage unit 108 (603), and the LV operation is continued. It is determined whether or not there is an imaging instruction from the user during the LV operation (604). When an image pickup instruction is given, it is confirmed whether or not the storage unit 108 has information on the occurrence of horizontal smear.

After that, the light-shielding unit 104 is shielded from light, and an image for detecting lateral smear that has passed only through the optical unit 101 is captured (505). Since the left image is shielded from light, an image corresponding to FIG. 3a is captured and stored in the storage unit 108. Further, the position of the high-brightness subject is detected by the high-brightness subject detection unit 106, and the position information is added.

After that, the light-shielding unit 103 is shielded from light, and an image for detecting lateral smear that has passed only through the optical unit 102 is captured (506). Since the right image is shielded from light, an image corresponding to FIG. 4a is captured and stored in the storage unit 108. Further, the position of the high-brightness subject is detected by the high-brightness subject detection unit 106, and the position information is added.

Next, the horizontal smear correction amount generation unit 109 generates the horizontal smear correction amount (608). Specifically, first, the horizontal smear component detection image (FIG. 3a) whose left side is shaded is read out from the storage unit 108, and the horizontal smear components of the horizontal smear regions SL and SO are acquired. Further, by calculating the horizontal smear component of the horizontal smear region SR from the changes in the horizontal smear components of SL and SO, the horizontal smear component (FIG. 3c) generated on the left side of the high-luminance subject AR is detected.

Next, the horizontal smear component detection image (FIG. 4a) whose right side is shaded is read out from the storage unit 108, and the horizontal smear region and the horizontal smear component of SR are acquired. Further, by calculating the horizontal smear component of the horizontal smear region SL from the change of the horizontal smear component of SR, the horizontal smear component (FIG. 4c) generated on the right side of the high-luminance subject AL is detected.

Then, both the left and right lateral smear components (FIG. 4d) of a single high-luminance subject are calculated from the left lateral smear component (FIG. 3c) and the right lateral smear component (FIG. 4c) of the high-luminance subject.

Since there are two high-intensity subjects at the HR and HL positions in the recording image (FIG. 2a),
From the horizontal smear component of the high-brightness subject at the HL position (Fig. 2c) and the horizontal smear component of the high-brightness subject at the HR position (Fig. 2d), the horizontal smear component (Fig. 2e) when there are two high-brightness subjects is calculated. Let this be the amount of horizontal smear correction.

Next, the imaging device captures a recording image (501). At this time, the light-shielding portions of 103 and 104 do not block light, pass through the optical units 101 and 102, and capture the image formed on the image sensor 105.

After that, the horizontal smear correction unit 110 corrects the image by subtracting the horizontal smear correction amount calculated by the horizontal smear correction amount calculation unit 109 from the recording image (FIG. 2a), and the horizontal smear correction unit 110 corrects the image (FIG. 2g). ) Is output (509).

The image output from the horizontal smear correction unit 110 is displayed on the display unit 112 after the camera signal processing unit 111 performs camera signal processing such as development, and is recorded on the machine f6 unit 113. The above steps 604 to 609 are performed within a sufficiently short time in which the position of the subject does not change.

In this way, in an image pickup device that has two optical systems and forms an image in parallel on one image sensor, the horizontal smear correction is performed with high accuracy by shielding the optical systems one by one and obtaining the horizontal smear component. Will be possible. Further, in this embodiment, the presence or absence of a high-brightness subject is detected in advance, and the horizontal smear correction is performed at the same time when the recording image is captured, so that the image with the horizontal smear corrected is displayed to the user at the time of shooting. Will be possible.

101, 102 Optical unit 103, 104 Light-shielding unit 105 Image sensor 106 High-intensity subject detection unit 107 Control unit 108 Storage unit 109 Horizontal smear correction amount generation unit 110 Horizontal smear correction unit 111 Camera signal processing unit 112 Display unit 113 Recording unit

Claims (2)

A plurality of optical units, a light-shielding unit that can independently block each optical unit, and a light-shielding unit.
One image sensor on which an image formed by the plurality of optical systems is projected, and
High-intensity subject detection means for detecting high-intensity subjects in captured images,
A storage unit that stores captured images and
A horizontal smear correction amount generator that detects the horizontal smear component from the image and calculates the horizontal smear correction amount,
In an imaging device having a horizontal smear correction unit that corrects a captured image using a calculated horizontal smear correction amount.
When a high-brightness subject is present in the captured image, some of the multiple optical systems are shielded, and the horizontal smear correction amount is calculated from the shaded image of the image sensor to perform horizontal smear correction. An image pickup device characterized by. The imaging device according to claim 1, wherein the optical unit has two and is arranged in parallel with parallax so that stereoscopic photography is possible.

Images