JP2022016112A - Imaging device - Google Patents

Abstract

To provide an imaging device that can correct a focus according to a cut-out region.SOLUTION: The imaging device of the present invention includes: imaging means; focusing means for driving a focus lens and performing a focusing operation; focus region setting means for setting a focus region for performing the focusing operation; cut-out region setting means for setting a cut-out region from an image acquired by the imaging means; and correction means for performing a focus correction on the basis of a focus displacement amount in the position of the focus region and in the position of the cut-out region when the cut-out region is cut out.SELECTED DRAWING: Figure 3

Description

The present invention relates to an image pickup apparatus.

In recent years, the number of pixels in cameras has been increasing, and the lineup has been increasing from full HD image quality to 4K and 8K compatible cameras. As described above, as the number of pixels increases, it becomes possible to shoot a subject with high definition, and even when a part of an image is cut out and enlarged and displayed, it is possible to display the image with sufficient resolving power. Therefore, it has been devised in various fields to cut out and use a part of an image of a multi-pixel camera.

Further, conventionally, it is known that the out-of-focus blur occurs due to the inclination when the sensor is attached to the housing or the lens is attached to the housing. As shown in FIG. 14A, when the lens in the camera and the sensor are facing each other, the above-mentioned out-of-focus does not occur. However, as shown in FIGS. 14B and 14C, when either the sensor or the lens is tilted, there may be a portion where the focus state is not good even if a subject at the same distance is photographed. For example, even when the subject is focused at the center of the image, the focus may be out of focus at the peripheral portion of the image. Specifically, with a camera in which the lens and the sensor are not tilted, an image in focus is taken in the entire imaging range as shown in FIG. 15A. However, in a camera in which the lens or sensor is tilted, the central portion is in focus as shown in FIG. 15B, but the upper right portion is not in good focus.

As described above, as the number of pixels in the camera increases, it becomes possible to shoot with high definition, but there is a problem that the focus state of a part of the image is not good due to the tilt of the sensor and the lens at the time of mounting. Is assumed.

In order to solve such a problem, Patent Document 1 includes a mechanism for adjusting the focus position for each cutout region.

JP-A-2017-129871

However, the technique disclosed in Patent Document 1 corrects the focus position so that the contrast values of the image center and the cut-out position are equal to each other based on the contrast value of each cut-out position, and the focus position is corrected for different subjects. It is difficult to match the contrast.

Therefore, an object of the present invention is to provide an image pickup apparatus capable of focus correction suitable for a region to be cut out.

The image pickup apparatus of the present invention includes an image pickup means, a focusing means for driving a focus lens to perform a focusing operation, and a focusing means.
The focusing area setting means for setting the focusing area for performing the focusing operation, the cutting area setting means for setting the area to be cut out from the captured image acquired by the imaging means, and the focusing area when cutting out the cutting area. It is characterized by having a correction means for performing focus correction based on the position of the region and the amount of focus shift at the position of the cutout region.

According to the present invention, it is possible to provide an image pickup apparatus capable of performing focus correction suitable for a region to be cut out.

Configuration diagram of the image pickup device Image of out-of-focus due to tilt when attaching the sensor or lens to the main body Flowchart in Example 1 The figure which shows the shooting scene in Example 2. Flowchart in Example 2 The figure which shows the shooting scene in Example 3. Flowchart in Example 3 The figure which shows the relationship between a focus lens and a sensor Flow chart in Example 4 The figure which shows the positional relationship in the movement of a focus lens and a zoom lens Flowchart in Example 5 The figure which shows the shooting scene in Example 6. Flowchart in Example 6 The figure which shows the relationship between a sensor and a lens Impact on captured images due to the presence or absence of out-of-focus due to tilt when the sensor or lens is attached to the main body Example of impact on captured images

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram of an image pickup apparatus according to the present invention.

The image pickup device includes a lens group 301, an optical filter 302, an aperture 303, a color filter 304, an image pickup element 305, an AGC 306, an A / D conversion 307, and a video signal processing unit 308. The image pickup apparatus further includes a video signal output unit 309, an exposure control unit 310, an optical control unit 311 and a control setting unit 313. Further, the image pickup apparatus is connected to an external setting means 312 and a monitor (not shown) for displaying the captured image via a network.

The lens group 301 is an optical system that collects the incident light from the subject on the image pickup device 305. The lens group 301 includes a focus lens that focuses on the subject, a zoom lens that adjusts the angle of view, and the like. The light that has passed through the lens group 301 passes through the optical filter 302 and enters the image pickup apparatus main body.

The aperture 303 adjusts the amount of light incident on the image sensor 305. The optical filter 302 is, for example, an infrared cut filter (IRCF) or the like. Then, the video information for which the amount of light has been adjusted passes through the color filters 304 arranged in a predetermined order for each pixel on the light receiving surface of the image sensor 305, and is received by the image sensor 305.

The image pickup device 305 outputs the captured image information of the image pickup target as an analog signal. The image formed on the image sensor 305 is gain-controlled by the AGC 306 to adjust the brightness of the image signal. Further, the A / D conversion unit 307 converts the analog image pickup signal of the image whose brightness is adjusted by the AGC 306 into a digital signal.

The video signal processing unit 308 performs predetermined processing on the digital image pickup signal from the A / D conversion unit 307 and outputs a brightness signal and a color signal for each pixel. The video signal processing unit 308 creates an image for output and controls the camera. Each parameter for doing is created. Examples of the parameters for controlling the camera include those used for aperture control, AF evaluation value which is a frequency component value for focusing, and white balance control for adjusting color.

The video signal output unit 309 outputs the video signal created by the video signal processing unit 308 to the outside. The exposure control unit 310 calculates the brightness information in the shooting screen from the brightness information output from the video signal processing unit 308, and controls the aperture 303 and the AGC 306 to adjust the shot image to a desired brightness. Further, the brightness may be adjusted by adjusting the accumulation time of the image pickup device 305 by adjusting the shutter speed.

The optical control unit 311 (focusing means) controls the lens group 301. Focus adjustment (focusing operation) is performed by controlling the lens group 301 by the optical control unit 311 so that the AF evaluation value becomes maximum. The high frequency component is extracted from the video signal created by the video signal processing unit 308, and the value of the high frequency component is used as the focus focus information (AF evaluation value).

The control setting unit 313 (focusing area setting means, cutting area setting means, priority setting means) is a means for actually performing control commands for the camera sent from the external setting means 312, and is used for exposure control and lens control. Make settings, etc. In addition, the control setting unit 313 also performs correction processing associated with the operation set by the external setting means 312. The control setting unit 313 sets the focusing area for performing the focusing operation, and sets the cutting area described later according to the user's instruction.

The external setting means 312 is connected to the image pickup apparatus via a network and is a setting means for operating the image pickup apparatus. For example, so-called general camera operations such as focusing, brightness specification, and zoom magnification specification are performed. The external setting means may be provided in the image pickup apparatus. Hereinafter, examples will be described based on the camera configuration shown in FIG.

In the first embodiment, the focus correction according to the cutting position will be described in detail. FIG. 2 is an example of an image in which out-of-focus occurs due to tilt when the sensor or lens is attached to the main body. FIG. 3 is a flowchart of the first embodiment.

As shown in FIG. 2, a case will be described in which the rectangular dotted line area 2 at the upper right of the screen is cut out and displayed or saved while the focus is on the substantially center position (rectangular dotted line area 1) of the screen.

As shown in FIG. 2, in the state of focusing at the substantially center position (rectangular dotted line area 1) of the screen, the focused state is not good in the rectangular dotted line area 2 on the upper right of the screen, and out-of-focus occurs. This is because, as described above, out-of-focus occurs due to the tilt of the lens or sensor when it is attached to the main body.

Therefore, a method for obtaining a more suitable image when cutting out an arbitrary area in an image (screen) will be described with reference to the flowchart of FIG.

First, the focusing area is set when shooting (S501). The focusing area may be set automatically by AF, or the focusing area may be set manually by the user. Then, the optical control unit 311 controls the lens group 301 to focus on the focusing area. Next, a cutout area to be cut out from the shot image within the shooting range is set (S502). The cutout area is designated by the user by the external setting means 312.

Then, the amount of focus shift between the in-focus area and the cut-out area is calculated (S503). At this time, as the amount of deviation, for example, the focus difference of a plurality of points is stored in advance as data inside the image pickup apparatus, and the focus difference is calculated by converting from the positional relationship between the focusing area and the cutting area. Alternatively, when the cutout area is determined, the focus difference may be obtained by actually driving the focus lens and obtaining the in-focus position for each area.

Next, it is determined whether or not to cut out the area (S504). Specifically, it is determined whether or not the user has set (permitted) the cutout function. When the area is not cut out, that is, when the cutout function is not set by the user, the focused state is maintained in focus (S505).

On the other hand, when it is determined in S504 that the area is to be cut out, the position of the focus lens is corrected by the amount of focus correction obtained in S503 (S505). When the area is not cut out (No in S504), the whole image is delivered to the external device, and when the area is cut out (Yes in S504), the whole image and the cut-out image or the cut-out image is the external device. Will be delivered to.

By performing the above processing, it becomes possible to perform focus correction for out-of-focus due to tilt when the sensor or lens is attached to the main body. Therefore, it is possible to perform focus correction so as to suppress blurring due to the influence of tilt when the sensor or lens is attached to the main body of an arbitrary part of the image, and suitable shooting becomes possible.

Further, the focus correction amount may be further added depending on the setting status of the image pickup apparatus. Specifically, more suitable shooting can be achieved by changing the focus correction amount according to the change in the focus position due to the change in the aperture and the zoom position.

Example 2 shows a method for performing a more suitable focus correction than that of the first embodiment. FIG. 4 shows an example of a shooting scene. In FIG. 4, two subjects are shown, the subject 601 in the foreground is in focus, and the distant subject 602 farther than the subject 602 is located outside the depth of field and is in focus. It's a bad scene. Consider a case where an attempt is made to focus on a distant subject 602 from a state in which the subject 601 in the foreground is in focus. Since the method of the first embodiment only corrects the focus by the tilt of the sensor and the lens, it is conceivable that the amount of movement of the focus lens is insufficient for shooting the subject 602 to be actually focused.

Therefore, in this embodiment, it is determined whether the focus is corrected for the tilt of the sensor or the lens or the focus is refocused, and shooting suitable for the subject to be shot at the time of cutting is performed. This method will be described with reference to FIG. FIG. 5 shows a flowchart of the second embodiment.

First, when shooting, the focusing area is set (S701). Then, the optical control unit 311 controls the lens group 301 to focus on the focusing area. Next, a cutout area to be cut out from the shot image within the shooting range is set (S702). Then, the difference between the subject distance in the focusing area and the subject distance in the cutout area is calculated (S703). As a method for calculating the subject distance, various methods such as a method using a distance sensor and a method for determining from an image are known, and it can be obtained by using a known method.

Next, it is determined whether or not to cut out the area (S704). When the area is not cut out, the focused area is maintained in focus (S705).

On the other hand, when it is determined in S704 that the area is to be cut out, it is determined whether or not the difference in the distances of the subjects obtained in S703 is equal to or greater than a predetermined amount. (S706). When the difference in the subject distance is smaller than a predetermined amount, it is considered to be affected by the tilt of the sensor or the lens, and the same focus correction as in the first embodiment is performed (S707). When it is determined that the difference between the subject distances is equal to or greater than a predetermined amount, it is determined that the subject 602 to be focused requires focus correction different from the influence of the tilt of the sensor or the lens. Therefore, the optical control unit 311 controls the lens group 301 to perform a refocusing process for focusing on the cutout area so as to focus on the subject 602 in the cutout area (S708). When the area is not cut out (No in S704), the whole image is delivered to the external device, and when the area is cut out (Yes in S704), the whole image and the cut-out image, or the cut-out image is the external device. Will be delivered to.

By performing the above processing, more suitable shooting according to the cutting area becomes possible.

Example 3 shows a method for performing suitable focus correction when there are a plurality of cutout areas. FIG. 6 is an example of a shooting scene in the third embodiment. Three cut-out areas are set, and it is assumed that a plurality of cut-out areas are delivered at the same time instead of switching the cut-out areas according to the positions specified by the user as in the past. That is, it is assumed that if only one of the regions is prioritized, the focus state will not be good in the other regions.

The processing in such a case will be described with reference to the flowchart of FIG. FIG. 7 shows a flowchart of the third embodiment.

First, the focusing area is set when shooting (S901). Then, the optical control unit 311 controls the lens group 301 to focus on the focusing area. Next, a cutout area to be cut out from the shot image within the shooting range is set (S902). In FIG. 6, this cutout area is shown by an area surrounded by three rectangles. Then, the difference between the subject distance in the focusing area and the subject distance in the cutout area is calculated (S903). As a method for calculating the subject distance (distance determination means), various methods such as a method using a distance sensor and a method for determining from an image are known, and it can be obtained by using a known method. .. Then, the focus correction amount for each of the three cutout areas is calculated from the distance difference information.

Next, it is determined whether or not to cut out the area (S904). When the area is not cut out, the focused area is maintained in focus (S905).

On the other hand, if it is determined in S904 that the three areas are to be cut out, the weighting of each cutout area is adjusted (S906). This weighting is a coefficient that determines the correction amount of which area is prioritized and focused on each cutout area. The weighting itself can be arbitrarily determined by the user, and of course, it is sorted by various factors such as the distance from the focusing area, the size of the cutout area, and the characteristics (person) of the subject. Further, based on this weighting, it is conceivable to set the intermediate focus position of a plurality of cutout areas. Then, the focus is corrected according to the weighting (S908). When the area is not cut out (No in S904), the entire image is delivered to the external device, and when the area is cut out (Yes in S904), the whole image and three cutout images, or 3 The cut-out image of the place is delivered to the external device.

As described above, when a plurality of areas are cut out at the same time, it is possible to cut out a video according to the intention of the user by giving priority to each area.

Example 4 shows a method for enabling more suitable focus adjustment by changing the focus movable range based on the amount of out-of-focus according to the cut-out area. FIG. 8 shows a simplified positional relationship between the focus lens and the sensor. Ideally, as shown in FIG. 14, it is desirable that the lens and the sensor face each other (FIG. 8A).

However, for example, as shown in FIG. 8B, when the sensor is tilted at an angle, the focus position shifts within the same image as described above. This shift in the focus position is assumed to be the case where the focus lens cannot move in the area at the end of the focus movable range and the focus cannot be adjusted (the part where the area is insufficient in the figure). In such a case, focus cannot be obtained within the preset movable range under the condition that there is no out-of-focus due to the tilt of the sensor.

Therefore, in this embodiment, a method of the focus adjustment mechanism in consideration of blurring due to the tilt of the sensor will be described with reference to FIG. FIG. 9 is a flowchart of the fourth embodiment.

First, the amount of out-of-focus due to the tilt of the lens and the sensor is calculated (S1101). This out-of-focus amount can be calculated by a method such as storing in advance the amount of deviation from the reference position in the shooting screen. For example, a configuration is conceivable in which the center position of the image is used as a reference position and the correction amount is stored in advance so that the tilt blur amount is provided for each image height.

Then, the cutout area in the captured image is set (S1102). The focus range of motion is changed based on the amount of out-of-focus according to the set cutout area (S1103). Then, the focus position is adjusted according to the change in the movable focus (S1104).

By doing so, it is possible to suppress the inability to focus due to out-of-focus due to tilt. Further, in this embodiment, an example of setting the focus movable range according to the setting of the cutout area has been described, but the focus movable range may be changed regardless of the cutout area from the amount of out-of-focus due to the tilt obtained in advance. ..

Example 5 shows a suitable method of zooming operation according to the cutting area. FIG. 10 shows the relationship between the moving positions of the focus lens and the zoom lens, and the horizontal axis indicates the zoom position and the vertical axis indicates the focus position.

By moving the focus lens and the zoom lens in accordance with the curve (cam curve) shown in FIG. 10, it is possible to maintain the in-focus state. Therefore, when the zoom operation is performed along the curve of the focusing area in FIG. 10, it is conceivable that the focus position of the cutout area shifts. Therefore, as shown in FIG. 10, the focus of the cut-out area is focused by changing this curve according to the focus correction amount considering the tilt of the sensor or the lens in the cut-out area with respect to the in-focus area to be focused. Can be maintained. It should be noted that the creation of the cam curve is not limited to any method because various methods are known from conventional cameras.

Next, the above will be described with reference to the flowchart of FIG. FIG. 11 is a flowchart of the fifth embodiment. First, the focusing area is set when taking a picture with the camera (S1301). Then, the optical control unit 311 controls the lens group 301 to focus on the focusing area. Next, a cutout area to be cut out from the shot image within the shooting range is set (S1302). Then, the amount of focus shift between the focusing area and the cutting area is calculated (S1303).

Next, it is determined whether or not to perform zooming at the cutout position of the area (S1304). When zooming is not performed at the cutout position of the area, the zooming operation is performed with reference to the focus of the focusing area (S1305).

On the other hand, in S1304, when it is determined that the zoom is performed at the cutout position of the area, the cam curve is determined based on the focus shift amount calculated in S1303 (S1306). After that, the lens is driven according to the curve determined in S1306 (S1307).

This makes it possible to perform a suitable zoom operation according to the cutting area.

The sixth embodiment shows an example in which the depth of field, which is the in-focus width, is adjusted according to the degree of the out-of-focus when there is out-of-focus due to the influence of the inclination when the sensor is attached.

An example is shown with reference to FIG. If there is no out-of-focus due to the influence of the tilt when the sensor is attached, the person or flower in the figure looks in focus because the vertical solid line of the depth of field width A looks in focus. On the other hand, if there is an influence of the inclination when the sensor is attached, even if the width A has the same depth of field, the width becomes a diagonal dotted line, so that the flower is out of focus. Therefore, by changing the width of the depth of field from A to B in consideration of the influence of the tilt when the sensor is attached, it is possible to shoot in focus in both cases.

Next, the flowchart of the sixth embodiment will be described with reference to FIG. FIG. 13 is a flowchart of the sixth embodiment. First, when taking a picture with the camera, the focusing area is set (S1501). After that, the optical control unit 311 controls the lens group 301 to focus on the focusing area. Next, a cutout area to be cut out from the shot image within the shooting range is set (S1502).

Then, the amount of focus shift between the focusing area and the cutting area is calculated (S1503). After that, the aperture diameter of the diaphragm is changed based on the amount of focus shift (S1504). That is, the imaging conditions are changed. Since it is possible to change the width of the depth of field by changing the aperture, it is possible to take a picture that is more in focus in both the in-focus area and the cut-out area.

In addition, although an example of adjusting the aperture is shown here, the position of the focus lens can be corrected from the amount of focus shift in the in-focus area and the cut-out area so that the focus position is set between the two areas. Is also possible.

By doing so, it becomes possible to take a more suitable image for a subject that is out of focus due to the influence of the tilt when the sensor is attached.

Claims (7)

Imaging means and
Focusing means that drives the focus lens to perform focusing operation,
Focusing area setting means for setting the focusing area for focusing operation,
A cutout area setting means for setting a region to be cut out from an image captured by the image pickup means, and a cutout area setting means.
An image pickup characterized by having, when cutting out the cutout region, a correction means for determining a focus correction amount based on a position of the focus region and a focus shift amount at the position of the cutout region and performing focus correction. Device. It has a distance determining means for determining the distance to the subject, and has
The image pickup apparatus according to claim 1, wherein when the difference between the subject distance in the in-focus region and the subject distance in the cut-out region is a predetermined amount or more, the in-focus operation is performed in the cut-out region. When the difference between the subject distance of the in-focus area and the subject distance of the cut-out area is smaller than a predetermined amount, the focus correction is performed based on the amount of focus shift between the position of the in-focus area and the position of the cut-out area. 2. The imaging device according to claim 2. It has a priority setting means for setting a priority for each cutout area.
The first aspect of claim 1, wherein when a plurality of cutout areas are set by the cutout area setting means, the focus correction amount is set based on the priority set by the priority setting means. Imaging device. The image pickup apparatus according to claim 1, wherein the movable range of the focus lens is changed according to the focus correction amount. The image pickup apparatus according to claim 1, wherein when the focus correction is performed, the focus position corresponding to each zoom position is changed. The imaging device according to claim 1, wherein the imaging conditions are changed based on the amount of focus shift.

Images