WO2024024375A1

WO2024024375A1 - Imaging device, imaging method, and imaging program

Info

Publication number: WO2024024375A1
Application number: PCT/JP2023/023851
Authority: WO
Inventors: 勇起吉留
Original assignee: ソニーグループ株式会社
Priority date: 2022-07-27
Filing date: 2023-06-27
Publication date: 2024-02-01

Abstract

This imaging device comprises a processing unit that controls an optical system for imaging. The control of the optical system by the processing unit includes adjusting the focusing position of the optical system so that the defocus values of each of a nearest subject and a farthest subject among a plurality of subjects approach an intermediate value of each of the defocus values.

Description

Imaging device, imaging method, and imaging program

The present disclosure relates to an imaging device, an imaging method, and an imaging program.

For example, as disclosed in Patent Document 1, various techniques regarding focus position adjustment when capturing images of a plurality of subjects have been proposed.

Japanese Patent Application Publication No. 2001-116980 Japanese Patent Application Publication No. 2022-16546

In order to include all subjects in the depth of field, it is necessary to increase the F value (aperture value), but increasing the F value too much may cause problems. Since the specific F value changes depending on the focus position, it is also necessary to consider adjusting the focus position. No specific studies have been conducted so far regarding focusing position adjustment that allows multiple subjects to be imaged at appropriate F-numbers.

One aspect of the present disclosure enables focus position adjustment suitable for imaging multiple subjects.

An imaging device according to an aspect of the present disclosure includes a processing unit that controls an optical system for imaging, and the processing unit controls the optical system for each of the nearest object and the farthest object among a plurality of objects. This includes adjusting the focusing position of the optical system so that the focus value approaches the intermediate value of the respective defocus values.

In an imaging method according to one aspect of the present disclosure, an optical system for imaging is configured such that defocus values of the closest object and the farthest object among a plurality of objects approach an intermediate value of the respective defocus values. This includes adjusting the focus position.

An imaging program according to one aspect of the present disclosure causes a processor to perform an imaging program such that defocus values of the closest object and the farthest object among the plurality of objects approach an intermediate value of the respective defocus values. A process for adjusting the focusing position of the optical system is executed.

1 is a diagram illustrating an example of a schematic configuration of an imaging device according to an embodiment. 1 is a diagram showing an example of a schematic configuration of an image sensor. FIG. 3 is a diagram illustrating an example of functional blocks of a processing unit. FIG. 3 is a diagram illustrating an example of subject selection. FIG. 3 is a diagram illustrating an example of subject selection. FIG. 3 is a diagram illustrating an example of subject selection. FIG. 3 is a diagram illustrating an example of subject selection. It is a figure which shows the example of a focus position and F value after adjustment. FIG. 3 is a diagram for explaining calculation of an F value. 2 is a flowchart illustrating an example of processing (imaging method) executed by the imaging device. 2 is a flowchart illustrating an example of processing (imaging method) executed by the imaging device.

Below, embodiments of the present disclosure will be described in detail based on the drawings. In addition, in each of the following embodiments, the same elements are given the same reference numerals to omit redundant explanation.

The present disclosure will be described according to the order of items shown below.
0. Introduction 1. Embodiment 2. Modification example 3. Example of effect

0. Introduction When photographing multiple subjects, until now, the focus position and F value have often been manually adjusted based on knowledge and experience. Such adjustments are not easy, and are almost impossible when shooting a video. Increasing the F value too large in order to include all multiple subjects in the depth of field may cause problems. For example, the background may be clearly reflected, or the image quality may deteriorate due to noise or the like because the image sensor is set to a high-sensitivity state. In particular, no specific solution has been found so far when the distance between subjects is large.

The optimal focusing position where multiple subjects can be imaged with the minimum F value may be in front of the intermediate position of each subject (on the imaging device side). It is also known to try to adjust the focus position to approximately the middle position of each subject, but in this case, the setting deviates from the optimal setting, especially when the front and rear positions of the subjects are far apart. Attempting to optimize the parameters of both the focus position and the F-number increases the amount of calculation, and it becomes difficult to maintain optimal parameters, especially when photographing a fast-moving subject or when shooting a video.

At least some of the issues described above are addressed by the disclosed technology. For example, the focus position is adjusted so that the minimum F value corresponding to the depth of field including the plurality of selected subjects is obtained. It is also possible to select a subject or, conversely, to exclude a subject from selection through a user operation. It is also possible to select a blur priority mode in which the F value is further reduced while the in-focus position remains the same.

1. Embodiment FIG. 1 is a diagram showing an example of a schematic configuration of an imaging device according to an embodiment. An object to be imaged by the imaging device 1 is referred to as a subject 9 and illustrated. The subject 9 may be any object that can be imaged by the imaging device 1. The object 9 may be a part of an object, and in that case, the objects 9 may be different parts of the same object. The subject 9 may be a moving object; examples of such a subject 9 are a person, an animal, a vehicle, etc. There may be multiple subjects 9, and four subjects 9 are illustrated in FIG. Among the plurality of subjects 9, the closest subject 9 and the farthest subject 9 to the imaging device 1 are referred to as the closest subject 9 _near and the farthest subject 9 _far .

The imaging device 1 includes an optical system 2, an image sensor 3, a drive section 4, a display operation section 5, a storage section 6, and a processing section 7.

The optical system 2 is an imaging optical system provided for the image sensor 3 and guides light from the subject 9 to the image sensor 3. The optical system 2 is configured to be able to adjust the focus position and F value. For example, the optical system 2 is configured to include a lens mechanism for adjusting the focal position, an aperture diaphragm mechanism for adjusting the F value, and the like.

The image sensor 3 images the subject 9 via the optical system 2. An example of the image sensor 3 is a solid-state image sensor such as a CMOS image sensor. An image within the field of view of the imaging device 1 is captured by the imaging element 3. Specifically, as schematically shown by the white arrow in FIG. 1, light from the subject 9 enters the image sensor 3 via the optical system 2, and the subject 9 is thereby imaged. The image sensor 3 is configured to obtain the DF value of each of the plurality of subjects 9. The DF value is a value indicating the amount of focus shift (defocus amount) when the subject 9 is out of focus. The image sensor 3 will be explained with reference to FIG. 2 as well.

FIG. 2 is a diagram showing an example of a schematic configuration of an image sensor. The imaging device 3 includes a plurality of pixels 31 arranged in an array so as to define an imaging surface 30. For example, one pixel 31 outputs a pixel signal according to the amount of received light of any one of red light, green light, and blue light.

The plurality of pixels 31 include a plurality of imaging pixels 311 and a plurality of phase difference detection pixels 312. An image is generated based on at least the pixel signal from the imaging pixel 311. Although this is not an essential configuration, the illustrated phase difference detection pixel 312 is different from the imaging pixel 311 in that half of the light receiving area is shielded from light. A pair of phase difference detection pixels 312 whose portions opposite to each other in the array direction are shielded from light are arranged in each part of the image sensor 3. Based on the amount of light received by each of the pair of phase difference detection pixels 312, that is, the pixel signal level, the DF value of the subject 9 corresponding to that portion can be calculated.

Note that the image may be an image of one frame of a video, and the image may be read as a video as appropriate as long as there is no contradiction. Imaging may be read as photographing as appropriate.

Returning to FIG. 1, the drive section 4 drives the optical system 2. The drive unit 4 moves the lenses included in the optical system 2 to adjust the focus position of the optical system 2, and moves the aperture to adjust the F value of the optical system 2. The drive unit 4 includes, for example, an actuator and the like.

The display operation unit 5 displays information handled by the imaging device 1 and receives operations on the imaging device 1. For example, the display operation unit 5 displays a captured image. The display operation unit 5 also displays an operation screen and receives user operations. The display operation unit 5 includes, for example, a touch panel display.

The storage unit 6 stores information used by the imaging device 1. An example of the information stored in the storage unit 6 is an imaging program 61. The imaging program 61 causes the processing unit 7 to execute various processes in the imaging device 1 . The imaging program 61 can be distributed collectively or separately via a network such as the Internet. Further, the imaging program 61 can be recorded collectively or separately on various computer-readable recording media, and can be executed by being read from the recording medium by the computer.

The processing unit 7 is configured to include, for example, one or more processors. The imaging program 61 stored in the storage unit 6 is read out and expanded into the memory, so that the processing of the processing unit 7 according to the imaging program 61 is executed. The processing unit 7 directly or indirectly controls other elements in the imaging device 1, including the optical system 2. The processing unit 7 will be further explained with reference to FIG. 3.

FIG. 3 is a diagram showing an example of functional blocks of the processing section. The processing unit 7 includes an image generation unit 71, an object recognition unit 72, a subject selection unit 73, a distance measurement unit 74, a DF value calculation unit 75, an F value calculation unit 76, an optical system adjustment unit 77, A movement detection section 78 is included.

The image generation unit 71 generates an image based on pixel signals from the image sensor 3. Unless otherwise specified, it is assumed that the image includes a plurality of subject candidates located within the field of view of the imaging device 1. The subject candidate is different from the subject 9 in that it is a subject that has not yet been selected by a subject selection unit 73, which will be described later.

The object recognition unit 72 recognizes multiple subject candidates in the image generated by the image generation unit 71. Various known image recognition algorithms may be used. By automatically recognizing subject candidates, user operations can be simplified accordingly.

The subject selection unit 73 selects a plurality of subjects 9 from the plurality of subject candidates recognized by the object recognition unit 72. Selection may be understood to mean selection of the subject 9 to be focused. The subject 9 may be automatically selected by the subject selection unit 73, or may be manually selected by the user of the imaging device 1. In the case of automatic selection, the user's operation can be simplified, and in the case of manual operation, it is possible to respond to the user's arbitrary wishes. The selection of the subject 9 may be performed by cooperation between the subject selection section 73 and the display operation section 5. Some specific examples will be described with reference to FIGS. 4 to 7.

4 to 7 are diagrams showing examples of subject selection. The display operation unit 5 of the imaging device 1 displays an image including a plurality of subject candidates.

In the example shown in FIG. 4, the plurality of subject candidates are heads (including facial parts) of different people. The display operation unit 5 displays the plurality of subject candidates in a manner that allows them to be selected as the subject 9. In this example, each of the plurality of subject candidates is highlighted with a rectangular frame to facilitate selection by the user. Some of the subject candidates are selected as the subject 9 by user operations such as touching the screen. For example, as shown by the thick rectangular frames in FIG. 4B, two people are selected as the subjects 9. By changing the display mode of the frame before and after selection, it becomes easier to distinguish between them.

For example, as described above, the subject 9 can be selected by a simple user operation. As mentioned earlier, the subject selection unit 73 may automatically select the subject 9. For example, data such as an image of the subject 9 may be prepared in advance and stored (registered) in the storage unit 6. The subject selection unit 73 may select the registered subject 9 from among the plurality of subject candidates in the image.

A plurality of parts of one object may each be selected as the subject 9. In the example shown in FIG. 5, as indicated by circular markers, the head, neck, belly, elbow, hand, and knee of one person are each taken as the subject 9. selected. A skeleton as shown by lines connecting each part may also be selected as the subject 9.

One object may be divided into a large number of parts, and some of these parts may be selected as the subject 9. In the example shown in FIG. 6A, one dog is subdivided into a number of square frames, and two of these parts are selected as the subject 9. In the example shown in FIG. 6B, one airplane is subdivided into a large number of rectangular frames, and two of them are selected as the subject 9. Based on the principle described below, it is possible to perform focus position adjustment suitable for imaging an entire object.

The subject 9 may be selected by specifying a range in the image. In the example shown in FIG. 7, a part of the area in the image is filled in by the user's operation, and a subject candidate that overlaps the area is selected as the subject 9.

Returning to FIG. 3, the distance measuring section 74 measures the distance of the subject 9 selected by the subject selecting section 73. The distance from the imaging device 1 to the subject 9, more specifically, for example, the distance from the imaging surface 30 of the image sensor 3 to the subject 9 is measured. Various known ranging techniques may be used. Examples of ranging methods include triangulation, ToF (Time Of Flight), and the like. When using triangulation, the image sensor 3 may be a plurality of image sensors 3 used as a stereo sensor. When using ToF, the image sensor 3 may be configured to include pixels that output distance measuring light. In the indirect ToF method, distance measurement is performed based on the distribution of charges within the pixel 31 and the difference. Among the plurality of objects 9, the large object 9 with the shortest measured distance and the longest object 9 are identified as the closest object 9 _near and the farthest object 9 _far .

The DF value calculation unit 75 calculates the DF value of each of the plurality of subjects 9 based on the pixel signal from the phase difference detection pixel 312 of the image sensor 3. More specifically, the DF value calculation unit 75 calculates the DF value of the subject 9 corresponding to that portion based on the difference in level of the pixel signals from the pair of phase difference detection pixels 312. Since the DF value calculation itself using the phase difference detection pixels 312 is well known, detailed explanation will be omitted. For example, a calculation method as shown in Patent Document 2 may be used. In the present embodiment, the DF value calculation unit 45 also calculates an intermediate value between the DF value of the closest subject 9 _near and the DF value of the farthest subject 9 _far .

The F value calculation unit 76 calculates the F value of the optical system 2. Details will be described later.

The optical system adjustment section 77 adjusts the optical system 2. Adjustment of the optical system 2 includes adjustment of the focus position and adjustment of the F value. For example, a control signal specifying these parameters is transmitted from the optical system adjustment section 77 to the driving section 4, and the driving section 4 drives the optical system 2 according to the control signal. Note that, hereinafter, the focus position will also be referred to as focus position FP. Specifically, the optical system adjustment section 77 adjusts the optical system 2 so that the DF values of the closest object 9 _near and the farthest object 9 _far approach the intermediate value calculated by the DF value calculation section 75 described above. Adjust the focus position FP. For example, the relationship between the amount of change in the DF value and the amount of drive of the optical system 2, that is, the amount of movement of the focus position FP, is known in advance, and therefore, such adjustment of the focus position FP is possible.

Based on the focus position FP of the optical system 2 adjusted by the optical system adjustment section 77, the F value calculation section 76 calculates the minimum depth of field that includes the closest object 9 _near and the farthest object 9 _far . Calculate the F value. Such an F value is, for example, based on the position of the focusing position FP of the optical system 2 adjusted by the optical system adjustment section 77 and the positions of the closest subject 9 _near and the farthest subject 9 _far (for example, the distance measuring section 74). distance measurement results). Note that hereinafter, the depth of field will also be referred to as depth of field x.

The optical system adjustment unit 77 adjusts the F value of the optical system 2 so that it approaches the F value calculated by the F value calculation unit 76. The adjusted focus position FP and F value will be explained with reference to FIG. 8 as well.

FIG. 8 is a diagram showing an example of the focused position and F value after adjustment. Regarding the F value, the corresponding depth of field x is illustrated. The focus position FP is located between the closest subject 9 _near and the farthest subject 9 _far , more specifically, in this example, closer to the nearest subject 9 _near than the farthest subject 9 _far . The depth of field x corresponding to the F value is determined to be the minimum range including the closest subject 9 _near and the farthest subject 9 _far . That is, the closest subject 9 _near and the farthest subject 9 _far are located at one end and the other end of the range of depth of field x (field range). Such a focus position FP can be said to be one of the focus positions FP suitable for imaging a plurality of subjects including the closest subject 9 _near and the farthest subject 9 _far .

For example, some conventional techniques can only focus on an intermediate position even when the difference between the front and back of the subject 9 is large, making it impossible to obtain the optimal focus position FP. According to the present embodiment, it is possible to immediately adjust the focus position to bring the plurality of subjects 9 within the depth of field x at the theoretical minimum F value under any situation.

Note that minimizing the F value is not essential. While keeping the same focus position FP, the F value of the optical system 2 is adjusted so that it becomes even smaller (so that the depth of field x becomes larger) than the F value calculated by the F value calculation unit 76. Good too. It is also possible to take an image in a mode (focus priority mode) that allows a certain degree of blur and increases the depth of field x.

The calculation of the F value by the F value calculation unit 76 will be further explained with reference to FIG. 9.

FIG. 9 is a diagram for explaining calculation of the F value. As a component of the optical system 2, a lens 21, which is a condensing lens, is exemplified. The diameter of the lens 21 is shown as a diameter D. The focal length of the lens 21 is shown as a focal length f. The distance between the lens 21 and the focus position FP is shown as a distance a. The distance between the lens 21 and the imaging surface 30 of the image sensor 3 is shown as a distance b. Of the depth of field x, the portion from the focus position FP to the closest subject 9 _near is referred to as front depth of field x _near and illustrated. The portion from the focus position FP to the farthest subject 9 _far is referred to as a rear depth of field x _far and is illustrated.

Three types of optical paths are schematically shown by solid lines, dashed lines and dash-dotted lines. The solid line indicates the optical path passing through the focus position FP. The broken line indicates the optical path passing through the closest object 9 _near . The dashed line indicates the optical path passing through the farthest object 9 _far .

The distance between the light from the closest subject 9 _near (the light along the optical path indicated by the dashed line) and the light from the farthest subject 9 _far (the light along the optical path indicated by the dashed-dotted line) on the imaging surface 30 is expressed as the allowable circle of confusion system δ. It is called and illustrated. The permissible circle of confusion system δ may be approximately the same as the resolution of the image sensor 3. The distance between the imaging surface 30 of the image sensor 3 and the focal point of the light from the closest subject 9 _near is referred to as front depth of focus X _near in the drawing. The distance between the imaging surface 30 of the image sensor 3 and the focal point of the light from the farthest object 9 _far is referred to as a rear depth of focus X _far .

Due to the similarity of triangles, the following equations (1) and (2) hold true.

When the ratio of the front focal depth X _near and the rear focal depth X _far is calculated based on the above equations (1) and (2), it becomes the following equation (3). Note that δD is treated as a value sufficiently smaller than 1 (for example, about 10 ⁻³ ).

The calculation of the F value will be described. In the following formula, the F value is indicated as Fno. By definition, Fno=f/D. First, the following equation (4) is derived from the above equation (1).

Since f is sufficiently larger than Fδ, the front depth of focus X _near is calculated as shown in equation (5) below.

The rear depth of focus X _far is calculated in the same way, and has the same value as the front depth of focus X _near . The DF value is the amount _of blur on the imaging surface 30, and since the values of the front _depth of focus A certain Fno is calculated.

For example, the F value calculation unit 76 of the processing unit 7 calculates the F value as described above. Note that the permissible circle of confusion system δ can be said to correspond to the permissible amount of blur, so this value is changed to select a trade-off between the degree of focus of multiple subjects 9 and the degree of blur of the background, such as in the blur priority mode. be able to.

Returning to FIG. 3, the movement detection section 78 detects the movement of the subject 9 selected by the subject selection section 73. Various known detection algorithms may be used. For example, even if the subject 9 moves, the subject 9 can be tracked. Detection may be performed for each image cycle. In the case of video shooting, the movement of the subject 9 may be detected for each frame, for example. Detection of movement by the movement detection unit 48 may include detection of a movement direction of the subject 9, for example, a direction toward the imaging device 1 and a direction away from the imaging device 1.

In one embodiment, the optical system adjustment section 77 controls the optical system 2 based on the detection result of the movement detection section 78. For example, each time movement of the subject 9 is detected by the movement detection unit 78, the focus position FP and F value of the optical system 2 may be adjusted (updated) as described above. By tracking the subject 9 and controlling the optical system 2, optimal adjustment of the focal position FP and F value of the optical system 2 can be maintained. For example, while shooting a video, the focus position FP and F value of the optical system 2 can be continuously changed smoothly.

FIG. 10 is a flowchart illustrating an example of processing (imaging method) executed by the imaging device. Since the specific contents of each process have been explained so far, detailed explanation will not be repeated.

In step S1, multiple subjects are selected. The subject selection unit 73 of the processing unit 7 selects a plurality of subjects 9 from a plurality of subject candidates.

In step S2, the focus position is adjusted so that the DF value becomes an intermediate value. The distance measuring section 74 of the processing section 7 measures the distances of each of the plurality of selected subjects 9. Based on the distance measurement results, the closest subject 9 _near and the farthest subject 9 _far are identified. The DF value calculation unit 75 calculates the DF value of the closest subject 9 _near and the farthest subject 9 _far , and also calculates an intermediate value thereof. The optical system adjustment unit 77 adjusts the focusing position FP of the optical system 2 so that the DF values of the closest subject 9 _near and the farthest subject 9 _far approach their intermediate values.

In step S3, the F value is adjusted. The F value calculation unit 76 of the processing unit 7 calculates the closest subject 9 _near and the farthest subject 9 _far based on the position of the nearest subject 9 _near , the position of the farthest subject 9 _far , and the adjusted focus position FP. Calculate the minimum F value that yields a depth of field x that includes The optical system adjustment unit 77 adjusts the F value of the optical system 2 so that it approaches the F value calculated by the F value calculation unit 76.

For example, as described above, the focal position FP and F value of the optical system 2 are adjusted so that the depth of field x that includes all of the plurality of selected subjects 9 is obtained. The F value is adjusted to the minimum value. Although not shown in the figure, imaging is performed with the parameters of the optical system 2 optimally adjusted in this way.

2. Modifications The disclosed technology is not limited to the above embodiments. Some modifications will be described.

The adjustable F value is limited by the configuration of the optical system 2 and the like. It may not be possible to include all of the selected subjects 9 in the depth of field x. In that case, the subject selection unit 73 of the processing unit 7 may exclude the farthest subject 9 _far from the selection targets. This will be explained with reference to FIG.

FIG. 11 is a flowchart illustrating an example of processing (imaging method) executed by the imaging device. The process in step S11 is the same as step S1 in FIG. 10 described above, so the explanation will be omitted.

In step S12, it is determined whether all of the selected subjects can be included in the depth of field. For example, the subject selection unit 73 of the processing unit 7 determines that the distance between the closest subject 9 _near and the farthest subject 9 _far is the upper limit of the F value of the adjustable optical system 2, that is, the upper limit of the depth of field x. If it is larger than the value, it is determined that all of the selected subjects 9 cannot be included in the depth of field x. If it is determined in this way (step S12: Yes), the process proceeds to step S14. If not (step S12: No), the process proceeds to step S13.

In step S13, the farthest subject is excluded from selection. This process is executed, for example, by the subject selection unit 73 of the processing unit 7. The excluded farthest subject 9 _far returns to the state before selection, that is, to a subject candidate. The remaining objects 9 remain as selection targets, and the farthest object 9 among them becomes the new farthest object 9 _far . After that, the process returns to step S12.

By going through the processing of steps S12 and S13 above, the number of subjects 9 to be selected is narrowed down until all the subjects 9 to be selected are included in the depth of field x. In this state, the process proceeds to step S14.

The processing in step S14 and step S15 is the same as the processing in step S2 and step S3 in FIG. 10 described above, so the explanation will be omitted.

According to the process in the above flowchart, it is possible to deal with the case where the F value of the optical system 2 is limited.

The adjustment of the focus position FP of the optical system 2 by the optical system adjustment unit 77 of the processing unit 7 is not limited to the adjustment described above (first adjustment), but also adjusts the focus position FP to a different position. It may also include a second adjustment. The second adjustment may be an adjustment of the focus position FP using various known methods. For example, in the second adjustment, the focus position FP of the optical system 2 is adjusted so that the focus position FP of the optical system 2 approaches the farthest subject 9 _far or the subject 9 located near the center of the image. be done.

The first adjustment and the second adjustment may be switched automatically or by user operation. By switching and using a plurality of adjustments, flexibility in adjusting the focus position can be increased. For example, the first adjustment and the second adjustment are switched depending on whether a specific subject 9 enters or exits the field of view. As an example, the first adjustment may be used when a particular subject 9 is within the field of view, and the second adjustment may be used when the subject 9 moves out of the field of view. For example, the second adjustment may be switched to the first adjustment after a certain period of time has passed since the subject 9 left the field of view. Changes in the focus position FP due to the entrance and exit of the subject 9 can be suppressed.

3. Examples of effects The techniques described above are specified as follows, for example. One of the techniques disclosed is an imaging device 1. As described with reference to FIGS. 1 to 9, etc., the imaging device 1 includes the processing section 7 that controls the imaging optical system 2. The optical system 2 is controlled by the processing unit 7 so that the DF values of the closest subject 9 _near and the farthest subject 9 _far among the plurality of subjects 9 approach the intermediate value of the respective DF values. This includes adjusting the focus position FP of. Thereby, focus position adjustment suitable for imaging a plurality of subjects 9 can be performed.

As described with reference to FIG. 1, FIG. 3, FIG. ₈ , FIG. 9, etc., the control of the optical system 2 by the processing unit 7 is based on the _depth of field may include adjusting the F-number of the optical system so that it approaches the minimum F-number that can be obtained. This enables optimal F-number adjustment based on optimal focus position adjustment.

As described with reference to FIGS. 1 and 2, the imaging device 1 includes an image sensor 3 that images a subject 9 via an optical system 2, and the image sensor 3 has a DF value of each of a plurality of subjects 9. A phase difference detection pixel 312 may be included to obtain the following. For example, using such an image sensor 3, the DF values of the closest subject 9 _near and the farthest subject 9 _far can be obtained.

As described with reference to FIGS. 3 to 7, etc., the processing unit 7 recognizes a plurality of subject candidates located within the field of view of the imaging device 1, and selects a plurality of subjects 9 from the recognized subject candidates. good. For example, the imaging device 1 may include a display operation unit 5 that displays a plurality of subject candidates located within the field of view of the imaging device 1 in a manner that allows them to be selected as the subject 9. Thereby, for example, a plurality of subjects 9 to be focused can be appropriately selected.

As described with reference to FIG. 3 and the like, the processing unit 7 may detect the movement of the plurality of subjects 9 and control the optical system 2 based on the detection results. Thereby, even if the subject 9 moves, optimal focus position adjustment can be maintained.

As described with reference to FIGS. 5 to 7, etc., each of the plurality of subjects 9 may be a different part of the same object. Thereby, it is possible to adjust the focus position suitable for imaging the entire object.

As explained with reference to FIG. 11 etc., when the depth of field cannot include all of the selected subjects 9, the processing unit 7 excludes the farthest subject 9 _far from the selection target. You may do so. This makes it possible to deal with the case where, for example, there is a limit to the adjustment of the F value of the optical system 2.

The processing unit 7 adjusts the focusing position FP of the optical system 2 so that the DF values of the closest subject 9 _near and the farthest subject 9 _far among the plurality of subjects 9 approach the intermediate value of the respective DF values. may include a first adjustment that adjusts the focus position FP of the optical system 2, and a second adjustment that adjusts the focus position FP of the optical system 2 to a position different from the first adjustment. . Thereby, for example, flexibility in adjusting the focus position can be increased.

The imaging method described with reference to FIG. 10 and the like is also one of the disclosed techniques. The imaging method is such that the focusing position of the imaging optical system 2 is adjusted so that the DF values of the closest subject 9 _near and the farthest subject 9 _far among the plurality of subjects 9 approach the intermediate value of the respective DF values. including adjusting the FP. With such an imaging method, as described above, it is possible to perform focus position adjustment suitable for imaging a plurality of subjects 9.

The imaging program 61 described with reference to FIG. 1 and the like is also one of the techniques disclosed. The imaging program 61 causes the processor (processing unit 7) to perform imaging so that the DF values of the closest object 9 _near and the farthest object 9 _{far among} the plurality of objects 9 approach the intermediate value of the respective DF values. A process of adjusting the focal position FP of the optical system 2 for use in the optical system 2 is executed. With such an imaging program 61, as described above, it is possible to perform focus position adjustment suitable for imaging a plurality of subjects 9. A computer-readable recording medium (storage unit 6) on which the imaging program 61 is recorded is also one of the techniques disclosed.

Note that the effects described in the present disclosure are merely examples and are not limited to the disclosed contents. There may also be other effects.

Although the embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the above-described embodiments as they are, and various changes can be made without departing from the gist of the present disclosure. Furthermore, components of different embodiments and modifications may be combined as appropriate.

Note that the present technology can also have the following configuration.
(1)
a processing unit that controls the optical system for imaging;
Equipped with
The control of the optical system by the processing unit includes focusing the optical system so that the defocus values of the closest object and the farthest object among the plurality of objects approach an intermediate value of the respective defocus values. including adjusting the position;
Imaging device.
(2)
The control of the optical system by the processing unit includes adjusting the F value of the optical system so that it approaches a minimum F value that provides a depth of field including the closest object and the farthest object. ,
The imaging device according to (1).
(3)
comprising an image sensor that captures an image of a subject through the optical system,
The image sensor includes a phase difference detection pixel so as to obtain a defocus value for each of the plurality of subjects.
The imaging device according to (1) or (2).
(4)
The processing unit recognizes a plurality of subject candidates located within a field of view of the imaging device, and selects the plurality of subjects from the recognized plurality of subject candidates.
The imaging device according to any one of (1) to (3).
(5)
comprising a display operation unit that displays a plurality of subject candidates located within the field of view of the imaging device in a manner selectable as the subject;
The imaging device according to any one of (1) to (4).
(6)
The processing unit detects movement of the plurality of subjects and controls the optical system based on the detection result.
The imaging device according to any one of (1) to (5).
(7)
Each of the plurality of subjects is a different part of the same object,
The imaging device according to any one of (1) to (6).
(8)
The processing unit excludes the farthest subject from the selection target if all of the selected plurality of subjects cannot be included in the depth of field.
The imaging device according to any one of (1) to (7).
(9)
Adjustment of the focusing position of the optical system by the processing unit includes:
A first adjustment that adjusts the focusing position of the optical system so that the defocus values of the closest object and the farthest object among the plurality of objects approach an intermediate value of the respective defocus values;
a second adjustment that adjusts the focusing position of the optical system to a position different from the first adjustment;
including,
The imaging device according to any one of (1) to (8).
(10)
adjusting the focusing position of the imaging optical system so that the defocus values of the closest object and the farthest object among the plurality of objects approach an intermediate value of the respective defocus values;
including,
Imaging method.
(11)
to the processor,
A process of adjusting the focus position of the imaging optical system so that the defocus values of the closest object and the farthest object among the plurality of objects approach an intermediate value of the respective defocus values;
to execute,
Imaging program.

1 Imaging device 2 Optical system 21 Lens 3 Imaging element 31 Pixel 30 Imaging surface 311 Imaging pixel 312 Phase difference detection pixel 4 Drive section 5 Display operation section 6 Storage section 61 Imaging program 7 Processing section 71 Image generation section 72 Object recognition section 73 Subject selection section 74 Distance measurement section 75 DF value calculation section 76 F value calculation section 77 Optical system adjustment section 78 Movement detection section 9 Subject 9 _nearest subject 9 _far farthest subject FP Focusing position x Depth of field

Claims

a processing unit that controls the optical system for imaging;
Equipped with
The control of the optical system by the processing unit includes focusing the optical system so that the defocus values of the closest object and the farthest object among the plurality of objects approach an intermediate value of the respective defocus values. including adjusting the position;
Imaging device.
The control of the optical system by the processing unit includes adjusting the F value of the optical system so that it approaches a minimum F value that provides a depth of field including the closest object and the farthest object. ,
The imaging device according to claim 1.
comprising an image sensor that captures an image of a subject through the optical system,
The image sensor includes a phase difference detection pixel so as to obtain a defocus value for each of the plurality of subjects.
The imaging device according to claim 1.
The processing unit recognizes a plurality of subject candidates located within a field of view of the imaging device, and selects the plurality of subjects from the recognized plurality of subject candidates.
The imaging device according to claim 1.
comprising a display operation unit that displays a plurality of subject candidates located within the field of view of the imaging device in a manner selectable as the subject;
The imaging device according to claim 1.
The processing unit detects movement of the plurality of subjects and controls the optical system based on the detection result.
The imaging device according to claim 1.
Each of the plurality of subjects is a different part of the same object,
The imaging device according to claim 1.
The processing unit excludes the farthest subject from the selection target if all of the selected plurality of subjects cannot be included in the depth of field.
The imaging device according to claim 1.
Adjustment of the focusing position of the optical system by the processing unit includes:
A first adjustment that adjusts the focusing position of the optical system so that the defocus values of the closest object and the farthest object among the plurality of objects approach an intermediate value of the respective defocus values;
a second adjustment that adjusts the focusing position of the optical system to a position different from the first adjustment;
including,
The imaging device according to claim 1.
adjusting the focusing position of the imaging optical system so that the defocus values of the closest object and the farthest object among the plurality of objects approach an intermediate value of the respective defocus values;
including,
Imaging method.
to the processor,
A process of adjusting the focus position of the imaging optical system so that the defocus values of the closest object and the farthest object among the plurality of objects approach an intermediate value of the respective defocus values;
to execute,
Imaging program.