JP2016208530A - Image generating apparatus, image generation method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To determine a focus position by considering an area to be imaged.SOLUTION: An imaging apparatus comprises: an imaging unit 16; a focus information acquisition unit 53; a frequency distribution calculation unit 54; and an imaging position determination unit 55. The focus information acquisition unit 53 calculates a focus position of a subject for each of a plurality of areas in an imaged image divided by the imaging unit 16. The frequency distribution calculation unit 54 counts the number of areas corresponding to respective subject focus positions obtained by the measurement. The imaging position determination unit 55 determines a focus position for imaging on the basis of depth of field of the imaging unit 16 in accordance with the number of areas corresponding to the focus positions of respective subjects calculated by the focus information acquisition unit 53.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program.

2. Description of the Related Art Conventionally, when imaging devices focus on all subjects from the rear to the front (hereinafter referred to as “all-focus images”), special imaging devices such as ultra-deep lenses (commonly referred to as “worm-eye lenses”) are used. Uses the correct lens.
However, the insect lens is very inconvenient to carry because of its very long overall length.
For this reason, an imaging device equipped with a normal lens captures a plurality of subjects with the same composition at a predetermined plurality of focus positions almost simultaneously by a single shutter operation, and synthesizes the data of the resulting captured images. Thus, a technique for generating omnifocal image data is disclosed (see Patent Document 1).

JP 2007-282152 A

  However, in Patent Document 1, the captured image used for generating the data of the omnifocal image captures a plurality of subjects having the same composition at a plurality of predetermined focus positions almost simultaneously. Take a picture without considering it. For this reason, there existed problems, such as acquiring the captured image which is not suitable for the production | generation of the data of an omnifocal image.

  The present invention has been made in view of such a situation, and an object thereof is to determine a focus position in consideration of a region to be photographed.

In order to achieve the above object, an imaging device of one embodiment of the present invention includes:
Imaging means;
Measuring means for measuring the focus position of the subject for each of a plurality of areas obtained by dividing the captured image captured by the imaging means;
The focus of the subject according to the depth of field of the lens of the imaging unit, including the focus position of the subject with high priority among the focus positions of the subject obtained by the measurement unit. A determination means for determining a focus position for shooting, which is different from the position;
It is characterized by providing.

  According to the present invention, it is possible to determine a focus position in consideration of a region to be photographed.

It is a schematic diagram explaining the outline | summary of focus bracketing imaging | photography in this embodiment. It is a block diagram which shows the structure of the hardware of the imaging device which concerns on one Embodiment of this invention. It is a functional block diagram which shows the functional structure for performing an omnifocal image generation process among the functional structures of the imaging device of FIG. It is a schematic diagram which shows the specific example of the method of determination of an imaging position and the imaging | photography number. It is a figure which shows the specific example of the method of arrangement | positioning of a depth of field, and the determination of an imaging position. It is a schematic diagram which shows the other specific example of determination of the imaging | photography position and imaging | photography number at the time of setting it as the imaging | photography area | region different from FIG. 4 is a flowchart for explaining a flow of omnifocal image generation processing executed by the imaging apparatus of FIG. 2 having the functional configuration of FIG. 3. It is a schematic diagram explaining the specific example of the weighting in frequency distribution.

  Hereinafter, an imaging apparatus according to an embodiment of the present invention will be described with reference to the drawings.

The generation of the omnifocal image in the present embodiment is performed by acquiring captured images constituting a plurality of omnifocal images by focus bracketing imaging and synthesizing the captured images.
Here, “focus bracketing shooting” refers to a position where a subject is focused (predetermined focus position) by simply pressing a shutter button, which is an input means mounted on the imaging device, and a lens. This is an imaging technique in which a plurality of captured images are continuously captured at each of positions where the driving means is driven and the focus is shifted before and after that (focus position shifted from a predetermined focus position).
In the present embodiment, focus bracketing shooting does not take a captured image at a focus position at a predetermined interval, but considers the distance of the subject included in the shooting region, that is, the position where the subject is focused. Shoot at the determined focus position.

  FIG. 1 is a schematic diagram for explaining an overview of focus bracketing shooting in the present embodiment. In the example of FIG. 1, a case where “flowers” and “grass” are photographed in “park” will be described.

  When shooting a shooting area as shown in FIG. 1 (a), “flowers” and “grass” blooming in front of the “park” scenery are included, and nearby “flowers” and “grass”. Then, “tiles” behind “flowers” and “grass” and distant “trees” can be identified.

The omnifocal image in the shooting region as in the example of FIG. 1A is an image in which “flowers”, “grass”, “tiles”, and “trees” that can be specified as subjects are in focus. preferable.
That is, as shown in FIG. 1B, a region R1 focused on “flower”, a region R2 focused on “grass”, a region R3 focused on “tile”, and “trees” It is desirable that the captured image in which each region of the region R4 in focus is set as the omnifocal image.

  However, in the case of conventional focus bracketing shooting, it is set to perform focus bracketing shooting regardless of the shooting situation. Specifically, in the conventional method, a predetermined number of images are taken with the focus position shifted by a predetermined interval. Then, an omnifocal image is generated by synthesizing the captured images acquired by the imaging.

  In the case of such a conventional method, in order to shoot at a predetermined focus position regardless of the shooting target, a poor-quality omnifocal image that has an out-of-focus part is generated, or the worst In this case, there is a problem that an omnifocal image cannot be generated.

  However, in the focus bracketing shooting according to the present embodiment, as shown in FIG. 1B, a high-quality omnifocal image is generated in order to determine the focus position and the number of shots in consideration of the imaging target. In addition, since unnecessary shooting is not required, the processing for generating the omnifocal image can be speeded up, the image quality is good, and the processing burden can be reduced.

Next, an imaging apparatus having a function of performing a focus bracketing shooting as described above and generating a suitable omnifocal image will be described.
FIG. 2 is a block diagram illustrating a hardware configuration of the imaging apparatus 1 according to an embodiment of the present invention.
The imaging device 1 is configured as a digital camera, for example.

  As shown in FIG. 2, the imaging apparatus 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, and an imaging. A unit 16, an input unit 17, an output unit 18, a storage unit 19, a communication unit 20, and a drive 21 are provided.

  The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 19 to the RAM 13.

  The RAM 13 appropriately stores data necessary for the CPU 11 to execute various processes.

  The CPU 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output interface 15 is also connected to the bus 14. An imaging unit 16, an input unit 17, an output unit 18, a storage unit 19, a communication unit 20, and a drive 21 are connected to the input / output interface 15.

  The imaging unit 16 includes an optical lens unit 41, an image sensor 42, and a lens driving unit 43.

The optical lens unit 41 includes various lenses such as a focus lens and a zoom lens in order to photograph a subject.
The focus lens is a lens that is driven by a lens driving unit 43 described later and forms an image of a subject on a light receiving surface of an image sensor 42 described later. The zoom lens is a lens that freely changes the focus distance within a certain range.

The image sensor 42 includes a photoelectric conversion element, an AFE (Analog Front End), and the like.
The photoelectric conversion element is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element or the like. A subject image is incident on the photoelectric conversion element from the optical lens unit 41. Therefore, the photoelectric conversion element photoelectrically converts (captures) the subject image, accumulates the image signal for a predetermined time, and sequentially supplies the accumulated image signal as an analog signal to the AFE.
The AFE performs various signal processing such as A / D (Analog / Digital) conversion processing on the analog image signal. The AFE generates a digital signal by various signal processing and outputs it as an output signal of the imaging unit 16. Such an output signal of the imaging unit 16 becomes captured image data. The captured image data is appropriately supplied to the CPU 11 or the like. In addition, information on the order of imaging and information on the focus position at the time of imaging are added to the captured image data as metadata. Hereinafter, unless otherwise specified, captured image data to which metadata is added is simply referred to as “captured image data”.

The lens driving unit 43 drives the focus lens of the optical lens unit 41 based on the result of AF (Auto Focus) processing by the CPU 11 and changes the focus position to focus on a predetermined subject.
In the image pickup apparatus 1 having such a focus mechanism, shooting in the shortest distance side direction, that is, in front of the subject, and infinite distance, that is, a distance far enough to eliminate the need for focus adjustment. It is possible to shoot with the subject as a subject.
Further, the lens driving unit 43 is driven so that focus bracketing imaging that sequentially performs imaging with different focus positions can be performed while changing the focus position from the position determined by the AF processing.

The input unit 17 includes various buttons including a shutter button, and inputs various types of information according to user instruction operations.
The output unit 18 includes a display, a speaker, and the like, and outputs images and sounds.
The storage unit 19 is composed of a hard disk, a DRAM (Dynamic Random Access Memory), or the like, and stores various image data.
The communication unit 20 controls communication performed with other devices (not shown) via a network including the Internet.

  A removable medium 31 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 21. The program read from the removable medium 31 by the drive 21 is installed in the storage unit 19 as necessary. The removable medium 31 can also store various data such as image data stored in the storage unit 19 in the same manner as the storage unit 19.

Next, a functional configuration for executing the omnifocal image generation process among the functional configurations of the imaging apparatus 1 will be described.
“All-focus image generation processing” is a method of performing focus bracketing shooting at the shooting position and the number of shots determined in consideration of the subject in the shooting area, and combining the acquired multiple captured image data into the shooting area. This is a series of processes from generation and output of omnifocal image data in which a plurality of included subjects are in focus.
FIG. 3 is a functional block diagram illustrating a functional configuration for executing the omnifocal image generation process among the functional configurations of the imaging apparatus 1.
In addition, the imaging apparatus 1 according to the present embodiment performs focus bracketing shooting and multi-area AF shooting for determining the shooting position and the number of shots in focus bracketing shooting. Here, “multi-area AF shooting” is a shooting method in which a captured image is divided into a plurality of focus areas, and the distance at which the subject included in each focus area is in focus is detected as the subject focus position for each focus area. . The subject focus position represents the distance at which the subject is focused as a value, and is represented as “∞” from a macro (for example, “10 cm”).
Note that the subject focus position is not limited to those measured by an AF method such as contrast AF or phase difference AF, or a lens driving method, but directly measures the distance to the subject using ultrasonic waves. It may be.

  When the imaging device 1 executes the omnifocal image generation process, as shown in FIG. 3, in the CPU 11, the imaging control unit 51, the captured image acquisition unit 52, the focus information acquisition unit 53, and the frequency distribution calculation unit. 54, the shooting position determination unit 55, the number of shots determination unit 56, and the image composition unit 57 function.

  In addition, as one area of the storage unit 19, an imaging information storage unit 71, a shooting condition storage unit 72, and a composite image storage unit 73 are provided.

In the imaging information storage unit 71, data of a captured image output from the imaging unit 16 and acquired by the captured image acquisition unit 52 is stored. More specifically, the imaging information storage unit 71 stores captured image data acquired by multi-area AF imaging and captured image data acquired by focus bracketing imaging.
In addition to the captured image data, the captured information storage unit 71 stores information on the subject focus position of each region in the captured image acquired by the focus information acquisition unit 53.

The shooting condition storage unit 72 stores shooting conditions in focus bracketing shooting. Specifically, in the shooting condition storage unit 72, as shooting conditions in focus bracketing shooting, the stop position of the focus lens (hereinafter referred to as “shooting position”) and the shooting conditions of the number of shots calculated from the shooting position are stored. Is memorized.
The shooting condition storage unit 72 stores information used for setting shooting conditions such as the characteristics of the depth of field of the lens. The characteristics of the depth of field of the lens are stored in the imaging condition storage unit 72 as table data.

  The synthesized image storage unit 73 stores data of the omnifocal image generated as a result of image synthesis of the captured image by the image synthesizing unit 57.

  The imaging control unit 51 controls imaging in the imaging unit 16 including the lens driving unit 43 and the like based on an instruction related to imaging from the input unit 17. Specifically, the imaging control unit 51 performs multi-area AF shooting when there is an instruction for multi-area AF shooting when the shutter button of the input unit 17 is half-pressed. The imaging unit 16 including the lens driving unit 43 and the like is controlled. In addition, when the shutter button of the input unit 17 is fully pressed, the imaging control unit 51 determines that there is an instruction for focus bracketing shooting and performs a lens bracketing unit so as to perform focus bracketing shooting. The imaging unit 16 including 43 and the like is controlled.

  The captured image acquisition unit 52 acquires captured image data for shooting condition determination by multi-area AF shooting, and captured image data for image synthesis (for generation of an omnifocal image) by focus bracketing shooting. Image data is stored in the imaging information storage unit 71.

  When the captured image acquisition unit 52 acquires the captured image, the focus information acquisition unit 53 acquires information on the subject focus position of the divided area of the captured image. The focus information acquisition unit 53 also acquires information on a focus area (hereinafter referred to as “AF area”) that is finally focused out of the subject focus positions. The captured image acquisition unit 52 stores these pieces of information in the imaging information storage unit 71 in association with the acquired captured image data for determining imaging conditions.

  The frequency distribution calculation unit 54 determines the subject from macro to infinity based on the subject focus position information of all the focus areas of the captured image acquired by the focus information acquisition unit 53 stored in the imaging information storage unit 71. The frequency distribution of the focus position (hereinafter referred to as “frequency distribution”) is calculated. The frequency distribution calculation unit 54 outputs the calculated frequency distribution to the imaging position determination unit 55.

  The shooting position determination unit 55 determines the shooting position based on the frequency distribution calculated by the frequency distribution calculation unit 54 and the characteristics of the depth of field of the lens stored in the shooting condition storage unit 72. For example, the shooting position determination unit 55 determines the shooting position so that a subject focus position with a high frequency of subject focus positions is set as a focus. By determining the shooting position in this way, it is possible to obtain a captured image in which at least a main subject is in focus. The shooting position determination unit 55 outputs the determined shooting position to the shooting number determination unit 56.

The shooting number determining unit 56 counts the shooting positions determined by the shooting position determining unit 55 to determine the shooting number. That is, for example, when there are five determined shooting positions, the number of shots determination unit 56 sets the number of shots to five.
In addition, the number-of-photographing number determination unit 56 outputs the information on the photographing position determined by the photographing position determination unit 55 and the information on the determined number of photographings to the photographing condition storage unit 72 for storage.

  The image synthesis unit 57 synthesizes data of a plurality of captured images acquired by focus bracketing shooting stored in the imaging information storage unit 71. The image synthesis unit 57 only needs to be able to synthesize the focus areas of a plurality of captured images, and generates omnifocal image data that is a synthesized image using various methods such as addition synthesis. The image composition unit 57 stores the generated omnifocal image data in the composite image storage unit 73.

Here, details of the method for determining the shooting position and the number of shots will be described.
FIG. 4 is a schematic diagram showing a specific example of a method for determining the shooting position and the number of shots. In the example of FIG. 4, the case where the imaging region of FIG. 1 is imaged is used as an example.

In determining the shooting position and the number of shots, the subject focus position in each focus area FR of the captured image is acquired by multi-area AF shooting as shown in FIG.
In the present embodiment, the focus area FR is partitioned into 5 × 5 areas in a grid pattern and is divided into a total of 25 focus areas FR in the captured image. In each of the focus areas FR, the subject focus position between the macro and infinity is determined.
In FIG. 4A, a region indicated by a solid line is an AF region AFR that is finally focused in the focus region FR, and a region indicated by a broken line is an AF region AFR of the focus region FR. A focus area FR that is not is shown.

The subject focus position of each focus area FR in the captured image of this example is “infinity ∞”, “3 m”, “2 m”, “1 m”, “50 cm”, “30 cm”, “20 cm”, “10 cm”. It has become.
In such a captured image, as shown in FIG. 4B, the focus area FR in which “infinity ∞” is the subject focus position is focused on the “trees” in FIG. The focus area FR corresponding to the area R4, in which “3m”, “2m”, and “1m” are the subject focus positions, corresponds to the area R3 focused on the “tile” in FIG. The focus area FR in which “50 cm”, “30 cm”, and “20 cm” are subject focus positions corresponds to the area R2 focused on “grass” in FIG. 1B, and “10 cm” is the subject focus. As a result, the focus area FR at the position corresponds to the area R1 focused on the “flower” in FIG. 1B, and the actual shooting target matches the subject focus position. It can be seen that you are.

  Next, in the determination of the shooting position and the number of shots, as shown in FIG. 4B, the focus areas FR arranged in the order of the subject focus position for each focus area FR are overlapped as a frequency distribution. Expressed as a frequency.

  Specifically, the frequency distribution includes five subject focus positions “10 cm”, one subject focus position “20 cm”, three subject focus positions “30 cm”, one subject focus position “50 cm”, and one subject. There are three focus positions “1 m”, one subject focus position “2 m”, two subject focus positions “3 m”, and nine subject focus positions “infinity ∞”.

  In the present embodiment, the subject focus position having a frequency equal to or greater than a predetermined number (three places) is the peak position PK that is preferentially set to the shooting position. In the case of FIG. 4B, the subject focus position “10 cm”, the subject focus position “30 cm”, the subject focus position “1 m”, and the subject focus position “infinity ∞” are the peak positions PK1 to PK4 (FIG. 4 ( c)).

  In such a frequency distribution, as shown in FIG. 4C, the depths of field DOF1 to DOF4 are arranged at positions including the peak positions PK1 to PK4, and in each of the arranged depths of field DOF1 to DOF4. The photographing positions SP1 to SP4 are determined as positions that cover the peak positions PK1 to PK4 and other subject focus positions. A method for determining the depth of field and the shooting position will be described later.

  In the example of FIG. 4C, four locations are determined as shooting positions according to the subject focus position. As a result, for example, the number of shots can be reduced compared to a case where five focus bracketing shots are taken as in the conventional method. Reducing the number of shots not only reduces the processing load of shooting, but also shortens the shooting time, shortens the time exposed to camera shake during shooting, and obtains higher quality captured images. Will contribute. In addition, since the number of images can be reduced, the image synthesis time can be shortened, and the time required to generate the final omnifocal image can be shortened from the viewpoint of shortening the shooting time and the image synthesis time. it can.

Next, a method of determining the depth of field and the shooting position will be described.
FIG. 5 is a diagram showing a specific example of a technique for determining the arrangement of the depth of field and the shooting position at the aperture value at the time of shooting. Note that the depth-of-field characteristic of the lens in this embodiment has such a characteristic that the depth-of-field range becomes wider from macro to infinity (∞).

  First, in the present embodiment, the imaging position of the peak position PK1 having the highest frequency among the peak positions PK1 to PK4 is determined. In this case, the depth of field DOF1, which is the depth of field covering the widest range including the peak position PK1, is selected. Then, the imaging position SP1 is determined as a position that emphasizes the peak position PK1 belonging to the range of the depth of field and covers another subject focus position belonging to the range of the depth of field DOF1.

  Next, of the peak positions PK2 to PK4, the imaging position of the next most frequent peak position PK2 is determined. In this case, the depth of field DOF2, which is the depth of field covering the widest range including the peak position PK2, is selected. Then, the imaging position SP2 is determined as a position that emphasizes the peak position PK2 belonging to the range of the depth of field and covers another subject focus position belonging to the range of the depth of field DOF2.

Next, of the peak positions PK3 and PK4, the photographing position of the peak position with the highest frequency is determined. In the case of this example, the frequency of the peak position PK3 and the peak position PK4 is the same as four places. When the frequency is the same, the peak position belonging to the deeper depth of field is preferentially determined. Since the depth of field of the lens of this example is such that the farther the subject focus position is, the wider the focus range and the deeper the depth of field, the peak position at which the subject focus position is farther than the peak position PK4. Determine PK3.
Here, a depth of field DOF3 having a deep depth of field that covers the widest range including the peak position PK3 is selected. Then, the photographing position SP3 is determined to be a position that covers the peak position PK3 belonging to the range of the depth of field and covers the other subject focus position.

Finally, the depth of field DOF4 having the same depth as the peak position PK3 and including the peak position PK4 belonging to the position where the depth of field is shallow and covering the widest range is selected. Then, the shooting position SP4 is determined as a position that covers the peak position PK4 that falls within the range of the depth of field and covers the other subject focus positions.
When the depth of field DOF selected to include the predetermined peak position PK includes another peak position PK, the depth of field DOF including only the predetermined peak position PK is selected again. Then, within the range of the selected depth of field DOF, the imaging position SP is determined to be a position that emphasizes the peak position PK and can cover other subject focus positions to which it belongs.

As a result, in the present embodiment, four shooting positions are determined in the order of (1) to (4) in which the peak positions PK1 to PK4 are frequent, and all eight subject focus positions are covered. The shooting position is determined at the position.
In this example, the subject focus position having a frequency equal to or higher than the predetermined number (three places) is set as the peak position, and the position for determining the photographing position is set, but the present invention is not limited to this. The shooting positions may be determined according to the frequency distribution of the subject focus positions (for example, in descending order of frequency) for all the subject focus positions. In this case, when the shooting position is determined so as to cover the other subject focus positions with the depth of field, including the subject focus position for which the shooting position has been determined as described above, this is covered. In addition, it is configured not to determine the shooting position of another subject focus position.

Next, another specific example of determining the shooting position and the number of shots when the shooting area is different from that in FIG. 1 will be described.
FIG. 6 is a schematic diagram showing another specific example of determining the shooting position and the number of shots when the shooting area is different from that in FIG.

  In the example of FIG. 6 (a), unlike the case of FIG. 1 (a), an attempt is made to photograph the sea from a cliff, with “grass” in the foreground and “sea” and “island” in the distance. There is a horizon ahead, and the “sky” spreads across the horizon.

  As shown in FIG. 6B, the subject focus position of each focus area FR is “3 m” in the focus area FR including “grass”, and other “sea”, “island”, and “sky” "Is included in the focus area FR is" infinity infinity ". The AF area AFR is a focus area FR including “grass” at the subject focus position “3 m”.

  As shown in FIG. 6C, the frequency distribution of each subject focus position has five subject focus positions “3m” and 20 subject focus positions “infinity ∞”.

In such a state, the shooting position is determined by selecting the depth of field DOF having the widest range, as shown in FIG.
In this example, all the subject focus positions are covered by selecting the widest depth of field DOF. That is. The subject focus position “3 m” and the subject focus position “infinity ∞” are covered by the selected depth of field DOF.
For this reason, the photographing position is determined to be a position where importance is placed on the subject focus position “3 m” and the subject focus position “infinity ∞” as the peak position PK.
As a result, in the photographing region as shown in FIG. 6A, an omnifocal image can be generated at one photographing position only at the hyperfocal position.

  In this way, since the shooting position and the number of shots can be determined depending on the shooting location, an omnifocal image can be suitably generated.

Next, a flow of omnifocal image generation processing executed by the imaging apparatus 1 having such a functional configuration will be described.
FIG. 7 is a flowchart for explaining the flow of omnifocal image generation processing executed by the imaging apparatus 1 of FIG. 2 having the functional configuration of FIG. In the flowchart of FIG. 7, an example in which FIG. 1 is used as an imaging region will be described.

  When the user performs a predetermined operation for executing the omnifocal image generation process using the input unit 17, the omnifocal image generation process is started with the operation.

  In step S <b> 11, the imaging control unit 51 detects a half-press operation to the shutter button of the input unit 17 from the user. Multi-area AF shooting is performed by half-pressing the shutter button.

In step S12, the imaging control unit 51 controls the imaging unit 16 to perform multi-area AF shooting. As a result, the captured image acquisition unit 52 acquires captured image data for determining imaging conditions and stores the acquired data in the imaging information storage unit 71. Then, the focus information acquisition unit 53 also acquires the subject focus position of each focus area FR in the acquired captured image. In addition, the focus information acquisition unit 53 also acquires information on an area that is finally focused (AF area). The focus information acquisition unit 53 stores the subject focus position and AF area information of each focus area in the imaging information storage unit 71 in association with captured image data for determining the imaging conditions.
The imaging information storage unit 71 stores, for example, information on the subject focus position and AF area in each of the captured image as shown in FIG. 1A and the focus area of the captured image as shown in FIG. Will do.

  In step S13, the frequency distribution calculator 54 calculates the frequency distribution of the subject focus position. That is, the frequency distribution calculation unit 54 calculates the frequency distribution of the subject focus position based on the information on the subject focus position of each focus area stored in the imaging information storage unit 71. In the present embodiment, for example, as shown in FIG. 4B, the frequency of the subject focus position from macro to infinity (∞) is counted and distributed for each subject focus position.

In step S14, the shooting position determination unit 55 and the shooting number determination unit 56 determine the shooting position and the number of shots based on the depth of field at each focus position and the frequency distribution of the subject focus position.
Specifically, as shown in FIG. 4C, the shooting position determination unit 55 determines the distribution of peak positions PK1 to PK4 that are subject focus positions calculated by the frequency distribution calculation unit 54, and the shooting conditions. Based on the characteristics of the depth of field of the lens stored in the storage unit 72, the depth of field DOF is selected, the peak position PK included in the depth of field DOF is emphasized, and other The shooting position SP is determined to be a position that covers the subject focus position.
Then, the number-of-shots determination unit 56 counts the number of shooting positions SP1 to SP4 determined by the shooting position determination unit 55 to determine the number of shots (four). Thereafter, information on the shooting position determined by the shooting position determination unit 55 and information on the determined number of shots are output from the shot number determination unit 56 and stored in the shooting condition storage unit 72.
In this example, it is usually possible to shoot four shots where it is necessary to shoot five. As a result, compared to the case of shooting five images, the shooting time is shortened and is not affected by camera shake, etc., and the image quality is good and the processing time for combining is reduced, so the combining time can be shortened. The generation time of the omnifocal image can be shortened.

  In step S <b> 15, the imaging control unit 51 detects an operation of fully pressing the shutter button of the input unit 17 from the user. Focus bracketing shooting is performed by fully pressing the shutter button.

  In step S16, the imaging control unit 51 controls the imaging unit 16 to perform focus bracketing imaging. The imaging control unit 51 controls the imaging unit 16 to perform shooting at the set shooting position based on the setting of the shooting conditions (shooting position and number of shots) stored in the shooting condition storage unit 72. To do. Thereafter, the captured image acquisition unit 52 acquires captured image data output from the imaging unit 16 by focus bracketing imaging, and stores the captured image data in the imaging information storage unit 71.

  In step S <b> 17, the image composition unit 57 synthesizes the captured image data obtained by the focus bracketing photographing that is photographed under the determined photographing conditions (the photographing position and the number of photographs) and stored in the photographing information storage unit 71. Processing (image composition processing) is performed. For example, the image composition unit 57 sequentially composes images near the focus position of the captured image to generate a composite image. As a result, data for one omnifocal image is generated. Thereafter, the image composition unit 57 stores the generated omnifocal image data in the composite image storage unit 73. Thereby, the omnifocal image generation process ends.

<Modification>
In the embodiment described above, the shooting position and the number of shots are determined according to the position of the subject included in the shooting area.
In this example, the shooting position and the number of shots are determined in consideration of the type of subject included in the shooting area. That is, the focus position is determined with priority.

Specifically, a shooting mode that matches the assumed shooting target is provided. In the shooting mode, the focus position (shooting position) can be preferentially determined according to the shooting target by changing the weighting of the frequency of the subject focus position corresponding to the shooting target.
Information on setting of weights for determining the focus position and the number of shots corresponding to such a shooting mode is stored in the imaging information storage unit 71. Then, the frequency distribution calculation unit 54 calculates the frequency distribution of the subject focus position in consideration of the weighting.

FIG. 8 is a schematic diagram illustrating a specific example of weighting in the frequency distribution.
As shown in FIG. 8A, the shooting modes include, for example, “landscape” shooting mode, “flower” shooting mode, “person” shooting mode, “selfie” shooting mode, and “center-weighted metering”. A mode such as a shooting mode is set. Of these shooting modes, the user changes the mode according to the situation where shooting is scheduled.

  The “landscape” shooting mode is a mode set when shooting a landscape, and weighting is set so that a shooting position is preferentially determined at an object focus position at infinity.

  The “flower” shooting mode is set when shooting close-ups of flowers, etc., so that the shooting position is preferentially determined at the subject focus position with a large number of areas on the macro end side excluding infinity. Is assigned a weight.

  The “person” shooting mode is set when shooting a person, and weighting is set so that the shooting position is preferentially determined to the focus position where the face is detected regardless of the number of areas of the subject focus position. The In the “person” shooting mode, when a plurality of faces are detected, priority is given to a subject focus position having a large number of detected face areas.

  “Self-portrait” shooting mode is set when shooting yourself, and weighting is set so that the shooting position is preferentially determined to the focus position where the face is detected regardless of the number of areas of the subject focus position. Is done.

  The “Center-weighted metering” shooting mode is a mode that is set when shooting is performed with an emphasis on the center of the captured image, and the shooting position is determined preferentially at the focus position at the center of the image. The weighting is set so that

  Further, in most cases, there is a strong tendency to place the main subject in the center of the shooting area and shoot, so that weighting that emphasizes the central portion may be set. In this case, as shown in FIG. 8B, for example, the weight of the subject focus position FP at the center is increased, and the weight is set lower as the distance from the center increases. That is, the subject focus position FP at the center is counted for four, the subject focus position FP around it is counted for two, and the surroundings are counted as normal.

In addition, when adjacent subject focus positions FP indicate the same value, the same subject is often included, and therefore, for example, weighting corresponding to the number of adjacent subject focus positions may be set. Good.
In this case, as shown in FIG. 8C, “infinity infinity” is counted for 10 because there are 10 adjacent subject focus positions FP, and “10 cm” has 6 adjacent subject focus positions. Therefore, 6 counts are performed, “30 cm” is counted for 3 because there are 3 adjacent subject focus positions FP, and “1 m” is 2 because there are 2 adjacent subject focus positions FP. A minute is counted. The other “2 m” and “3 m” are counted normally because there is no adjacent subject focus position FP.

According to the imaging apparatus 1 as described above, the imaging unit 16, the focus information acquisition unit 53, the frequency distribution calculation unit 54, and the imaging position determination unit 55 are provided.
The focus information acquisition unit 53 acquires the subject focus position for each of a plurality of areas obtained by dividing the shooting screen of the captured image captured by the imaging unit 16.
The frequency distribution calculation unit 54 counts the number of regions corresponding to each subject focus position acquired by the focus information acquisition unit 53.
The shooting position determination unit 55 sets the shooting position (focus stop position, for shooting) based on the depth of field of the imaging unit 16 according to the number of areas corresponding to each subject focus position counted by the frequency distribution calculation unit 54. Focus position).
Therefore, the imaging apparatus 1 can determine the focus position in consideration of the subject included in the shooting area. Note that when the focus position for shooting is determined in consideration of the entire subjects included in the shooting area, the number of focus positions for shooting is smaller than the number of subject focus positions.
Thereby, in the imaging device 1, it is possible to determine an appropriate focus position at the time of the focus bracket for acquiring the captured image used for the omnifocal image.
In the imaging apparatus 1, the determination can be used as a material for determining whether or not the user performs imaging.

In addition, the imaging apparatus 1 includes a shooting number determination unit 56 that determines the number of shots of the focus bracket according to the focus position for shooting determined by the shooting position determination unit 55.
Thereby, in the imaging device 1, for example, unnecessary shooting can be reduced, and the time taken to generate an omnifocal image can be shortened.

In addition, the shooting position determination unit 55 gives priority to the subject focus position counted by the frequency distribution calculation unit 54 and determines a focus position for shooting corresponding to the subject focus position.
Thereby, in the imaging device 1, it becomes possible to acquire the captured image which can produce | generate a high quality omnifocal image.

The shooting position determination unit 55 determines the focus positions for a plurality of shootings in consideration of overlapping depth of field of the imaging unit 16 so that the focus state at each subject distance becomes a predetermined state.
Thereby, in the imaging device 1, for example, since the focus position at the position where the depth of field overlaps is not determined, unnecessary shooting can be reduced, and the time required for generating the omnifocal image can also be shortened.

The shooting position determination unit 55 determines a focus position that has priority according to the shooting mode.
As a result, since the imaging apparatus 1 can determine the shooting position in consideration of the shooting situation that can be understood in advance, the captured image used for generating a higher-quality omnifocal image, that is, the focus position is determined at an appropriate position. It is possible to acquire a captured image.

The shooting position determination unit 55 detects a face in a shooting mode among the shooting modes, and prioritizes the focus position of the detected face.
Thereby, in the imaging device 1, it is possible to acquire a captured image in which the face is in focus suitable for photographing a person, and a high-quality omnifocal image can be generated.

When a plurality of faces are detected, the shooting position determination unit 55 gives priority to a focus position with a large number of areas.
As a result, the imaging apparatus 1 can acquire a captured image focused on the front or a larger face when photographing a plurality of persons, and can generate a high-quality omnifocal image.

The shooting position determination unit 55 determines the focus position for shooting by giving priority to the subject focus position weighted according to the distribution state of the region of the same focus position.
As a result, in the imaging apparatus 1, for example, in the case of the same focus position, the same subject is likely to be the same subject, so that a captured image considering the subject can be acquired, and high quality all A focus image can be generated.

The shooting position determination unit 55 performs high weighting when the areas having the same focus position are adjacent to each other.
Thereby, in the imaging device 1, since it is highly likely that the same subject is the same subject when the regions of the same focus position are adjacent to each other, a captured image that takes the subject into consideration can be acquired, and high quality An omnifocal image can be generated.

In addition, the imaging apparatus 1 includes an image composition unit 57 that synthesizes images.
The image composition unit 57 synthesizes captured images photographed with a predetermined focus bracket (focus bracketing photographing) to generate an omnifocal image.
Thereby, since the imaging apparatus 1 can perform focus bracketing imaging according to the state of the imaging region, for example, unnecessary imaging can be reduced and the time required to generate an omnifocal image can be reduced. Then, it is possible to acquire a captured image in which the focus position is determined at an appropriate position and generate a high-quality omnifocal image.

  In addition, this invention is not limited to the above-mentioned embodiment, The deformation | transformation in the range which can achieve the objective of this invention, improvement, etc. are included in this invention.

In the above-described embodiment, the shooting position determination unit 55 determines the focus stop position in consideration of all subject focus positions, but is not limited thereto. For example, the shooting position determination unit 55 may be configured to determine a shooting position (focus stop position) by providing a threshold and considering only subject focus positions equal to or greater than a predetermined number of subject focus positions.
Specifically, the shooting position determination unit 55 is configured to provide a threshold value for the number of areas and to determine the shooting position (focus stop position) with priority given to the focus position in an area equal to or greater than the threshold value.
Thereby, in the imaging device 1, for example, since the focus position at the position where the depth of field overlaps is not determined, unnecessary shooting can be reduced, and the time required for generating the omnifocal image can also be shortened.

In the above-described embodiment, the shooting position that is the focus stop position is determined by giving priority to the frequency of the subject focus position. However, the present invention is not limited to this. For example, determine the shooting position of the focus position belonging to the depth of field with the widest range of depth of field, and then determine the AF position of focus and the shooting position of the focus position arbitrarily set by the user, Thereafter, the photographing position may be determined according to the frequency of the focus position. Alternatively, the photographing position (focus stop position) may be determined by giving priority to a focus position with a wide depth of field.
With this configuration, in the imaging device 1, for example, since the focus position at a position where the depth of field overlaps is not determined, unnecessary shooting can be reduced and it is necessary to generate an omnifocal image. Time can be shortened.

In the above-described embodiment, the omnifocal image is generated based on the determined shooting position and the number of shots. However, the present invention is not limited to this. For example, based on the determined shooting position and the number of shots, the output unit 18 may display and output a determination result as to whether or not an omnifocal image can be generated and notify the user. In this case, it is notified that an omnifocal image cannot be generated based on the captured image, or that a suitable omnifocal image can be generated.
Further, it may be configured such that a live view image acquired sequentially is used for determination of generation of an omnifocal image, and generation of an omnifocal image is determined every time a live view image is acquired. In this case, the user determines the generation of the omnifocal image (determination of the shooting position and the number of shots) by pointing the imaging device in a predetermined direction, and the determination result is output from the output unit 18. The user can determine whether or not to execute the omnifocal image generation process based on the result of this determination. Further, when the determination result is that the omnifocal image cannot be generated, unnecessary processing may not be performed by not executing the omnifocal image generation processing.
Specifically, a determination unit is provided for determining whether a captured image suitable for generating an omnifocal image can be acquired based on the shooting position determined by the shooting position determination unit 55 or the like. It can be configured by providing an output control means for controlling the output unit 18 so as to notify and output the result.
Thereby, it can be left to a user whether the area | region where the present imaging | photography plan is suitable for an omnifocal image.
The output in the output unit 18 can be performed by screen display, sound, lamp lighting, or the like.

In the above-described embodiment, the imaging apparatus 1 to which the present invention is applied has been described using a digital camera as an example, but is not particularly limited thereto. The present invention can also be applied to an image processing apparatus that does not have an imaging function and performs an omnifocal image generation process based on an image captured externally.
For example, the present invention can be applied to general electronic devices having an omnifocal image generation processing function. Specifically, for example, the present invention can be applied to a notebook personal computer, a printer, a television receiver, a video camera, a portable navigation device, a mobile phone, a smart phone, a portable game machine, and the like.

The series of processes described above can be executed by hardware or can be executed by software.
In other words, the functional configuration of FIG. 3 is merely an example, and is not particularly limited. That is, it is sufficient that the imaging apparatus 1 has a function capable of executing the above-described series of processing as a whole, and what functional blocks are used to realize this function is not particularly limited to the example of FIG.
In addition, one functional block may be constituted by hardware alone, software alone, or a combination thereof.

When a series of processing is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  The recording medium including such a program is not only constituted by the removable medium 31 of FIG. 2 distributed separately from the apparatus main body in order to provide the program to the user, but also in a state of being incorporated in the apparatus main body in advance. It is comprised with the recording medium etc. which are provided in this. The removable medium 31 is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disc is configured by, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disc), a BD (Blu-ray (registered trademark) Disc), or the like. The magneto-optical disk is configured by an MD (Mini-Disk) or the like. In addition, the recording medium provided to the user in a state of being preliminarily incorporated in the apparatus main body includes, for example, the ROM 12 in FIG. 2 in which the program is recorded, the hard disk included in the storage unit 19 in FIG.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series along the order, but is not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.

  As mentioned above, although several embodiment of this invention was described, these embodiment is only an illustration and does not limit the technical scope of this invention. The present invention can take other various embodiments, and various modifications such as omission and replacement can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are included in the invention described in the claims and the equivalent scope thereof.

The invention described in the scope of claims at the beginning of the filing of the present application will be appended.
[Appendix 1]
Imaging means;
Measuring means for measuring the focus position of the subject for each of a plurality of areas obtained by dividing the captured image captured by the imaging means;
Counting means for counting the number of areas corresponding to the focus position of each subject obtained by measurement by the measuring means;
A determining unit that determines a focus position for shooting based on a depth of field of the imaging unit according to the number of areas corresponding to the focus position of each subject counted by the counting unit;
An imaging apparatus comprising:
[Appendix 2]
The number of focus positions for shooting is less than the number of focus positions of the subject;
The imaging apparatus according to appendix 1, characterized by:
[Appendix 3]
A calculating means for calculating the number of shots of the focus bracket according to the focus position for shooting determined by the determining means;
The imaging apparatus according to appendix 1 or 2, characterized by:
[Appendix 4]
The determination means prioritizes a focus position of a subject having a large number of areas counted by the counting means, and determines a focus position for photographing corresponding to the focus position of the subject;
The imaging apparatus according to any one of supplementary notes 1 to 3, wherein:
[Appendix 5]
The determining unit determines a focus position for a plurality of photographing in consideration of an overlap of depth of field of the imaging unit so that a focus state at each subject distance becomes a predetermined state;
The imaging apparatus according to any one of appendices 1 to 4, characterized by:
[Appendix 6]
The determining means provides a threshold value for the number of areas, and determines a focus position for shooting by giving priority to a focus position of a subject in an area equal to or greater than the threshold value;
The imaging apparatus according to any one of appendices 1 to 5, characterized by:
[Appendix 7]
The determination means prioritizes a focus position of a subject having a large depth of field to determine a focus position for shooting;
The imaging apparatus according to any one of appendices 1 to 6, characterized by:
[Appendix 8]
The determining means determines a focus position for shooting by giving priority to a focus position of a subject according to a shooting mode;
The imaging apparatus according to any one of appendices 1 to 7, characterized by:
[Appendix 9]
The determining means detects a face in the shooting mode among the shooting modes, determines a focus position for shooting by giving priority to the focus position of the detected face,
The imaging apparatus according to appendix 8, characterized by:
[Appendix 10]
The determination means prioritizes a focus position of a face having a large number of areas when a plurality of faces are detected;
The imaging apparatus according to appendix 9, characterized by:
[Appendix 11]
The determining means determines a focus position for photographing with priority given to a focus position of a subject weighted according to a distribution situation of a focus position area of the same subject;
The imaging apparatus according to any one of appendices 1 to 10, characterized by:
[Appendix 12]
The determining means performs high weighting when the focus position areas of the same subject are adjacent to each other,
The imaging apparatus according to appendix 11, characterized by:
[Appendix 13]
Determination means for determining whether or not a captured image suitable for generating an omnifocal image can be acquired based on the focus position for shooting determined by the determination means;
Output control means for controlling the output means so as to output the determination result of the determination means;
The imaging apparatus according to any one of supplementary notes 1 to 12, further comprising:
[Appendix 14]
Image combining means for combining images, and
The image synthesizing means generates an omnifocal image by synthesizing images taken with a predetermined focus bracket;
The imaging device according to any one of appendices 1 to 13, characterized by:
[Appendix 15]
In an imaging method executed by an imaging device including an imaging unit,
A measurement step of measuring the focus position of the subject for each of a plurality of areas obtained by dividing the captured image captured by the imaging unit;
A counting step of counting the number of areas corresponding to the focus position of each subject obtained by the measurement step;
A determination step for determining a focus position for photographing based on a depth of field of the imaging means according to the number of areas corresponding to the focus position of each subject counted in the counting step;
An imaging method comprising:
[Appendix 16]
A computer for controlling an imaging apparatus including an imaging means;
Measuring means for measuring the focus position of the subject for each of a plurality of areas obtained by dividing the captured image captured by the imaging means;
Counting means for counting the number of regions corresponding to the focus position of each subject obtained by measurement by the measuring means;
Determining means for determining a focus position for photographing based on the depth of field of the imaging means according to the number of areas corresponding to the focus position of each subject counted by the counting means;
A program characterized by functioning as

  DESCRIPTION OF SYMBOLS 1 ... Imaging device, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Bus, 15 ... Input-output interface, 16 ... Imaging part, 17 ... Input unit 18 ... Output unit 19 ... Storage unit 20 ... Communication unit 21 ... Drive 31 ... Removable media 41 ... Optical lens unit 42 ... Image Sensor, 43... Lens drive unit, 51... Imaging control unit, 52... Captured image acquisition unit, 53... Focus information acquisition unit, 54. Position determining unit 56... Shooting number determining unit 57... Image compositing unit 71... Imaging information storage unit 72.

The present invention relates to an image generation apparatus, an image generation method, and a program.

Conventionally, an image generating apparatus, an image focused on all the subjects from the rear to the front (hereinafter referred to as "all-in-focus images") when generating, such ultra-deep lenses (commonly known as "worms eye lens") Ru performing imaging using a special lens.
However, the insect lens is very inconvenient to carry because of its very long overall length.
For this reason, an image generation apparatus equipped with a normal lens shoots a plurality of subjects having the same composition at a predetermined plurality of focus positions almost simultaneously by a single shutter operation, and obtains data of a plurality of captured images obtained as a result. A technique for generating omnifocal image data by synthesizing is disclosed (see Patent Document 1).

However, in Patent Document 1, the captured image used for generating the data of the omnifocal image captures a plurality of subjects having the same composition at a plurality of predetermined focus positions almost simultaneously. Take a picture without considering it. For this reason, since a captured image that is not suitable for generation of omnifocal image data is acquired and synthesized , there is a problem in that the image quality of the generated omnifocal image is affected .

The present invention has been made in view of such a situation, and an object thereof is to generate an omnifocal image taking image quality into consideration.

In order to achieve the above object, an image generation apparatus according to an aspect of the present invention includes:
An image generation device that generates an omnifocal image,
Imaging means;
A plurality of measuring means for measuring the focus position of the Utsushitai included in the captured image captured by the imaging means,
When the focus position of the subject measured by the measuring means is included in the range of the depth of field of the lens included in the imaging means in a captured image obtained by one capturing, one capturing is performed. If not included, an image generating means for generating an omnifocal image by combining a plurality of captured images obtained by capturing at a plurality of different focus positions;
It is characterized by providing.

According to the present invention, it is possible to generate an omnifocal image that takes image quality into consideration.

Claims (13)

Imaging means;
Measuring means for measuring the focus position of the subject for each of a plurality of areas obtained by dividing the captured image captured by the imaging means;
The focus of the subject according to the depth of field of the lens of the imaging unit, including the focus position of the subject with high priority among the focus positions of the subject obtained by the measurement unit. A determination means for determining a focus position for shooting, which is different from the position;
An imaging apparatus comprising: A counting means for counting the number of areas in which the focus positions of the subjects obtained by measurement by the measuring means are regarded as equivalent;
The determining means determines the focus position for the photographing by giving priority to a focus position of a subject having a large number of counted areas;
The imaging apparatus according to claim 1. The determination means determines the focus position for the photographing in preference to the focus position of the subject having a large depth of field when there are a plurality of focus positions of the subject having the same counted number of areas. To do,
The imaging apparatus according to claim 2. The determining means prioritizes a focus position of a subject according to a shooting mode to determine a focus position for the shooting;
The imaging apparatus according to claim 1. The determination means detects a face in the shooting mode among the shooting modes, and determines the focus position for the shooting by giving priority to the focus position of the detected face;
The imaging apparatus according to claim 4. A counting means for counting the number of areas in which the focus positions of the subjects obtained by measurement by the measuring means are regarded as equivalent;
The determination means prioritizes a focus position of a face having a large number of counted areas when a plurality of faces are detected;
The imaging apparatus according to claim 5. The determining means determines the focus position for the shooting by giving priority to the focus position of the subject weighted according to the distribution state of the focus position area of the same subject;
The imaging apparatus according to claim 1. The determining means performs high weighting when the focus position areas of the same subject are adjacent to each other,
The imaging apparatus according to claim 7. The number of focus positions for shooting is less than the number of focus positions of the subject;
The imaging apparatus according to any one of claims 1 to 8, wherein Determination means for determining whether or not a captured image suitable for generating an omnifocal image can be acquired based on the focus position for shooting determined by the determination means;
Output control means for controlling the output means so as to output the determination result of the determination means;
The imaging apparatus according to claim 1, further comprising: The determining means determines a focus position for a plurality of shootings for generating an omnifocal image,
Image synthesizing means for synthesizing images,
The image synthesizing unit generates an omnifocal image by synthesizing images photographed at a plurality of focus positions for photographing determined by the determining unit;
The imaging apparatus according to claim 1, wherein In an imaging method executed by an imaging device including an imaging unit,
A measurement process for measuring the focus position of the subject for each of a plurality of regions obtained by dividing the captured image captured by the imaging unit;
The focus of the subject according to the depth of field of the lens of the imaging unit, including the focus position of the subject with high priority among the focus positions of the subject obtained by the measurement process. A determination process for determining a focus position for shooting, which is different from the position;
An imaging method comprising: In a computer that controls an imaging apparatus including an imaging means,
A measurement function for measuring a focus position of a subject for each of a plurality of regions obtained by dividing a captured image captured by the imaging unit;
The focus of the subject according to the depth of field of the lens of the imaging unit, including the focus position of the subject with high priority among the focus positions of the subject obtained by the measurement function. A determination function that determines the focus position for shooting, which is different from the position;
A program characterized by realizing.

Images