JP2013186293A - Image generation device and image display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology capable of accurately displaying a subject in a depth of field.SOLUTION: An image generation device comprises: a light-receiving sensor that receives light transmitted through a lens; a distance measuring sensor provided on the same substrate for the light-receiving sensor; and an image generation unit that generates image data according to the amount of light received by the light-receiving sensor. The image generation device measures a distance between a subject photographed by the light-receiving sensor and a reference position on the basis of the output of the distance measuring sensor; obtains lens information including diaphragm value, a focal distance, and a subject distance of the lens; specifies the subject included in a depth of field on the basis of the lens information, and the distance between the subject and the reference position; and displays information indicating the subject included in the depth of field, and an image indicated by the image data on a display unit.

Description

  The present invention relates to an image generation apparatus and an image display method.

  2. Description of the Related Art Conventionally, a technique for facilitating discrimination of a focused subject by enhancing a focused region in an image showing the subject is known. For example, Patent Document 1 discloses a technique for deriving an AF evaluation value of each area of a subject image and emphasizing an area having an AF evaluation value at the same level as the in-focus area as the in-focus area.

JP 2003-153043 A

In the conventional technology, even if the focus is not exactly in accordance with the depth of field, it can be photographed so that it is in focus, but in this way, it can be photographed so that it is in focus (in this application, this is referred to as “ "It is included in the depth of field.") The subject could not be specified accurately. In the technique disclosed in Patent Document 1, an in-focus area is specified by an AF evaluation value, and the AF evaluation value is generally specified based on a phase difference or contrast of a subject. Therefore, the AF evaluation value does not directly indicate that the subject exists within the depth of field, but indirectly evaluates the possibility that the subject exists within the depth of field. Only. For this reason, when the contrast of the subject is small, the degree of focus cannot be accurately determined by the AF evaluation value.
The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique capable of accurately showing a subject within the depth of field.

  An image generating apparatus according to the present invention is an image generating apparatus that generates image data according to the amount of light received by a light receiving sensor, and the light receiving sensor is based on the output of a distance measuring sensor provided on the same substrate as the light receiving sensor. Measure the distance between the subject to be photographed and the reference position. Then, the image generation device identifies the subject included in the depth of field based on the lens information including the lens aperture, the focal length, and the subject distance, and the distance between the subject and the reference position. Information indicating the subject included in the depth of field is displayed on the display unit together with the image indicated by the image data.

  That is, in the image generation apparatus, the distance between the subject and the reference position is measured by a distance measuring sensor provided on the same substrate as the light receiving sensor, and the subject within the depth of field is specified. In the configuration in which the distance measuring sensor is provided on the same substrate as the light receiving sensor, the distance measuring sensor receives the light that has passed through the lens as well as the light receiving sensor that receives the light that has passed through the lens. . Therefore, the distance measuring sensor can measure the distance between the subject and the reference position based on the light traveling on the optical path that is substantially the same as the optical path toward the light receiving sensor. For this reason, in the image generation apparatus, it is possible to accurately specify whether or not the subject exists within the depth of field, and the display unit can accurately indicate the subject within the depth of field. is there.

  Here, the light receiving sensor may be a sensor that receives light that has passed through one or more lenses. For example, the light receiving sensor can be configured by a sensor in which light receiving elements that output information according to the amount of received light are two-dimensionally arranged. is there.

  The distance measuring sensor only needs to be able to measure the distance between the subject photographed by the light receiving sensor and the reference position based on the output of the distance measuring sensor. The reference position only needs to be a reference when measuring the distance between the subject and one point in the image generating apparatus, and the position of the light receiving surface of the light receiving sensor in the optical axis direction, the position of the principal point, or the like may be used as the reference position. I can do it. The distance between the subject and the reference position may be measured directly or indirectly. For example, in a configuration in which the distance between the subject and the light receiving sensor can be measured, the distance between the subject and the reference position can be directly measured if the light receiving surface of the light receiving sensor is used as the reference position. On the other hand, in a configuration that can measure the distance between the subject and the light receiving sensor, if the principal point is the reference position, the distance between the subject and the reference position is obtained by adding the distance between the principal point and the light receiving sensor to the measured value. Can be measured indirectly.

  In addition, the distance measuring sensor only needs to be able to specify the distance between the target subject indicating whether or not it exists within the depth of field and the reference position, and within the depth of field in the image indicated by the image data. It is only necessary to be able to specify the distance between the subject and the reference position in the portion where the subject to be identified is present. Therefore, in order to determine whether or not a subject existing at a specific location in the image exists within the depth of field, the distance measuring sensor is present at the specific location on the same substrate as the light receiving sensor. It is only necessary that the distance between the subject and the reference position can be measured. In addition, if the distance between the subject and the reference position is to be measured with one distance measuring sensor for each part, the same number of distance measuring sensors as the number of parts may be provided in the light receiving sensor.

  The image generation unit only needs to be able to generate image data according to the amount of light received by the light receiving sensor. That is, it is only necessary to generate image data indicating an image of a subject based on the amount of light received by each light receiving element within a predetermined exposure time in a state where light from the subject is irradiated to the light receiving sensor. Of course, various types of image processing may be executed as the image data is generated.

  The lens information acquisition unit only needs to acquire lens information. That is, it is only necessary to acquire information regarding the lens that is a parameter for calculating the depth of field. The lens information is information including a lens aperture, a focal length, and a subject distance. The lens information may be configured to identify lens information based on a setting value in the image generation apparatus, or may be information output from an interchangeable lens. The lens information may be specified based on the lens information. Of course, any information that can calculate a distance equivalent to the focal length or the subject distance may be used, and various expressions can be adopted. That is, as long as a distance substantially equivalent to the focal length and the subject distance is used, it is within the scope of the present invention. For example, when the distance between the focused object and the principal point is the subject distance, the shooting distance that is the distance between the focused object and the light receiving sensor is the distance from the principal point to the light receiving sensor. Since the value is obtained by adding

  The subject specifying unit only needs to be able to specify the subject included in the depth of field based on the distance between the subject and the reference position and the lens information. The depth of field can be specified by lens information including the hyperfocal length, the focal length of the lens, and the subject distance that can be specified by the lens aperture, the focal length, and the diameter of the allowable circle of confusion, according to its optical definition. is there. If the diameter of the permissible circle of confusion is a predetermined fixed value, whether or not the subject exists within the depth of field can be specified based on the lens information and the distance between the subject and the reference position. It is.

  The display control unit only needs to display information indicating the subject included in the depth of field and an image indicated by the image data on the display unit. That is, information is displayed on the display unit so that the subject included in the depth of field can be visually recognized together with the subject image. The position of the information indicating the subject included in the depth of field is not limited, but the information indicating the subject included in the depth of field is preferably displayed at the position of the image of the subject. For example, it is possible to employ a configuration in which information indicating the subject included in the depth of field is displayed by highlighting the subject included in the depth of field.

  Further, the distance measuring sensor only needs to be able to output information for measuring the distance between the subject photographed by the light receiving sensor and the reference position, and is based on the light from the subject that has passed through the lens. A sensor or the like that measures the distance can be used. More specifically, the electromagnetic wave output unit is configured to output an electromagnetic wave, and after the electromagnetic wave output unit outputs the electromagnetic wave, the electromagnetic wave is reflected by the subject and is measured by the distance measuring sensor. It is good also as a structure which measures distance. That is, if the speed at which the electromagnetic wave travels in the space is known, the distance of the entire optical path can be specified based on the time from the output of the electromagnetic wave to the measurement by the distance measuring sensor. Therefore, the distance between the subject and the reference position can be acquired based on a distance that is ½ of the total optical path distance.

  The electromagnetic wave may be output from the electromagnetic wave output unit included in the image generation device, may be reflected by the subject and can be detected by the distance measuring sensor, and is different from light for generating image data received by the light receiving sensor. An electromagnetic wave in the wavelength range is preferable. Examples of such electromagnetic waves include infrared rays.

  Further, a distance measuring sensor may be provided in each pixel of the light receiving sensor. According to this configuration, it is possible to measure the distance between the portion of the subject photographed by each pixel of the light receiving sensor and the reference position. As a result, it is possible to show in detail whether or not the subject exists within the depth of field in all the subjects included in the image indicated by the image data.

  Furthermore, a distance measuring sensor may be provided for each divided area obtained by dividing the light receiving surface of the light receiving sensor. According to this configuration, it is possible to measure the distance between the subject photographed by the light receiving sensor in the divided area and the reference position. As a result, it is possible to indicate whether or not the subject exists within the depth of field for each subject photographed by the light receiving sensor in each divided region.

  Further, as in the present invention, the depth of field is included based on the distance between the subject photographed by the light receiving sensor and the reference position, which is measured by the distance measuring sensor provided on the same substrate as the light receiving sensor. The method for specifying and displaying the subject can also be applied as a program or method. In addition, the apparatus, program, and method as described above may be realized as a single apparatus, or may be realized using a shared component in an apparatus having multiple functions. Is included.

It is a block diagram of an imaging device including an image generation device according to an embodiment of the present invention. (2A) is a schematic diagram showing an imaging unit, and (2B) is an explanatory diagram for explaining the depth of field. It is a flowchart which shows an imaging | photography process.

Here, embodiments of the present invention will be described in the following order.
(1) Configuration of image generation apparatus:
(2) Shooting process:
(3) Other embodiments:

(1) Configuration of image generation apparatus:
FIG. 1 shows an imaging apparatus 1 including an image generation apparatus according to an embodiment of the present invention. The imaging apparatus 1 includes an optical system 10, an imaging unit 14, a storage unit 15, a display unit 20, and a recording unit 30. , An operation unit 40, an aperture adjustment unit 45, a shutter control unit 50, an imaging unit control unit 60, a CPU 70, an image generation unit 80, and an electromagnetic wave output unit 90. The CPU 70 includes a diaphragm adjustment unit 45, a shutter control unit 50, an imaging unit control unit 60, a storage unit 15, an image generation unit 80, an electromagnetic wave output unit 90, a display unit 20, a recording unit 30, and an operation unit 40 according to a predetermined program. Control the behavior.

  The operation unit 40 includes a shutter button, an operation unit for switching a shooting mode, an operation unit for switching an aperture, an operation unit for switching a shutter speed, and an operation unit for operating various setting menus. The user can give various instructions to the photographing apparatus 1 by operating the operation unit 40. In this embodiment, the stroke amount of the shutter button can be detected in two stages. That is, it is possible to distinguish and detect a state where the shutter button is half-pressed and a state where the shutter button is fully pressed. In the present embodiment, when the shutter button is half-pressed, it is considered that an instruction to adjust the focus while continuing the live view display is given. When the shutter button is fully pressed, an image is recorded on the recording medium. The instruction to record is considered to have been given.

  The display unit 20 includes an interface circuit (not shown), a liquid crystal panel driver, a liquid crystal panel, an eyepiece (not shown), and the like. In the present embodiment, the display unit 20 is an EVF (Electronic View Finder) that displays an image showing a subject to be photographed and allows the user to grasp information such as the state of the subject before photographing and photographing conditions. The imaging device 1 according to the embodiment is a mirrorless digital camera provided with an EVF.

  The recording unit 30 can insert a recording medium (not shown), and records information on the recording medium and reads information from the recording medium with the recording medium inserted in the recording unit 30. Can do. That is, the image data indicating the photographed image can be recorded on the recording medium.

  The optical system 10 includes a lens 11 that forms a subject image on a light receiving sensor, an aperture 12, and a shutter 13. In the present embodiment, the lens 11 and the diaphragm 12 are provided in a lens barrel, and the lens barrel is attached to a housing (not shown) in a replaceable manner. In the present embodiment, the lens 11 includes a plurality of lenses arranged along a direction parallel to the optical axis, but in FIG. 1, only one lens is represented for the sake of simplicity. Each lens is supported by an outer edge portion, and one or all of the lenses can be moved in the optical axis direction by a user operating a focus ring provided in the lens barrel or by a focus adjustment instruction from the photographing apparatus 1. This makes it possible to adjust the focus position. Further, an optical zoom operation is performed by allowing the user to operate a zoom ring provided in the lens barrel or to move one or all of the lenses in the optical axis direction in response to a zoom adjustment instruction from the photographing apparatus 1. It is possible. That is, in this example, a manual focus lens is attached to the photographing apparatus 1. Note that the manual focus lens in the present embodiment can output information indicating the focal length and subject distance by communicating with the CPU 70. Here, the subject distance is the distance between the subject in focus by the lens 11 and the photographing surface of the principal point, and a value corresponding to the rotation angle of the focus ring is output. Therefore, the CPU 70 can specify the subject distance corresponding to the state of the lens 11 after being operated by the focus ring at an arbitrary timing together with the focal length. In addition, the configuration of the optical system may adopt other configurations, for example, a configuration in which the lens is configured by a liquid lens, the focus position is adjusted by deforming the lens, and an optical zoom operation is performed. It is good.

  The diaphragm 12 is composed of a plurality of shielding plates that are rotatably supported in a plane perpendicular to the optical axis of the lens 11, and the plurality of shielding plates rotate in conjunction with each other to rotate relative to the optical axis. It is possible to change the area of the unshielded part in the vertical plane. The aperture diameter of the aperture 12 is configured to be controlled by an aperture adjuster 45. When the aperture adjuster 45 is instructed about the aperture diameter of the aperture 12, the aperture adjuster 45 drives the aperture 12 to Set to the indicated opening diameter.

  In this embodiment, the shutter 13 is a mechanical focal plane shutter, and includes a plurality of open / close (folding) light shielding curtains as planar plate-shaped light shielding portions parallel to the light receiving surface of the light receiving sensor. . The light-shielding curtain is configured to move in a direction perpendicular to the optical axis in accordance with a control signal from the shutter control unit 50. Normally, the light-shielding curtain does not block the optical path in the direction parallel to the optical axis. Is held by. In addition, in a state where the light shielding curtain is held without blocking the light path, when a predetermined trigger is given, the state where the light shielding curtain is held without blocking the light path is released, and the light shielding curtain is moved with respect to the optical axis. Thus, the blades are driven in the vertical direction, and the plurality of blades block the optical path. In FIG. 1, the moving direction of the shutter 13 is indicated by a dashed arrow Am.

  The imaging unit 14 includes RGB light receiving sensors each including an RGB color filter and a light receiving element that accumulates a charge corresponding to the amount of light for each pixel by photoelectric conversion, and a distance measuring sensor. In the present embodiment, a light receiving sensor and a distance measuring sensor are provided for each pixel. The light receiving surface on which the light receiving sensor and the distance measuring sensor are formed in the imaging unit 14 is a rectangular region, and the light receiving surface of the imaging unit 14 (the surface directed toward the lens 11) is seen on the left side of FIG. 2A. The rectangular area | region visually recognized in the case is shown typically.

  The right side of FIG. 2A shows an enlarged view of the light receiving sensor and the distance measuring sensor formed in the imaging unit 14. In this enlargement, the light receiving sensor and the distance measuring sensor are indicated by rectangles. As illustrated in FIG. 2A, the imaging unit 14 includes RGB light receiving sensors 14a, 14b, 14c and a distance measuring sensor 14d on the light receiving surface. A plurality of light receiving sensors corresponding to RGB colors and one distance measuring sensor constitute one pixel. In the enlarged view of FIG. 2A, one pixel is surrounded by a thick rectangle. The light receiving sensors 14a, 14b, and 14c are sensors that output information indicating the intensity of light received from the subject for each of RGB, and an image that indicates an image of the subject based on the outputs of the light receiving sensors 14a, 14b, and 14c. Data is generated. The generated image data is stored in the storage unit 15.

  The storage unit 15 is a memory that temporarily records image data (information indicating the amount of light received for each light receiving element) output from the light receiving sensors 14a, 14b, and 14c. The light receiving sensor in the present embodiment is a CMOS (Complementary Metal Oxide Semiconductor) image sensor, but the light receiving sensor may of course be another sensor such as a CCD (Charge Coupled Device) image sensor.

  The light receiving sensors 14a, 14b, and 14c according to the present embodiment perform a reset operation for resetting the accumulated charge corresponding to the amount of light received by the light receiving element when the imaging unit control unit 60 issues a reset instruction, and exposure at each light receiving element. Can start. In addition, the light receiving sensors 14a, 14b, and 14c can end the exposure by reading out information indicating the amount of light received by the light receiving element when the imaging unit control unit 60 issues a reading instruction. Furthermore, in the light receiving sensors 14a, 14b, and 14c, the start and end of these exposures can be controlled for each line or for each screen.

  When a recording image is captured by the imaging apparatus 1 according to the present embodiment (normal imaging), the exposure time is controlled by a combination of the shutter 13 that is a mechanical shutter and the electronic shutters of the light receiving sensors 14a, 14b, and 14c. To do. That is, in the case of normal shooting in the present embodiment, the exposure time is determined by the electronic front curtain-mechanical rear curtain shutter system in which exposure is started by the electronic shutters in the light receiving sensors 14a, 14b, and 14c and the exposure is ended by the shutter 13. Be controlled. Specifically, in the case of normal shooting, exposure is started in a line-sequential manner with an electronic shutter, and the light is shielded by a mechanical shutter so that each line is shielded at the timing at which the exposure time for each line reaches a set shutter speed. Is started. In the photographing apparatus 1, live view display can be performed on the display unit 20, and when an image for performing the live view display is photographed, the exposure time is controlled by an electronic shutter method. That is, both the front curtain and the rear curtain are controlled by the electronic shutter.

  The distance measuring sensor 14d is a sensor that receives infrared rays reflected by the subject and outputs information indicating the distance between the subject and the light receiving sensors 14a, 14b, and 14c. That is, the electromagnetic wave output unit 90 included in the photographing apparatus 1 is an apparatus that outputs infrared rays (Ir indicated by a one-dot chain line in FIG. 1) over a wide range in the visual field direction of the optical system 10. It exists on the extended line of the light receiving surface of the unit 14. When the infrared light periodically output from the electromagnetic wave output unit 90 is reflected by the subject and reaches the distance measuring sensor 14d, the distance measuring sensor 14d outputs information indicating that the infrared light has arrived to the imaging unit control unit 60. . The imaging unit control unit 60 further passes information indicating that the infrared rays have arrived to the CPU 70. The distance measuring sensor 14d and the light receiving sensors 14a, 14b, and 14c are formed on the same substrate, and the distance measuring sensor 14d and the light receiving sensors 14a, 14b, and 14c are arranged so that the portions exposed to the light receiving surface are the same surface. Is formed. Therefore, in this embodiment, the distance from the subject to the distance measuring sensor 14d and the distance from the subject to the light receiving sensors 14a, 14b, and 14c are regarded as the same distance.

  Therefore, the CPU 70 outputs a control signal for outputting infrared rays to the electromagnetic wave output unit 90 and measures an elapsed time after the infrared rays are output. When it is detected by the distance measuring sensor 14d that infrared rays have arrived, information indicating the detection is output to the CPU 70 by the imaging unit control unit 60. Therefore, the CPU 70 receives the infrared sensor from the subject based on the time from when the infrared ray is output from the electromagnetic wave output unit 90 until the infrared ray is reflected by the subject and measured by the distance measuring sensor 14d and the speed at which the infrared ray travels. The distance to 14a, 14b, 14c is specified.

  The image generation unit 80 is configured by a circuit that executes various processes on image data output from the light receiving sensors 14a, 14b, and 14c according to a predetermined procedure. Specifically, the image generation unit 80 acquires the image data stored in the storage unit 15 by the processing of the CPU 70, performs predetermined image processing such as γ conversion on the image data indicated by the image data, and displays the image data. And a function of generating data indicating a recording image and a recording image. Data indicating the display image is transferred to the display unit 20, and the captured image is displayed on the display unit 20. Data indicating an image for recording is transferred to the recording unit 30, and the data is recorded on a recording medium inserted into the recording unit 30.

  The image processing executed by the image generation unit 80 includes processing for outputting an evaluation value for performing AE (Automatic Exposure) processing. That is, the image generation unit 80 evaluates the brightness of pixels included in a predetermined photometry area set within the photographing range by the light receiving sensors 14a, 14b, and 14c (for example, an average value of luminance). Can be specified and output as an evaluation value for performing the AE process.

  The CPU 70 outputs a control signal to the aperture adjustment unit 45, the shutter control unit 50, and the imaging unit control unit 60 based on the evaluation value for performing the AE process, and the aperture diameter and the shutter speed of the aperture 12 so as to achieve proper exposure. Adjust. That is, the CPU 70 has a function of specifying shooting conditions when shooting image data based on an instruction or default setting in the operation unit 40 and setting each unit according to the shooting conditions. Specifically, the CPU 70 specifies shooting conditions necessary for achieving proper exposure based on the evaluation value for performing the above-described AE processing, and the aperture 12 and the shutter speed are set according to the shooting conditions. The control signal is output to the aperture adjustment unit 45, the shutter control unit 50, and the imaging unit control unit 60. Note that the shooting conditions necessary for achieving proper exposure can be specified under various preconditions (for example, aperture priority, shutter speed priority, etc.).

  Here, the shutter speed indicates the time during which the light receiving sensors 14a, 14b, and 14c are exposed to light, and the CPU 70 exposes each line of the light receiving sensors 14a, 14b, and 14c for a time corresponding to the shutter speed. The exposure start timing in the light receiving sensors 14a, 14b, and 14c is adjusted as shown.

  In the present embodiment, the user can adjust the focus position by the manual focus lens. In order to focus on the subject selected by the user himself / herself as the focus target, the focus ring is adjusted so that the subject selected by the user himself / herself as the focus target exists within the depth of field. It is necessary to In a general photographing apparatus, the focus position is adjusted by operating the focus ring so that the edge of the subject is in an appropriate state by the user's own eyes. However, it is generally difficult to determine whether or not the edge is in an appropriate state by viewing the display unit 20 in an arbitrary subject.

  Therefore, in the present embodiment, the CPU 70 is a subject existing within the depth of field in a state where the subject is visually recognized by the EVF (a state where live view display is performed) before the image data for recording is captured. Perform processing to emphasize. That is, in this embodiment, the CPU 70 outputs a control signal for outputting infrared rays to the electromagnetic wave output unit 90 and measures the elapsed time after the infrared rays are output, so that the infrared rays are output by the distance measuring sensor 14d after the infrared rays are output. The time until detection is specified, and the distance from the subject to the light receiving sensors 14a, 14b, and 14c is specified for each pixel by the speed of infrared rays. In this state, the distance from the subject photographed by the light receiving sensors 14a, 14b, and 14c of each pixel to the light receiving sensors 14a, 14b, and 14c is specified.

  Further, the CPU 70 specifies the distance (which may be the subject distance and the depth of field) that can be photographed so as to be in focus according to the state of the lens 11 and the state of the diaphragm 12 that is set at the time of photographing. It is determined for each subject whether or not the subject is included in the depth of field. The depth of field is a hyperfocal distance specified based on the focal length corresponding to the state of the lens 11, the F value corresponding to the aperture diameter of the stop 12, and the value of the diameter of the allowable circle of confusion specified in advance. It is calculated using. That is, the CPU 70 specifies the front end and the rear end of the depth of field based on the hyperfocal distance, the focal length, and the subject distance, and the range between the front end and the rear end is the object scene. It is regarded as the range of the distance between the object included in the depth and the principal point. Then, the CPU 70 determines that the subject is included in the depth of field when the distance from the subject to the light receiving sensors 14a, 14b, and 14c exists between the front end and the rear end. The calculation regarding the depth of field will be described in detail later. The determination is performed for each pixel, and the CPU 70 performs a process of emphasizing the subject (for example, binarization or bordering) on the image data of the pixel obtained by imaging the subject determined to be included in the depth of field. Do.

  Then, the CPU 70 outputs the image data that has been subjected to the process of emphasizing the subject to the display unit 20, and causes the display unit 20 to display the subject based on the image data. As a result, the subject in the field of view of the optical system 10 is displayed on the display unit 20 in a state where the subject included in the depth of field is emphasized. The user can identify the subject within the depth of field by visually recognizing the display unit 20, and therefore focus until the subject that he wants to focus on is highlighted on the display unit 20. By adjusting, it is possible to easily adjust the focus position so that the subject exists within the depth of field.

  When the user gives an instruction to shoot image data for recording, the CPU 70 outputs a control signal to the aperture adjustment unit 45 to set the aperture diameter of the aperture 12 to a size for shooting, and to the imaging unit control unit 60. A control signal is output to start exposure for each line of the light receiving sensors 14a, 14b, and 14c, and the control signal is output to the shutter control unit 50 at a timing such that exposure is performed for a time corresponding to the set shutter speed. The exposure is terminated by the shutter. As a result, image data indicating the image of the subject is output to the storage unit 15, and the CPU 70 transfers the image data to the recording unit 30 and records the image data on a recording medium (not shown).

  In the above configuration, the distance between the subject and the light receiving sensors 14a, 14b, and 14c is measured for each pixel by the distance measuring sensor 14d provided on the same substrate as the light receiving sensors 14a, 14b, and 14c. Whether each pixel exists within the depth of field is specified for each pixel. Similarly to the light receiving sensors 14a, 14b, and 14c that receive the light that has passed through the lens 11, the distance measuring sensor 14d also receives the light that has passed through the lens 11. Accordingly, the distance measuring sensor 14d can measure the distance between the subject and the light receiving sensors 14a, 14b, and 14c based on the light that travels in substantially the same optical path as the light path toward the light receiving sensors 14a, 14b, and 14c. . For this reason, compared with the case where the distance measuring sensor is provided in a completely different place, in the photographing apparatus 1, it is possible to more accurately identify which subject is present within the depth of field. The unit 20 can more accurately indicate the subject within the depth of field.

(2) Shooting process:
Next, a photographing process in the present embodiment will be described in detail with an example. FIG. 3 is a flowchart of the photographing process. The CPU 70 normally executes processing for performing live view display on the display unit 20. Here, an example in which shooting processing is performed in the aperture priority mode will be described. In the present embodiment, an instruction to execute the AE process and the process of displaying the subject included in the depth of field on the display unit 20 is performed by pressing the shutter button halfway. Therefore, first, the CPU 70 determines whether or not the shutter button has been half-pressed, and waits until it is determined that the shutter button has been half-pressed (step S100). When the shooting mode is started in the aperture priority mode as in this example, the CPU 70 next adjusts the aperture 12 so that the aperture diameter according to the set value of the aperture 12 set by the user is obtained (step S105). ). That is, the user sets the F value of the diaphragm 12 in advance by the operation unit 40. The CPU 70 outputs a control signal for adjusting the diaphragm 12 to the set F value to the adjustment unit 45, and the diaphragm adjustment unit 45 controls the diaphragm 12 so that the aperture diameter corresponds to the F value of the diaphragm 12. .

  When the aperture diameter of the diaphragm 12 is controlled to be a set value set by the user, the AE process is executed next. That is, the CPU 70 evaluates the brightness of the pixels in the photometry area set in advance within the photographing range of the light receiving sensor in a state where the aperture diameter of the diaphragm 12 becomes the set value set by the user. An evaluation value for performing the AE processing is specified for the CPU 70 and output as an evaluation value for performing the AE process (step S110). Note that the shutter speed at this stage is a default value (or a set value at the time of the previous shooting).

  Next, the CPU 70 adjusts the shutter speed so as to achieve proper exposure based on the evaluation value for performing the AE process (step S115). That is, the CPU 70 compares the evaluation value for performing the AE process with a predetermined appropriate exposure range, and specifies the shutter speed necessary for the evaluation value for performing the AE process to fall within the proper exposure range. . Then, the CPU 70 outputs a control signal indicating the shutter speed to the imaging unit controller 60. As a result, the imaging unit control unit 60 adjusts the shutter speed of the electronic shutter in the light receiving sensor so that the evaluation value for performing the AE process becomes a shutter speed necessary for entering the appropriate exposure range.

  With the AE process as described above, the CPU 70 transfers the image data generated by the image generation unit 80 to the display unit 20 and continues the live view display on the display unit 20 in a state where the exposure of the photographed image is appropriate. . In such a live view display state, the user can adjust the focus position by adjusting the focus ring, but the display unit 20 simply displays the image of the subject. In this case, since it is often difficult to determine an appropriate focus position, the CPU 70 performs a process for clearly indicating the focus position.

  That is, the CPU 70 acquires the distance L between the subject and the light receiving sensor for each pixel (step S120). That is, the CPU 70 outputs a control signal that causes the electromagnetic wave output unit 90 to output infrared rays, and causes the electromagnetic wave output unit 90 to output infrared rays in the visual field direction of the optical system 10. At the same time, the CPU 70 starts measuring the elapsed time after infrared rays are output from the electromagnetic wave output unit 90. Note that the electromagnetic wave output unit 90 in the present embodiment can output infrared rays in a wide range determined in advance so that the infrared rays reach a subject within the field of view of the optical system 10. FIG. 2B schematically shows a positional relationship among the electromagnetic wave output unit 90, the imaging unit 14, and the subject O, and schematically shows an infrared output range Rir by a one-dot chain line.

  Further, the CPU 70 outputs information indicating that infrared rays are detected from the distance measuring sensor 14d, and when the information is delivered from the imaging unit control unit 60, the infrared rays are output and then detected by the distance measuring sensor 14d. The distance of the entire optical path from when the infrared ray is output until it reaches the distance measuring sensor 14d is determined by the product of the time until and the velocity of the infrared ray (= light velocity). Infrared light output from the electromagnetic wave output unit 90 is reflected by the subject O and reaches the imaging unit 14 via the optical system 10, but the distance between the electromagnetic wave output unit 90 and the imaging unit 14 is usually the subject and the imaging unit. In this embodiment, the optical path of light reaching the imaging unit 14 from the subject O is considered to be substantially parallel to the horizontal direction. Further, the infrared ray output port of the electromagnetic wave output unit 90 is on an extension line of the light receiving surface of the imaging unit 14. Therefore, the optical path until the infrared ray output from the electromagnetic wave output 90 reaches the subject O is substantially equal to the optical path until the infrared ray reflected by the subject O reaches the light receiving sensor.

  Therefore, the CPU 70 specifies 1/2 of all the optical paths specified as described above as the distance L from the subject to the light receiving sensors 14a, 14b, 14c. This process is performed for all pixels in which infrared rays have been detected by the distance measuring sensor 14d.

  Next, the CPU 70 acquires lens information (step S125). That is, the CPU 70 communicates with the manual focus lens and corresponds to the information indicating the subject distance s which is the distance between the subject in focus and the principal point corresponding to the rotation angle of the focus ring, and the rotation angle of the zoom ring. And information indicating the focal length f of the obtained lens. In addition, the CPU 70 acquires information indicating the F value of the aperture 12 set by the user through the operation unit 40.

Next, the CPU 70 specifies the depth of field (step S130). That is, the depth of field is specified by specifying the front end _DN and the rear end _{DF of the} depth of field. 2B illustrates the positional relationship between the front end _DN and rear end _{DF of the} depth of field and the light receiving sensor. That is, in this example, the front end _DN is the distance between the point closest to the photographing apparatus 1 and the principal point in the in-focus range, and the rear end _DF is the distance from the photographing apparatus 1 in the most in-focus range. This is the distance between the far point and the principal point.

In order to specify the depth of field, the CPU 70 first specifies the hyperfocal distance H based on the following equation.
H = f ² / Nc
Here, f is the focal length of the lens 11, N is the F value of the diaphragm 12, and c is the diameter of a predetermined allowable circle of confusion. The diameter of the permissible circle of confusion may be determined according to the size of the light receiving sensor and the user's designation. For example, if the size of the light receiving sensor is 22.5 mm × 15.0 mm, the allowable confusion circle diameter may be set to 0.019 mm, and if the size of the light receiving sensor is 36 mm × 24 mm, the value of 0.026 mm may be used. it can.
Next, the CPU 70 specifies the front end _DN and the rear end _DF of the depth of field by the following formula based on the hyperfocal distance H, the focal distance f, and the subject distance s.
D _N = sH / (H + s−f)
D _F = sH / (H + fs)

Next, the CPU 70 identifies a subject included in the depth of field (step S135). That is, the CPU 70 reduces the distance between the principal point of the lens 11 and the light receiving sensors 14a, 14b, and 14c from the distance L between the subject photographed by each pixel and the light receiving sensors 14a, 14b, and 14c, thereby removing the principal point from the subject. To obtain the distance Ls. Then, the CPU 70 determines whether or not Ls> D _N and Ls <D _F based on the front end _DN and the rear end D _F, and Ls> D _N and Ls <D _F. When it is determined that there is a subject, it is determined that the subject photographed with the pixel is included in the depth of field. For example, as shown in FIG. 2B, it is determined that a subject photographed with a pixel having a distance Ls between the front end _DN and the rear end _DF of the depth of field is included in the depth of field. The

  Next, the CPU 70 superimposes and displays information indicating the subject included in the depth of field on the image (step S140). That is, in step S135, the CPU 70 performs a process of enhancing the image data of the pixel obtained by photographing the subject determined to be included in the depth of field, thereby superimposing information indicating the subject included in the depth of field. Generated image data is generated. Then, the CPU 70 delivers the generated image data to the display unit 20. As a result, the display unit 20 displays an image in which the subject included in the depth of field is emphasized. The above processing is repeatedly executed every time the image on the display unit 20 is updated while the shutter button is half-pressed. As a result, when the shutter button is pressed halfway during the live view display in the photographing apparatus 1, an image in which the subject included in the depth of field is emphasized is displayed in the live view on the display unit 20.

  Therefore, the user visually recognizes the display unit 20 so that the subject is in focus when the subject that the user desires to focus is emphasized, and the subject when the subject is not emphasized. Can be determined to be out of focus. That is, by visually recognizing the display unit 20, the user can determine whether to shoot a recording image based on the focused state of the subject or to continue focus adjustment.

  In the present embodiment, the CPU 70 is configured to issue an instruction to shoot a recording image by fully pressing the shutter button, so the CPU 70 releases the half-press of the shutter button in the operation unit 40, or It is determined whether or not it has been fully pressed (step S145). If it is determined in step S145 that the half-press of the shutter button has been released, the processes in and after step S100 are repeated. That is, the enhancement of the image is canceled and the live view display on the display unit 20 is continued.

  On the other hand, if it is determined in step S145 that the shutter button has been fully pressed, the CPU 70 captures an image for recording and records it on the recording medium (step S150). That is, the CPU 70 outputs a control signal to the image pickup unit controller 60 in a state where the focus position and the aperture diameter of the diaphragm 12 set in step S100 are maintained, and the exposure of the light receiving sensor is sequentially performed line by line. Let it begin. Further, the CPU 70 outputs a control signal to the shutter control unit 50, and starts driving the shutter 13 at a timing such that the exposure time of each line of the light receiving sensor becomes a time corresponding to the shutter speed set in step S110. Let As a result, since the image data is output from the light receiving sensor and stored in the storage unit 15, the CPU 70 transfers the image data to the recording unit 30 and records it on a recording medium (not shown). When shooting is completed, the process returns to S100 for the next shooting.

(3) Other embodiments:
The above embodiment is an example for carrying out the present invention, and is based on a distance between a subject photographed by the light receiving sensor and a reference position measured by a distance measuring sensor provided on the same substrate as the light receiving sensor. As long as the subject included in the depth of field is specified and displayed, the following modifications may be combined as appropriate, and various other embodiments may be employed.

  For example, in the above-described embodiment, the display unit 20 is an EVF using a liquid crystal panel, but the display unit 20 is a display unit other than the EVF, for example, a display unit using a liquid crystal panel attached to the back of the photographing apparatus 1. It may be, or a system using a system other than the liquid crystal panel may be used. The photographing apparatus 1 may be a single-lens reflex camera provided with a mirror, a movie camera, or a device such as a mobile phone having a photographing function. Furthermore, the arrangement of the light receiving sensors 14a, 14b, and 14c described above may be an arrangement other than the arrangement shown in FIG. 2B. Further, the present invention may be applied to an image generation apparatus using a 3CCD system or a 3-layer type sensor.

  Furthermore, in the above-described embodiment, the manual focus lens is used. However, the auto focus lens is attached to the photographing apparatus 1, the distance between the subject and the reference position is specified based on the output of the distance measuring sensor, Based on the distance and lens information, it is determined whether or not the subject to be focused is included in the depth of field, and the lens position is automatically moved until the subject to be focused is included in the depth of field. A configuration may be adopted.

  Further, the distance measuring sensor may be provided at a rate of one for every N pixels instead of every pixel. For example, a distance measuring sensor is provided for each divided region obtained by dividing the light receiving surface of the light receiving sensor on the same substrate as the substrate on which the light receiving sensor is formed. The distance may be measured by a distance measuring sensor in each divided area. According to this configuration, the number of distance measuring sensors can be suppressed.

Further, the distance measuring sensors may not be arranged uniformly over the entire light receiving surface of the light receiving sensor, but the distance measuring sensor may be provided in a specific area and the distance measuring sensor may not be provided in other areas. Examples of the specific area include areas predetermined as the focus area, such as the center of the light receiving surface and a plurality of locations around the center.
Alternatively, each of the light receiving sensors 14a, 14b, and 14c may be a pixel, and an interpolation or the like may be performed based on the distances measured by the plurality of distance measuring sensors 14d to determine the distance from the subject to each light receiving sensor. .
Further, the distance from the subject to each pixel may be determined by taking into account the distance between the electromagnetic wave output unit 90 and the imaging unit 14.
In the above-described embodiment, the distance is measured using infrared rays. However, the distance may be measured by other methods such as using electromagnetic waves of other wavelengths.
In addition, the process of acquiring the distance between the subject and the light receiving sensor for each pixel (step S120) and the process of acquiring lens information and specifying the depth of field (step S125 and step S130) may be reversed in order. However, parallel processing may be performed.
The distance calculated based on the output of the distance measuring sensor 14d is not limited to the distance between the subject and the light receiving sensor, but the distance between the subject and the lens mounting position, or the distance between the subject and the display unit 20. Other distances may be used, and these distances may be displayed on the display unit 20 by switching from the display in S140. As a display method, the distance to the subject displayed at the center of the display unit may be displayed as a number, or a color corresponding to the distance may be displayed on each subject.
As information on the F value of the aperture 12, information indicating the current F value may be acquired from the lens, a value set by the user may be acquired from the memory, or the aperture may be determined by AE processing. May acquire the value determined by the AE process from the memory.
Further, as the equations for describing the front end _DN and the rear end _DF , in addition to the above-described equations, equations obtained as a result of deformation using various approximations can be used, and the subject distance and the object distance can be substantially used. The front end _DN and the rear end _DF may be described using the same parameter, for example, the shooting distance. That is,
Front end D _N = (hyperfocal length × shooting distance) / (hyperfocal length + shooting distance)
Rear end D _F = (hyperfocal distance × shooting distance) / (hyperfocal distance−shooting distance)
A configuration that specifies the front end _DN and the rear end _DF may be adopted.

DESCRIPTION OF SYMBOLS 1 ... Imaging device, 10 ... Optical system, 11 ... Lens, 12 ... Aperture, 13 ... Shutter, 14 ... Imaging unit, 14a, 14b, 14c ... Light receiving sensor, 14d ... Distance sensor, 15 ... Memory | storage part, 20 ... Display 30 ... Recording unit, 40 ... Operation unit, 45 ... Adjustment unit, 50 ... Shutter control unit, 60 ... Imaging unit control unit, 70 ... CPU, 80 ... Image generation unit, 90 ... Electromagnetic wave output unit

Claims (6)

A light receiving sensor that receives light passing through the lens;
A distance measuring sensor provided on the same substrate as the light receiving sensor;
An image generating unit that generates image data according to the amount of light received by the light receiving sensor;
A distance measuring unit that measures a distance between a subject photographed by the light receiving sensor and a reference position based on an output of the distance measuring sensor;
A lens information acquisition unit that acquires lens information including the aperture, focal length, and subject distance of the lens;
A subject identifying unit that identifies the subject included in the depth of field based on the lens information and a distance between the subject and a reference position;
A display control unit for displaying information indicating the subject included in the depth of field and an image indicated by the image data on a display unit;
An image generation apparatus comprising: Further equipped with an electromagnetic wave output unit for outputting electromagnetic waves,
The distance measuring unit, between the subject and a reference position, is based on a time from when the electromagnetic wave output unit outputs the electromagnetic wave until the electromagnetic wave is reflected by the subject and measured by the distance measuring sensor. Measure distance,
The image generation apparatus according to claim 1. The distance measuring sensor is provided in each pixel of the light receiving sensor,
The image generation apparatus according to claim 1. The distance measuring sensor is provided for each divided region obtained by dividing the light receiving surface of the light receiving sensor.
The image generation apparatus according to claim 1. The distance between the subject photographed by the light receiving sensor and the reference position is a distance from the subject to the principal point of the lens,
The subject specifying unit specifies a hyperfocal distance based on the aperture and focal length of the lens and a diameter of a predetermined allowable circle of confusion, and determines the hyperfocal distance, the focal length of the lens, and the subject distance. Based on whether the distance from the subject to the principal point of the lens is included in the range of the distance between the object included in the depth of field and the principal point of the lens,
The image generation apparatus according to claim 1. A light receiving sensor that receives light passing through the lens;
A distance measuring sensor provided on the same substrate as the light receiving sensor;
An image display method in an image generation apparatus comprising an image generation unit that generates image data according to the amount of light received by the light receiving sensor,
A distance measuring step of measuring a distance between a subject photographed by the light receiving sensor and a reference position based on an output of the distance measuring sensor;
A lens information acquisition step of acquiring lens information including the lens aperture, focal length, and subject distance;
A subject specifying step of specifying the subject included in the depth of field based on the lens information and a distance between the subject and a reference position;
A display control step of displaying information indicating the subject included in the depth of field and an image indicated by the image data on a display unit;
An image generation method including:

Images