JP2013153321A - Imaging apparatus, imaging method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable a clear image to be easily obtained, even if there is an obstacle that hides a part of an imaging target subject in front of the subject, focused on the subject and from which the obstacle is removed.SOLUTION: An imaging unit 6, after focusing on an obstacle in front of an imaging target subject on the basis of an image projected on an imaging sensor 6d of the imaging unit 6 and detecting an image part area of the obstacle, focuses on the imaging target subject on the basis of an image other than the image part area of the obstacle, captures an image projected on the image sensor 6d at that moment as a captured image, and interpolates the captured image on the basis of the image part area of the obstacle.

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program suitable for imaging in a case where there is an obstacle that hides a part of the subject in front of the subject that is an imaging purpose.

In general, in a digital camera that employs contrast AF (autofocus), the contrast (color or brightness difference) is large while the focus lens is moved based on the image displayed on the image sensor (AF evaluation value). The point where is the maximum is detected as the focus position, and the focus is set to that position. However, the subject (for example, animals or humans) that is the object of shooting is located behind the wire mesh or net (obstacle). In such a case, the contrast of the wire mesh or the net is higher than that of the subject, and the focus may be inadvertently in front of the subject. Also, even in this case, even if you can focus on the subject that you want to shoot, a wire mesh or net will be reflected in front of the subject, so part of the subject will be hidden and the entire image will be hidden. Becomes difficult to see.
Therefore, conventionally, there has been disclosed a technique that enables a subject in a desired range to be focused even when shooting a subject through a wire mesh by controlling the focus range (Patent Document). 1).

Japanese Patent Laid-Open No. 2005-202064

  However, in the technique of the above-described patent document, it is possible to focus on a target subject from a wire mesh (obstacle), but it is limited to focusing on a subject within a narrow focus range. And it was a technology that stayed in focus until the subject was focused.

  An object of the present invention is to easily obtain a clear image in which an object is focused and the obstacle is removed even if an obstacle that hides a part of the subject exists in front of the subject that is a photographing purpose. Is to be able to.

In order to solve the above-described problem, one aspect of the present invention is as follows.
Imaging means comprising an imaging element;
First focusing means for focusing an object located in front as an obstacle based on an image shown on the image sensor;
Detecting means for detecting an area of an image portion of the obstacle from an image focused on the obstacle by the first focus means and reflected on the image sensor;
Second focusing means for focusing a subject located on the rear side as a subject to be photographed based on an image excluding the area of the image portion of the obstacle detected by the detecting means;
Capture means for capturing an image shown on the image sensor as a captured image in a state in which the second focus means is focused on the subject to be photographed;
Interpolation means for performing processing to interpolate the captured image captured by the capture means based on the area of the image portion of the obstacle;
It is an imaging device characterized by comprising.

In order to solve the above-described problem, another aspect of the present invention provides:
Based on the image shown on the image sensor, focusing the object located in front as an obstacle,
Detecting an area of an image portion of the obstacle from an image focused on the obstacle and reflected on the imaging element;
Based on an image excluding an area of the detected image portion of the obstacle, focusing a subject located on the rear side as a subject for photographing;
Capturing an image reflected on the image sensor as a captured image in a state in which the subject to be photographed is focused;
Performing a process of interpolating the captured image captured based on the area of the image portion of the obstacle;
It is an imaging method characterized by comprising.

Furthermore, in order to solve the above-described problem, another aspect of the present invention provides:
Against the computer,
Based on the image shown in the image sensor, the function to focus the subject located in front as an obstacle,
A function of detecting a region of an image portion of the obstacle from an image focused on the obstacle and reflected on the imaging element;
Based on the image excluding the area of the detected image portion of the obstacle, a function for focusing a subject located on the rear side as a subject for photographing,
A function of capturing an image reflected on the image sensor as a captured image in a state in which the object to be photographed is focused;
A function of performing the process of interpolating the captured captured image based on the area of the image portion of the obstacle;
It is a program for realizing.

  According to the present invention, even if there is an obstacle that hides a part of the subject in front of the subject to be photographed, it is possible to easily obtain a clear image in which the subject is focused and the obstacle is removed. Thus, even a subject that is difficult for the user to photograph can be clearly photographed.

FIG. 3 is a block diagram illustrating basic components of a digital camera as an imaging device. The flowchart which showed the operation | movement started according to the switch to a metal-mesh / net imaging mode. The figure which showed the relationship between a to-be-photographed object distance and contrast value (AF evaluation value) at the time of starting contrast AF process from the near distance side (Near side). (1) is the figure which showed the image (image before edge detection) which image | photographed the animal in the cage of a zoo, (2) is the figure which showed the image after performing edge detection. The relationship between the subject distance and the contrast value (AF evaluation value) when contrast AF processing is started from the far side (Far side) based on the image (processed image) after deleting the obstacle image portion is shown. Figure. 9 is a flowchart that starts execution in response to switching to a wire mesh / net shooting mode in the second embodiment. 7 is a flowchart for explaining in detail processing (step B6 in FIG. 6) for acquiring an image focused on a subject after AF processing. The figure which showed the relationship between a photographer's movement and the to-be-photographed object in 2nd Embodiment. The figure which illustrated the state which piled up each image while showing each image when the same subject was each image | photographed in the some position A, B, and C as an example.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(First embodiment)
First, a first embodiment of the present invention will be described with reference to FIGS.
The present embodiment is applied to a digital camera (compact camera) as an imaging device, and FIG. 1 is a block diagram showing basic components of the digital camera.
This digital camera is a compact camera having a contrast AF (autofocus) function, and has a configuration with the control unit 1 as a core. The control unit 1 operates by supplying power from the power source unit (secondary battery) 2 and controls the overall operation of the digital camera (hereinafter abbreviated as a camera) according to various programs in the storage unit 3. The control unit 1 is provided with a CPU (Central Processing Unit) and a memory (not shown).

  The storage unit 3 has a configuration including, for example, a ROM, a flash memory, and the like, and a program memory in which a program and various applications for realizing the present embodiment are stored according to an operation procedure illustrated in FIG. M1 and a work memory M2 that temporarily stores various information necessary for the operation of the camera (for example, captured images, flags, timers, and the like). Note that the storage unit 3 may be configured to include a removable portable memory (recording medium) such as an SD card or an IC card, and is connected to a network via a communication function (not shown). May include a storage area on a predetermined external server side.

  Although not shown in the drawings, the operation unit 4 is a variety of push button types that perform shooting operations (release operations) for instructing shooting, setting operations for setting shooting conditions such as exposure, and playback operations for instructing playback of captured images. Keys (shutter key, setting key, reproduction start key, etc.). The control unit 1 performs, for example, shooting processing, setting of shooting conditions, etc. as processing according to the input operation signal from the operation unit 4. Performs playback processing and data input processing. The touch input display unit 5 has a display unit and a touch sensor (not shown), senses a touch operation with a finger and inputs data corresponding to the touch operation, and monitors a live view image during shooting. Functions as a finder screen (monitor screen) to be displayed.

  The imaging unit 6 can shoot a subject with high definition by forming a subject image from an optical lens on an image sensor (CCD, CMOS, etc.). It is also possible to shoot. The imaging unit 6 subjects a subject image from a lens unit 6c including a zoom lens 6a, a focus lens 6b, and the like to an image sensor (CCD or CMOS) 6d via an aperture or a shutter (not shown). Is provided with an analog processing unit 6e and an image signal processing unit 6f in addition to various sensors. The image signal (analog value signal) after photoelectric conversion by the image sensor 6d is subjected to color separation, gain adjustment for each RGB color component, and the like, and then converted to digital value data. The digitally converted image data is subjected to color interpolation processing (demosaic processing) and sent to the touch input display unit 5 to be displayed in full color as a live view image.

  In this embodiment, AF processing, zoom processing, exposure adjustment processing (AE processing), auto white balance adjustment processing (AWB), aberration correction processing, image compression processing, image restoration processing, and the like can be executed. The captured image captured by the imaging unit 6 is recorded and stored in the storage unit 3 after being subjected to image compression processing for compression (reversible compression) by reducing the size of the image to form a file. In the present embodiment, various shooting modes suitable for the subject when shooting by the imaging unit 6, such as an aperture priority mode, a shutter speed priority mode, a landscape mode, a night view mode, and a wire mesh / net shooting mode described later. The shooting mode suitable for the subject is selected from the above and the like to perform shooting.

  In the wire mesh / net shooting mode, for example, when shooting an animal in a cage at a zoo or when shooting a subject (player) over the net at a baseball stadium or the like, Even if there is an approximately grid-like wire net or net (obstacle) in front of the player, etc., it is equivalent to a clear image focused on the subject to be photographed and without the obstacle This is a shooting mode in which an image can be obtained. Here, the “obstacle” includes not only a wire mesh or a net but also a fence and a gate iron grid, and hides a part of the subject (for example, about 1/5). In other words, the obstacle in the first embodiment means that the subject is hidden to the extent that the entire subject can be estimated by the remaining portion of the subject that is not hidden by the obstacle.

  In the above-described wire mesh / net shooting mode, contrast AF is used, and the contrast (color or brightness difference) is large while moving the focus lens 6b based on the image shown on the image sensor 6d (AF evaluation). The point where the (value) is maximum is detected as the focal position and focused. In this case, since the subject to be photographed is located behind the wire mesh or net (obstacle), the contrast of the obstacle is higher than that of the subject, and the subject in front of the subject (obstacle) In the first embodiment, the image portion of the obstacle is deleted from the image shown in the image sensor 6d, and then the region other than the region of the image portion of the obstacle is selected. Focusing is performed based on the image. In this way, the subject to be photographed is focused. Then, an image shown on the image sensor 6d in a state where the subject to be photographed is focused is captured as a captured image, and the captured image is interpolated based on the area of the image portion of the obstacle.

  Thus, in the first embodiment, after detecting an area of the image portion of the obstacle by focusing on the obstacle in front of the subject to be photographed based on the image reflected on the image sensor 6d, The subject to be photographed is focused on the basis of an image other than the area of the obstacle image portion, and the image shown on the image sensor 6d at that time is captured as a captured image, and the area of the obstacle image portion is used as a base. However, such a series of processes may be performed independently by the imaging unit 6 or may be performed jointly by the control unit 1 and the imaging unit 6.

  Next, the operation concept of the camera in the first embodiment will be described with reference to the flowchart shown in FIG. Here, each function described in this flowchart is stored in the form of a readable program code, and operations according to the program code are sequentially executed. Further, it is possible to sequentially execute the operation according to the above-described program code transmitted via a transmission medium such as a network. The same applies to other embodiments described later. In addition to the recording medium, an operation peculiar to the present embodiment can be executed using a program / data supplied externally via a transmission medium. FIG. 2 is a flowchart showing an outline of the operation of the characteristic part of the first embodiment of the overall operation of the camera. When the flow of FIG. 2 is exited, the main flow of the overall operation (not shown) Return to).

FIG. 2 is a flowchart showing an operation started in response to switching to the wire mesh / net shooting mode.
Photographers can take pictures of animals in cages and wire nets at zoos, etc., or when shooting subjects (players) over the net at baseball stadiums, etc., from the menu screen of the shooting mode. After selecting the shooting mode and switching the shooting mode, the camera is pointed at the subject (animal, player, etc.) that is the shooting purpose, and the shooting range is determined. In this state, the control unit 1 displays an image shown on the imaging element 6d of the imaging unit 6 as a preview image on the touch input display unit 5 (step A1), and then checks whether a release operation has been performed ( In step A2), it is checked whether an operation for instructing the end of shooting in the wire mesh / net shooting mode is performed (step A3), or whether other operations are performed (step A4).

  Here, as other operations, for example, when exposure setting and zoom setting are performed (YES in step A4), processing such as exposure setting and zoom setting (step A5) is executed as processing corresponding to the operation. After that, the process returns to step A1 described above. Also, when an operation of canceling the wire mesh / net shooting mode and switching to another shooting mode is performed, or an operation of switching from the shooting mode to the playback mode is performed (YES in Step A3), the flow of FIG. 2 is exited. .

  Now, when a release operation is performed in the wire mesh / net shooting mode (YES in Step A2), the first contrast AF process is started from the near distance side (Near side) (Step A6). Then, the contrast value (AF evaluation value) is calculated while moving the focus lens 6b from the short distance side (step A7). FIG. 3 is a diagram showing the relationship between the subject distance and the contrast value (AF evaluation value) when contrast AF processing is started from the near distance side (Near side). Here, when there is a metal mesh or net in front of the subject to be photographed, the contrast AF processing from the short distance side is started because the contrast of the obstacle is higher than that of the subject. It is valid. Thus, the maximum point of the AF evaluation value is the position of (obstacle) such as a wire mesh or a net. Therefore, the focus lens 6b is moved in order to set that point as the focal position, and the object is the object before photographing. The subject located at is focused as an obstacle, and the focus is adjusted to the position of the obstacle (step A8).

  In the state where the position of the obstacle is focused in this way, edge detection is performed on the image reflected on the image sensor 6d, and the edge portion where the pixel value changes sharply is set as the region of the obstacle image portion. For example, the area (for example, outline: outline) of the image portion of the obstacle is recognized and temporarily stored (step A9). FIG. 4 (1) is a diagram showing an image (image before edge detection) of an animal (tiger) in a zoo cage. In this case, an obstacle (a shark The focus is on the wire mesh. FIG. 4 (2) shows an image after edge detection is performed on the image of FIG. 4 (1), and the outline of the obstacle is clear. After the obstacle is detected in this way, the image portion of the obstacle is deleted based on the image processing technique (step A10).

  As described above, the second contrast AF process is started based on the image (processed image) after the area of the obstacle image portion is deleted, that is, the image other than the area of the obstacle image portion (step). A11). Then, the contrast value (AF evaluation value) is calculated while moving the focus lens 6b from the far distance side (Far side) (step A12). FIG. 5 shows the subject distance and contrast value (AF evaluation) when contrast AF processing is started from the far side (Far side) based on the image (processed image) after deleting the area of the image part of the obstacle. It is a transition diagram showing a relationship with (value). Here, the image after processing the area of the image portion of the obstacle (processed image) has a high contrast only in the vicinity of the subject to be photographed, so that the contrast AF process is started from the far side. It is valid. In this case, since the maximum point of the AF evaluation value is in the vicinity of the subject to be photographed, the focus lens 6b is moved to set the point as the focal position, and the subject located behind the obstacle is photographed for the purpose of photographing. Focus as a subject and focus on the subject to be photographed (step A13).

  In this manner, an image shown on the image sensor 6d in a state where the subject to be photographed is focused is captured as a capture image (step A14), and the captured image is positioned around the area of the obstacle image portion. The surrounding image (shape, color, pattern, brightness, etc.) is analyzed (step A15). Then, the image portion of the subject hidden by the obstacle is estimated from the analysis result (step A16), and the area of the image portion of the obstacle is interpolated by the estimation result (step A17). Note that such an interpolation function is a technique that is generally used in image processing technology, and in this embodiment, the well-known technique is used. However, the interpolation function of the first embodiment is the following: It features the following points.

  That is, in general, in an image focused on an obstacle in the foreground, there is almost 100% obstacle image information in the obstacle image portion, so even if the obstacle image portion is removed. It is difficult to restore the image portion of the subject to be photographed behind. On the other hand, in the image focused on the target subject behind, the image portion of the obstacle is blurred and spread, and therefore, for example, 10% is in the area where the image portion of the obstacle is blurred and spread. The remaining 90% of the image information of the obstacle is in a state where the image information of the subject to be photographed and the image information of the obstacle are mixed, such as the image portion of the subject to be photographed. If only the image component corresponding to the image information of the obstacle can be removed from the mixed image, it is possible to restore the image information of the subject to be photographed. However, in the image focused on the subject to be photographed behind, the image portion of the obstacle is blurred and widened, so the area of the image portion of the obstacle and the color information of the image portion are accurately identified. Since it is not possible to do this, it is not clear which image component of which color should be removed in which range. Therefore, an image focused on the target subject behind based on the area of the image part of the obstacle acquired from the image focused on the obstacle in front and the color information of the image part of the obstacle. By specifying the range, color components, and the like that are the targets of image processing for restoring the image information, more accurate image restoration (image interpolation) can be performed.

Therefore, in the first embodiment, a state (for example, outline: outline) of an obstacle image portion is recognized and temporarily stored by edge detection after performing the first contrast AF process described above. In this case, the region of the image portion of the obstacle is read out as a reference pattern, and a portion corresponding to the obstacle in the captured image is searched based on the reference pattern. In this case, the part corresponding to the obstacle in the captured image has a unique shape in which the same pattern continues and is similar to the reference pattern, but the area of the obstacle image part is blurred and widened. , It is one size larger than the reference pattern. Assuming that it is similar to the reference pattern with a unique shape and one size larger than the reference pattern, the continuity of colors and brightness patterns can be adjusted while matching the position of the reference pattern in the captured image. In consideration of this, an area portion (for example, outline: outline) corresponding to the obstacle in the captured image is specified. Then, an interpolation process is performed on the part specified as described above to process the image without the obstacle.
When the above-described interpolation processing is performed, the interpolated image is compressed (reversible compression) and converted into a file, and then recorded and stored in the storage unit 3 (step A18). Thereafter, the process returns to the first step A1.

  As described above, in the first embodiment, after focusing on the obstacle based on the image shown on the image sensor 6d, the object in the foreground is focused on the obstacle, Based on the image to be excluded, the subject behind is taken as a subject to be photographed, and the subject is focused. At the same time, the image shown on the image sensor 6d is captured as a captured image, and this captured image is captured as an obstacle image. Since interpolation is performed based on the area of the part, even if there is an obstacle that hides a part of the subject in front of the subject that is the object of shooting, it is clear that the subject is focused and the obstacle is removed Therefore, even a subject that is difficult for the user to shoot can be captured clearly.

  Since the area of the image part of the obstacle is detected by the high-contrast part, if there is a wire mesh or net in front of the subject to be photographed, the contrast of the obstacle is higher than that of the subject. Therefore, it is possible to easily detect the area of the image portion of the obstacle.

  In addition, since the image portion of the obstacle is deleted and the object to be photographed is focused on the basis of the image after the deletion, the object to be photographed is focused on the image having no obstacle. This makes it easy to focus on the shooting purpose.

  Since the obstacle is an obstacle formed in a substantially lattice shape, for example, the obstacle is suitable for photographing a cage, an animal in a wire net, or a baseball player over the net.

  When performing the interpolation process, the image part of the subject hidden by the obstacle is estimated and interpolated from the analysis result obtained by analyzing the peripheral image of the area of the obstacle image part. It is possible to expect. In this case, since the obstacle is an obstacle formed in a substantially lattice shape, for example, the estimation is easier than in the case where the portion hidden by the obstacle is the lower half and the right half. It becomes.

  The first contrast AF process, the process for detecting the area of the obstacle image part, the second contrast AF process, the process for capturing the image focused on the subject to be photographed as the captured image, the area of the obstacle image part Since the interpolating process is performed by a single photographing operation, it is possible to obtain a clear image that does not include an obstacle by focusing on the subject to be photographed by performing only one photographing operation.

  The first contrast AF process focuses on the obstacle located in front of the subject to be photographed by focusing from the short distance side based on the image shown on the image sensor 6d. It is possible to focus efficiently.

  In the second contrast AF process, focusing is performed on the subject to be photographed by focusing from the far distance side based on the image shown on the image sensor 6d, so that the subject can be focused efficiently. .

  Since the region of the image portion of the obstacle is detected by detecting the edge where the pixel value changes sharply, the contour of the obstacle can be recognized with high accuracy.

  When the captured image is interpolated based on the area of the image part of the obstacle, an area similar to the area of the image part of the obstacle and larger than that is detected as an area part corresponding to the obstacle in the captured image, Since the interpolation process is performed on the detection area, even if the image part of the obstacle is blurred and spread in the captured image, the area corresponding to the obstacle can be accurately identified. Becomes easy.

  Since the interpolated image is recorded and stored, it is possible to record a clear image that does not include an obstacle by focusing on the subject to be photographed.

  In the first embodiment described above, the region of the image portion of the obstacle is detected from the image reflected on the image pickup device 6d in a state where the obstacle is in focus, and the portion is deleted. However, the area of the image portion of the obstacle may be masked or ignored.

Further, in the first embodiment described above, the image portion of the subject hidden by the obstacle is estimated and interpolated from the analysis result obtained by analyzing the peripheral image of the region of the obstacle image portion. What kind of subject is specified by image recognition. For example, in the case of a tiger, interpolation processing may be performed based on the result of image analysis and the type of subject specified by image recognition. In addition, when an image focused on a subject to be photographed is captured as a capture image and the area of the obstacle image part is interpolated, the area of the obstacle image part is deleted and blanked, and then the object Interpolation may be performed by embedding the image part, or interpolation may be performed by superimposing the image part of the subject by overwriting the area of the obstacle image part. Is optional.
Further, in the first embodiment described above, contrast AF is used as the AF processing, but other AF methods such as phase difference AF may be used.

(Second Embodiment)
A second embodiment of the present invention will be described below with reference to FIGS.
In the first embodiment described above, the image portion of the subject hidden by the obstacle is estimated and interpolated from the result of analyzing the peripheral image of the area of the obstacle image portion. In the second embodiment, a portion of the subject that is hidden by the obstacle at the time of the first shooting is performed for the second and subsequent times by performing a plurality of continuous shootings while shifting the shooting position (changing the viewpoint) of the subject. In this way, the area of the image part of the obstacle is interpolated. In the first embodiment described above, “obstacle” means that the subject is hidden to the extent that the entire subject can be estimated by the remaining portion of the subject that is not hidden by the obstacle. In the second embodiment, the subject may be hidden more greatly than in the first embodiment. Here, in both the embodiments, the same or the same names are denoted by the same reference numerals, the description thereof will be omitted, and the following description will focus on the features of the second embodiment. .

FIG. 6 is a flowchart that starts execution in accordance with switching to the wire mesh / net shooting mode in the second embodiment. Note that steps B1 to B5 in FIG. 6 are processes corresponding to steps A1 to A5 in FIG. 2, and steps B6 (steps B61 to 69) are processes corresponding to steps A6 to A14 in FIG.
First, the control unit 1 displays a preview of the image shown on the image sensor 6d (step B1), and then checks whether a release operation has been performed (step B2), or ends the shooting in the wire mesh / net shadow mode. It is checked whether an instructing operation has been performed (step B3), or whether other operations have been performed (step B4).

  If another operation is performed (YES in step B4), a process corresponding to the operation (step B5) is executed, and then the process returns to step B1 described above. Also, when the end of shooting in the wire mesh / net shadow mode is instructed (YES in step B3), the flow exits from FIG. When a release operation is performed in the wire mesh / net shooting mode (YES in step B2), the process proceeds to a process of acquiring an image focused on the subject after the contrast AF process (step B6).

FIG. 7 is a flowchart for explaining in detail the process for acquiring an image focused on the subject after the contrast AF process (step B6 in FIG. 6).
First, the first contrast AF process is started from the near distance side (Near side) (step B61). Then, the contrast value (AF evaluation value) is calculated while moving the focus lens 6b from the short distance side (step B62), and the focus is adjusted to the position of an obstacle (such as a wire mesh or net) (step B63). In this state, edge detection (step B64) is performed on the image reflected on the image sensor 6d, and then the image portion of the obstacle is deleted based on the image processing technique (step B65). Then, the second contrast AF process is started based on the deleted image (processed image) (step B66), and an AF evaluation value is calculated from the long distance side (step B67), and the position of the subject to be photographed is set. The focus is adjusted (step B68), and the image at that time is captured as a captured image (step B69).

  In this way, after the process of acquiring an image focused on the subject after the AF process (step B6) is finished, the process stands by until a continuous shooting operation (release operation) is performed (step B7). When the second release operation is performed (YES in step B7), the process proceeds to a process (step B8) for acquiring an image focused on the subject after the AF process, and the above operation is performed again according to the flowchart of FIG. Run. As a result, the obstacle is focused on the obstacle in the first AF process to delete the image portion of the obstacle, and then the focus is focused on the subject to be photographed in the second AF process, and the image at that time is used as the capture image. take in.

  Then, the image captured at the time of the second photographing is superimposed on the image captured at the time of the first photographing (step B9). FIG. 8 illustrates the case where the shooting position is changed three times and the same subject is shot for each position. The movement direction of the photographer (the direction of the arrow in the figure) and the stationary subject are illustrated. Showing the relationship. In this case, the first shooting is performed at a position A in a 45 ° right oblique direction toward the subject, and then the photographer moves to a position B facing the subject to perform a second shooting, and further, as illustrated in FIG. The example shows a case where the third shooting is performed at a position C in a 45 ° left oblique direction toward the subject. In the example of FIG. 8, the case where the photographing is performed three times is illustrated, but the number of photographing may be two times or four times or more.

  In this way, by shifting the shooting direction (shooting position) of the same subject, that is, by shooting multiple times while changing the viewpoint, even if there is an obstacle in front of the subject, Since the position of the obstacle changes relatively, the portion of the subject that was not seen by the obstacle in one direction can be seen in the other direction. For example, the portion of the subject that was not visible in the first shooting at position A can be seen in the second shooting at position B, and the portion of the subject that was visible in the first shooting is reversed in the second shooting. Become invisible. For this reason, by superimposing the first image taken from the position A in the 45 ° diagonal direction on the subject and the second image taken from the front B, the part hidden by the obstacle is It disappears completely and you can see the whole, or it's much less.

  Here, when a plurality of images taken from each direction are superimposed, the position of the subject is specified by an image processing technique, and the images are superimposed so that the joints of the subjects are aligned. Then, the contour of the subject is detected based on the superimposed images, the image portion of the subject is detected from each captured image, and is acquired as subject detection information (step B10). The region of the image portion of the object is interpolated (step B11). As a result, it is checked whether all areas of the image portion of the obstacle have been interpolated (interpolation is completed) (step B12). If the interpolation has not been completed (NO in step B12), the number of continuous shots is the limit value ( For example, it is checked whether or not (3 times) has been exceeded (step B15). Now, since it is the second continuous shooting operation (release operation) (NO in step B15), it is in a standby state until the next continuous shooting operation is performed (step B7).

  Accordingly, when the third continuous shooting operation is performed (YES in step B7), the same operation as described above is performed (steps B7 to B11). FIG. 9 is a diagram showing each image taken at a plurality of positions A, B, and C as in the example of FIG. 8 and a state in which the images are superimposed. When interpolation can be completed by photographing from three directions and superimposing the images in this way (YES in step B12), a buzzer (not shown) is driven to notify the photographer to that effect. Then, a notification sound is generated (step B13). Then, after performing an image compression process for compressing (reversibly compressing) the interpolated image into a file, the image is recorded and saved in the storage unit 3 (step B14). Thereafter, the process returns to the first step B1.

  If the interpolation cannot be completed even if the limit value (for example, 3 times) is exceeded, the interpolation can be performed as in the first embodiment described above. The peripheral image of the remaining part that has not been analyzed is analyzed, and the part is estimated and interpolated based on the analysis result (step B16). Then, a notification sound indicating completion of interpolation is generated (step B13), the interpolated image is compressed and recorded in the storage unit 3 (step B14), and then the process returns to the first step B1.

  As described above, in the second embodiment, every time a plurality of continuous shooting operations are performed while changing the shooting direction by moving the camera, the first contrast AF process, the area of the obstacle image portion is changed. A process for detecting, a second contrast AF process, and a process for capturing an image focused on a subject to be photographed as a captured image, and a process for interpolating the area of the image part of the obstacle in the last photographing operation, At that time, since the area of the image part of the obstacle is interpolated by overlapping the captured images acquired by multiple continuous shooting, just by performing multiple continuous shooting while changing the shooting direction, Interpolation can be performed using an actually captured image portion, and an image equivalent to an image when no obstacle is present can be obtained.

  Since the interpolation completion is notified when all the regions of the image portion of the obstacle can be interpolated, in order to be able to notify the photographer of the completion of interpolation, for example, two consecutive times If the shooting is sufficient, the shooting can be stopped at that time, and if the two continuous shootings are not enough, the third continuous shooting may be performed.

  In each of the above-described embodiments, the zoom operation is not mentioned. However, when the zoom operation is performed, the contents of the focus table that stores the zoom lens position and the focus lens position in association with each other change. In order to achieve this, the start position of the contrast AF process may be appropriately changed according to the zoom operation.

  In each of the above-described embodiments, the present invention is applied to a digital camera (compact camera). However, the present invention is not limited thereto, and is not limited to a single-lens reflex camera, a personal computer with a camera function, a PDA (personal portable information communication device), It may be a smartphone, other mobile phone, tablet terminal device, music player, or the like.

  Further, the “apparatus” and “unit” shown in each of the above-described embodiments may be separated into a plurality of cases by function, and are not limited to a single case. In addition, each step described in the above-described flowchart is not limited to time-series processing, and a plurality of steps may be processed in parallel or separately.

The embodiment of the present invention has been described above. However, the present invention is not limited to this, and includes the invention described in the claims and the equivalent scope thereof.
Hereinafter, the invention described in the claims of the present application will be appended.
(Appendix)
(Claim 1)
The invention described in claim 1
Imaging means comprising an imaging element;
First focusing means for focusing an object located in front as an obstacle based on an image shown on the image sensor;
Detecting means for detecting an area of an image portion of the obstacle from an image focused on the obstacle by the first focus means and reflected on the image sensor;
Second focusing means for focusing a subject located on the rear side as a subject to be photographed based on an image excluding the area of the image portion of the obstacle detected by the detecting means;
Capture means for capturing an image shown on the image sensor as a captured image in a state in which the second focus means is focused on the subject to be photographed;
Interpolation means for performing processing to interpolate the captured image captured by the capture means based on the area of the image portion of the obstacle;
It is an imaging device characterized by comprising.
(Claim 2)
The invention according to claim 2 is the imaging apparatus according to claim 1,
The detection means detects an area of the image portion of the obstacle by a high-contrast portion;
The imaging apparatus is characterized by the above.
(Claim 3)
According to a third aspect of the present invention, in the imaging device according to the first or second aspect,
A deletion unit that deletes an image part area of the obstacle detected by the detection unit;
The second focus unit focuses on the subject to be photographed based on an image after the obstacle is deleted by the deletion unit.
The imaging apparatus is characterized by the above.
(Claim 4)
The invention according to claim 4 is the imaging apparatus according to any one of claims 1 to 3,
The obstacle is an obstacle formed in a substantially lattice shape.
This is an imaging apparatus.
(Claim 5)
The invention according to claim 5 is the imaging apparatus according to any one of claims 1 to 4,
Analysis for analyzing a peripheral image located in the vicinity of the area of the image portion of the obstacle among the images photographed by the imaging means in a state in which the object to be photographed is focused by the second focus means. Further comprising means,
The interpolation means performs a process of interpolating based on the region of the image portion of the obstacle and the analysis result by the analysis means.
The imaging apparatus is characterized by the above.
(Claim 6)
The invention according to claim 6 is the imaging apparatus according to any one of claims 1 to 5,
The first focus means, the detection means, the second focus means, the capture means, and the interpolation means operate in one shooting operation;
The imaging apparatus is characterized by the above.
(Claim 7)
The invention according to claim 7 is the imaging apparatus according to any one of claims 1 to 4,
Each time a shooting operation is performed by a plurality of continuous shootings while changing the shooting direction by moving the imaging unit, the first focus unit, the detection unit, the second focus unit, and the capture unit are respectively Work,
The interpolation means operates at the last photographing operation, and superimposes the captured images captured by the plurality of continuous shootings, thereby superimposing the area of the obstacle image portion on the basis of the area of the obstacle image portion. To perform interpolation processing,
The imaging apparatus is characterized by the above.
(Claim 8)
The invention according to claim 8 is the imaging apparatus according to claim 7,
Discriminating means for discriminating whether or not all the regions of the image part of the obstacle have been interpolated,
A notification means for notifying completion of interpolation when it is determined that all the interpolation is performed by the determination means;
Further comprising
The imaging apparatus is characterized by the above.
(Claim 9)
The invention according to claim 9 is the imaging apparatus according to any one of claims 1 to 4,
The first focusing means focuses on an obstacle located in front of the subject to be photographed by focusing from a short distance side based on an image shown on the image sensor.
The imaging apparatus is characterized by the above.
(Claim 10)
The invention according to claim 10 is the imaging apparatus according to any one of claims 1 to 4,
The second focusing means focuses on the subject to be photographed by focusing from a long distance side based on an image shown on the image sensor.
The imaging apparatus is characterized by the above.
(Claim 11)
The invention according to claim 11 is the imaging apparatus according to any one of claims 1 to 4,
The detection means detects an area of the image portion of the obstacle by detecting an edge where the pixel value changes sharply.
The imaging apparatus is characterized by the above.
(Claim 12)
The invention according to claim 12 is the imaging apparatus according to any one of claims 1 to 11,
The interpolation means, when interpolating the captured image captured by the capture means based on the area of the obstacle image portion, captures an area similar to and larger than the area of the obstacle image portion. Detect as an area corresponding to an obstacle in the image, and perform interpolation processing within the detection area.
An imaging apparatus characterized by being configured as described above.
(Claim 13)
The invention according to claim 13 is the imaging apparatus according to any one of claims 1 to 12,
It further comprises recording means for recording the image after being interpolated by the interpolation means.
The imaging apparatus is characterized by the above.
(Claim 14)
The invention according to claim 14
Based on the image shown on the image sensor, focusing the object located in front as an obstacle,
Detecting an area of an image portion of the obstacle from an image focused on the obstacle and reflected on the imaging element;
Based on an image excluding an area of the detected image portion of the obstacle, focusing a subject located on the rear side as a subject for photographing;
Capturing an image reflected on the image sensor as a captured image in a state in which the subject to be photographed is focused;
Performing a process of interpolating the captured image captured based on the area of the image portion of the obstacle;
It is an imaging method characterized by comprising.
(Claim 15)
The invention according to claim 15 is:
Against the computer,
Based on the image shown in the image sensor, the function to focus the subject located in front as an obstacle,
A function of detecting a region of an image portion of the obstacle from an image focused on the obstacle and reflected on the imaging element;
Based on the image excluding the area of the detected image portion of the obstacle, a function for focusing a subject located on the rear side as a subject for photographing,
A function of capturing an image reflected on the image sensor as a captured image in a state in which the object to be photographed is focused;
A function of performing the process of interpolating the captured captured image based on the area of the image portion of the obstacle;
It is a program for realizing.

DESCRIPTION OF SYMBOLS 1 Control part 3 Memory | storage part 6 Image pick-up part 6a Zoom lens 6b Focus lens 6c Lens part 6d Image pick-up element 6e Analog processing part 6f Image signal processing part M1 Program memory

Claims (15)

Imaging means comprising an imaging element;
First focusing means for focusing an object located in front as an obstacle based on an image shown on the image sensor;
Detecting means for detecting an area of an image portion of the obstacle from an image focused on the obstacle by the first focus means and reflected on the image sensor;
Second focusing means for focusing a subject located on the rear side as a subject to be photographed based on an image excluding the area of the image portion of the obstacle detected by the detecting means;
Capture means for capturing an image shown on the image sensor as a captured image in a state in which the second focus means is focused on the subject to be photographed;
Interpolation means for performing processing to interpolate the captured image captured by the capture means based on the area of the image portion of the obstacle;
An imaging apparatus comprising: The detection means detects an area of the image portion of the obstacle by a high-contrast portion;
The imaging apparatus according to claim 1, which is configured as described above. A deletion unit that deletes the image portion of the obstacle detected by the detection unit;
The second focus unit focuses on the subject to be photographed based on an image after the obstacle is deleted by the deletion unit.
The imaging apparatus according to claim 1, wherein the imaging apparatus is configured as described above. The obstacle is an obstacle formed in a substantially lattice shape.
The imaging device according to any one of claims 1 to 3, wherein Analysis for analyzing a peripheral image located in the vicinity of the area of the image portion of the obstacle among the images photographed by the imaging means in a state in which the object to be photographed is focused by the second focus means. Further comprising means,
The interpolation means performs a process of interpolating based on the region of the image portion of the obstacle and the analysis result by the analysis means.
The imaging apparatus according to claim 1, wherein the imaging apparatus is configured as described above. The first focus means, the detection means, the second focus means, the capture means, and the interpolation means operate in one shooting operation;
The imaging apparatus according to claim 1, wherein the imaging apparatus is configured as described above. Each time a shooting operation is performed by a plurality of continuous shootings while changing the shooting direction by moving the imaging unit, the first focus unit, the detection unit, the second focus unit, and the capture unit are respectively Work,
The interpolation means operates in the last shooting operation, and performs a process of interpolating based on the region of the image portion of the obstacle by superimposing the captured images captured by the plurality of continuous shootings.
The imaging apparatus according to claim 1, wherein the imaging apparatus is configured as described above. Discriminating means for discriminating whether or not all the regions of the image part of the obstacle have been interpolated,
A notification means for notifying completion of interpolation when it is determined that all the interpolation is performed by the determination means;
Further comprising
The imaging apparatus according to claim 7, which is configured as described above. The first focusing means focuses on an obstacle located in front of the subject to be photographed by focusing from a short distance side based on an image shown on the image sensor.
The imaging apparatus according to claim 1, wherein the imaging apparatus is configured as described above. The second focusing means focuses on the subject to be photographed by focusing from a long distance side based on an image shown on the image sensor.
The imaging apparatus according to claim 1, wherein the imaging apparatus is configured as described above. The detection means detects an area of the image portion of the obstacle by detecting an edge where the pixel value changes sharply.
The imaging apparatus according to claim 1, wherein the imaging apparatus is configured as described above. The interpolation means, when interpolating the captured image captured by the capture means based on the area of the obstacle image portion, captures an area similar to and larger than the area of the obstacle image portion. Detect as an area corresponding to an obstacle in the image, and perform interpolation processing within the detection area.
The imaging apparatus according to claim 1, wherein the imaging apparatus is configured as described above. It further comprises recording means for recording the image after being interpolated by the interpolation means.
The imaging apparatus according to claim 1, wherein the imaging apparatus is configured as described above. Based on the image shown on the image sensor, focusing the object located in front as an obstacle,
Detecting an area of an image portion of the obstacle from an image focused on the obstacle and reflected on the imaging element;
Based on the image excluding the area of the detected obstacle image portion, focusing a subject located on the rear side as a subject for shooting;
Capturing an image reflected on the image sensor as a captured image in a state in which the subject to be photographed is focused;
Performing the process of interpolating the captured image captured based on the area of the image portion of the obstacle;
An imaging method comprising: Against the computer,
Based on the image shown in the image sensor, the function to focus the subject located in front as an obstacle,
A function of detecting a region of an image portion of the obstacle from an image focused on the obstacle and reflected on the imaging element;
Based on an image excluding the area of the detected obstacle image portion, a function for focusing a subject located on the rear side as a subject for photographing,
A function of capturing an image reflected on the image sensor as a captured image in a state in which the object to be photographed is focused;
A function of performing processing for interpolating the captured image captured based on the area of the image portion of the obstacle;
A program to realize

Images