JP2010246033A - Camera, image processing device, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem in conventional image correction that mainly performs shift correction, and requires a complicated image process to obtain a tilt-shift effect without actually tilting and shifting a shooting lens. <P>SOLUTION: An α-blend module 303 receives a development image signal P4 read out from a development image storage region M2 of a buffer memory 301, a low-pass process image signal P7 read out from a low-pass process image storage region M3 of the buffer memory 301, and a mask image signal P6 read out from a mask image processing part 306, and executes a predetermined operation to obtain a composite image. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a camera, an image processing apparatus, and an image processing program.

As conventionally known, it is widely known to correct a building image viewed from a fixed viewpoint by performing so-called tilt correction. Japanese Patent Application Laid-Open No. 2004-228561 describes switching the tilt correction method by referring to shooting condition data recorded in an image file.
The tilt correction mentioned here refers to image correction (that is, shift correction) for obtaining a shot image similar to that obtained when the shooting lens is shifted by a tilt shift lens or a large format camera.

Japanese Patent Laid-Open No. 2006-163497

  As described above, conventionally known image correction mainly performs shift correction, and in order to obtain a reverse tilt photographing effect without actually reverse tilting the photographing lens, a complicated image processing process is required. there were. In particular, it has not been possible to easily obtain a reverse tilt photographed image desired by the user using a digital camera equipped with a normal photographing lens.

The invention according to claim 1 is a developing unit that forms a developed image based on an image signal obtained from an image sensor, and a high-frequency removing unit that forms a low-pass processed image by removing a high-frequency component of the developed image. Mask image information output means for outputting, as mask image information, an alpha value group for performing alpha blend processing on the developed image and the low-pass processed image, and a plurality of the mask image information corresponding to the shooting state of the camera. Mask image information selecting means for selecting any one of them, combining means for performing alpha blend processing based on the mask image information output from the mask image information selecting means, the developed image, and the low-pass processed image; And outputting a composite image having an inverse tilt photographing effect from the combining means.
According to a seventh aspect of the present invention, there is provided a high-frequency removing unit that removes a high-frequency component from a captured image to form a low-pass processed image, and an alpha blend process for the captured image and the low-pass processed image. Mask image information output means for outputting an alpha value group as mask image information, and mask image information selection means for selecting any one of the plurality of mask image information based on information attached to the captured image And a synthesizing unit that performs alpha blend processing based on the mask image information output from the mask image information selection unit, the captured image, and the low-pass processed image, and has an inverse tilt imaging effect. An image processing apparatus that outputs a synthesized image from the synthesizing means.
The invention according to claim 9 is a high-frequency removal step of removing a high-frequency component from a captured image to form a low-pass processed image, and an alpha blending process for the captured image and the low-pass processed image A mask image information output step of outputting an alpha value group as mask image information, and a mask image information selection step of selecting any one of the plurality of mask image information based on information attached to the captured image And an alpha blend process based on the mask image information selected in the mask image information selection step, the captured image, and the low-pass processed image, thereby obtaining a composite image having an inverse tilt imaging effect. An image processing program for causing a computer to execute a synthesis process to be formed.

According to the camera of the present invention, it is possible to obtain a photographed image having a desired reverse tilt photographing effect without actually tilting the photographing lens.
According to the image processing apparatus of the present invention, a desired reverse tilt photographing effect can be given to an image after completion of photographing.
According to the image processing program of the present invention, it is possible to realize image processing for giving a desired reverse tilt photographing effect to an image after photographing is completed.

1 is a cross-sectional configuration diagram of a digital camera to which the present invention is applied. It is a figure explaining the attitude | position of the camera main body. It is a block diagram which shows the electric system of the digital camera shown in FIG. FIG. 4 is a block diagram illustrating a signal processing unit for performing α blend processing according to the first embodiment in the ASIC 13 illustrated in FIG. 3. 7 is a flowchart showing an image processing procedure for providing an inverse tilt photographing effect using the ASIC 13. It is a figure which shows the mask image for alpha blend processes. It is the figure which illustrated the relationship between AF area position and a mask image. It is a figure which shows the state from which a mask image changes substantially according to the focal distance of a lens.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Embodiment 1>
FIG. 1 is a cross-sectional configuration diagram of a digital camera to which the present invention is applied. In this figure, 100 is a detachable lens barrel, and 200 is a camera body. In the first embodiment, a reverse interchangeable photographing effect is obtained by image processing to be described in detail later without actually reverse tilting the photographing lens, so that a normal interchangeable lens can be used.
Here, the reverse tilt shooting effect means that an image in a specific area in a captured screen is an in-focus image, but an image in another area is in a so-called out-of-focus state. Note that the boundary between the focused image and the so-called out-of-focus image includes not only a case where the boundary is gradually changed but also a case where the boundary is changed step by step.

  The camera body 200 is provided with a camera control unit 205 and controls all camera operations. Reference numeral 201 denotes a main switch which performs a power on / off operation. Reference numeral 204 denotes a release button having a half-press detection function and a full-press detection function.

  Reference numeral 206 denotes an inclination sensor that detects the photographing posture of the camera body 200. The tilt sensor 206 is in a state where the camera body is held horizontally as shown in FIG. 2 (1), in a state where the grip is on the upper side as shown in FIG. 2 (2), and (3) in FIG. ) To detect the state where the grip portion is on the lower side. Based on the camera posture information output from the tilt sensor 206, a mask image in which the alpha value for alpha blending processing is stored as a binary image is selected (described in detail later with reference to FIG. 6).

  FIG. 3 is a block diagram showing an electrical system of the digital camera shown in FIG. In this figure, 10 is a CCD as an image sensor, 11 is an analog signal processing circuit, 12 is an A / D conversion circuit, 16 is a CCD driver, and 17 is a timing generator. The mechanism for driving the photographic lens to the in-focus position so that the subject light incident on the photographic lens forms an image on the CCD 10 by using the focus detection device 26 and the like is the same as that of a normal digital camera. Omitted. The timing generator 17 controls the operation timing of the CCD driver 16, the analog signal processing circuit 11, and the A / D conversion circuit 12. The photometry device 25 starts measurement when a half-press operation is detected by the release button 204. When the full-press operation is performed subsequent to the half-press operation, signal charges corresponding to the brightness of the subject image are accumulated in the CCD 10. The signal charges accumulated in the CCD 10 are discharged under the control of the driver 16 and input to the analog signal processing circuit 11 including an AGC circuit and a CDS circuit. The analog signal processing circuit 11 performs analog processing such as gain control and noise removal on the input analog image signal, and outputs the processed analog signal to the A / D conversion circuit 12.

  The signal digitally converted by the A / D conversion circuit 12 is input to an ASIC (Application Specific Integrated Circuit) 13 that is an image processing circuit. In the ASIC 13, contour compensation, gamma correction, white balance adjustment, and the like are performed and temporarily stored in the buffer memory 15. The image data stored in the buffer memory 15 is subjected to data compression at a predetermined ratio by the JPEG method by the compression circuit 14 and recorded in a storage medium 27 such as a flash memory. The buffer memory 15 stores image data for live view display and displays it as a live view on the display 21. After shooting is completed, a shooting confirmation screen is displayed on the display 21.

Reference numeral 40 denotes a reverse tilt setting switch, which designates a reverse tilt shooting mode while viewing a menu screen (not shown) displayed on the display 21 of the camera body. In the first embodiment, the “reverse tilt photographing mode A” in which “the horizontal region (horizontal region) including the central portion of the developed image is focused and the image is blurred as it goes to the upper and lower end regions of the developed image” is used. Or set “Reverse Tilt Shooting Mode B” to “focus the vertical area (vertically long area) including the central part of the developed image and blur the image as it goes to the left and right edge areas of the developed image” Is selected by the reverse tilt setting switch 40.
The reverse tilt setting switch 40 may also be used as a cross key (not shown) for performing general menu settings.

  FIG. 4 is a block diagram illustrating a signal processing unit for performing the α blend process according to the first embodiment in the ASIC 13 illustrated in FIG. 3. FIG. 5 is a flowchart showing an image processing procedure for providing an inverse tilt photographing effect using the ASIC 13, and steps S1 to S5 are stored in the form of a program in a memory (not shown). FIG. 6 shows a mask image in which the alpha value for alpha blend processing is stored as a binary image. The four types of mask images shown in FIG. 6 have pixel value distribution patterns that are black when α = 1 and white when α = 0.

  The operation of the ASIC 13 will be described with reference to FIGS.

§Procedure 1: Step S1 is executed according to the following procedure.
The imaging signal P1 input to the ASIC 13 via the CCD 10, the analog signal processing circuit 11, and the A / D conversion circuit 12 is accumulated in the input image storage area M1 in the buffer memory 301. Thereafter, the image pickup signal P2 read out from the input image storage area M1 is input to the image processing unit 305 and subjected to predetermined interpolation processing, color processing, contour processing, and the like to become a developed image signal P3, which is stored in the buffer memory 301. The image is stored in the developed image storage area M2.

§Procedure 2: Step S2 is executed according to the following procedure.
The developed image signal P4 read from the developed image storage area M2 is input to the low-pass module 302 and is output as a low-pass processed image (so-called out-of-focus image) P5 from which high-frequency components have been removed. This low-pass processed image (so-called out-of-focus image) P5 is stored in the low-pass processed image storage area M3 in the buffer memory 301.

§Procedure 3: Step S3 is executed according to the following procedure.
First, the setting contents of the reverse tilt setting switch 40 shown in FIG. 1 are taken into the mask image processing unit 306 in the ASIC 13 via the CPU 18. As described above, this setting content indicates which one of “reverse tilt shooting mode A” and “reverse tilt shooting mode B” is selected by the user.

  In the “reverse tilt photographing mode A”, the horizontal mask image A1 or the vertical mask image A2 shown in FIG. 6 is used. Image processing for blurring the image as it goes to the upper and lower end regions is performed. Accordingly, in both the horizontally long mask image A1 and the vertically long mask image A2, the central portion of the developed image is α = 1 (black), and the α value gradually increases as the center portion shifts upward and downward. = 0 approaching. The α value distribution curve is drawn on the left side of the horizontally long mask image A1 and the vertically long mask image A2. In the “reverse tilt photographing mode A”, the α value gradually decreases toward the upper end and the lower end of the developed image, but the α value in the lateral direction of the developed image has a constant value.

  In the “reverse tilt shooting mode B”, the horizontal mask image B1 or the vertical mask image B2 shown in FIG. 6 is used, and “the vertical region (vertical region) including the central portion of the developed image is focused, and the developed image Image processing for blurring the image as it goes to the left and right end regions. Accordingly, in both the horizontally long mask image B1 and the vertically long mask image B2, the central portion of the developed image is α = 1 (black), and the α value gradually increases as α shifts from the central portion to the left side and the right side. It approaches 0. The α value distribution curve is drawn below the horizontally long mask image B1 and the vertically long mask image B2. In the “reverse tilt shooting mode B”, the α value gradually decreases toward the left end and the right end of the developed image, but the α value in the vertical direction of the developed image has a constant value.

The alpha blend process performed using any one of the mask images shown in FIG.
When α takes a value of 0 to 1, it means that the calculation according to the following equation is performed for each pixel of the developed image.
Composite pixel value X _ij after α blend processing = (pixel value A _{ij of} developed image) × α + (pixel value B _ij of low-pass processed image) × (1-α) (Equation 1)

  The mask image processing unit 306 selects the mask image A1 and the mask image A2 as candidate mask images A ′ when the “reverse tilt photographing mode A” is designated by the user. Similarly, when the “reverse tilt photographing mode B” is designated by the user, the mask image B1 and the mask image B2 are selected as candidate mask images B ′. Next, the mask image processing unit 306 receives a detection signal from the tilt sensor 206 (see FIG. 3) and detects the posture of the camera body (see FIG. 2). As a result, if the posture of the camera body is horizontal, the mask image processing unit 306 selects the mask image A1 or B1 from the candidate mask image A ′ or the candidate mask image B ′. On the other hand, when the posture of the camera body is not horizontal (that is, when it is detected that the grip portion is on the upper side or the lower side), the mask image processing unit 306 determines from the candidate mask image A ′ or the candidate mask image B ′. Select mask image A2 or B2.

  The mask image shown in FIG. 6 is stored in a storage area (not shown) of the mask image processing unit 306, but may be stored in a mask image storage area (not shown) of the buffer memory 301. .

  Although four types of mask images are shown in FIG. 6, a mask image B2 is obtained by rotating the mask image A1 by 90 degrees, and similarly, a mask image B1 is obtained by rotating the mask image A2 by 90 degrees. Therefore, two types of mask images may be actually stored. Furthermore, although the term “mask image” conventionally used in the field of alpha blending processing is used in the first embodiment, it is natural that the α value corresponding to each pixel of the developed image is a generic name. And it's just called a mask image.

  From the mask image processing unit 306, the mask image signal P6 selected in this way is sent to the α blend module 303.

§Procedure 4: Step S4 is executed according to the following procedure.
The α blend module 303 includes a developed image signal P4 read from the developed image storage area M2 in the buffer memory 301, a low pass processed image signal P7 read from the low pass processed image storage area M3 in the buffer memory 301, and a mask image processing unit. The mask image signal P6 read from 306 is input, and the above (Equation 1) is calculated to obtain a composite image. The composite image is sent from the α blend module 303 to the buffer memory 301 as a composite image signal P8 and stored in the composite image storage area M4.

§Procedure 5: Step S5 is executed according to the following procedure.
The composite image stored in the composite image storage area M4 in the buffer memory 301 is subjected to a predetermined compression process, stored in the compressed image storage area M5, and then transferred to the recording module 309. In the first embodiment, a display image is created from the composite image by a live view processing circuit (not shown) and displayed on the display 21. As a result, the user can confirm an image (that is, a composite image) obtained by reverse tilt imaging in real time. It should be noted that a normal live view image may be displayed on the display device 21 and a composite image that has undergone alpha blending processing after completion of shooting may be displayed for a predetermined time.

-Actions and effects of the first embodiment-
According to the first embodiment, the following operations and effects can be achieved.

  (1) The camera according to Embodiment 1 includes an image processing unit 305 that forms a developed image based on an image signal obtained from the CCD 10, and a low-pass module 302 that forms a low-pass processed image by removing high-frequency components of the developed image. And an alpha value group for performing alpha blend processing on the developed image and the low-pass processed image are stored as mask image information, and any one of a plurality of mask image information depending on the shooting posture (horizontal / vertical) of the camera is stored. A mask image processing circuit 306 that selects one of them, and an α blend module 303 that performs alpha blend processing based on the mask image information output from the mask image processing circuit 306, the developed image, and the low-pass processed image. Therefore, even though the shift lens is not used, it is actually shot like a reverse tilt shot. Images can be obtained.

  (2) Reference mask images A1 and A2 in which alpha values corresponding to pixel positions are stored in advance, and another mask image is obtained by rotating the reference mask images A1 and A2 by 90 degrees according to the posture of the camera. Since B2 and B1 can be formed, the storage capacity can be halved and the reading time can be shortened.

  (3) When performing the α blending process, one of the mask images is selected from a plurality of mask images according to the reverse tilt photographing mode set in advance by the user and the posture of the camera that changes from time to time. The reverse tilt shooting effect intended by the user can always be obtained regardless of the shooting posture of the camera.

  (4) Since an α blended image is displayed for each frame during live view display, a composite image having a reverse tilt shooting effect can be confirmed in real time.

-Modification in Embodiment 1-
(Modification 1)
In the first embodiment described above, the reverse tilt shooting modes A and B have been described as shooting modes set by the user. However, a mask image corresponding to the position of the AF frame displayed by the camera of FIG. 1 may be automatically selected. Is possible.
FIG. 7 is a diagram showing the relationship between the AF area position and the mask image. The left column of the figure shows the mask image used for the α blend process, and the right column shows the position of the AF area with respect to the image at the time of photographing. As is apparent from FIG. 7, the center position of the mask image (position where α = 1) is changed so that the position of the AF area becomes α = 1.

  FIGS. 7A, 7B, and 7C are modifications of A1 shown in FIG. That is, in FIG. 7A, the center position of the mask image (position where α = 1) is arranged in accordance with the upper AF area position. In FIG. 7B, when the AF area is the center or left / right, the center of the mask image in the vertical direction (position where α = 1) is aligned with the position of the AF area as the center. In FIG. 7C, the lower AF area position is the center position of the mask image (position where α = 1).

  FIGS. 7D and 7E are modified examples of A2 shown in FIG. That is, FIG. 7D is a mask image when the camera is held on the grip or under the grip (see FIG. 2). At this time, if the AF area is on the lower side, the center position of the mask image (position where α = 1) is also moved downward. In FIG. 7E, when the camera is held on or below the grip and the AF area is on the upper side, the center position of the mask image (position where α = 1) is moved upward.

(Modification 2)
In the first embodiment, two types of reverse tilt shooting modes A and B can be set by selecting a menu screen of the camera. However, more detailed reverse tilt shooting modes A _n and B _n can be set. It is. Here, for example, n = 1, 2, 3. Moving More specifically, the reverse tilt shooting mode A ₁ when moving the mask image shown in A1 of FIG. 6 only slightly upward, the mask image shown in reverse by A1 in FIG. 6 only slightly downwards is a reverse tilt shooting mode a ₃ when to. The same applies when the mask image shown in B1 of FIG. 6 is finely moved in the left and right directions.

(Modification 3)
The tilt sensor 206 shown in FIG. 1 detects whether the posture of the camera is horizontal or vertical (see FIG. 2). However, by setting the resolution of the detection angle to 45 degrees, the mask image can be rotated by 45 degrees. As a result, reverse tilt photography suitable for various camera angles can be performed.

(Modification 4)
Although the mask images described so far have been stored in advance, the α value for each pixel can be calculated based on a predetermined arithmetic expression. As a result, a variety of mask images can be generated by preparing a plurality of arithmetic expressions.

(Modification 5)
When the α blend process is performed, it is possible to change the blur area of the composite image by changing the focal length of the photographing lens. That is, even when a specific mask image is selected, it is possible to obtain the same effect as when the α value distribution is substantially changed by utilizing the original imaging characteristics of the lens itself.

FIG. 8 shows a state in which the mask image substantially changes according to the focal length of the lens. 8A is taken at the wide end, FIG. 8B is taken at the intermediate position, FIG. 8C is taken at the tele end, and the left and right blur areas gradually expand. It shows how it goes. More specifically, the mask image shown in FIG. 8A is a basic mask image, and blur can be generated above and below the composite image. This corresponds to the mask image in the “reverse tilt photographing mode A” described in the first embodiment. However, as the image is advanced to the wide side, a blur area can be generated on the left and right.
As described above, the range of the left and right blur regions can be set to (a: wide end) <(b) <(c: tele end) according to the focal length. The same can be said for the mask image in the “reverse tilt photographing mode B” described in the first embodiment.

<Embodiment 2>
In the second embodiment, an image processing apparatus for inputting photographed photograph data and obtaining an image having a reverse tilt photographing effect is provided. This image processing apparatus includes the ASIC 13 shown in FIG. 4, and executes the processing procedure shown in FIG. That is, the image processing apparatus according to the second embodiment is
A high-frequency removing unit that removes high-frequency components from the captured image to form a low-pass processed image;
A mask image information output unit that outputs, as mask image information, an alpha value group for performing alpha blend processing on the captured image and the low-pass processed image;
A mask image information selection unit that selects any one of a plurality of mask image information based on information attached to the captured image;
A synthesizing unit that has a mask image information output from the mask image information selection unit, a captured image, and a synthesizing unit that performs an alpha blending process based on the low-pass processed image, and has a reverse tilt imaging effect. Output from.
By adopting such a configuration, it is possible to realize an image processing apparatus that inputs an image after completion of imaging and outputs an image that gives an inverse tilt imaging effect.

  Here, it is preferable to use Exif information as information attached to a captured image. For example, by reading out the vertical / horizontal direction of the image, the tilt angle information of the camera, the focus point, and the like, it is possible to select the mask image used for the α blend process and to control the rotation of the mask image.

<Embodiment 3>
In the third embodiment, an image processing program for inputting photographed photograph data and obtaining an image having a reverse tilt photographing effect is provided. This image processing program is stored in a program storage area (not shown) in the ASIC 13 shown in FIG. 4, and executes the processing procedure shown in FIG. That is, the image processing program according to Embodiment 3 is
A high-frequency removal step of removing a high-frequency component from the captured image to form a low-pass processed image;
A mask image information output step for outputting, as mask image information, an alpha value group for performing alpha blend processing on the captured image and the low-pass processed image;
A mask image information selection step for selecting any one of a plurality of mask image information based on information attached to the captured image;
A synthesis step for forming a composite image having an inverse tilt imaging effect by performing an alpha blending process based on the mask image information selected in the mask image information selection step, the captured image, and the low-pass processed image; Is executed by a computer.

The above description is merely an example, and the present invention is not limited to the above-described embodiments and modifications unless the features of the present invention are impaired.
It is also possible to combine the embodiment and one of the modified examples, or to combine the embodiment and a plurality of modified examples.
It is possible to combine the modified examples in any way.
Furthermore, other forms conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention.

10 CCD image sensor 11 Analog signal processing circuit 12 A / D conversion circuit 13 ASIC (image processing circuit)
16 CCD driver 17 Timing generator 25 Photometry device 26 Focus detection device 100 Lens barrel 200 Camera body 201 Main switch 204 Release button 205 Camera control unit 206 Tilt sensor 301 Buffer memory 302 Low pass module 303 α blend module 305 Image processing unit 306 Mask image Processing unit 309 Recording module

Claims (9)

Developing means for forming a developed image based on an image signal obtained from the image sensor;
High-frequency removing means for removing a high-frequency component of the developed image to form a low-pass processed image;
Mask image information output means for outputting, as mask image information, an alpha value group for performing alpha blend processing on the developed image and the low-pass processed image;
Mask image information selection means for selecting any one of the plurality of mask image information according to the shooting state of the camera;
Compositing means for performing alpha blend processing based on the mask image information output from the mask image information selecting means, the developed image, and the low-pass processed image;
A camera characterized in that a composite image having an inverse tilt photographing effect is output from the combining means. The camera of claim 1,
The mask image information output means includes a reference mask image storing an alpha value corresponding to a pixel position,
The camera according to claim 1, wherein the mask image information selection means outputs specific mask image information by rotating the reference mask image by a predetermined angle according to a photographing state of the camera. The camera of claim 1,
The mask image information output means is a processor that calculates the alpha value for each pixel by calculation to form the mask image information,
The mask image information selecting means outputs specific mask image information from the processor in accordance with a photographing state of the camera. The camera according to any one of claims 1 to 3,
The mask image information selection means selects any one of a plurality of the mask image information according to a photographing condition preset by a user and a posture of the camera. The camera according to any one of claims 1 to 4,
The mask image information selection means selects any one of a plurality of mask image information according to the selected AF area position. The camera according to any one of claims 1 to 4,
A camera characterized in that the composite image is displayed as a live view screen. High-frequency removing means for removing high-frequency components from the captured image to form a low-pass processed image;
Mask image information output means for outputting, as mask image information, an alpha value group for performing alpha blend processing on the captured image and the low-pass processed image;
Mask image information selection means for selecting any one of a plurality of mask image information based on information attached to the captured image;
Compositing means for performing alpha blend processing based on the mask image information output from the mask image information selecting means, the captured image, and the low-pass processed image;
An image processing apparatus, wherein a composite image having an inverse tilt photographing effect is output from the composite means. The image processing apparatus according to claim 7.
An image processing apparatus using Exif information as information attached to the photographed image. A high-frequency removal process that removes high-frequency components from the captured image to form a low-pass processed image;
A mask image information output step of outputting, as mask image information, an alpha value group for performing alpha blend processing on the captured image and the low-pass processed image;
A mask image information selection step of selecting any one of the plurality of mask image information based on information attached to the captured image;
By performing alpha blend processing based on the mask image information selected in the mask image information selection step, the captured image, and the low-pass processed image, a composite image having a reverse tilt imaging effect is formed. The synthesis process,
An image processing program that is executed by a computer.

Images