JP5441747B2 - Imaging apparatus and image processing method - Google Patents

Description

  The present invention relates to a blurring process for an image obtained by an imaging apparatus using a solid-state imaging device.

  Conventionally, diorama-like shooting in which the top and bottom of an image is blurred has been optically realized. In order to realize optically, for example, overhead view photographing is performed using a tilt shift lens which is a special lens. Further, the blurring effect of the image is not limited to the diorama style, and can be realized by image processing on a normal captured image. Japanese Patent Application Laid-Open No. 2004-151561 discloses a technique that can obtain an effect like soft focus by mixing an image obtained by blurring a normal captured image and an original image at a certain ratio. Moreover, in patent document 2, when displaying the picked-up image of the vehicle periphery by a vehicle-mounted camera on a monitor screen, the technique which makes the sharpness of an image display different in the other area except the said partial area and the some area of a picked-up image. Is disclosed.

JP 2000-69277 A Japanese Patent No. 3944926

  In order to realize a diorama-like still image by image processing, a plurality of stages of blurring processing in which the degree of blur is changed in a predetermined direction is necessary. If the relationship between the blur processing region and direction and the composition of the photographed image is appropriate, a good image in which the effect of the diorama effect can be visually recognized can be obtained. However, if the posture of the camera is changed while the blur processing area and direction are fixed, the direction and composition of the blur effect of the photographed image also change, which may result in an image that does not exhibit the expected diorama effect.

  The present invention is intended to solve the above-described problems, and enables an image having an expected diorama effect to be obtained even when the attitude of the imaging apparatus is changed.

In order to solve the above-described problems, an image processing apparatus of the present invention is an image processing apparatus that performs image processing on a captured image input from an imaging unit, and a region and directionality not to be blurred from the captured image. image processing means for producing an image having a have been blurred area, has a setting means for setting the direction of the blurring of the image to be generated by the image processing means, the setting means, sets a realm not blurred and, when changing the direction of setting of the blur, and sets the realm of the blurred not after the change according to the setting of the realm that is not blurred before change the.
The image processing apparatus of the present invention is an image processing apparatus that performs image processing on a photographed image input from the means, and includes a first area and a first area in a predetermined direction from the photographed image. Image processing means for generating an image having a second area that is more blurred, and setting means for setting the predetermined direction and the first area, and the setting means includes the predetermined direction. When the setting is changed, the first area after the change is set according to the setting of the first area before the change.
Also, the image processing method of the present invention is an image processing method for performing image processing on the obtained captured image by the imaging device, from the captured image, and a region blurred with areas and direction not blurred an image processing step of generating an image having, anda setting step of setting the direction of the blurring of the image generated by the image processing step, and in the setting step sets the realm not blurred, the blurring when changing the direction of setting, and sets the realm without the blurred after the change according to the setting of the blurred not realm before the change.

  According to the present invention, it is possible to obtain an image in which an expected diorama-like effect appears even when the posture of the imaging apparatus is changed.

The figure which shows the structure of the imaging device Operation flow for explaining diorama-like image processing according to an embodiment of the present invention Operation flow for explaining the photographing operation of the embodiment of the present invention Schematic diagram showing the relationship between the attitude of the imaging device and diorama processing Schematic diagram showing the relationship between the attitude of the imaging device and diorama processing

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 is a diagram illustrating a configuration of an imaging apparatus according to the first embodiment.

  The optical system 101 images a subject by forming incident light on the image sensor 102. The optical control unit 111 controls exposure, zoom, focus, optical blur correction, and the like of the optical system 101. The image sensor 102 converts the formed optical image into an electrical signal. The imaging control unit 112 is a control system for driving the imaging element. The analog signal processing unit 103 performs processing such as clamping and gaining on the output of the solid-state imaging device. An analog / digital (A / D) conversion unit 104 performs digital conversion on an analog signal and outputs it as image data. In the present embodiment, the imaging unit including the optical system 101 to the A / D conversion unit 104 is used. In addition, the imaging apparatus enables still image shooting by recording image data of a captured image output from the imaging unit on the recording medium 108 in a single frame. In addition, video recording is possible by continuously recording a plurality of frames. The signal processing unit 105 performs image processing described later on the captured image of the A / D converted digital signal to generate an output image. The internal storage unit 106 temporarily stores image data when generating an output image. The signal processing unit 105 is configured to be able to read out image data stored in the internal storage unit 106 according to an instruction from the system control unit 113, and performs diorama-like image processing such as generation of a blurred image, trimming processing, and image synthesis processing. I do. Details of the diorama-style image processing will be described later.

  The recording / reproducing unit 107 records the generated image data on the recording medium 108. As the recording medium 108, various removable recording media such as an optical disk recording medium and a semiconductor recording medium can be used. The image display device 110 displays the generated image data as a live image via the display control unit 109. The display device 110 having a function as an electronic viewfinder also performs in-focus frame display, in-focus display, and out-of-focus warning display according to the execution of the program in the system control unit 113. The system control unit 113 basically controls the overall operation of the imaging apparatus, and usually includes a CPU. For example, a predetermined calculation process is executed by the signal processing unit 105 using image data obtained by the imaging unit, and the optical control unit 111 and the imaging control unit 112 are controlled based on the calculation result. The optical control unit 111 performs control to execute automatic focus control (AF). The optical control unit 111 controls the diaphragm and shutter, and the imaging control unit 112 controls the electronic shutter of the image sensor 102 to execute automatic exposure control (AE).

  The posture detection unit 115 detects the posture of the imaging apparatus using an acceleration sensor and / or a gyro sensor used for the optical blur correction. The system control unit 113 determines the posture of the imaging apparatus based on the posture detection result output from the posture detection unit 115, displays setting information according to the posture, and changes the input direction of the cursor key according to the posture. . Reference numeral 114 denotes an operation unit. The operation unit 114 includes a shutter SW1 for instructing the imaging apparatus to perform a preliminary operation before photographing, a shutter SW2 for instructing the imaging apparatus to perform a photographing operation, a cursor key for instructing a position for performing image processing, and the like. The system control unit 113 performs AF and AE by detecting the shutter SW1, and performs a photographing operation by detecting the shutter SW2. The configuration of the imaging apparatus illustrated in FIG. 1 is an example, and the configuration of the imaging apparatus according to the present invention is limited to the configuration illustrated in FIG. 1 as long as the operations described below can be performed. It is not something.

  The diorama style image processing of this embodiment will be described with reference to FIG. In the present embodiment, an image in which the effect of the diorama effect can be visually recognized can be generated by performing the diorama image processing in any of the shooting operations of still image shooting, moving image shooting, and live display. In the present embodiment, common diorama-like image processing is performed in the shooting operation. However, since the image quality and frame rate required for each shooting operation are different, the contents of the diorama-like image processing may be changed according to the shooting operation.

  In FIG. 2, an image 200 shows the entire captured image input to the signal processing unit 105. An object of the present embodiment is to obtain an image in which a diorama effect appears mainly on a person in a scene including a building, a person, a tree, and the like. A frame 201 displays a range and position for focusing specified in the angle of view of the captured image. A person is included in the frame 201, and an image showing a diorama effect is obtained by performing a process of blurring the upper and lower regions. The strength of blurring the area can be arbitrarily set by the signal processing unit 105, and can be gently blurred.

  In the present embodiment, the frame 201 is also displayed on the live image displayed on the display device 110, the vertical position thereof is fixed at a position slightly below the center line 202, and the width 203 is also constant. It has become. The signal processing unit 105 continuously performs diorama-like image processing on each frame according to the position and width of the frame 201 so as to update the live image.

  Next, an operation flow of diorama-like image processing will be described with reference to FIG. This operation flow is realized by the system control unit 113 controlling each processing block.

  First, the system control unit 113 responds to the fact that one frame of a photographic image to be processed is obtained by performing development processing such as white balance, which is image data input by the imaging unit. Is started (S201). Here, at the start of the operation flow, fixed values for the position and width of the frame 201 are already set.

  The system control unit 113 determines the amount of blur in an area other than the frame 201 based on the degree of blur set by the user (S202). In the present embodiment, the degree of blur can be selected from “strong” and “weak” even during the photographing operation.

  Next, the system control unit 113 stores the image data of the captured image obtained by performing development processing such as white balance, which is image data input by the imaging unit, in the internal storage unit 106 (S203). This is the same as the photographed image when photographing in the normal mode.

  Then, the signal processing unit 105 creates an image in which the entire image is blurred from the image data of the captured image (S204). In the present embodiment, a blurred image is generated by reducing a captured image by a blinking process and then enlarging again to the original size. For example, if the selected degree of blur is “strong”, the reduction rate of the reduction process is 1/8, and if it is “weak”, the reduction rate of the reduction process is 1/4. The present invention is not limited to this method, and a blurred image may be generated using an image that has been optically captured in advance at an out-of-focus position.

  The signal processing unit 105 trims the portion of the frame 201 from the image data of the captured image stored in the internal storage unit 106 (S205). Then, the signal processing unit 105 aligns and pastes the trimmed image to the blurred image created in step S204, thereby generating an image in which the upper and lower sides are blurred and the sense of resolution remains in the center (S206). . In addition, when pasting image data of a photographed image, the pasting boundary portion is made inconspicuous by gently mixing the blurred image and the trimmed photographed image. That is, the process of setting the combined ratio of the captured image and the blurred image to 0: 1 at the boundary portion of the frame 201 and increasing the combined ratio of the captured image toward the center line of the frame 201 is performed.

  FIG. 3A is an operation flow for changing the contents of the diorama-like image processing in accordance with the posture change of the imaging device. This operation flow is realized by the system control unit 113 controlling each processing block. The content change operation flow can also be applied to moving image shooting and still image shooting. Here, the operation from the state of performing a live image display operation to the still image shooting will be described.

  In the state where the live image is displayed, this operation flow is started in response to the setting of the diorama-like shooting mode by the user's mode designation. After the start of the operation flow, the system control unit 113 determines the attitude of the imaging device based on the output signal of the attitude detection unit 115 (S301). Next, the system control unit 113 compares the current posture detected in step S301 with the stored previous posture. Here, the current posture is stored for use as the previous posture in the next posture comparison (S302). Next, the system control unit 113 determines whether the posture has been changed according to the comparison result (S303). If it is determined in step S303 that the posture has not been changed, the process proceeds to step S305. On the other hand, if it is determined that the posture has been changed, the process proceeds to step S304.

  In step S304, as shown in FIG. 3A, FIG. 3B, and FIG. 3C, the display direction of the frame 201 and diorama-like image processing are performed according to the orientation of the imaging device determined in step S301. Switch the direction.

  If the posture has not been changed (step S303: No), the direction is as it is, and if the posture has been changed (step S303: Yes), the direction is changed in step S304 in FIG. 2B. The described diorama-style image processing is performed (S305).

  Then, it is determined whether or not the shutter button has been pressed (S306). If it is determined that the shutter button has not been pressed, the process returns to step S301, and the display of the live image and the posture detection based thereon are continued, and it is determined that the shutter button has been pressed. Performs still image shooting processing (S307). Then, the diorama-like image processing described with reference to FIG. 2B is performed on the obtained still image (S305). Here, the image quality should be prioritized over the frame rate for the still image, so that the process of steps S204 to S206, which is performed using one blurred image, is performed using two or more blurred images, so that a diorama can be obtained. A still image with a wind effect may be obtained.

  The processing direction change processing in step S304 in FIG. 3A will be described in detail. FIG. 3B is a diorama-like image displayed on the display device 110 when the user holds the imaging device in the normal position. This is the same as that described with reference to FIG. 2A, and the range in focus and the position of the display frame are the frame 201. Here, the normal position is a position where the user holds the imaging device as shown in FIG.

  FIG. 3C shows a position where the user holds the grip position of the imaging apparatus downward as shown in FIG. 4B and rotates 90 ° to the right with respect to the normal position. FIG. 4B shows the processing direction at this time and that the frame 201 is rotated 90 ° to the left with respect to the normal position.

  FIG. 3D is a position rotated 90 ° to the left with respect to the normal position. The position of the frame 201 at this time has a symmetrical relationship with FIG.

  If the frame is fixed even if the imaging device is rotated 90 degrees in the left and right directions, the composition changes with respect to the scene 200 in FIG. 2A and the frame is displayed in the vertical direction, and the diorama processing is performed. Therefore, the expected diorama effect cannot be obtained. On the other hand, as in this embodiment, by rotating the direction of the frame as shown in FIGS. 3C and 3D according to the posture change, the effect of the diorama effect can be obtained even if the composition changes. Since the direction can be maintained in the state before the rotation, a good image can be obtained.

(Second Embodiment)
In the present embodiment, during the live image display operation, the user can set the vertical position and width of the frame 201 using the operation unit 114. The signal processing unit 105 executes diorama-like image processing according to the set position and width of the frame 201 so as to update the live image.

  At this time, when the direction of the diorama-like image processing is changed due to the posture change, it is preferable that the diorama-like image is continuously displayed with the position and width of the frame 201 set by the user.

  A method for setting the frame 201 (the range and position where the image is in focus) when rotated 90 ° to the right from FIG. 4 (a) from the normal position in FIG. 4 (a) will be described with reference to FIG. Although description about the case of 90 ° rotation to the left is omitted, since it can be realized with the same configuration, in the following description, it is assumed that the rotation is 90 ° to the right or 90 ° to the left. Express.

  When the scene shown in FIG. 4C similar to that of the first embodiment is photographed to obtain a diorama effect, the user photographs with a composition of either the normal position or the vertical position. Since the in-focus range and position (frame 201) are set according to the person who is the main subject, regardless of the normal position (402) and vertical position (401), the user's specification is reflected and the width is almost the same. It is necessary to set it. Further, the position of the frame is also set so that the vertical balance is within a certain region regardless of the normal position (402) and the vertical position (401) so that the balance of the composition is not lost.

  FIG. 5 is a diagram illustrating an example in which the above setting is performed according to a change in posture.

  FIG. 5A shows an example in which the width of the frame 201 at the normal position is set to M, and shows the case of the composition 402 in FIG. 4C. Normal position When the posture is changed from the vertical position shown in FIG. 5A to the vertical position shown in FIG. 5B, the width of FIG. 5B in the vertical position according to the width M of FIG. 5A in the normal position. Set P. Here, there is a relationship of width M: vertical width X of the captured image = width P: horizontal width Y of the captured image. Using the same ratio relationship, the width O and the width Q in FIG. 5B in the vertical position are set according to the width L and the width N in FIG. Vertical Position Even when the posture is changed from FIG. 5B to the normal position FIG. 5A, the width L, the width M, and the width N are set according to the width O, the width P, and the width Q.

  FIG. 5C is an example in which the position of the frame 201 at the normal position is set to the center position S, and shows the case of the composition 402 in FIG. 4C. The center position S of the frame 201 in FIG. 5C at the normal position is set as a ratio to the height X of the entire screen. When the posture is changed from FIG. 5C to FIG. 5D in the vertical position, the center position U of the frame 201 can be easily obtained from the width Y of the entire screen by the formula: U = Y × S ÷ X. Is possible. Further, when the posture is changed from FIG. 5D in the vertical position to FIG. 5C in the normal position, it is calculated from the equation S = X × U ÷ Y. Here, an example in which the center position U, the center position S, and the like are inherited at a ratio has been shown, but the position may be calculated from the upper end of the screen or the number of pixels from the screen center.

  The setting of the frame 201 has been described separately for the width and the position. However, if the width M of FIG. 5A is set from the center position S of the frame 201 of FIG. 5C, FIG. ) Can be set from the center position U of FIG. In this case, as shown in FIG. 4D, when the user sets the frame 201 near the lower end of the screen at the vertical position 404 and resets it to the normal position 405, the center position U of the frame is increased when the set width P is wide. And S may correspond to each other, but the frame range P may protrude from the composition. Therefore, when the frame is set at the position that touches the lower end, the center position relationship is not inherited as it is even if the posture is changed, and the center position is set so that the lower end position is maintained while maintaining the width of the frame. To do. Although not shown, the same processing is performed when the frame 201 is set near the upper end.

  As described above, in the present embodiment, in the specification in which the user can set the position and width of the frame for performing the diorama processing, even when the orientation of the imaging apparatus changes, the expected diorama processing An image showing the effect can be obtained.

(Other embodiments)
In the above-described embodiment, the display of live images and the diorama-like image processing for still image recording have been described. However, the present invention is not limited to this, and live display in the case of performing moving image recording or for moving images to be recorded Is also applicable.

  In the above embodiment, the term “diorama-like image processing” is used as a special effect. However, this is for convenience, and the captured image and one or more blurred images are given directions and each region is changed. The present invention can be applied to other special effect processing as long as it includes processing to synthesize.

  In the above embodiment, the case where the present invention is applied to an imaging apparatus has been described. However, the present invention is not limited to this, and a personal computer, a digital television, or the like that can input a captured image from an externally connected imaging apparatus. It may be realized in the image processing apparatus. In this case, the posture is determined from the posture detection information included in the header information in the image file of the photographed image, and diorama-like image processing corresponding to the posture is performed. In the case of a personal computer, it is realized by obtaining and installing application software for executing the steps shown in FIG. 3 from a storage medium such as a CD-ROM or a server on a network.

105 signal processing unit 106 internal storage unit 113 system control unit 115 attitude detection unit

Claims (7)

An imaging apparatus that performs image processing on a captured image input from the imaging unit, and that generates an image having an area that is not blurred and an area that is blurred with directionality from the captured image;
Setting means for setting a direction of blur of an image generated by the image processing means, and the setting means sets a position or width of a region not to be blurred, and changes the setting of the direction of blur. An imaging apparatus, wherein the position or width of the non-blurred area after change is set according to the setting of the position or width of the non-blurred area before change. Detecting means for detecting the posture of the imaging device;
The imaging apparatus according to claim 1 , wherein the setting unit sets a blur direction of an image generated by the image processing unit in accordance with an attitude detected by the detection unit. The setting means, when changing the setting of the blur direction, based on the relationship between the length of the captured image in the direction before the change and the position or width of the non-blurred area, does not blur the change in the direction after the change the imaging apparatus according to claim 1 or 2, characterized in that setting the position or the width of the region. When the setting unit changes the setting of the blur direction, the ratio of the width of the non-blurred area and the height of the blur direction of the entire photographed image is not blurred before and after the change. the imaging apparatus according to any one of claims 1 to 3, characterized in that setting the width. An image processing method for performing image processing on a captured image obtained by an imaging apparatus, and generating an image having a region not blurred and a region blurred with directionality from the captured image;
A setting step for setting a blur direction of the image generated in the image processing step. In the setting step, a position or a width of an area not to be blurred is set, and the setting of the blur direction is changed. An image processing method comprising: setting the position or width of the non-blurred area after the change according to the setting of the position or width of the non-blurred area before the change. A computer-executable program in which the steps of the image processing method according to claim 5 are described. A computer-readable storage medium storing a program for causing a computer to execute the steps of the image processing method according to claim 5.

Images